Monitoring, Evaluation and Learning Systems

My name is Nicky Bowman and I am the President of Bowman Performance Consulting (BPC)

BPC is a nationally award winning grant writing and scientific consulting firm.  I’ve not only written grants but more importantly I’ve WON them for clients. I’m so happy to share some lessons learned, quick tips, and resources with you so you are equipped to win at grant writing too!

Lesson Learned:  Don’t Believe the Man in the Dollar Suit!

Remember Matthew Lesko who ran around in a dollar suit and screamed enthusiastically about all the “free money” that anyone could have?  Decades later, many people still think there is a ton of “free money” available.  Given the economy, government budget cuts, and highly competitive nature of grant writing, there is much less funding to go around today.

Hot Tip: Patient, consistent, and methodical grant writing will help you win and sustain funding. 

Lesson Learned:  Be Strategic About Which Grants you Write!

Understand what grants are best suited for your organization or program and what is most realistic based on the funder’s history of grant awards.  Consider your mission, vision, and goals.  Choose grants that best align with the purpose of your work.

Rad Resource:  Use a funding research worksheet to find the appropriate grants.

Hot Tip:  Your organization has a grant writing resume. Start at an entry level position, gain experience and skills, and work your way up. If you are a beginner do not go for a complex multi-million dollar award as an “entry level” effort.

Lesson Learned:  Get Real and Get Real Busy!

Make an honest assessment about the capacities and expectations you have for grant writing and supporting the implementation of grant awards.  Know who you are by strategically completing your grant research, and carefully select the grants which are best suited.

Rad Resource:  Allow enough time to write the grant by backwards mapping each step.

Hot Tip:  Plan a year in advance.  Look for free online webinars, YouTube channels/videos, Twitter feeds, and listserves to help you better understand grant writing.  Contact program officers and ask for exemplary sample proposals.  Find federal offices that need grant reviewers (who “score” applications to decide who gets funding). This gives you the inside track on what the funding agency expects.

Follow aea365, attend my second AEA Coffee Break Webinar on June 6th  and visit the BPC website. Request resources from me via e-mail, nicky@bpcwi.com or social media: Twitter @NBPC1 and LinkedIn.

Rad Resource: Check out my materials in the AEA Public eLibrary.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Susan Kistler on Funding Opportunities for Evaluators
2. Laura Sefton on Searching for Grant Funding Opportunities
3. Susan Kistler on Locating Grants and RFPs

Peter Sims, author of Little Bets: How Breakthrough Ideas Emerge from Small Discoveries, recently spoke at a plenary session moderated by Grant Oliphant, president & CEO of The Pittsburg Foundation, during CEP’s national conference  on Tuesday, May 21. Little Bets breaks down the elements of design thinking by sharing stories and revealing the mindsets and practices of some of the most creative thinkers we know, from Beethoven to Pixar Studios.

So what is design thinking? Tim Brown, president and CEO of IDEO, describes it as such:

Design thinking is a deeply human process that taps into abilities we all have but get overlooked by more conventional problem-solving practices. It relies on our ability to be intuitive, to recognize patterns, to construct ideas that are emotionally meaningful as well as functional, and to express ourselves through means beyond words or symbols. Nobody wants to run an organization on feeling, intuition, and inspiration, but an over-reliance on the rational and the analytical can be just as risky. Design thinking provides an integrated third way.

Brown identifies three spaces working in relationship to one another not in sequence but overlapping that support design thinking:

1. Inspiration – the problem or opportunity that motivates the search for solutions.
2. Ideation – the process of generating, developing, and testing ideas.
3. Implementation – the path that leads from the project stage into people’s lives.

Sims’ research findings build on and simplify Brown’s definition to a set of often-counterintuitive experimental methods that innovators not just practice but embrace:

• Failing quickly to learn fast
• Trying imperfect ideas
• Engaging in highly immersed observation—that free their minds, opening them up to making unexpected connections and perceiving invaluable insights

He notes that these methods unshackle innovators from the constraints of conventional planning, analytical thinking, and linear problem solving often overemphasized at the expense of creativity.  And in his mind, the philanthropic sector is uniquely positioned to integrate a design thinking mentality to its core way of being.

What are the implications for philanthropy? As with many things, it depends.  And even more so, it depends on the foundation. And with that in mind, I offer the following questions, based on my close to 20 years of working with philanthropic sector organizations, to those of you wondering if you are ready to make little bets:

1. What is your risk tolerance?  Not just at the staff level but at the Board level. In short, organization wide.
2. Can you create an environment and organizational culture that provides the space to “try?”
3. What current processes and practices can you adapt or adopt to be able to use the information learned from trying?
4. How might you shift your use of or understanding of data, information and evaluation to support design thinking?
5. Do you have staff/advisors who either have or can build the capacity to be creative?
6. Which is more important to you, the means or the ends? If it is the former, think twice.
7. Given responses to the previous six questions, now what?

Jara Dean-Coffey is the principal and founder of jdcPartnerships. 

Hello, I am Kerry Bruce, the Director of Results and Measurement at Pact.  I’m currently based in Madagascar and support Pact programs in Africa, Asia and Eastern Europe. I am part of Pact’s central technical team that provides monitoring and evaluation support to more than 20 country offices and more than 70 projects around the world.  In 2012 we started to roll out the use of mobile technology in our programs including evaluation.  This is the second post on mobile technology and focuses on choosing your platform. Check out my first post on this topic: Getting Started with Mobile Phones.

Hot Tips:

• Look at a wide range of available platforms and ask yourself:
  • What is my budget for phones?  Some platforms work better than others with entry level (vs. Android) smartphones.
  • What is my budget for the data collection?  Will my data collection reoccur frequently (on-going evaluation) or is this a one time event?  Each platform has a different pricing structure and each lends itself to different types of data collection.
  • Platform operators will promise you the moon – but will their platform deliver?  Test basic issues such as skip logic, ease of set-up and use, how data download and dashboards work before you buy.  Most platforms have a trial version that you can use and some allow small data collection projects for free.
  • Will I need help to set up my survey, or do I have the skill set to set it up in house?  Some platforms offer survey set up and technical support (useful for complicated data collection exercises) and some are all do-it-yourself.
  • What language will the survey be in and can the platform support it?  This is especially important for non-Latin alphabets.
  • Get a reference.  All these platforms should be able to provide you with a reference from someone who has used them before and can tell you what is good and what needs work.

Lesson Learned: Evaluate two or more platforms before you decide which one to use. 

• Some have recurrent or annual costs and others only charge for the data that you collect.  Others are free up to a certain level of data collection.
• Each platform has its strengths (and weaknesses) – you’ll need to understand what you need it to do and shop around until you find it.
• Just because a platform can not do something today does not mean they won’t be able to do it tomorrow, check back and give feedback.  This technology is rapidly adapting.

Rad Resources: Here is a list of some of the mobile technology platforms that are commercially available today.

*These are platforms I have used.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Susan Kistler on Lessons Learned Using Online Survey Software
2. Kerry Bruce on Getting Started with Mobile Phones
3. Shortcut Week: Corinne Poth on Skip Logic for Online Surveys

I’m Chad Green, Program Analyst at Loudoun County Public Schools in Ashburn, VA. For over seven years I’ve served as an internal evaluator of instructional initiatives sponsored by central office administrators.

Do you have an interest in understanding school-based professional development from a sociocultural learning perspective?  Read on! Years ago I evaluated two school-wide improvement initiatives using an integrated conceptual framework.  The first component was Learning Forward’s original context standards which today serve as its first three standards for professional learning. The purpose of this framework was to constrain the data to essential long-term staff development outcomes.  The second component (Honig, 2008) operationalized the first one into six overlapping sociocultural learning practices, two for each context standard (see below).

Framework for High-Quality, School-Based Professional Development

I.   Skillful leadership is evidenced when school and central office staff:

1. Model high quality teaching and learning practices
2. Boundary span to connect staff with new sources of expertise

II.  Professional learning communities are evidenced when school and central office staff:

1. Interact at a high level of collaborative inquiry
2. Engage in joint work on authentic tasks that are meaningful and sustained over time

III. Dedicated resources are evidenced when school and central office staff:

1. Provide access to ongoing, job-embedded learning opportunities that increase the level of participation in shared work practices (i.e., from novice to expert)
2. Develop common conceptual and practical tools (e.g., principles, frameworks, routines, language, protocols, templates, materials)

Lesson Learned:  The patterns that emerged from the data were surprising on two levels.  At a superficial level they revealed a continuum of leadership approaches to program implementation ranging from a top-down, hierarchical structure on one end to a more subtle, heterarchical structure on the other. Coincidentally, these leadership structures aligned with the level of diversity (i.e., complexity) of the school’s student populations.  At a deeper level, the findings suggested a connection between each school’s sources of power and knowledge (i.e., truth).  In the top-down structure, tacit knowledge was concentrated in the principal and specialist roles (i.e., authority) whereas in the heterarchical setting knowledge was more explicit in the form of online repositories of co-created tools and resources.

Hot Tip:  Since then, I have learned that I am much more effective when I help central office administrators integrate their prepackaged conceptual frameworks (i.e., programs) into coherent strategic thinking portfolios which facilitate increased experimentation and interconnectedness system-wide.

Rad Resource: Check out Honig’s journal article on district central office as learning organizations.

Final Word: Both schools’ staff development programs were equally effective in the short run with respect to implementation and outcomes.  Which school structure do you think will be more sustainable in the long run?

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. PD CoP Week: Donna Campell on Evaluating Professional Development: Guskey Level 3 – Organizational Support and Learning
2. PD CoP Week: Marguerite Dimgba and Sheila Robinson Kohn on Evaluating Professional Development: Guskey Level 4 – Participant’s Use Of New Knowledge And Skills
3. PD CoP Week: Anne D’Agostino, Jennifer Coyle, and June Gothberg on Building a Cross-Community Collaborative Evaluation Community of Practice

With its recent report, Foundation Transparency: What Nonprofits Want, the Center for Effective Philanthropy examines the nonprofit perspective, outlining the benefits of foundation transparency to the partners we support. As we continue to explore the merits of greater transparency in philanthropy, I propose we add another dimension to the discussion by asking: What about transparency in foundation boardrooms?

In my view, one of the fundamental duties of a foundation CEO is to construct time with the board in ways that engage trustees in substantive ways and that add value to the work of the foundation. Information sharing certainly deserves its place on board meeting agendas, but it should not comprise the majority of the board’s time together. Rather, we should be building into our board agendas opportunities for strategic dialogue, robust engagement, and, yes, even debate and disagreement. And, as we do so, a transparency mindset is essential.

In the decade I have been privileged enough to lead a foundation, philanthropy as a sector has come a long way with respect to transparency. But, as the CEP report points out, we have much more work to do, especially when it comes to providing more insight into our internal processes, the ways we make decisions, and how we measure and assess our performance as foundations.

We also have more work to do with our boards, so let me offer three questions that we should be asking in order to bring greater transparency to foundation boardrooms:

• What’s not working?
• How can the board learn more?
• Who are our partners, allies, and critics?

What’s Not Working? Reading any business publication or blog these days reminds us of the accepted conventional wisdom that we learn a great deal from failure. Indeed, in some circles, failures are trumpeted with pride and viewed as badges of honor. Alas, this is a much more nascent trend in philanthropy.

Paul Brest, former president of the William and Flora Hewlett Foundation, often spoke of engaging his program colleagues in robust discussions each year of failed grants—and even failed grantmaking strategies—at Hewlett. He reported that the staff found these discussions to be especially engaging and instructive. What if we brought this kind of conversation and candor to our boardrooms?

There is often a fear in exposing the board to failure for a variety of reasons, including the prospect of having them wonder whether they have the right staff. But, it has been my experience that talking about failures counter-intuitively lowers the stakes because the board appreciates the openness by the staff and honestly doesn’t want to hear that everything is going just fine (they are smart enough to know it’s not). By engaging the board in constructive conversation about what’s not working, we collectively solve problems and engage them in the real (and challenging) work of the foundation. This not only contributes more usefully to the foundation’s work itself, but it leads board and staff to work more effectively as partners.

How Can the Board Learn More? In preparing for our board meetings, we often aim to provide sufficient information for the board to understand the context for the decisions we ask them to make. This is what leads to the voluminous board books that do their share of damage to the environment.

Today, technology enables board members to learn much more about the subjects we discuss together, and a transparency mindset can harness this technological power to great effect. At Irvine, we will introduce a new grants database this summer, powered by a Salesforce platform. This is not only going to provide our staff with much greater functionality, but now we will be able to provide our board members with greater insight and background on our grant recommendation. We made the transition to electronic board materials a few years ago, which has served as a useful first step. Now, with this new database, we intend to provide our board with access to background documents, proposals, budgets, and other materials that will help them to have the necessary context for our work together. They can explore as much as they want, and we can presume that they have the necessary background information to inform substantive and strategic discussions at our meetings.

Who Are our Partners, Allies, and Critics? Another way to advance transparency in the boardroom is to bring more voices to the discussion. Without having done a scientific review of our board agendas at Irvine, I suspect that 80 percent of board meeting time each year is spent with one another or with our staff. In retrospect, this may not be the right balance.

Exposing the board to other voices and perspectives—those of our partners, allies, and even critics—should be a higher priority. Doing so enables the board to understand the context and even complexity of our work. And if we are able to structure these conversations in a way that encourages candor and downplays the power dynamic between a funder and grantee, then the conversations can be even richer. While we have not done enough of this at Irvine, we have invited guest speakers, both to broaden the board’s understanding of our context and to have them meet key partners in the field. We have also brought other foundation colleagues to meet with the board, so they could understand how our peers approach their work and strategies. And we have made an effort to bring other Irvine staff, both formally and informally, to our meetings so that the board can meet and engage with the broad diversity of the Irvine team. These are good first steps, as bringing these other voices and perspectives may serve to enrich our discussions.

***

I am not necessarily arguing here that transparency equals philanthropic effectiveness. Indeed, some have thoughtfully questioned whether we are making that conceptual leap too quickly. (On that note, for a careful articulation of this argument, I commend John Tyler’s Transparency in Philanthropy: An Analysis of Accountability, Fallacy, and Volunteerism, published by The Philanthropy Roundtable.)

What I am arguing, however, is that adopting a transparency mindset in the boardroom does engender greater authenticity in the relationship with our boards.  And such authenticity—and the honesty and trust that comes with it—can only be an asset as the boards and staffs of our country’s foundations grapple with some of the biggest challenges facing our society today. Those we aim to serve by our philanthropy deserve no less, and our institutions will be healthier as a result.

Jim Canales is President and CEO of The James Irvine Foundation. You can find him on Twitter @jcanales.

Hear Jim speak about getting the best from foundation boards at our national conference, Pursuing Results, tomorrow and join the discussion at #CEP13.

Our names are Marc Wheeler and Salem Valentino and we are internal evaluators for Big Brothers Big Sisters of America.

There is a lot of buzz today about infographics.  Many of you may have thought about using infographics in your evaluation reports to try to translate your findings more effectively.  We recently took the plunge and incorporated infographics in our 2013 Youth Outcomes Report .

For example, we created the infographic below to clarify the theoretical connection between the short-term outcomes we currently measure for each youth and those long-term outcomes of interest to many stakeholders. The graphic summarized a large quantity of research literature in a single infographic that was easily interpretable and concise.

Lessons Learned:

1. High-quality infographics require a certain level of expertise; as we didn’t have the relevant experience in house, we contracted with an external graphic designer who delivered great results.
2. Beyond experience, it also takes time to develop your infographics and get them right.  First, you need to figure out the story behind your data.  Then, what are the best ways to illustrate this story, while remaining true to the data?  For instance, we wanted to communicate effect sizes but didn’t want to take up space in our report explaining what they meant to a lay audience.  So instead, we developed the visualization below to better illustrate our story of the magnitude of youth outcomes.  Lastly, infographics require a number of iterations and can benefit from the input of diverse audiences.  Budget your time accordingly.

1. Understanding your project needs will help you choose the right designer. We looked at several resources on the internet to find a graphic designer. For us it was important that our work with the designer was collaborative; we wanted to ensure the quality of the evaluation content. Due to our timeline, it was also important that our designer could design the entire report and not merely the infographics.  You may also want to ask your designer how comfortable they are with Excel or other types of data you will be using in the report.

Rad Resources:

Elissa Schloesser at Visual Voice – our designer’s 5 Steps for Translating Evaluation Findings into Infographics

Visual.ly’s Marketplace service will find a designer for you and help you create an infographic for one price.

Easel.ly is a website where you can create your own infographic for free.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Sarah Rand on Using Visual.ly’s Marketplace Service for Infographics
2. Elissa Schloesser on 5 Steps for Translating Evaluation Findings into Infographics
3. Susan Kistler on visual.ly and #eval

As organized philanthropy descends upon Detroit for this year’s conference of the Center for Effective Philanthropy, we will confront first-hand one of the most challenging urban environments in our nation. One of the trickiest issues for foundations is how to balance the way we represent the grim reality of cities like Detroit with inspiring accounts of heroic efforts to revive the city.

Likewise, journalists and documentary filmmakers face a similar dilemma: how to tell hard truths they observe in their reporting, without having to tailor their accounts to advance the public relations demands of philanthropic benefactors, which are becoming increasingly prominent players in an otherwise declining news industry.

Nowhere is this tension more clear than the case of Detropia, which will be seen in a national broadcast on PBS next week. Participants in the CEP conference in Detroit will have the chance to get a sneak preview tonight, after the opening reception.

Detropia recently won the Henry Hampton Award for social documentary film in the 46th Annual Film and Video Festival, a joint project of the Council on Foundations and Media Impact Funders.

But a previous screening last year with foundation and nonprofit leaders in Detroit touched a raw nerve.  “They just rejected the film outright,” according to Rachel Grady, who directed the film with Heidi Ewing. “They told us we missed the story; we missed the dozens of rays of light that were being planted all over Detroit.”

Grady says that they did look for stories of gritty determination – and indeed some of those stories are included in the film – but she says that those stories were overshadowed by the enormity of the problems facing Detroit.

Foundation leaders will have a chance to judge for themselves how well Detropia balances the story of modern Detroit in tonight’s screening, while everyone else will have a chance next week. And funders who would like to engage more directly with the filmmakers will hear from Grady and Ewing in person at the Media Impact Forum, which will take place June 4 at the Ford Foundation.

This post is excerpted from the current edition of The Chronicle of Philanthropy, where I write an occasional column.

Vincent Stehle is the executive director of Media Impact Funders and a member of the board of directors at the Center for Effective Philanthropy. You can find him on Twitter @VinceDaily.

Hear Vince speak at our national conference, Pursuing Results, and join the discussion at #CEP13.

Greetings! I’m Nichole Stewart, a doctoral student in UMBC’s Public Policy program in the evaluation and analytical methods track. I currently work as an analyst, data manager, and evaluator across a few different sites including Baltimore Integration Partnership, Baltimore Workforce Funders Collaborative, and Carson Research Consulting Inc.

Lessons Learned: The Growing Role of Data Science for the “Little” Data in Program Evaluation. Evaluators are increasingly engaged in data science along every step of the evaluation cycle. Collecting participant-level data and developing indicators to measure program outputs and outcomes is now only a small part of the puzzle. Evaluators are working with more complex data sources (administrative data), navigating and querying data management systems (ETO), exploring advanced analytic methods (propensity score matching), and using technology to visualize evaluation findings (R, Tableau).

Evaluators Also Use Big Data.  Large secondary datasets are appropriate in needs assessments and for measuring population-level outcomes. Community-level data, or data available for small levels of geography, provide context and can be used to derive neighborhood indicators. Evaluators must be able to not only access and manipulate this and other kinds of Big Data but to ultimately learn to use data science to maximize the value of the data.

Rad Resource: The American Community Survey (ACS)  is an especially rich, although recently controversial, Big Data resource for evaluators. The survey offers a wide range of data elements for areas as small as the census block and as specific as the percent of carpoolers working in service occupations in a census tract.

Hot Tips:

• View the latest technical document for the ACS 5-Year estimates for a list of available data elements (p. 43).
• Visit the National Historical Geographic Information System to download Census and ACS data as well as GIS boundary files for mapping.

Rad Resource: The Census Bureau’s OnTheMap application is an interactive web-based tool that provides counts of jobs and workers and information about commuting patterns that I explored in an AEA Coffee Break webinar.

Lessons Learned: Data Science is Storytelling: Below is a map of unemployment rates by census tract from the ACS for Baltimore City and surrounding counties.  This unemployment data is overlaid with data extracted from OntheMap depicting job density and the top 25 work destinations for Baltimore City residents.  The map shows that 1) there are high concentrations of unemployed residents in inner-city Baltimore compared to other areas, 2) jobs in the region are concentrated in Downtown Baltimore and along public transportation lines and the beltway, and 3) many Baltimore City workers commute to areas in the surrounding counties for work.  Alone, these two datasets are robust but their power lies in visualizing data and interpreting relevant intersections between them.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Taj Carson on Visualizing Neighborhood-Level Data with Baltimore DataMind
2. MaryAnn Sorensen Allacci on Spatial Analysis
3. DVR Week: Chris Lysy on Data Sources that Embrace Data Visualization

I’m Clara Hagens. I work for Catholic Relief Services (CRS) as the Regional Technical Advisor for Monitoring, Evaluation, Accountability and Learning in Asia. I’d like to share with you a guidance document we have developed to support project teams to operationalize monitoring and evaluation (M&E) plans, big and small, in various contexts.

Rad Resource: CRS’ Guidance on Monitoring and Evaluation covers a range of topics related to basic M&E concepts and to designing and implementing M&E activities. The topics include gender in M&E, random and purposeful sampling, developing qualitative data collection tools, M&E in emergencies, and community participation in M&E to name a few. Each topic is grounded in a set of standards to guide our M&E practice. The standards are accompanied by narrative to explain how each can be achieved, tips and good practices, examples, and planning tables and templates.

For example, the Guidance provides standards for Community Participation in M&E that state that M&E systems track the changes most important to communities and communities participate in data collection and in the interpretation of M&E results and includes tips for each step in the process. The standards for Planning and Conducting an Evaluation refer to the importance of developing project-specific evaluation questions related to the relevance, effectiveness, efficiency, impact and sustainability. The Guidance also includes a simple evaluation planning table which helps teams to link the methods for data collection and respondents to the evaluation questions.

The CRS Guidance on Monitoring and Evaluation is appropriate for project teams who are looking for additional hands-on support to further engage with their M&E systems. I hope you will find this useful!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. PD CoP Week: Donna Campell on Evaluating Professional Development: Guskey Level 3 – Organizational Support and Learning
2. Chris Camillo on Improving Monitoring Systems from an Auditor’s Perspective
3. Greg Lestikow and Fatima Frank on Three Basic Evaluation Presentations

Changes are afoot! My name is Susan Kistler and I am just about to become the American Evaluation Association’s Executive Director emeritus. I’ve written before about my stepping down and my last official day in this position is this coming Monday, May 20. I’ll continue to write for aea365, and will be working on the Evaluation 2013 conference program, but will be moving on to new and exciting opportunities and serving AEA primarily in a volunteer capacity. The new staff team coming on board brings a breadth and depth of knowledge of management and communications that represents a significant increase in AEA’s organizational capacity. I can’t wait to see what new things come about in the months ahead.

Rad Resource – Denise Roosendaal: AEA is searching for a permanent Executive Director (see the position description here). In the meantime, Denise Roosendaal, CAE, is the interim ED and may be reached at droosendaal@eval.org. Denise brings to the position twenty-five years of experience in running nonprofits and associations and she can’t wait to get to know more about AEA and evaluation. If you are attending the Summer Institute next month in Atlanta, be sure to stop by the registration desk and to say “hello.”

Lessons Learned – What to Expect: You likely won’t see major changes in AEA’s programs and services in the near-term and only improvements in the long-term.  aea365 will continue to come out daily under the watchful eye of Sheila Robinson, our lead volunteer curator (she may be reached at aea365@eval.org). The association will have a new phone number (1-202-367-1166) but the old one will forward until the end of June. There will be new voices on the other end of the phones, but with the same commitment to serving AEA. We ask that you are patient in the next few weeks as the new staff gets up to speed. If they don’t know the answer, they’ll find it and get back to you expeditiously.

Get Involved – Share Your Ideas: Have a great idea for something new for AEA? Email Denise at droosendaal@eval.org or put it in the comments on aea365.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. FIE/MME Week: Donna Podems on Applying Feminist Evaluation for Non-feminist Evaluators

Hello, I am Vanessa Hiratsuka, secretary of the Alaska Evaluation Network (AKEN) and a senior researcher at Southcentral Foundation (SCF), a tribally owned and managed regional health corporation based in Anchorage, Alaska, which serves Alaska Native and American Indian people.

As part of Commitment to Quality, a key organizational value, Southcentral Foundation (SCF) prioritizes continuous quality improvement (CQI), quality assurance, program evaluation, and research.

Although the strategies and tools used in CQI, quality assurance, program evaluation, and research are similar, we do different things. One of our challenges is to help staff across the organization understand who does what. Because these four fields differ in aim and audience, exploring the goals of a project (aim) and who will use its findings (audience) provides a useful framework to determine where a project fits.

At SCF, improvement staff work directly with SCF department and clinic processes to develop and implement project performance measures and outcome indicators as well as help staff (audience) improve processes to better meet customer-owner needs and inform business directions (aim).  Quality Assurance staff conduct quality monitoring to ensure programs are complying (aim) with SCF processes and the requirements of our accrediting bodies (internal and external audiences).

SCF internal evaluators measure programs’ performance (aim) and provide feedback to programmatic stakeholders — including staff, leadership, and funders (audience). The SCF research department’s projects address questions of clinical significance to contribute to generalizable knowledge (aim) for use within SCF and for dissemination in the scientific literature around American Indian and Alaska Native health (audience).

Lessons Learned:

-        Define the aim and intended audience early in the process! This helps identify the stakeholders, level of review, and oversight needed during all stages of a project, including development, implementation, and dissemination of findings.

-        Broadly disseminate findings! Findings and recommendations from all disciplines are only useful when they are shared. At SCF, findings are shared at interdivisional committee meetings and with staff who oversee the work of departments. Multipronged dissemination ensures involvement from all levels of SCF and supports innovation and the spread of new knowledge.

-        Project review can be complicated!  At SCF, research projects must be vetted through a tribal concept review phase, an Institutional Review Board review, and finally a tribal review of the proposal.  Later, all research dissemination products (abstracts for presentation, manuscripts, and final reports) are also required to undergo a tribal research review process. These take time, so it is important to understand the processes and timelines and build review time into your project management timelines.

Check out these posts on understanding evaluation:

1. 1.    Gisele Tchamba on Learning the Difference between Evaluation and Research
2. 2.    John LaVelle on Describing Evaluation

The American Evaluation Association is celebrating Alaska Evaluation Network (AKEN) Affiliate Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. ACA Week: Jennifer Novak-Leonard on Audience Impact
2. Sara Plachta Elliott on Sparking Collective Learning
3. Wilder Research Week: Dan Swanson on Data Quality

As I reviewed “Foundation Transparency: What Nonprofits Want,” the latest publication from the Center for Effective Philanthropy (CEP), I had an overwhelming sense of déjà vu. So I dug deep into the archives to find reports on the subject produced by the organization I now lead, the National Committee for Responsive Philanthropy (NCRP).

In May 1980, NCRP released Foundations & Public Information: Sunshine or Shadow? It was a scathing report that took foundations to task for their reticent approach to sharing information, and it launched a decades-long commitment by NCRP to promote increased transparency. The report explored why foundations should be accountable and transparent, and also the inadequate government requirements at that time. It ranked and scored 208 of the largest philanthropies using a rigorous methodology and found that 60 percent of foundations in the sample did not meet an acceptable standard of transparency. Just 4 percent were found to be “excellent.”

The methodology included a heavily weighted assessment of whether foundations provided the kinds of information that nonprofits most desired, including information about grantmaking interests and policies, and how grant applications were evaluated and decisions made about which organizations to fund.

I see many parallel findings between that report and CEP’s excellent new report. A full 33 years later, nonprofits are still clamoring for more information about how foundations make funding decisions and they want clear and open communication about priorities. They want to know whether it’s worth their time to cultivate a relationship and pursue funding. And despite an explosion of glossy annual reports and fancy websites, leaders of grant-seeking organizations are still highly frustrated by the lack of clear communication about a central element of foundation activity, namely how foundations decide which organizations to fund.

Foundation Transparency surveyed 138 nonprofit leaders, and I was unsurprised to see many of the respondents reference a desire to know how foundations assess their own performance and the impact they have. It only seems fair that since foundations are requiring this from grantees, that they be willing to be accountable for articulating impact, too.

Some of the findings suggest to me that nonprofits really want foundations to function as true partners. For example, the fact that an overwhelming majority of respondents wanted to know more about what foundations are learning indicates that grantees want learning to go both ways.

The CEP report doesn’t explore the regulatory framework for foundation transparency, nor does it explore in-depth the arguments for why greater transparency may be warranted. But another report released this year does revisit those questions. The Philanthropy Roundtable published in March 2013 Transparency in Philanthropy: An Analysis of Accountability, Fallacy, and Volunteerism.

As I reviewed Foundations & Public Information in light of the Roundtable’s current offering, I was struck by how little the arguments in favor of greater foundation transparency have changed since 1980. The original NCRP report looks at the partially-public nature of philanthropy, which is revisited by the Roundtable (though our organizations obviously come down on different sides). The partially-public dollars argument asserts that because of the preferential tax treatment afforded to foundations, a high level of transparency and accountability is owed to the public and grantees. NCRP repeated and expanded on this argument in our 2009 publication Criteria for Philanthropy at Its Best: Benchmarks to Assess and Enhance Grantmaker Impact.

In 1980, NCRP devoted some attention to why greater transparency is in the self-interest of foundations and how it might improve their effectiveness. This topic is explored robustly in the Roundtable’s new report, Criteria, and is touched on in the CEP report. Because I see additional regulation as unlikely in the near future, the link between effectiveness and voluntary transparency merits further exploration.

Speaking of regulation, there has been some increase in activity around this in recent years, though nothing has actually changed for more than 20 years in terms of mandated disclosures. Most philanthropy insiders are familiar with efforts by the Greenlining Institute to pass AB624, which would have required new disclosures for the largest foundations in California. Fewer are aware of quieter efforts by the Philanthropy Roundtable to pass legislation in several states banning efforts similar to AB624.

The last substantive change that shaped the current required information disclosure in the IRS form 990-PF can be traced to when NCRP worked with Senator Dave Durenberger (R-Minn.) to influence the IRS to change what it required in the form. Those changes contributed to helping the Foundation Center produce the best data available about the sector. An abbreviated version of how NCRP’s efforts on transparency evolved, including the Durenberger intervention with the IRS, can be found on page 10 of this look back at NCRP’s history.

What I’m left with is a sense that, on the issue of transparency, the more things change, the more they stay the same.

Coincidentally, around the same time as I was reviewing the new CEP publication and beginning to think about crafting this blog post, Bob Bothwell invited me to join him on a Friday evening for a baseball game at Nationals Park. Bothwell was NCRP’s executive director from its inception in 1976 until 1998. I am reminded again of how important it is for those of us from a new generation who are leading nonprofits and foundations to intentionally nurture connections to our history, even while we attempt to take our organizations in new directions.

And in case you’re wondering, the Washington Nationals beat the Cincinnati Reds 1-0, and Jordan Zimmerman pitched a one-hitter.

Aaron Dorfman is executive director of the National Committee for Responsive Philanthropy (NCRP). He frequently blogs about the role of philanthropy in society. Follow NCRP on Twitter @ncrp.

Greetings! My name is kas aruskevich and I am principal of Evaluation Research Associates LLC. I live in Fairbanks and work primarily in rural Alaska. Alaska is known for its great natural beauty, extreme temperatures, and unique context of diverse and far-flung communities assessable only by air. Alaska is the largest state in the U.S.

Rural communities often have a small population and rarely have a local evaluator for hire. Consequently, a program evaluator is most often hired from outside the community or region. Helicopter evaluation is a depreciating term used to describe a drop in – evaluate – depart approach. Today’s post talks about methods to strengthen and add depth to evaluations that involve distance between evaluator and evaluand.

Hot Tip: First, context is important. Familiarize yourself with the community and region before you travel. Gather demographic data of the community, leading industry, and cultural composition. Learn about the organization hosting the program, before your first contact. Plan your site-visit around a community event so you can see the community in a broader context.

Rad Resource: The importance of context is discussed in New Directions for Evaluation Fall 2012, Issue 135.

Hot Tip: Next, work to build open communication with program staff. Begin with a teleconference to provide an opportunity to meet staff and organization and discuss program status. Teleconferences also give you a chance to describe your evaluation style and see if you are a ‘fit’ for the organization and the evaluation project.

ALWAYS include participatory methods. I don’t ‘come in’ as the expert with an unchangeable evaluation design, but instead write up suggestions for the evaluation to negotiate before a plan is finalized. As an itinerant evaluator you can’t be on site as often as you might like. Using a participatory evaluation approach, program staff can be involved in the evaluation through taking photos or identifying program participants or stakeholders to interview.

Rad Resource – Read more about participatory evaluation in Cousins and Chouinard’s new book Participatory Evaluation Up Close.

Hot Tip: Lastly, work to build a friendly relationship based on mutual interests with at least one person in the organization or community. After years of conducting evaluations, friendly relationships have evolved into continuing friendships. These friendships have mutual benefits, in-part, they are a bridge for the evaluator to learn community specific cultural protocols–very important to conduct evaluations in cross-cultural settings – which in turn can strengthen the program through appropriate evaluation.

Lesson Learned: Itinerant evaluation can be much more than a helicopter site-visit approach. Regular communication and working together with program staff as a team can expand the evaluative evidence collected and increase report credibility, relevance, and use by the program staff.

The American Evaluation Association is celebrating Alaska Evaluation Network (AKEN) Affiliate Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Internal Eval Week: Kathleen Norris on the Internal Evaluation Boa
2. CP TIG Bonus: Susan Wolfe on Using Performance Data to Empower Staff and Build Capacity
3. Internal Evaluation Week: Debbie Cohen on Working with External Evaluators

We’re data junkies at CEP, and we spend a lot of time collecting information about foundations through online research and scrolling through foundation websites. It’s always interesting to see how some foundations share detailed information on what they are trying to do, while others—even those that give hundreds of millions of dollars in grants annually—share little to nothing about their work besides the broad program areas they fund.

Of course, foundations are free to be as transparent as they want (with the exception of reporting the mandatory 990 PF form), so conversations about the extent to which foundations share information often highlight that foundation transparency is voluntary. However, the concern over whether transparency is voluntary or mandatory misses the main point: What do we really mean by “transparency” and why does it matter?

While all of us might like to know more about foundations, there’s one group that relies upon the information foundations are willing to share: the nonprofits who seek foundation support.

To learn more about what nonprofits think about foundation transparency, we surveyed our Grantee Voice panel, a sample of nonprofits of varying sizes and focuses that receive support from large foundations. We asked them what foundation transparency means to them, in what ways would they like to see foundations be more transparent, and why this matters. Our newly released report, Foundation Transparency: What Nonprofits Want, summarizes that research.

From the 138 nonprofit leaders who responded to our survey, my colleagues Ellie Buteau, Ramya Gopal, and I found that these nonprofits certainly have a perspective on how transparent foundations are and why this matters to their organizations.

One takeaway from our research is that to nonprofits, foundation transparency means being clear, open, and honest about information that nonprofits care about, which is information about foundations’ processes and decisions that have implications for their work.

We also learned that there are key areas where nonprofits desire greater foundation transparency:

• What foundations are learning through their work;
• How foundations assess performance and the impact they are having;
• Foundations’ selection processes and funding decisions.

Nonprofit leaders view information about these issues as important to their organizations. These leaders also report that the foundations that are more transparent are more helpful to their organization’s ability to work effectively, easier to develop good relationships with, and more credible.

To my earlier questions of what is transparency and why does transparency matter, the data we collected from nonprofit leaders indicates that nonprofits want meaningful, clear information that will help them plan, assess, and spend their time wisely. Such transparency is in the interests of both the foundations and the nonprofits they support.

Andrea Brock is a Manager on the Research team at the Center for Effective Philanthropy. You can find her on Twitter @CEP_Andrea.

Join the conversation about the findings featured in Foundation Transparency: What Nonprofits Want on Twitter using the hashtag #granteevoice. Hear Andrea speak about this topic at our upcoming national conference, Pursuing Results, and join the discussion at #CEP13.

We are Alexandra Hill and Diane Hirshberg, and we are part of the Center for Alaska Education Policy Research at the University of Alaska Anchorage.  The evaluation part of our work ranges from tiny projects – just a few hours spent helping someone design their own internal evaluation – to rigorous and formal evaluations of large projects.

In Alaska, we often face the challenge of conducting evaluations with very small numbers of participants in small, remote communities. Even in Anchorage, our largest city, there are only 300,000 residents. We also work with very diverse populations, both in our urban and rural communities. Much of our evaluation work is on federal grants, which need to both meet federal requirements for rigor and power, and be culturally responsive across many settings.

Lesson Learned: Using mixed-methods approaches allows us to both 1) create a more culturally responsive evaluation; and 2) provide useful evaluation information despite small “sample” sizes. Quantitative analyses often have less statistical power in our small samples than in larger studies, but we don’t simply want to accept lower levels of statistical significance, or report ‘no effect’ when low statistical power is unavoidable.

Rather, we start with a logic model to ensure we’ve fully explored pathways through which the intervention being evaluated might work, and those through which it might not work as well.  This allows us to structure our qualitative data collection to explore and examine the evidence for both sets of pathways.  Then we can triangulate with quantitative results to provide our clients with a better sense of how their interventions are working.

At the same time, the qualitative side of our evaluation lets us lets us build in measures that are responsive to local cultures, include and respect local expertise, and (when we’re lucky) build bridges between western academic analyses and indigenous knowledge. Most important, it allows us to employ different and more appropriate ways of gathering and sharing information across indigenous and other diverse communities. 

Rad Resource: For those of you at universities or other large institutions that can purchase access to it we recommend SAGE Research Methods.  This online resource provides access to full text versions of most SAGE research publications, including handbooks of research, encyclopedias, dictionaries, journals, and ALL the Little Green Books and Little Blue Books.

Rad Resource: Another Sage-sponsored resource is Methodspace, an online network for researchers. Sign-up is free, and Methodspace posts selected journal articles, book chapters and other resources, as well as hosting online discussions and blogs about different research methods.

Rad Resource: For developing logic models, we recommend the W.K. Kellogg Foundation Logic Model Development Guide.

The American Evaluation Association is celebrating Alaska Evaluation Network (AKEN) Affiliate Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. AKEN Week: Amelia Ruerup on Understanding Indigenous Evaluation in an Alaskan Context
2. AKEN Week: Corrie Whitmore Introducing the Alaska Evaluation Network
3. AKEN Week: Alda Norris on Interdisciplinary Approaches to Extension Evaluation in Alaska

The Center for Effective Philanthropy’s 2013 national conference, Pursuing Results: Effective Foundation Practice, kicks off in less than a week! We look forward to hosting nearly 300 leaders from the field of philanthropy in Detroit for two days of hard-hitting programming, May 21-22.

In this video, President Phil Buchanan previews the agenda for our time in Detroit and highlights several of the exciting sessions on our program. From outside experts armed with insights for philanthropy to seasoned foundation leaders sharing lessons from their experience to the unveiling of new CEP research, this conference will provide attendees a chance to learn and connect.

Hope you’re as excited for this conference as we are. See you in the Motor City next week!

Mark Russell is the Director of Communications & Programming at the Center for Effective Philanthropy.

Greetings from the Last Frontier. I’m Alda Norris, webmaster for the Alaska Evaluation Network (AKEN) and evaluation specialist for the University of Alaska Fairbanks Cooperative Extension Service (CES).

The faculty and staff I work with at CES are experts in a variety of fields, from horticulture, entomology and forestry to economics, nutrition and child development. That adds up to quite an interdisciplinary organization! Our diversity makes for fantastic collaborations, as well as complicated syntheses. Lucky for me, my PhD is in interpersonal communication, which applies across the board.

Lessons Learned:  Ask people to tell you the inspiration behind their projects. Every group has a story to tell.What common goals bring these people together?Inquiring about the “why” and not just the “what” of a program really benefits capacity building efforts. I got to know CES better while writing a Wikipedia entry. Hearing and reading about the contributions Extension has made in Alaska since the 1930s deepened my understanding of what led up to each of our program’s current priorities and logic models.

• Help yourself with history. Too often we are mired in a static view of where an organization is now, rather than having an appreciation for how it has changed, and continues to change, over time. Even in a “young” state like Alaska, there is rich historical data we can learn from.

• Boost your evaluation planning by gathering information on your/the client organization’s “story” from a variety of sources. Talk to emeritus professors, compare the org chart of today to past decades, and comb through newspaper archives. Becoming familiar with past waves of change is very helpful in understanding the meaning behind current missions, goals and structures (and people’s attachments to them).

Hot tip: Communicate about communication! Add a question about communication preferences to your next needs assessment. Don’t assume you know what level of technology and form(s) of interaction your colleagues and clients are comfortable with. Before you do a survey, figure out what modes of communication the target population values. For example, if oral history is a large part of a sample group’s culture, how well will a paper and pencil form be received?

Rad Resources:

1. The National Communication Association (NCA) can help you step up your message design game. Take advantage of free advice from experts on verbal and nonverbal communication by reading NCA’s newsletter, Communication Currents.
2. AnyMeeting is a freetool that you can use to reach a wider audience. With it, you can host online meetings and make instructional videos, both of which are really handy when working in a geographically diverse setting. AnyMeeting also has screenshare clarity in its recordings that Google Hangouts lacks.

The American Evaluation Association is celebrating Alaska Evaluation Network (AKEN) Affiliate Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. AKEN Week: Amelia Ruerup on Understanding Indigenous Evaluation in an Alaskan Context
2. AKEN Week: Corrie Whitmore Introducing the Alaska Evaluation Network
3. Glenn O’Neil on Evaluating Communications Programs Using ToC

Hello! My name is Amelia Ruerup, I am Tlingit, originally from Hoonah, Alaska although I currently reside in Fairbanks, Alaska.  I have been working part-time in evaluation for over a year at Evaluation Research Associates and have spent approximately five years developing my understanding of Indigenous Evaluation through the mentorship and guidance of Sandy Kerr, Maori from New Zealand.  I consider myself a developing evaluator and continue to develop my understanding of what Indigenous Evaluation means in an Alaska Native context.

I have come to appreciate that Alaska Natives are historic and contemporary social innovators who have always evaluated to determine the best ways of not only living, but thriving in some of the most dynamic and at times, harshest conditions in the world.  We have honed skills and skillfully crafted strict protocols while cultivating rich, guiding values.  The quality of our programs, projects, businesses and organizations is shaped by our traditions, wisdom, knowledge and values.  It is with this lens that Indigenous Evaluation makes sense for an Alaska Native context as a way to establish the value, worth and merit of our work where Alaska Native values and knowledge both frame and guide the evaluation process.

Amidst the great diversity within Alaska Native cultures we share certain collective traditions and values.  As Alaska Native peoples, we share a historical richness in the use of oral narratives.  Integral information, necessary for thriving societies and passing on cultural intelligence, have long been passed on to the next generation through the use of storytelling. It is also one commonality that connects us to the heart of Indigenous Evaluation.  In the Indigenous Evaluation Framework book, the authors explain that, “Telling the program’s story is the primary function of Indigenous evaluation…Evaluation, as story telling, becomes a way of understanding the content of our program as well as the methodology to learn from our story.” To tell a story is an honor.  In modern Alaska Native gatherings, we still practice the tradition of certain people being allowed to speak or tell stories.  This begs the question: Who do you want to tell your story and do they understand the values that are the foundation and framework for your program?  

Hot Tip: Context before methods.  It is essential to understand the Alaska Native values and traditions that are the core of Alaska Native serving programs, institutions and organizations.  Indigenous Evaluation is an excellent approach to telling our stories.

Rad Resource: The Alaskool website hosts a wealth of information on Alaska Native cultures and values.  This link will take you to a map of “Indigenous Peoples and Languages of Alaska”

The American Evaluation Association is celebrating Alaska Evaluation Network Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. AKEN Week: Corrie Whitmore Introducing the Alaska Evaluation Network
2. Corrie Whitmore on Framing an Evaluation Conversation for Programs with Fuzzy Goals
3. Video in #Eval Week: Kas Aruskevich on Telling the Story Through Video

I’m Corrie Whitmore, president of the Alaska Evaluation Network (AKEN) and an internal evaluator working for Southcentral Foundation (SCF).  SCF is an Alaska Native owned and operated health care organization serving approximately 60,000 Alaska Native and American Indian people living in Anchorage, the Matanuska-Susitna Valley, and 60 rural villages in the Anchorage Service Unit. SCF has had program evaluation in-house since 2009. We are a small department with two evaluators, so it is important for us to build our skills and keep up to date with changes in evaluation practice by staying engaged with the American Evaluation Association (AEA) and local evaluation practitioners.

The Alaska Evaluation Network (AKEN), which is a great resource for all Alaskans interested in evaluation, was founded in 2012 with an emphasis on improving the quality of evaluation research, theory and practice in Alaska and creating forums for dialogue, relationship–building, learning, and collaboration.

Alaska offers a unique environment for evaluation.  According to the 2010 census, we have 730,000 people spread over an area larger than Texas, New Mexico, and Arizona combined. Population density is low, some communities are only accessible by airplane or boat, and many evaluators work in tribal contexts. Building a community of practice encourages AKEN’s members to support evaluation practices that are responsive to the uniqueness of Alaska’s geographic, social, cultural, and administrative context; encourage effective evaluation; improve evaluation capacity within the state; and advocate for evaluation leadership.

AKEN’s goals are to: increase the understanding of evaluation’s purpose and use in Alaska; build evaluator and organization capacity around evaluation approaches, methods, and cultural competency; promote evaluation as a profession; and support the contribution of evaluation to the generation of theory and knowledge about effective human action in Alaska and the circumpolar north.

To date, AKEN has more than 50 members spread from Fairbanks, Alaska, to Southern California (spanning 3000 miles). This geographic dispersal is a strength – we are committed to including evaluators across the state and those living elsewhere who work in Alaska – and a challenge. To address that reality, all of our meetings have a teleconference or web conferencing option, meeting minutes are posted to our website, and much business is done by email.

Lessons Learned:  Connecting with other evaluators in your region can enrich your work and support capacity building. While it would be great to sit across the table from each other, it can be just as valuable to connect using technology!

Get Involved: by joining AKEN or your own local affiliate. These groups are online at the AEA Affiliate List. If your area doesn’t have an affiliate yet – start one! We’d be glad to share our experience with you.

The American Evaluation Association is celebrating Alaska Evaluation Network (AKEN) Affiliate Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Corrie Whitmore on Framing an Evaluation Conversation for Programs with Fuzzy Goals
2. Video in #Eval Week: Kas Aruskevich on Telling the Story Through Video

I am Susan Kistler, the American Evaluation Association’s Executive Director and aea365’s regular Saturday contributor.

Guess what just moved into the #1 spot for the most read aea365 article of all time? The April 6, 2013 post announcing the collaboration with BetterEvaluation to produce a series of publicly available Coffee Break Webinars! We’re well under way with the webinar series focusing on the RainbowFramework for planning, managing, and implementing an evaluation, and I want to share an update:

Hot Tip – Sign Up to Attend the Remaining Offerings Live: There are still six more webinars to go in the series, to be offered every Tuesday and Thursday through until the end of May. If you attend live, you have an opportunity to raise questions at the end, and to hear directly from the presenters. Register for the remaining free webinars here – note that you must register for each one separately.

Hot Tip – Subscribe to AEA’s YouTube Channel and Check Out the First Webinar in the Series: The recording of the first webinar in the series is already available online. Go to http://www.youtube.com/user/AmEvalAssn and click the Subscribe button just below the header to receive notices as each subsequent recording is posted.

Hot Tip – We’re having each of the videos in the series professional transcribed: The transcription is useful in at least four ways: (1) it enables reading the video content for those who may have hearing limitations, (2) it allows viewers to read along in case the video’s audio is not perfect or the viewer is more comfortable with written rather than spoken English, (3) it allows for easily extracting references and written quotes from the video, and (4) it serves as the basis for improved translation to other languages.

Cool Trick – Automated Translation Is Available and Human Translations Are Coming: You can select “Translate Captions” (see the screenshot below) and Google will create a machine translation of the captions and transcript. Working from a professional transcription improves the quality of the machine translation, but the quality is still quite variable. I am excited to announce that we are working with a team of volunteers to translate this video series into multiple languages.

Lessons Learned (Coming Soon): I’ll share a note once the human translations are available. And, in a future post I will also cover (a) how to contract for transcriptions, (b) how to work with colleagues to create video translations (it’s easier than you think), and (c) how to upload/incorporate transcriptions and translations into your videos since I know that increasingly we’re seeing evaluators sharing video on YouTube.

Hot Tip – Make the Most of YouTube:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

1. Susan Kistler on Four Free Coffee Break Demonstration Recordings on International Monitoring and Evaluation
2. CC Week: Cindy Crusto on Introduction to AEA Public Statement on Cultural Competence in Evaluation Week
3. Video in #Eval Week: Corey Newhouse on Using Video as an Elicitation Technique