Resource Feeds

The Collective Impact Forum Online Community is now Live!

FSG - Mon, 03/24/2014 - 05:00
Join the community at collectiveimpactforum.org.
Categories: Resource Feeds

CP TIG Week: Debra Rog on Historical Perspective on Evaluability Assessment

American Evaluation Association 365 Blog - Mon, 03/24/2014 - 01:15

Hi! My name is Debra Rog and I have been interested in EA’s application and utility since grad school. I’m thrilled to contribute this blog on evaluability assessment (EA).   In fact, EA was the topic of my dissertation nearly 30 years ago!  Now, as an evaluator working at Westat, I’ve been able to use EA both formally and informally in a range of efforts.

I’ve watched EA rise in use in the last decade or so after a long period of diminished use in the late 1980s – 1990s.  It is a tool in the evaluator’s toolkit that can improve the targeting of our evaluation efforts.  With an eye toward maximizing our evaluation funding, EA can help reduce waste on premature evaluations and improve the focus and planning of those that do occur.

EA was developed to assess a program’s ‘readiness’ to be evaluated against its outcomes.  Joseph Wholey and colleagues in the late 1970s discovered that many federal evaluations were not useful to managers, in part because they were yielding null or negative results with little information to make decisions.  Upon investigation, Wholey and colleagues found a number of reasons for these results, including evaluations being conducted: on programs that were not fully developed and some not even in place; against goals that were stated primarily for obtaining funding and were often very vague and  unrealistic; and with measures of outcomes that were not fully agreed upon by key stakeholders.  Therefore, Wholey and colleagues developed EA as a tool to assess these features and others BEFORE undertaking an evaluation.

Lessons Learned: EA is a practical tool that can be used as is or modified for many pre-evaluation situations.  In addition to using EA to assess the readiness of a program for an evaluation, I’ve found it be useful in my own work in:

- selecting program sites to include in a multisite outcome evaluation;

- providing quick information to program funders to guide technical assistance and other supports (especially in programs with multiple sites)

- guiding the development of new programs and initiatives.

Even in situations where funding has not been specifically allocated for EA, I have used an abbreviated approach (typically involving only key document review and key informant telephone calls) to learn more about a program’s goals, level of implementation, context, and so on to help in the planning of an evaluation.  In many ways, ‘evaluability’ is a perspective that is helpful to have before engaging in an evaluation.

Rad Resources: A few relatively recent useful resources:

Evaluability Assessment to Improve Public Health Policies, Programs, and Practices, (2010) by Laura Leviton et al.

Planning Evaluability Assessments: A Synthesis of the Literature with Recommendations (2013) by Rick Davies.

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Related posts:

  1. CP TIG Week: Julianne Manchester on Evaluability Assessment in the Health Professions
  2. EPE Week: Valerie Williams on Evaluating Environmental Education Programs
  3. EPE Week: Jane Peters on Engagement in the Field of Energy Evaluation

Randomized control trials (RCTs): interesting, but a marginal tool for governments

ODI general feed - Mon, 03/24/2014 - 00:00
Are Randomized Control Trials all they're cracked up to be for informing government policy in developing countries?
Categories: Resource Feeds

What have we learnt on getting public services to poor people? What's next?

ODI general feed - Mon, 03/24/2014 - 00:00
'Politics is not only part of the problem, but also of the solution. Working around politics rather than with it does not work: meaningful education reform cannot happen despite teachers’ unions, but in negotiation with them. Equally, efforts to stimulate the voice of citizens and their demand for services only work when they are met with equal efforts to better understand the incentives and decision-making logic of the politicians and civil servants responsible for the delivery of those services.'
Categories: Resource Feeds

Zero poverty... think again: the impact of climate change on development efforts

ODI general feed - Mon, 03/24/2014 - 00:00
Climate change will affect the achievability of any future development goals through its impacts on areas such as agriculture, water and health, and through its many indirect impacts. This paper reviews what is known about the impacts of climate change on potential post-2015 development goal areas and shows that it is essential for climate change to be addressed in order not to compromise development efforts.
Categories: Resource Feeds

CP TIG Week: Julianne Manchester on Evaluability Assessment in the Health Professions

American Evaluation Association 365 Blog - Sun, 03/23/2014 - 01:15

My name is Julianne Manchester, Co-Program Chair for the Community Psychology TIG and the PI at Case Western Reserve University- School of Medicine for an evaluation capacity building initiative with health professionals planning educational programs.  I am pleased to be discussing Evaluability Assessment in this kick-off blog post for AEA365.

What is Evaluability Assessment (EA)? According to the oft-cited founder of EA, Joseph Wholey, it is (in a nutshell) a series of steps with stakeholders to assess the probability that programs will achieve measurable objectives.  In this role, I’ve had the (I think valuable) experience of seeing programs not conducting an EA become stuck as stakeholders (in this case, from clinical settings) experience shifts in organizational priorities toward continuing education of staff.

These have included unanticipated changes to data collection access with electronic medical records or senior hospital leadership priorities.  Perhaps advanced work with these stakeholders through an EA process could have prevented the educational programmers from scrambling to find new sites mid-stream.  But, this was necessary in order to train nurses and measure the provider changes with patients by the federal reporting deadlines.

My challenge is to disseminate an EA framework within the health professions community, particularly those implementing continuing education programs with multiple disciplines (nursing, social work, pharmacy).  I hope to develop a model I can put forth within this context.

Lesson Learned: Different fields have different names for what is essentially an evaluability assessment. In healthcare-oriented research, I couldn’t even find the term until I started looking under implementation research (driven by implementation theory).  This seems to be the appropriate umbrella for these and other planning evaluation activities (developing logic models, so forth) when translating evidence-based programs into practice.

Rad Resource: I found a wonderful guide to EA related to public health (and other areas) in 2010’s Evaluability Assessment to Improve Public Health Programs, and Practices available open-access through this website: http://www.annualreviews.org/journal/publhealth

(Share Clip)

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. CP TIG Week: Debra Rog on Historical Perspective on Evaluability Assessment
  2. Nicola Dawkins on Evaluability Assessment and Systematic Screening Assessment
  3. EPE Week: Valerie Williams on Evaluating Environmental Education Programs

Sheila B. Robinson on a Really Rad Resource for Summer Evaluation Learning

American Evaluation Association 365 Blog - Sat, 03/22/2014 - 01:15

Hello Evaluation Learners! I’m Sheila B. Robinson, aea365′s Lead Curator and sometimes Saturday contributor. Today, I’m writing about a fabulous opportunity for summer. Don’t worry – there’s no need to skip the vacation, the beaches, or time with the kids.

AEA’s Summer Evaluation Institute is a great opportunity to continue learning about evaluation, hone your skills, and network with other evaluators. I attended and taught a course there last year and I learned a ton and had a blast.

Registration is now open for the 2014 AEA Summer Evaluation Institute, to be held June 1-4 in Atlanta, Georgia. Here’s a quick preview of just a few of the high quality courses offered. Note: Descriptions are truncated, so please visit the site for complete descriptions:

Rad Resources:

Every Picture Tells a Story: Flow Charts, Logic Models, LogFrames, Etc. What They Are and When to Use Them with Thomas Chapel: A host of visual aids are in use in planning and evaluation. This session will introduce you to some of the most popular ones—with an emphasis on flow charts, logic models, project network diagrams, and log frames…

Translating Evaluation Findings to Actionable Recommendations with Lori Wingate: In this workshop, participants will learn how to convert evaluation results into sound recommendations that align with the purpose of an evaluation and its intended use. The workshop presents and demonstrates 14 strategies for developing, presenting, and following up on evaluation recommendations—taking into account the myriad reasons that evaluation recommendations are often not accepted or acted upon…

Conflict Resolution Skills for Evaluators with Jeanne Zimmer: Unacknowledged and unresolved conflict can challenge even the most skilled evaluators. Conflict between evaluators and clients and among stakeholders creates barriers to successful completion of the evaluation project. This workshop will delve into ways to improve listening, problem solving, communication and facilitation skills and introduce a streamlined process of conflict resolution that may be used with clients and stakeholders…

Popping the Question: Developing Quality Survey Items with Susan Kistler: Developing reliable and valid surveys that gather useable, informative, data requires crafting high quality questions. High quality questions result in increased reliability and decreased ambiguity in survey responses. You will leave this session with a range of example survey questions, a guide to assist with the selection of question types, and a set of resources for further investigation of surveys and question wording…

Write to the Point: Effective Writing for Evaluators with Joy Quill: The Write to the Point workshop is an abbreviated version of a 2-day effective writing course that the presenter has taught to evaluators in several federal agencies.  It covers important writing principles that evaluators can follow to improve the effectiveness of their reports and proposals…

I wish I could share the rest of the 25 great courses here!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

 

 

 

Related posts:

  1. CEA Week: Mimi Doll on How to Handle Scope Creep
  2. Susan Kistler on Taking a Professional Development Break This Summer
  3. Linda Cabral and Judy Savageau on Improve Your Surveys by Conducting Cognitive Interviews

Thailand Needs Friends To Help It Through Its Crisis

"Usually, in the past, we would have had a coup by now," said one retired senior Thai official this week about his country's travails. He didn't mean the absence of a coup marked progress; he was reflecting a widespread resignation that without action by some external force – such as the military – the crisis being played out in Bangkok risked running the country into the ground.

Announcing the Shared Value Channel on Business Fights Poverty

FSG - Fri, 03/21/2014 - 02:24
Content by the field, for the field, will help advance the thinking of shared value.
Categories: Resource Feeds

Harlan Luxenberg on When to Use a Database Solution instead of Excel

American Evaluation Association 365 Blog - Fri, 03/21/2014 - 01:15
Hi, I’m Harlan Luxenberg from Professional Data Analysts, Inc., a public he alth evaluation firm in Minneapolis, and I’d like to share some thoughts about certain situations where databases may be more useful than Microsoft Excel.  Excel is great for quickly crunching data and managing small datasets; however, using Excel in the wrong situations can actually make your data management tasks trickier.

Below are problems that our colleagues have encountered in Excel and reasons why we think that databases would be better solutions in these cases.

Hot Tip: Know which situations to use a database.

Rad Resource: Anyone can learn databases!

While the thought of learning databases can sound intimidating, anyone can learn them and there are tons of resources that will help you get going! There are numerous blogs and websites, and even free online classes such as Coursera.

Rad Resource: Start with Microsoft Access

A good database to start with is Microsoft Access (http://office.microsoft.com/en-us/access/), which is part of the Microsoft Office Professional suite, and which may already be installed on your computer. Microsoft Access allows users to build reports, create data collection forms, visually create tables, and integrates seamlessly with Excel (which you can still use to create beautiful charts).

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Lisa R. Holliday on Using Microsoft Access to Code Qualitative Data
  2. Pei-Pei Lei on Using InfoPath as a Project Management/Data Collection Tool
  3. Michael Harnar on Filemaker Pro

Zero Poverty... think again

ODI general feed - Fri, 03/21/2014 - 00:00

An end to extreme poverty is unthinkable unless we act now on climate change, warns our new report.

Categories: Resource Feeds

The economics of climate change adaptation in Africa’s water sector: a review and a way forward

ODI general feed - Fri, 03/21/2014 - 00:00
A summary of existing work on the costs and benefits of climate change adaptation for the water sector in Africa
Categories: Resource Feeds

World Water Day theme ignores the ‘nuts and bolts’ of progress

ODI general feed - Fri, 03/21/2014 - 00:00
​I sometimes wonder whether we’re re-living past debates on water and dressing them up in fancy new clothes.

Take the theme of this year’s World Water Day: the water, energy and land nexus.     

I can’t argue with the principle: we need to recognise the links between energy, land and water investments, as action in one area can have spillover effects in others. 

Nor do I underestimate its importance as we made clear in the 2011/12 European Report on Development, Confronting scarcity: managing water, energy and land for inclusive and sustainable growth

After all, the projected water demands for electric power generation needed to meet carbon emissions targets are roughly twice those for conventional power generation, and the trend towards alternative transportation fuels – biofuels, oil shales, oil sands, etc  – can be three to ten times more water intensive than traditional fossil fuels. 

Green growth? Perhaps not that green. 

People rather than systems  

So what's the problem? If ‘the nexus’ can get us all thinking about the inter-connections between water, energy and land, so much the better Better still if the result is enlightened policy that strengthens the claims of poor people in the management of natural resources. 

But my worry is this: what if sector professionals once again get hung up on integration and process, and lose sight of objectives? Is the adoption (by whom?) of a nexus approach an end goal, or a means to an end?     

We’ve been here before. And the vision of how water resources should be managed alongside other resources has been debated for decades. Water should be treated as both a social and an economic good – allocated to its most valuable uses whilst protecting the environment and ensuring basic needs are met. 

Water withdrawals, use and waste disposal should be integrated to account for interdependencies and externalities, and administered through licensing and pricing systems. And more broadly, management and governance should be integrated across sectors and scales, moving away from decision-making silos in land, water and energy to greater policy coherence. 

The problem is, implementing it remains elusive in most countries, and is still couched in language that suggests optimisation can be achieved at zero cost, and without prickly and politically contentious tradeoffs. Indeed a focus on ‘integration’ as an end goal rather than a means to an end can get in the way of more pragmatic, problem-focussed solutions that put people rather than systems or sectors centre stage.     

China and the more ‘crop per drop’ example

China might seem like an odd place to start in a discussion on sound environmental policy. After all, Premier Li Keqiang has just declared a ‘war against pollution’ at the National People’s Congress in Beijing. However, new work by ODI is looking at how China’s ‘learning by doing’ approach to water reform is playing out at the urban-rural interface.

In a series of pilot schemes along the Yellow River, farmers and irrigation managers release water to downstream cities because they are compensated for doing so, with government investing heavily in land and water management (more ‘crop per drop’) to ensure farmers don’t lose out.

 In some cases, the money has come directly from downstream industries and municipalities keen to use or bank savings. A similar problem-focussed approach has been adopted within irrigation districts, with a variety of contracting and user association (WUA) models piloted to see what works best for local operation, maintenance and cost-recovery. 

China is hardly a paragon of sound land and water stewardship. But in each case we see engagement with the ‘nuts and bolts’ issues of rewards and incentives, the clarification of roles, responsibilities and accountabilities and, above all, getting results in terms of improved services, better fee collection and more crop and income per drop. 

Sub-Saharan Africa: a different challenge

Despite misleading metrics on water scarcity, many countries in the region have generous endowments of surface and groundwater. The challenge is to harness those water assets for power, food production, industry, livestock and rapidly growing towns and cities in ways that strengthen rather than undermine the claims of poorer people – those with a stake but little voice in allocation decisions. A subject we'll be discussing further in our all-day event on 21 March - available to watch live

 This will require investment in water resources management – long neglected by donors and governments – as well as investment in the infrastructure of water storage, distribution and supply

But investment in the institutional plumbing of water allocation needs to start from an understanding of problems and pragmatic solutions, not from an idealised vision of how management should work in nexus nirvana. Let’s move on. 

Categories: Resource Feeds

Food prices update March 2014

ODI general feed - Fri, 03/21/2014 - 00:00
ODI’s Food Price Updates focus on tracking international prices of key staple cereals maize, rice, and wheat, and provide commentary on events in markets that affect these prices. They also follow international food and commodity price indices.
Categories: Resource Feeds

Adaptation to Climate Change in Water, Sanitation and Hygiene - assessing risks, appraising options in Africa

ODI general feed - Fri, 03/21/2014 - 00:00
This report presents the findings of research into the risks to delivery of WASH results posed by climate change in Africa.
Categories: Resource Feeds

Building natural capital: how REDD+ can support a Green Economy

ODI general feed - Fri, 03/21/2014 - 00:00
This report, on the current status and future potential of REDD+, describes the many benefits of forests and other ecosystems as a way of demonstrating that forests have multiple values beyond carbon sequestration and indeed are a foundation for sustainable societies. In doing so it provides a summary of the elements necessary for integrating REDD+ into a green economy, providing policymakers with innovative ideas for supporting economic development while maintaining or increasing forest cover.
Categories: Resource Feeds

Africa's 'hidden water': can it secure agricultural growth and food security?

ODI general feed - Fri, 03/21/2014 - 00:00
On the eve of World Water Day, we investigate media claims that African agriculture sits atop a vast reserve of untapped water that could fuel a green revolution. 
Categories: Resource Feeds

Africa's 'hidden water': can it secure agricultural growth and food security?

ODI general feed - Fri, 03/21/2014 - 00:00
Twelve months ago a paper on Africa’s water resources made the news headlines. African agriculture, so the story went, sits atop a vast reserve of untapped water that could fuel a green revolution. Can groundwater really fuel an agrarian transformation similar to that of South Asia, but without the ‘race to the bottom’ of the continent’s aquifers?

The event is held to mark World Water Day, and will be hosted by the ODI Water Policy Programme, Global Water Initiative East Africa and UEA’s Water Security Research Centre. Join us at ODI's offices in London or follow online.
Categories: Resource Feeds

From ground to growth: can Africa's hidden water resource help secure future agricultural growth and food security?

ODI general feed - Fri, 03/21/2014 - 00:00
An ODI Water Policy Programme, Global Water Initiative East Africa and University of East Anglia event marking World Water Day
Categories: Resource Feeds

Cultivating System Change Webinar Changed

Networking Action - Thu, 03/20/2014 - 11:34
Thursday, March 27:  Cultivating system change – a practitioners companion

NOTE:  The day has been changed to Thursday, and times are different for US/Canada because of daylight savings irregularities:  8am PT/ 11am ET/ 3pm UK/ 4pm CET (1 hour)

Presenter: