American Evaluation Association 365 Blog

Syndicate content
A Tip-a-Day by and for Evaluators
Updated: 12 min 13 sec ago

YFE Week: Anne Gleason and Miranda Yates on Practical Tools for a Youth Research Camp

10 hours 39 min ago

Hi, we are Anne Gleason and Miranda Yates from the Program Evaluation and Planning Department at Good Shepherd Services in New York City and we would like to share some tools we put together for a youth research camp. Two summers ago, we partnered with youth in one of our afterschool programs to conduct research on what youth think it means to be successful, a topic that the students selected, which ultimately culminated in a student-produced documentary. Drawing on techniques we learned at the Critical Participatory Action Research (CPAR) Summer Institute offered by CUNY’s Public Science Project, we facilitated a series of research camp days with a group of twenty 10-14 year olds. The days were organized as follows: What Is Research, Survey Design Parts I and II, Data Entry, and Data Analysis. Check out the camp schedule for more details.

The project provided an enriching learning experience for everyone involved.  Youth gained a unique first-hand experience conducting research by playing a lead role in the design and implementation of the study and the data analysis. In turn, their insider perspective helped us to better understand how to form more meaningful questions and interpret results. For example, one survey question presented a list of resources and asked respondents to rate their importance to achieving life goals. For the goal of attending college, we saw that older students rated having a supportive family/supportive teachers as less important when compared with younger students. We initially were perplexed as to why older students would hold less value for supportive adults. The youth participants posited that older youth may feel more independent and, thus, be more confident in their own ability to achieve success. This insight underscored the benefit of partnering with youth in research.

Lessons Learned:

  • If you plan to conduct a full youth participatory project, allow yourself plenty of time. Ideally, we would have liked a few extra days to delve deeper into research techniques and data analysis.
  • If you’re limited with time or resources, you don’t have to give up the idea of drawing on participatory techniques. We have also found ways to incorporate youth voice into our evaluation activities that are less time intensive, but inspired by a participatory approach. For example, we routinely conduct focus groups throughout our programs to gather feedback on surveys and other evaluation tools and develop action plans.

Rad Resources: Our camp curriculum included activities, role playing and group discussions. Here are two handouts that might be useful to those considering a camp of their own: Survey Development 101 and Survey Administration 101.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. YFE Week: Rob Shumer on Involving Youth in School-Based Participatory Evaluation
  2. Ann Gillard on Measuring Fun in Summer Camp
  3. Rob Shumer on Conducting Youth Led Evaluation

YFE Week: Jessica Manta-Meyer, Jocelyn Atkins, and Saili Willis on Creative Ways to Solicit Youth Feedback

Mon, 09/15/2014 - 01:36

Our names are Jessica Manta-Meyer, Jocelyn Atkins and Saili Willis and we are evaluators at Public Profit, an evaluation firm with a special focus on out-of-school time programs for youth.

We usually evaluate networks of after school programs as a whole (some of which serve more than 20,000 youth, where a survey is indeed one of the best approaches). However, we particularly enjoy opportunities to build the capacity of youth programs to solicit feedback through creative ways that align with best youth development practices.

Here are some of the methods that have been most popular with these programs:

Cool Trick – Journals: At the start of a program, provide journals for all youth in the program and ask them to write something related to the program goals. Is one of the program’s goals to develop leadership skills? They can ask the youth to respond to this question: “In what ways are you a leader?” Is one of the goals to increase enjoyment of reading? “What do you like about reading?” Then, at the end of the program, youth can read what they wrote the first day and write “How would you answer that question differently, now?” or some other question to get them to reflect on how they’ve changed in the program.

Cool Trick – Candy surveys: Ask students to answer surveys questions by putting certain colors of candy in a cup then tally the candy colors to get your responses. Have the youth tally the results themselves. They can even make a bar chart on chart paper by taping the actual candy to the paper. The youth can then eat the candy after they’ve tallied the results.

Hot Tip – used wrapped candy! Starburst works well and is what this summer program used:

Cool Trick – 4 Corners Activity: Youth leadership programs do this all the time. They ask youth to “take a stand” next to signs that are marked Strongly Agree, Agree, Disagree or Strongly Disagree in response to a statement like “youth should be able to vote at age 16.” Once the youth stand next to one of the signs, the group can talk out their different perspectives. Programs can also use this to collect both quantitative (how many stand where) and qualitative (what they say about why they are standing where they are) data.

Hot Tip: For more Creative Ways, come to our Skill-Building Workshop Saturday at 8am. Yes, it’s early, but we promise to have you moving, interacting and creating. Plus, there will be candy.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Related posts:

  1. Kim Sabo Flores on Engaging Youth in Evaluation
  2. YFE Week: Mariah Kornbluh on Addressing Adultism & Being an Ally in Youth-Focused Evaluations, Part 2
  3. Susan Kistler on AEA’s Evaluation Journals

YFE Week: Kim Sabo Flores and David White on New Perspectives and Voices in Youth Focused Evaluation

Sun, 09/14/2014 - 01:31

Hello, my name is Kim Sabo Flores and I am David White. We are honored to serve as co-chairs of the Youth Focused Evaluation Topical Interest Group (YFE TIG). As we prepare for the 2014 conference and our annual business meeting, we have to acknowledge how truly remarkable it is that a loosely knit group of like-minded individuals could grow into a burgeoning group of over 300 individuals who have begun to define and unify the field and practice of youth focused evaluation within the Association. As a group, we are exploring evaluations focused on youth and positive youth development in a variety of settings. However, at our core, many of us are interested in the practice and outcomes of youth participation in evaluation. This focus has been a key part of our history as a TIG because we understand youth participation to be a pillar of positive youth development.

Hot Tip: Youth-Adult Partnerships

When youth are equal partners with adults in the evaluation process, they share equally the decision making power and responsibility. What does this look like? Here are a few key considerations:

  • Evaluation questions are jointly developed.
  • Evaluation activities are performed by youth and adults.
  • Data are analyzed by youth and adults.
  • Youth and adults receive significant benefit from involvement and from the evaluation findings.

Rad Resource: Youth-Adult Partnerships in the Evaluation Process

This 2005 chapter from the Innovation Center for Community and Youth Development outlines best practices in youth youth-adult partnerships in evaluation.

Our TIG is taking the first of several steps necessary to provide a genuine, inclusive, and participatory space for all evaluators, regardless of age. The TIG will host two sessions at AEA this year designed to ignite conversation about how to best include youth in our yearly conference.

See you in Denver!

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. YFE Week: Mary Arnold on Getting Youth Participatory Evaluation Projects off to a Solid Start
  2. YFE Week: Kim Sabo Flores and David White on A New Youth Focused Topical Interest Group
  3. YFE Week: Mariah Kornbluh on Addressing Adultism & Being an Ally in Youth-Focused Evaluations, Part 2

Sheila B Robinson on Learning at the Beach with MOOCs

Sat, 09/13/2014 - 06:12

Hi! I’m Sheila B Robinson, aea365′s Lead Curator and sometimes Saturday contributor. While I cringe at describing myself with the heavily clichéd “lifelong learner,” I’m afraid it’s all too accurate. In fact, when I dream of winning the lottery I don’t think about whiling away my days on a beach reading cheesy novels, but rather sitting in a classroom, taking all the courses I missed out on in college. Problem is, my other half Larry has the beach dream, and I’d really miss him!

Lessons Learned:

Try MOOCs – Massive Open Online Courses. MOOCs have been mentioned a few times on aea365 (see here) and many evaluators (including me) have taken a course on data visualization and infographcis from the Knight Center for Journalism from the author of The Functional Art: An Introduction to Information Graphics and Visualization, Alberto Cairo. But, did you know there are additional MOOCs that might appeal to evaluators?

Image credit: AJC1 via Flickr

MOOCs offer flexibility. They’re free, distance learning (read: you can take courses from the beach!), and you can do your coursework any time of day. Most MOOCs have a video lecture component, some exercises or homework to be completed between sessions, and a final project. There are usually discussion boards where students can communicate with each other, pose questions to the professor, give feedback on each other’s work, or just have conversations. Some courses allow you to work collaboratively on the final project.

With a MOOC, there’s no pressure. While I don’t encourage registering for a MOOC with no intention of finishing, I must admit I am one of the over 90% of MOOC starters who have not finished one. I’m not proud of this, but realities of life called for me to drop something each time, and my MOOC always got the axe. The caveat is, however, that I feel as if I learned a great deal from the weeks I did participate in each course!

MOOCs can offer high quality, rigorous coursework. I took the first few weeks of a course on data analysis and statistical inference (offered by Duke University through Coursera), just to brush up on my skills and see if I could pick up anything new, and before I knew it, we were deep into conditional probabilities, Bayesian inference, and using R! Many offer certificates of completion.

Rad Resources: While Coursera is one of the most popular MOOC sites, look at MOOC aggregators like Class Central or Course Talk to search multiple sites. You’ll find courses on statistics, data analysis, R programming, research methods, writing, problem solving, and much, much more.

So if you see me at the beach under an umbrella with a laptop…

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Susan Kistler on an Amazing, Free, Lifelong Learning Opportunity
  2. Sheila B. Robinson on Being an AEA365 Sponsored Weeks Archaeologist!
  3. Best of aea365: Sheila B Robinson on Being an AEA365 Sponsored Weeks Archaeologist!

Susan Kistler on Padlet: A Free Virtual Bulletin Board and Brainstorming Tool

Fri, 09/12/2014 - 01:14

Hello aea365ers! I’m Susan Kistler, Executive Director Emeritus of the American Evaluation Association, professional trainer and editor, and all around gregarious gal. Email me at susan@thesmarterone.com if you wish to get in touch.

Rad Resource – Padlet: The last time I wrote about Padlet for aea365, exactly two years ago on September 12 of 2012, it was still called Wallwisher. One name change, two years, and a number of upgrades since then, this web-based virtual bulletin board application is worth a fresh look.

Padlet is extremely easy to set up – it takes under 10 seconds and can be done with or without an account; however, I highly recommend that you sign up for a free account to manage multiple bulletin boards and manipulate contributions.

Padlet is even easier to use, just click on a bulletin board and add a note. You can add to your own boards, or to other boards for which you have a link. I’ve set up two boards to try.

Hot Tip – Brainstorming: Use Padlet to brainstorm ideas and get input from multiple sources, all anonymously. Anonymously is the keyword here – the extreme ease of use (no sign in!) is balanced by the fact that contributions only have names attached if the contributors wish to add their names.

Hot Tip – Backchannel: Increasingly, facilitators are leveraging backchannels during courses and workshops as avenues for attendees to discuss and raise questions. Because Padlet is a platform/device independent application (PIA) accessed through the browser, and does not require a login to contribute, it can make an excellent backchannel tool.

The uses are almost endless – any time you might try sticky notes, Padlet may be a virtual alternative.

***IF YOU ARE READING THIS POST IN EMAIL, PLEASE CLICK BACK TO THE AEA365 WEBSITE TO TRY IT OUT!***

This board illustrates the linen background (there are 15+ backgrounds from which to choose) with contributions added wherever the contributor placed them (the owner may then move them). Just click to give it a try. Please.

Created with Padlet

This board illustrates the wood background with contributions organized as tiles (a new option).

Created with Padlet

The size is small when embedded on aea365, go here to see the same board in full page view.

Hot Tip – Multimedia: Padlet can accommodate pictures, links, text, files, and video (when hosted elsewhere).

Hot Tip – Export: A major improvement to Padlet’s functionality has been the addition of the capacity to export the contributions to Excel for analysis, sharing, etc.

Rad Resource – Training: I’ll be offering an estudy online workshop in October on collaborative and participatory instrument development. We’ll leverage Padlet as an avenue for stakeholder input if you’d like to see it in action. Learn more here.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Susan Kistler on Wallwisher, Your Free Online Bulletin Board
  2. Susan Kistler on a Fast and Free Option for Creating a Webpage from a Word Document
  3. Susan Kistler on Getting More From Google

Samantha Grant on Baby Steps to Building Capacity

Thu, 09/11/2014 - 01:15

Hi, I’m Samantha Grant, and I work for the University of Minnesota Extension as a Program Evaluator for the Center for Youth Development. As an internal evaluator, building evaluation capacity is crucial for my organization (and my mental health!)

Building capacity doesn’t happen overnight, but with a few tactical strategies, you will be on your way.

Hot Tips:

Start where the learner is at. Before embarking on capacity building, gain a good understanding of the organization’s and staff’s competency in evaluation. Tailor training to the readiness of the group. Some learners may be ready for more advanced training while others are just getting a handle on the basics. Try to break people up into mini-cohorts to make the learning experience customized for your audience.

Build Confidence and Affirm Expertise. I work with an incredibly skilled group of youth workers who are naturally building evaluation into their practice without even realizing. We talk about all the ways that they are evaluating or reflecting in their program; how they present data to stakeholders; and how they improve their programs with participant feedback. Knowing that they already act like an evaluator helps to build their confidence in gaining more skills.

Get Creative. Use creative, hands-on strategies to get people engaged in the materials. I’ve found resources from people conducting youth focused evaluations to be especially hands- on. Materials created for use with youth often work with learners of all ages.

Structure capacity building as an entry to greater growth. As your audience becomes savvier with evaluation concepts, they will naturally make connections about how they could grow in the future. (This is without you having to tell them what’s next!) Capacity building has helped me to build trust and relationships with my colleagues, so we can ask hard questions in our evaluation. People begin to respect your skills and see you as a resource and not a threat.

Good luck with your future capacity building!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Samantha Grant on Getting Program Staff Invested in Evaluation
  2. EEE Week: Mary Arnold on Building Capacity
  3. YFE Week: Krista Collins on Supporting Positive Developmental Outcomes

New Board Member At Large Corrie Whitmore on the Importance of Aim and Audience in Internal Evaluation

Wed, 09/10/2014 - 01:15

I’m Corrie Whitmore, one of the new At Large Board Members for AEA. I live in Anchorage, Alaska where I am president of the Alaska Evaluation Network and an internal evaluator for Southcentral Foundation, an Alaska Native owned and operated health care organization serving approximately 60,000 Alaska Native and American Indian people each year.

In my current role, I evaluate everything from space utilization to nurse home visiting programs and provide results to both internal operations staff and external funders. I enjoy the diverse work and opportunity to teach evaluation principles as part of our organization’s focus on capacity building. My experience in indigenous organizations and rural environments has deeply enriched my practice and I look forward to sharing my understanding of these important contexts during my service on the AEA Board.

Lesson Learned: discussing the aim and audience of evaluation work is a great way to help people understand what evaluation is and why it is important. As an internal evaluator, the audience for my work is usually program funders, operations staff, and decision makers. Working with the intended audience (aka “stakeholders”) to agree on the aim of our work together early in the project gets us all on the same page, saving time and building understanding.

Rad Resource: The CDC Framework for Program Evaluation is a simple, appealing framework that can anchor conversations about the evaluation process with an audience. I use the circle graphic showing the steps of program evaluation with operations folks to outline our project and help explain the process we will work through together.

2015 is the International Year of Evaluation and an exciting time to join the board. I look forward to learning about the infrastructure that keeps our complex organization and conference functioning and helping AEA build relationships with policymakers and organizations. I’m proud to be part of our socially responsible organization dedicated to supporting “effective and humane organizations and ultimately to the enhancement of the public good!”

See you in Denver!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. AKEN Week: Vanessa Hiratsuka on Continuous Quality Improvement, Quality Assurance, Evaluation, and Research: Where does my project fit?
  2. Corrie Whitmore on Framing an Evaluation Conversation for Programs with Fuzzy Goals
  3. AKEN Week: Amelia Ruerup on Understanding Indigenous Evaluation in an Alaskan Context

Celeste Brubaker on Designing a Standardized Monitoring and Evaluation Framework Applicable across Multiple Programs

Tue, 09/09/2014 - 01:15

Hi! My name is Celeste Brubaker and I am a Monitoring and Evaluation Coordinator at IREX. IREX is a US-based nonprofit organization working to improve lives through international education, professional training, and technical assistance. In our education programs division we have a portfolio of seven student programs (in which international young leaders complete intensive U.S. based learning experiences), each similar but also unique. To understand the outcomes of the programs as a whole we created one standardized monitoring and evaluation framework. From start to finish the process took about half a year. M&E staff led the design with feedback solicited from program managers at each stage of the process. At this point, the first round of data has been collected. Some of our results are visualized in the graph at the bottom of this post. Here are some hot tips and lessons learned we picked up along the way.

Hot Tip: Clearly define the purpose of standardization. At IREX, our aim was to create a framework for gathering data that would allow us to report on our portfolio of student programs as a whole and also to streamline the data collection and information management process. We wanted to achieve these goals while still accounting for the unique aspects of each program. Understanding these goals and parameters guided our decision to create a common framework with room for a small quantity of customized components.

Hot Tip: Start by identifying similarities and differences in expected results. To do this we literally cut apart each of our existing results frameworks. We then grouped similar results, stratified by type of result – output, outcome, objective or goal. The product of this activity was useful in helping us to visualize overlaps across our multiple evaluation systems and provided a base from which to draft an initial standardized results framework. Check out the activity in the picture to the right.

Lesson Learned: It’s an iterative process. There will be lots of rewrites and that’s a good thing! During the process we learned that soliciting feedback in multiple settings worked best. Meeting with the collective group of program managers was useful in that dynamic discussion often led to ideas and points that would not have necessarily come out of individualized input. At the same time, one-on-one meetings with managers provided a useful space for individualized reflection.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Related posts:

  1. Internal Eval Week: Anthony Kim on Balancing Dual Roles as an Internal Evaluator
  2. MIE TIG Week: Andrea Guajardo on Culturally Responsive Evaluation in a Health Setting
  3. Katye Perry on Teaching Ethics in Evaluation through Case Examples

Adam Kessler on Why Evaluations Fail – The Importance of Good Monitoring

Mon, 09/08/2014 - 01:15

My name is Adam Kessler, and I work with the Donor Committee for Enterprise Development (DCED). The DCED has developed a monitoring framework called the “DCED Standard for Results Measurement”, which is currently used by over a hundred private sector development projects across five continents. This blog provides some lessons learned on why evaluators need good monitoring systems, and why implementing staff need good evaluators.

My experience working with private sector development programmes has shown me that they can become an evaluator’s worst nightmare. In private sector development, staff attempt to facilitate change in complex market systems, which change quickly and unpredictably for all sorts of reasons. As a consequence, staff often modify their activities and target areas mid-way through implementation, potentially rendering your expensive baseline study useless. Moreover, links between outputs and outcomes (let alone impact) are unpredictable in advance, and hard to untangle after the event.

Lesson learned: If you want to evaluate a complex programme, ensure that it has a good monitoring system. A good private sector development programme relies on continual, relentless experimentation, in order to understand what works in their context. If staff are not collecting and analysing relevant monitoring data, then they’ll just end up with a lot of small projects which seemed like a good idea at the time. Not easy to evaluate. You’re going to need to see the data they used to make their decisions, and make your own judgement about its quality.

Hot Tip: Good evaluation and good monitoring aren’t all that different, after all. Do you want a robust theory of change, critically interrogating assumptions, outlining activities and examining how they interact with the political and social context to produce change? Guess what – programme staff want that too, though they might use shorter words to describe it. Good quality data? Understanding attribution? Useful for both evaluators and practitioners. Although incentives vary (hence the jealously-guarded independence of many evaluators), in effective programmes there should be a shared commitment to learning and improving.

Incredible Conclusion: Monitoring and evaluation are often seen as different disciplines. They shouldn’t be. Evaluators can benefit from a good monitoring system, and implementation staff need evaluation expertise to develop and test their theories of change.

Rad Resources:

1)     I recently co-authored a paper called “Why Evaluations Fail: The Importance of Good Monitoring” which develops this theme further. It uses the example of the DCED Standard for Results Measurement, a results measurement framework in use by over a hundred projects that helps to measure, manage, and report results.

2)     For an evaluation methodology that explores the overlap between monitoring and evaluation, see Developmental Evaluation.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. SEA Week: Dr. Fred Seamon on Evaluation Careers in the Private Sector
  2. Frank Meintjies on Embedding M&E Within Organisations
  3. GOVT Week: David Bernstein on Top 10 Indicators of Performance Measurement Quality

Lily Zandniapour and Nicole Vicinanza on The Social Innovation Fund Evaluation Plan Guidance Document: A Tool for Building Shared Understanding of Rigorous Impact Evaluation Designs

Sun, 09/07/2014 - 07:25

Greetings! We are Lily Zandniapour of the Corporation for National and Community Service (CNCS) and Nicole Vicinanza of JBS International.   We work together with our colleagues at CNCS and JBS to review and monitor the evaluation plans developed and implemented by programs participating in the CNCS Social Innovation Fund (SIF).   The SIF is one of six tiered- evidence initiatives introduced by President Obama in 2010. The goals of the SIF are two-fold: 1) to invest in promising interventions that address social and community challenges and, 2) to use rigorous evaluation methods to build and extend the evidence base for funded interventions.

Within the SIF, CNCS funds intermediary grantmaking organizations that then re-grant the SIF funding to subgrantee organizations. These subgrantees implement and participate in evaluations of programs that address community challenges in the areas of economic opportunity, youth development, or health promotion.

Rad Resource: Go to http://www.nationalservice.gov/programs/social-innovation-fund to see more about the work of the Social Innovation Fund.

SIF grantees and subgrantees are required to evaluate the impact of their programs, primarily using experimental and quasi-experimental designs to assess the relationship between each funded intervention and the impact it targets. To date, there are over 80 evaluations underway within the portfolio.

Lesson Learned: A key challenge we’ve encountered is making sure that CNCS, JBS, intermediaries, subgrantees and external evaluators all know what is required for a plan to demonstrate rigor in the SIF. To address this, CNCS and JBS worked together to develop the SIF Evaluation Plan (SEP) Guidance document based on a checklist of criteria that evaluators, participating organizations, and reviewers for intermediaries and CNCS could all use when developing and reviewing a plan.

Over the past three years, this Guidance document has been used to structure and review over 80 evaluation plans, and it has proved highly valuable in helping evaluators, programs, and funders to build a shared understanding of what this type of impact evaluation plan includes.

Rad Resource: Have a look at the SIF Evaluation Plan (SEP) Guidance ! It includes a detailed checklist for writing an impact evaluation plan, references and links to resources for each section of the plan, and sample formats for logic models, timelines, budgets, and a glossary of research terms.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Clara Hagens on Guidance on Monitoring and Evaluation
  2. Deepa Valvi on the Strategic Evaluation Planning Process
  3. Systems Week: Susan Wolfe on Adding an Ecological Perspective to Evaluation

Dan McDonnell on Staying Focused On The Web

Sat, 09/06/2014 - 12:19

Hi, my name is Dan McDonnell and I am a Community Manager at the American Evaluation Association. The worldwide web is a treasure trove of useful information and knowledge. Where productivity is concerned, this can often be a double-edged sword: where reading Twitter, blogs and Wikipedia can be immensely rewarding, it is quite easy to fall down the rabbit hole. If you’re a Google Chrome user, there’s a super-easy way to help limit the time you spend on your favorite website, to help you focus on the task at hand: the appropriately named Chrome extention, Stay Focusd.

 

Stay Focusd

Hot Tip: Choose Sites To Block

Once you’ve installed the extension, you’ll notice a small blue clock icon in the upper right corner of your toolbar. Click this, then select ‘Settings’ along the bottom of the pop-up window. and select ‘Blocked Sites’ from the sidebar. Here, type in the addresses of any websites on which you’d like to limit your browsing time: for example, www.facebook.com or en.wikipedia.org. Once you’ve put together your list, click ‘Add Blocked Site(s).’

Hot Tip: Set Access Times

Now that you have your blocked sites listed, it’s time to set limited acccess up. In the Stay Focusd sidebar, select’ Max Time Allowed,’ and choose the amount of time per day you’d like to have access to your ‘rabbit-hole’ websites – the default is 10 minutes, but you may want to be a little more generous than that and give yourself 30 or 60 minutes a day. Or if you want to be hard on yourself, 5 minutes.

You can also select individuals days and times of day during which you’d like the extension to be active. Maybe you give yourself some leeway on nights or weekends, but want to limit the amount of time you spend on Twitter during the work week.

Hot Tip: Be Devious

Is your web browsing willpower low? You can take a couple of quick steps to add an extra layer of protection to stop yourself from simply going into Stay Focusd and changing the settings to give yourself more browsing time or just uninstalling it altogether. For the first, go to the ‘Requires Challenge’ tab: once this option is enabled, you’ll be forced to manually type out challege text to be allowed to change any settings. The text is customizable, so you may find yourself shying away from changing settings if you know you’ll have to type out the entire first chapter of War and Peace first.

You can also block the Chrome extension site, to stop yourself from uninstalling Stay Focusd, or take ‘The Nuclear Option’ which will block a website for a period of time with no going back. Now that’s productivity.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Jara Dean-Coffey on Visual Facilitation and Graphic Recorders
  2. Eric Graig on Requall
  3. Shortcut Week: John Paul Manning on RescueTime for Time Tracking

p2i Week: Ann K. Emery and Johanna Morariu on Message, Design, and Delivery for Webinars

Fri, 09/05/2014 - 01:15

Greetings, we’re Ann K. Emery and Johanna Morariu, Innovation Network evaluators and p2i Advisory Board Members. We train foundations and nonprofits on everything from Evaluation 101 concepts to logic models to data visualization through both in-person trainings and online webinars.

Lesson Learned: Want to rock your next webinar? We’ve adapted p2i’s preparation, design, and delivery strategies for our webinars, plus created a few of our own strategies.

Message: Structure (and Time!) Your Webinar Content. First, outline your content. Don’t sit down to a blank PowerPoint file and just start typing; your webinar will be much better if you structure, chunk, and organize first.

Second, consider the p2i Messaging Model. I ask myself, “How much time does each particular story, example, or resource really deserve? 30 seconds, 1 minute, 5 minutes?”

Finally, create a Pacing Schedule by writing the main headers from your outline and their corresponding time allocations onto a large sheet of paper. During the live webinar, display the Pacing Schedule somewhere visible so you can glance up and make sure you’re on track.

Design: Structure Your Slides. As you sit down to design your slides, don’t forget about your original outline. Through Stephanie Evergreen’s Design Demo slidedeck for p2i, we learned about creating divider slides to alert the audience that new sections are beginning. We use this design strategy in live workshops as well as online webinars so that participants can better parse and digest the new information.

Can you spot our divider slides below? We use big font against dark backgrounds, which contrast from the main body slides.

Delivery: Structure Your Physical Space. Deliver your best webinar ever by carefully structuring your physical space.

As shown below, we use three laptops. Laptop #1 is for viewing your slides and speaking points (rather than clumsily flipping through hard copies of notes). Laptop #2 is the “live” webinar laptop, which is registered for the webinar in the Presenter role. Laptop #3 is registered for the webinar in the Participant role so you can spy on yourself and make sure your slides are progressing smoothly.

Learn more about structuring your physical space at http://annkemery.com/webinar-command-center/.

How have you adapted p2i strategies for your webinars? Do you have additional tips to share? Comment below or connect with us on twitter: @annkemery and @j_morariu.

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. James Coyle on Use of the Potent Presentations Initiative (p2i) in Evaluation Contexts Outside of the AEA Conference
  2. Stephanie Evergreen on p2i at Eval12
  3. John Nash on Creating Outstanding Presentation Slides

p2i Week: Laura Beals on Applying p2i to Presentations at Work

Thu, 09/04/2014 - 01:15

Hello evaluation folks! I am Laura Beals, Director of Evaluation at Jewish Family and Children’s Service, a large multi-service nonprofit in Waltham, MA. Last year was my first AEA annual conference and I was fortunate to be able to present. As I was preparing my presentation, I was alerted to p2i resources; while at first I was (admittedly) not quite sure how to apply some of the tips, they have been instrumental in how I have improved my presentation style.

Hot Tips:

  • One of my favorite p2i tips is to state your key take-aways at the beginning of the presentation, as described in the “Messaging” tutorial on the p2i homepage. Lately, especially when I am presenting evaluation findings and I want an audience-driven discussion, I also state upfront what I am asking of people (e.g., “I will be asking you to provide me feedback on the methodology”).
  • My second favorite p2i tip is that handouts do not have to be printouts of your slides; in fact, handouts should be created separately to complement the presentation. Once I mentally separated the presentation from the handouts, I found myself having more freedom in my slides, since I knew they didn’t have to be understood out of the context of the presentation. For example, below is a side-by-side comparison of two slides and the handout from a literature review training I gave at my agency:

  • I will be honest—presentations that are primarily visual take time to prepare, so allot extra time, especially when you are first learning. It has taken time and practice for me to undo the default “bulleted PowerPoint style.” While now I can more easily envision a visual presentation from the outset, I often have to make my presentation the “old-school” way (bullets) to start, which then serves as an outline of what content I want to make sure to address on each slide. Then, I go through each slide and think about the key take-away and how I can present it visually instead.
  • If you are feeling stuck about how to design your slides, poster, or handout, be inspired by others! I recently listened to a NPR TED Radio Hour show on Originality—in it the guests reflected on how we borrow ideas from others. I find that when I am stuck with where to begin, I like to use others’ as inspiration (and I stress “inspiration”—be respectful of the copyrights of other artists—only use materials that are released for re-use and always attribute!). For example, I love COLOURlovers for color palettes and I have been inspired by Stephanie Evergreen’s “rule of thirds” template and the “Fab Five” reboots on the p2i website.

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Susan Kistler on Tips and Tools for Presentations From Ignite AEA
  2. James Coyle on Use of the Potent Presentations Initiative (p2i) in Evaluation Contexts Outside of the AEA Conference
  3. John Nash on Creating Outstanding Presentation Slides

p2i Week: Meredith Haaf on Applying p2i to Conference Posters

Wed, 09/03/2014 - 01:15

My name is Meredith Haaf and I work for the Evaluation Studies Unit in the Faculty of Medicine at the University of British Columbia. I used the p2i poster guidelines to create a conference poster illustrating the findings of an evaluation.

Although I had some ideas for creating effective presentations, I wanted to learn new strategies to make this poster stand out in the right way.

Rad Resources: The poster guidelines provide an overview of suggestions for creating an effective poster. Topics include font size, language use, and choice of colour and graphics.  It was a great resource to use as a starting point and for reference during the editing stages.

This How-to Guide uses visuals in a step-by-step guide to revamping research posters. It helped me to apply the poster guidelines with easy-to-follow instructions for reducing the amount of text, adding graphics, choosing colour schemes, and more.

Before

The first iteration of my poster used a solid dark background and white text. The charts included all of the survey items related to a given topic, in addition to call-out boxes containing qualitative data. The choice of colours made it difficult to distinguish between the charts and the background (e.g., blue on blue). Moreover, the large amount of text and data seemed difficult for the reader to digest in a short amount of time. The images are intentionally blurry to mask our data, but you can still get the gist of the layout.

After

After reviewing the p2i resources, I realized it was possible to greatly reduce the amount of data presented, while still effectively communicating the results of the evaluation.

I reduced the number of survey items presented in each section, focusing only on those directly related to my key message. I also summarized the findings in bullet points. Having less text allowed me to increase the size of the graphics and the amount of white space.

The p2i poster examples showed me that a light-coloured background with dark text and brightly coloured graphics are more easily viewed from a distance (i.e., higher contrast). My new poster used white columns with a solid grey background to emphasize the charts and tables:

Lesson Learned: The most important take-home message I gathered from the p2i resources on poster presentations is to “keep it simple”. Your poster does not need to illustrate all findings related to a particular project; rather, ensure you are presenting key findings in the most clear and engaging manner. Your audience will appreciate being able to easily grasp the basis of your project, and any further questions can be addressed by you during the poster presentation or in follow-up communications!

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Related posts:

  1. Stephanie Evergreen on the Potent Presentations Initiative Website
  2. Ian Shadrick on Accessibility During and After Your Potent Presentation
  3. Stephanie Evergreen on p2i at Eval12

p2i Week: Kate Haley Goldman on Using p2i Outside of AEA

Tue, 09/02/2014 - 01:14

I’m Kate Haley Goldman, and I’m the co-owner of Audience Viewpoints Consulting. We work with informal science and arts and cultural institutions.

Last year, I applied p2i Design principles to a presentation I gave at the Visitor Studies Association conference. It was the best way to showcase my content for several reasons. First, it fit my topic – Nimble Evaluation, which focused on the overlap between design thinking and rapid iteration. Second, I was trained in the academic-style approach to reporting but that clashed with the key approaches of the technology developers I work with. My developer friends found our reports and presentations too dense, too dull, and non-actionable. My academic colleagues were concerned that without graphs and extensive background, our complex work would be taken out of context and watered down. The p2i approach to presenting helped bridge these ideas.

While I’d made similar presentations within the tech field for years, I hadn’t tried it with researchers and evaluators. Nonetheless, I took a leap and employed the lessons from the Potent Presentations Initiative and my graphic designer friends and presented slides in their style when I talked about Nimble Evaluation at the VSA conference.  No graphs, few words, mostly pictures. Full-bleed color pictures help the audience deepen their connection to the context and the lack of bullets allow them to focus on the speaker.

Rad Resource: Since then I’ve moved the majority of my presentations to this style. I’ve also discovered the p2i Rad Resources, which give not only examples of how to present in this more effective manner, but they do so in ways that academic, data-driven evaluators can appreciate. In particular, I’d like to draw attention to the Design webinar session hosted by Stephanie Evergreen, which can be found on the p2i homepage.

Because Stephanie offers the rationale for the design choices made, you can not only make more effective presentations but understand why they are effective. You’ll still need the same amount of preparation time (or perhaps even more) for your slides, as fewer words on the screen forces you to be an organized and engaging speaker. We’ve found the Rad Resources at p2i help support and reinforce better speaking in all our presentations, at conferences and elsewhere.

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Stephanie Evergreen on the Potent Presentations Initiative Website
  2. Stephanie Evergreen on p2i at Eval12
  3. Susan Kistler on the (Mostly) Joys of Using (Free) Haiku Deck to Make Gorgeous Slides

p2i Week: Taj Carson on Hacking the Ignite Presentation Format

Mon, 09/01/2014 - 13:14

I’m Taj Carson and I’m the CEO of CRC, where we help organizations to tell their stories using data and data visualization. Last year, the Data Visualization & Reporting TIG hosted an Ignite session at the annual conference. I knew our work in mapping, including the development of the Baltimore DataMind, would be a great way to introduce people to the beauty and wonder of maps. The Ignite format involves a five minute presentation, using only 20 slides. That means I had better get to the point quickly.

Lessons Learned: Trying to be funny (haha) makes you feel funny (strange). I rehearsed my presentation and co-wrote it with two colleagues, Sheila Matano and Jill Scheibler. We laughed so hard making this presentation. But the whole time I kept saying “What if WE think it’s funny, but no one else does? What if other people don’t get it?” Trying to do a dynamic presentation in a new format is scary. Lucky for me the other evaluators in the room liked it as much as we did.

Hot Tips: Practice like your life depends on it. I practiced my Ignite presentation so many times that I had it unintentionally (but not robotically) memorized.

  • Figure out what you want to say, what’s your message? For me it was to convey our enthusiasm and excitement about maps and mapping data and to show others how beautiful that can be, while making them laugh at the same time. We wanted to be funny and irreverent because that’s how we roll here at CRC, so it was a good fit for us, and entertaining for the audience.
  • No more than a handful of words per slide! Words are over-rated. Seriously, don’t read your slides. Doing this in a regular presentation is bad enough, you will be shunned if you do this (or have giant tables full of tiny numbers) in an Ignite presentation.
  • Use the opportunity to show, not tell, what you want to say. We showed good maps and bad maps. We mocked the bad maps and pointed out why the good maps were so easy to understand.

 

  • Don’t be afraid to hack the format. Layer multiple photos onto one slide. For me it meant that the timing had to be down to within 2-3 seconds as we went from image to image, but it was still only 20 slides!

Rad Resources:

  • Then read Jon Udell’s thoughts on how to actually practice the presentation.

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Susan Kistler on Tips and Tools for Presentations From Ignite AEA
  2. John Nash on Creating Outstanding Presentation Slides
  3. Sheila Robinson Kohn on Igniting the Morning!

p2i Week: Jane Davidson on p2i’s Messaging Model for Workshops

Sun, 08/31/2014 - 13:06

Potent Presentations aren’t just for paper sessions! This week’s posts, curated by dataviz dynamo Stephanie Evergreen, highlight ways that AEA members have used the Potent Presentations Initiative beyond paper sessions at the annual conference. If you think p2i doesn’t apply to your session type or situation, read on for inspirational guidance!

Kia ora (greetings, from New Zealand), evaluators! I’m Jane Davidson and I’m an evaluation coach, trainer, facilitator, and consultant (check me out at EvaluationCoaching.com).

Looking forward to the AEA conference in Denver? Had a presentation or workshop accepted? Now is the time to start thinking about how to ROCK that room!

Lessons Learned: Ever had one of those “Where do I start?” moments when trying to create a conference presentation? I have had more than one of those, and p2i snapped me out of my mental block! Better still, it’s useful for demo sessions and workshops too!

Rad Resource: Check out my all-time favorites: the p2i Messaging Video and the Messaging Model Handout. These have literally saved me again and again!

Hot Tips: Just this one diagram was pure gold – stick it up on your wall right this second:

Now, be honest, people. Have you ever nailed the background piece of a 15-minute presentation in just 45 seconds? And then gone straight to the bottom line? No, I hadn’t either!

But here’s the really cool thing. I use the same guidelines to plan demonstration sessions, mini-workshops, and even my big 2-day preconference workshop at AEA. When I’m explaining a particular concept, method, or example, I use roughly the same time proportions to make sure I get the message across in the most understandable way.

I particularly love that all-important part at the end: What do you want people to DO differently once they leave your presentation or workshop? For example, the next time someone tells one of my workshop attendees that it’s “all just subjective” whether a result is good or not, I want them to totally nail the response with the great explanation I give them. The p2i Messaging Model reminds me to say so!

The American Evaluation Association is celebrating p2i Week with AEA members who have used our Potent Presentations Initiative. The contributions all this week to aea365 come from members who have used p2i strategies in their presentations. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Related posts:

  1. Susan Kistler on Remembering the Ides of March and Thinking About Your Message
  2. Susan Kistler on Three Free (+1) Resources for Communications Quality
  3. MNEA Week: Angie Ficek on Learning about Potent Presentations

Dan McDonnell on Time Saving Twitter Tools

Sat, 08/30/2014 - 05:09

My name is Dan McDonnell and I am a Community Manager for the American Evaluation Association. With the vast amount of different social analytics and measurement tools that exist – particularly for Twitter – it can be time consuming to research the ins and outs.  Today, I’ll be giving a brief overview of a handful of tools, in a quick hits-like format to help you find the best tools to enhance your use of social media – without the need for hours of research!

Rad Resource: Twazzup – Evaluate Hashtag Data
Twazzup lets you dig into keyword or hashtag data on Twitter with a quick search. While the typical Twitter search just returns the most recent (and popular) related Tweets, Twazzup pulls a list of the top influencers who Tweet with your searched term – a great way to find new, interesting people to follow.

Rad Resource: Commun.it – Manage Your Network

Commun.it makes it easy to keep in touch with your Twitter followers by providing helpful suggestions. In addition to daily insights, Commun.it reminds you to @reply to followers, and updates you on suggested people to follow or unfollow based on recent activity. With one click, you can give a shout out to your top followers to say ‘Thanks!’

Rad Resource: Topsy – Listen Better

Topsy is like Google Alerts, but for Twitter. Set up monitoring filters to comb Twitter and email you updates on when certain keywords or hashtags are mentioned. It’s a great way to get by the minute updates on a particular topic on Twitter. Quick tip – set up a topsy mention to monitor your blog URL, so you can see when people link to you on Twitter!

Rad Resource: Buffer + Feed.ly – Share Cool Stories on Twitter, Later!

I’ve mentioned Buffer before, but I thought it worth mentioning again in tandem with Feed.ly, an RSS feed reader, as it’s one of my favorite tool combinations. Add your favorite blogs and feeds to Feed.ly, and if you have a Buffer account set up (on your PC or your phone), a Buffer Icon will appear as a share icon below each post in the feed. Simply click the Buffer button on any article that you want to share on Twitter, and Buffer will let you customize your Tweet, and automatically schedule it to go out at a later time. It takes just two clicks to share each news item, blog post or story with your network, so this is a tremendous time saver.

Related posts:

  1. Dan McDonnell on Using Buffer to Save Time While Tweeting
  2. Dan McDonnell on Making New Friends and Mastering Lesser-Known Twitter Features Without Third Party Apps
  3. Dan McDonnell on 5 Social Media Tools For Curation and Visualization

SEA Week: Dr. Moya Alfonso on the Benefits of Being A Grant Reviewer

Fri, 08/29/2014 - 01:15

My name is Moya Alfonso, and I’m an Assistant Professor at Georgia Southern University and University Sector Representative and Board Member for the Southeast Evaluation Association (SEA), a regional affiliate of the American Evaluation Association (AEA).

So, you need to improve (or develop) your grant writing skills and perform service. A perfect way to address both of these needs is to serve as a grant reviewer!

Lesson Learned: I have honed my grant writing skills by reviewing for local nonprofits, the Centers for Disease Control and Prevention, and the Department of Education, and learning what is expected and seeing the mistakes made by others.  At the same time, I performed an important service to the fields of public health and educational research and evaluation.

Hot Tip: Select the Right Opportunity. When looking for opportunities to be a grant reviewer, consider where your strengths lie. If you’re a program evaluator with a background in education, for example, the Department of Education might be a good place to start. Targeting opportunities will increase your odds of being selected for a review panel – even if you are new to reviews.

Hot Tip: Know What You’re Getting Into. So you’ve found an opportunity that is right up your alley. Now what? It’s time to determine logistics. If detailed information is not provided in the call for reviewers, contact the review administrator about in-person versus remote reviews, estimates of time required, number of applications assigned, grant review dates or time periods, and travel reimbursement or stipends.

Hot Tip: Be Critical Yet Constructive. There’s nothing worse than receiving a “Great Job!” back from a reviewer. No one is perfect. Read (and reread) each application with an eye toward both its strengths and weaknesses. Keep feedback constructive; there is no room for personal insults in grant reviews.

Hot Tip: Know You’re Not Alone. Grant reviewers typically serve on panels comprised of individuals with a variety of perspectives and skill sets. You are not expected to know everything! Feel free to draw upon the wisdom of your grant review administrator and your fellow reviewers.

Hot Tip: Don’t Trust Technology. Technology is amazing – when it works! When completing reviews, you will likely need to learn new technology to complete your reviews. Don’t trust it! Perform your reviews in a word processing program, save your files to your computer, and use the copy and paste functions to complete your reviews.

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. Leslie Goodyear on Serving as a Reviewer for the National Science Foundation
  2. STEM TIG Week: Susan Eriksson on Scientists Becoming Evaluators
  3. SEA Week: Sean Little on Book Reviewing for Skill Growth

SEA Week: Jason Lawrence on Making the Most of Graduate Education in the Evaluation Profession

Thu, 08/28/2014 - 01:15

My name is Jason Lawrence, Grants Manager at the Florida Office of the Attorney General and Student Sector Representative and Board Member for the Southeast Evaluation Association (SEA), a regional affiliate of the American Evaluation Association (AEA). I am also a second-year Masters of Public Administration (MPA) student in the Askew School of Public Administration and Policy, at Florida State University.

A career in evaluation often begins with solid graduate-level education. But the classroom isn’t the only place where you can prepare for the field. You’ll need additional skills to complement your degree before entering the job market. And those competencies can be acquired in three ways: internships or volunteer assignments; joining professional organizations; and presenting research at conferences.

Hot Tip: If you’re not already working as an evaluator or in a related position, then acquiring practical skills is a critical first step. Finding internship programs can help you overcome this deficiency, but acquiring one can be difficult and highly competitive.

Hot Tip: There is a shortcut. Instead of going through the painstaking application and interview process for an internship, you may consider volunteering your time with a local non-profit organization. These organizations spend a great deal of time measuring the results and effectiveness of their services, but may not have the resources to conduct rigorous evaluations.

As a volunteer, you may inquire how you could be part of their evaluation process. This quid-pro-quo gives the organization the human capital it needs to be effective and equips you with practical skills you need to advance your evaluation career.

Rad Resources: Many times such arrangements take shape through budding professional relationships.  As a member of AEA, you have access to a cadre of evaluation professionals who joined the organization for the very purpose of making connections and sharing skills of the trade. Memberships in professional organizations also afford opportunities to present your academic research at conferences. This is an impressive addition to a fledgling professional resume. Both SEA and AEA offer presenting opportunities year-round.

These are just a few of the ways an aspiring evaluator can break into the field. If you haven’t managed to do any of the above, there is still time. Having a year or even a semester left in graduate study means you have plenty of time to develop the skills needed to land your dream job. Enrolling in a graduate program is only a starting point.

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Related posts:

  1. SEA Week: Sheena Horton on Breaking Into the Evaluation Field
  2. Tamara Bertrand Jones on Finding and Working With a Mentor
  3. SEA Week: Jennifer Johnson and Melanie Meyer on Using Mentoring and Training Programs to Develop Knowledge and Skills