For many non-profit program staff, planning the program evaluation for a grant application can be the most difficult part of the grant proposal. We all know it is important to communicate results of our project to the grant-maker and be accountable, but if program evaluation is not your expertise, the language and process can be daunting.
Beginning with the right evaluation questions will make the whole evaluation process run more smoothly. Here are some tips for working with your evaluator, proposal writer, and other stakeholders to formulate the right questions.
Tip 1: Limit the number of evaluation questions
In working with many clients to craft grant proposals and manage grants they’ve received, one of the most common problems we see is trying to answer too many questions during the evaluation. Instead, prioritize the questions you want to answer and select the top 5-7 questions.
If you have difficulty prioritizing your questions, discuss with your stakeholders how you will use the findings from the evaluation (see Tips 2 and 3 below). Important elements of an evaluation might be effectiveness, relevance, impact, sustainability, relevance, or equity. Look again at the description of your program to decide which one of these evaluation elements most relates to the program you are implementing FutureOn.
Tip 2: Plan now for how you will use the answers
In the evaluator’s world, this idea is often called “utilization-focused evaluation.” Non-profits that want to make the best use of resources should evaluate only those questions for which they will use the answers. Common uses of evaluation results (other than satisfying your funder) are:
Deciding which program(s) to expand and which to eliminate.
Improving an existing program.
Adding or eliminating specific services.
Reaching an underserved population in a different way.
Planning for evaluation utilization involves identifying who will use the results. It’s all too easy to think people not involved in your program or the evaluation planning will use the results in their decision-making. Our experience, however, shows that unless you involve your intended users in the evaluation planning, it is unlikely that they will use the results. Thinking consciously about who has a stake in the results and inviting their input to the evaluation greatly increases the probability that your evaluation will be used. (See Tip 3 below.)
Tip 3: Engage your stakeholders in planning the evaluation questions
Non-profits routinely engage their stakeholders on advisory boards, in program planning, and in fundraising. Sometimes, however, they forget to include stakeholders such as clients, donors, and first-line program staff in planning the evaluation questions. One of the best evaluations I’ve had the pleasure to work on included clients in the evaluation planning. The evaluation questions were highly relevant and the results were used throughout the project to make improvements.
Your grant-makers are also important stakeholders to include in conversations about evaluation questions. Read the websites of prospective funding agencies and request for proposal (RFP) instructions for information about what the prospective funder thinks is important. Some grant-makers will accept calls or emails from potential grantees. If so, you can ask their opinion. If you have already received a grant for the program, do invite your project officer to give his or her suggestions on important evaluation questions. Since grant-makers work will multiple non-profits at the same time, they often have a broader view of what questions need to be answered to improve programs and services in your field.
Tip 4: Define every word in every question
Another common problem with evaluation questions is when important words have not been defined. Words such as “success,” “improving,” “popular,” and many others, are important and frequently evaluated concepts, but can mean wildly different things to different stakeholders. For example, is “success” feeding more people at a soup kitchen or helping people get jobs so fewer people need food? The best evaluation questions come to the evaluator with specific definitions. These definitions are what the evaluator uses to determine the data collection needs of the evaluation.
Tip 5: Refer to your logic model
Finally, compare your evaluation questions to your logic model. Do your evaluation questions target key activities and outcomes you identified in your model? Look for mismatches in both directions-are there key processes on the logic model that are not being evaluated by your questions and have you suggested evaluation questions that are not relevant to your logic model? If you answer “yes” to either of these questions, we suggest taking your logic model and evaluation questions back to your stakeholders to revise and re-prioritize (see Tips 3 and 1 above).
So, while evaluation planning can be complex, there are several things non-profits can do to help their evaluation and grant-writing consultants start with the right questions. Experienced consultants will have more tips, checklists, and suggestions to help you plan your evaluation. Don’t hesitate to ask!