Social work case management software and human services software give nonprofits the opportunity to answer important questions about the effectiveness of their programs by using data, empirical proof, evidence, and statistical significance to validate program hypotheses and assumptions.
We interact with organizations both large and small and in various stages of strategic and operational growth. Our partners use data in varying ways, but we’ve found that all of them fall on a continuum (non-scientific nor validated) of program performance measurement. As a result, we believe this continuum can be applied to any organization hoping to achieve higher levels of program performance and impact measurement.
The Program Performance Measurement Continuum
This continuum has five levels. Each level is identified by a question and the question identifies how your organization is using performance measurement and data to demonstrate impact.
- Do we serve a need?
- Are we making a difference anecdotally?
- Are we making a difference empirically?
- How do we grow the difference we make?
- The Next Step – How do we scale our impact?
The purpose of the continuum is to move to higher levels. For example, if you can prove that you are “serving a need,” then the next step is to prove that you are “making a difference anecdotally.”
You can move up the continuum if you know where your organization falls on this continuum and you know what questions to ask to achieve higher levels.
Level 1: Do we serve a need?
This is the foundational level of nonprofit program performance measurement. Output data is used to answer this question. Output data defines the following three things.
- Are we serving?
- Who are we serving?
- How are we serving?
While this is the lowest level of nonprofit program performance measurement, it is an important one. Answering this question confirms that needs are being addressed.
How to reach this level:
- Count outputs for the number of people you serve and the number of services or supports you provide.
- Monitor participant demographics to make sure you are addressing the needs of your target population(s).
- Monitor services and supports delivered to make sure you are meeting expectations for the number of services and supports offered.
- Maintain a pulse on the needs in your community with outreach or advocacy, and confirm your mission addresses an identified need.
Level 2: Are we making a difference anecdotally?
Most organizations go past needs and begin to answer the question, “are we making a difference?” However, there are two variations on this question. One version of this question is anecdotal and the other is empirical. The anecdotal version uses stories and subjective assessments, while the empirical version uses objective data.
While many organizations want to answer the “are we making a difference” question, actually answering it can be a challenge. The easier method of answering this question is to use subjective assessments, stories, or anecdotes to outline the difference your programs, services and supports make on your target population. It is more challenging to validate those stories empirically.
Anecdotal information is useful for fundraising, development, and communicating your impact external to the organization, as well as building internal buy-in.
How to reach this level:
- Track subjective assessments or evaluations of program participants from enrollment to exit.
- Conduct program participant feedback surveys and track results.
- Develop stories of progress made by program participants including “big gains” and transformational shifts.
Level 3: Are we making a difference empirically?
Elevating beyond anecdotal evaluations of program impact requires a systematic approach and discipline to that approach. Moving beyond simple anecdotes qualifies the “are we making a difference” question by asking, “how do we know we are making a difference?”
Being able to answer the “how do we know” question requires objective proof and validation. Validation comes from tracking, monitoring, analyzing, and acting on performance data related to your programs and services.
In our experience, having a data system that tracks and collects data doesn’t mean you’ve reached this level of nonprofit program performance measurement. You must apply the data by answering the question, “How do we know we are making a difference?”
Note: This particular level includes both outcomes measurement and impact measurement. An organization that has reached this level is commonly operating with a Theory of Change logic model or a robust Performance-Based methodology.
How to reach this level:
- Develop a methodology that identifies what, how, when, and why for your programs, services, and supports.
- Maintain a data management platform that allows you to objectively collect, analyze, and act on program data and information.
- Build a culture of asking “why,” where curiosity into the “how do we know” question drives your management processes.
Level 4: How do we grow the difference we make?
Ultimately making a difference isn’t the end game. The ultimate goal is to make a consistent difference and improve program performance over time.
Growth in the “difference” your organization makes happens in two ways.
The first is growth of programmatic reach. If you have a model that works, it is important to expand its reach to touch more of those in need.
The second form of growth is in performance improvement (which is also a function of growth). Managing, analyzing, and acting on data with objective evaluations of program performance is a quantifiable leap in this continuum, but improving the outcomes and impact (and true “difference”) you make over time is the ultimate goal.
An organization that is expanding its reach while consistently improving performance can achieve growth in impact.
How to reach this level:
- Implement performance improvement methodologies that maintain positive performance trends over time.
- Build a culture that pushes for bigger and bigger results from your programs and services.
- Produce highly effective management systems that set goals, report on progress, and act on analysis.
- Tie program-related goals, strategies, and tactics to larger organizational strategic planning, budgeting, fundraising, and administration (build an ecosystem for organizational growth).
How to apply the continuum to your organization
This post outlines a continuum, which means you can evolve your capacities and capabilities to answer each question with time. As a continuum, it is hard to skip over steps because each level represents a different evolution for your organization. Therefore, by using this concept you can move your organization up the continuum, step by step, to higher levels of impact measurement, performance, and effectiveness.