Program Assessment and Evaluation

Programmatic evaluations measure the effectiveness and success of programs, policies, services, or organizations, and identify areas of potential improvement. A programmatic evaluation can be used to measure what is working and what is not. Some of the questions, among many others, that a programmatic evaluation can answer include the following:

• What are the positive and negative outcomes of the program, service, or organization?
• What factors (historical, environmental, organizational, political, financial, etc.) are associated with the success or failure of the program, service, or organization to date?
• To what extent are the needs of all members of the stakeholder community effectively served?
• How can implementation be improved?
• How, if at all, should the goals and objectives of the program, service, or organization be revised?
• Relative to cost, how valuable are the results of the program?
• Will the contributions of the program be sustained?

The key to a good programmatic evaluation is to evaluate the program, service, or organization from all angles. Programmatic evaluations make use of a variety of methods—including literature reviews, case studies, site visits, focus groups, personal interviews, economic analyses, and quantitative surveys—to ensure a thorough external and internal review of the program, service, or organization. Programmatic evaluations can include any or all of these methods, depending on the scope and focus of the study.

A needs assessment is a systematic exploration and analysis of “the way things are” and “the way things ought to be.” A need is not a want or desire, but rather a gap between the current situation and the optimal situation. Using both qualitative and quantitative research, a needs assessment identifies gaps in training, programs, services and/or outreach efforts. Needs assessments can be used to identify and solve performance problems in order to direct an agency or organization’s future planning efforts. Needs assessments can address a variety of issues and can answer many important questions including:

• Problems or deficits: Are there problems in the organization that might be solved by training or other activities? Are certain outreach efforts reaching the desired audience?
• Opportunities: Could the organization improve itself by taking advantage of or offering new technologies, training programs, or services?
• Strengths: How can the organization take advantage of its strengths, as opposed to reacting to its weaknesses?
• New directions: Can the organization take a proactive approach to move to new levels of performance and service? What can the organization do to make more informed decisions to meet the needs of important stakeholder groups?

The general strategy employed by Responsive Management to conduct needs assessments is to couple both internal assessments, an “inside-out” approach, with external assessments, an “outside-in” approach. Assessment from the “inside-out” means an organization takes a detailed look at where it wants to go as an organization, sets realistic goals and measurable objectives, evaluates its mission, and undertakes the job of better understanding the organization’s internal attitudes, values, and vision for the future. Assessment from the “outside-in” means an organization gains a better understanding of and working relationship with its various constituents and the general public by learning the opinions, attitudes, program priorities, and needs of those external constituents.

There are various methodologies used to conduct needs assessments that target both “inside-out” and “outside-in” assessments. An “inside-out” assessment may include focus groups and/or quantitative surveys of employees. Employee focus groups and quantitative surveys garner feedback from the “inside-out” about employees’ attitudes toward agency and organization program priorities and needed future directions. An “outside-in” assessment can include focus groups and/or quantitative surveys of stakeholders and/or the general population. The goal of the “outside-in” assessment is to gain an understanding of the attitudes of external constituents toward the agency or organization and its values.

The knowledge gained from a thorough assessment of both internal and external constituents provides agencies and organizations with valuable knowledge to improve programs and to build strong partnerships. Awareness of the program priorities of employees and the general population produces long-term benefits to agencies and organizations and facilitates the development of plans that address important program priorities.

Responsive Management works with agencies and organizations to implement programmatic evaluations and needs assessments to measure the effectiveness and success of various programs, policies, services, or initiatives. These evaluations and assessments can identify areas for improvement, quantify impacts, and determine the short- and long-term effects of programs. Evaluations and assessments measure what is working and what is not. Responsive Management also has substantial experience using confidential qualitative and quantitative research to assess employee morale. Our recent major evaluations and assessments include the following: