Module 6: HIV Prevention Evaluation for Native Communities

Goal of Module 6: To provide an evaluation framework for your HIV prevention intervention program.

In order to provide an effective HIV prevention intervention for Native people, you will need to ensure that a solid, culturally relevant evaluation plan informs your program from beginning to end. Program evaluation can help determine whether your HIV prevention intervention program is accomplishing what it set out to accomplish. This module includes an overview of helpful cultural components that effect evaluation, hands-on considerations, evaluation steps and tools, and avenues to share lessons learned to help you build an effective evaluation for your HIV prevention intervention program.

Contents 3c Focus on Evaluation Design

This module helps you to create and conduct evaluation of your HIV/AIDS prevention and intervention program.

It includes information on:

1.


Key Concepts to Guide Evaluation

2.
Hands-On Considerations
3.
Evaluation Steps

An evaluation design is created through a participatory process among all stakeholders wishing to be part of the evaluation team—which can be composed of your community Planning Committee members and/or the community members you brought together from your community forum meeting. It is often helpful to have an evaluation consultant or a funder evaluator on your evaluation team too. The evaluation team decides the purpose of the evaluation and the questions that should be answered via the evaluation. Once the purpose and evaluation questions are decided, then decisions must be made about:

  • what will be measured and what indicators will be used to measure;
  • what methods will be used (and methods should include both scientific rigor and cultural competency);
  • who will perform each activity of the evaluation;
  • how the results of the evaluation will serve to inform and improve the program; and dissemination of results.

Process Evaluation
A process evaluation can assess whether a program has been implemented as planned (implementation fidelity), reveal strengths and weaknesses of a program, identify strategies for overcoming implementation problems or contextual realities, and suggest changes in program activities. They should be conducted periodically throughout the life–cycle of the program to insure implementation fidelity and to improve implementation based upon evaluation findings. Process evaluations provide information on strengths and weaknesses of the implementation activities. Types of indicators that are measured for effectiveness by process evaluation can include:

  • The number of participants receiving services
  • Attrition rates of participants receiving services
  • The number of "train the trainer" sessions - and the number of trainers who complete the sessions
  • Amount of money the program is using on each type of activity
  • Amount of money the program is using on each type of activity
  • The number of programs or services offered and the number of clients accessing each service

Outcome Evaluation
Outcome evaluations assess the impact of a program based upon its stated objectives (short-term, intermediate, and long-term). While outcome evaluations should be conducted only when the program is mature enough to have effected changes in knowledge, attitudes, beliefs, and/or behaviors (KABB), the reality is that most funders require some type of outcome evaluation by the end of the funding cycle. Depending upon the Logic Model that you create for your program, you can posit that your program will perform a short-term and intermediate outcome evaluation and that future public health surveillance data will provide data on your long-term outcomes.

Typically, you can evaluate early results of your program through short-term objectives. Changes in knowledge, attitudes, beliefs, and/or behaviors may be a short-term indicator of the effectiveness of your program. These can be measured through pre-and post-tests that are given before and after HIV education classes.

Intermediate outcomes are generally measurable 3 to 5 years post-implementation of a program. An example of an intermediate outcome is to record a decrease in the number of MSM reported as transmission modes in your service area 3 years after implementation of program.

Process Evaluation: Consider your program. Provide details about the way it was initially implemented: who was the intended audience? How did you plan to reach the intended audience? How was cultural competency incorporated into your program? What were the original interventions? Now, provide details about the evolution of your program: did the original implementation plans work? If so, explain. If not, explain what improvements/changes were incorporated?

Outcome Evaluation: Again, think about your program. Provide three examples through which you can support the success of the program in changing knowledge, attitudes, beliefs, behaviors, condition or status.