The authors have declared that no competing interests exist.
Established in 1996 with funding from CDC and NIH, the HIV/AIDS Prevention Program Archive (HAPPA) is now the biggest private sector collection of HIV-related evidence-based behavioral interventions (EBIs). Each EBI in HAPPA has been determined by a distinguished Scientist Expert Panel to have demonstrated efficacy in preventing HIV or its risk-related behaviors in the United States. The multimedia replications kits contain everything that a new site would need to implement an EBI such as a user guide that gives an overview of the program and the evidence of its effectiveness; a facilitator’s manual that gives step-by-step implementation protocols for each session; and session implementation materials referenced in the facilitator's manual such as slides, video clips, participant handouts, activity masters, checklists, and homework assignments for the next session. The program packages also contain evaluation materials such as surveys and questionnaires that were used in the original demonstration of effectiveness and that may be used to re-evaluate the program as implemented in a new setting. Recently, we have expanded HAPPA’s scope to include HIV EBIs developed globally and to include evidence-based structural interventions (effective in modifying the physical, social, cultural, political, economic, legal, and/or policy aspects of the HIV risk environment). This paper describes HAPPA’s procedures for identifying, selecting, acquiring and packaging HIV EBIs. It also provides comprehensive lists of evidence-based HIV behavioral and structural interventions and gives information on how to access EBI program packages for implementation in new settings.
To make the best use of limited resources, researchers, practitioners, and funding agencies have increasingly emphasized the importance of disseminating and implementing evidence-based behavioral interventions (EBIs) for HIV prevention
The original HAPPA collection included 11 EBIs for which evaluation studies had been published through 2001
Historically, EBIs for HIV prevention have been delivered by health professionals working in schools, clinics, and community-based organizations. Recent advances in information technology have enabled the development of a new generation of computer-delivered programs that incorporate a variety of interactive media elements such as websites, multimedia presentations, animations, audio and video clips, games, and/or social networking applications
In this article, we describe the methods used and results of our efforts to update and expand HAPPA with new EBIs that have had evaluation studies published since our original article in 2001. We examine trends regarding key characteristics of EBIs over the last decade. We conduct a brief tour of the 24 EBIs available from HAPPA. We look at recent developments that are updating and expanding HAPPA to include computer-delivered interventions, global interventions, and effective structural interventions aimed at changing the contexts or environments that shape individual behavior.
In 1996 Sociometrics Corporation, under terms of Small Business Innovation Research (SBIR) Grants from the Centers for Disease Control and Prevention and the National Institute of Allergy and Infectious Diseases, began development of the HIV/AIDS Prevention Program Archive (HAPPA). HAPPA is a collection of HIV prevention program packages (also known as replication kits) containing all the materials needed to replicate and re-evaluate EBIs that have been shown to be efficacious in preventing HIV infection or its risk-related behavioral antecedents among adults
In 2006, a new set of SBIR grants from the National Center on Minority Health and Health Disparities was received to update and augment the set of program packages comprising HAPPA. A similar, if slightly updated, 7-step procedure was used to select programs
1. Establish an independent Scientist Expert Panel to select programs. The HAPPA Panel currently consists of four nationally recognized experts in HIV/AIDS prevention research: Don Des Jarlais (Beth Israel Medical Center), Ralph DiClemente (Emory University), Don Morisky (University of California Los Angeles), and Gina Wingood (Emory University).
2. Articulate the selection criteria to assess program efficacy. HAPPA staff worked with the Scientist Expert Panel to update the objective criteria used for assessing the evidence of effectiveness of diverse HIV/AIDS prevention programs. In keeping with the evolving state of the science regarding evaluation of HIV prevention programs, including the shift toward the development of programs focused on secondary-in addition to primary-prevention, two updates were made to the selection criteria from those used to select programs evaluated through 2001. First, the required follow-up time period for evaluation assessment was increased from three months to six months. Programs had to demonstrate that their positive outcomes persisted for at least six months after the end of the intervention in order to be included in HAPPA. Second, additional criteria were added that were relevant for prevention programs targeting PLH, such as improved antiretroviral therapy (ART) adherence and decreased viral load. In addition to these two updated criteria, programs were evaluated according to: quality of program implementation; scientific rigor of evaluation; and positive effects on HIV risk behavior (sexual, drug injection, or prenatal/perinatal), antiretroviral therapy adherence, or biological markers (STI/HIV rates or viral load). For a complete list of HAPPA EBI-selection criteria, please refer to the table titled HAPPA EBI-Selection Criteria provided on the Sociometrics website (
3. Identify candidate programs. HAPPA staff identified candidate programs via a comprehensive search of the peer-reviewed literature using databases such as Academic Search Complete, JSTOR, and PubMed. To ensure that important programs were not missed, nominations and literature pointers were solicited from funders, researchers, program directors, and evaluators working in the field.
4. Prepare briefing materials for candidate programs. To assist HAPPA’s Scientist Expert Panel in making selection decisions, HAPPA staff prepared a 4-6 page summary for each candidate program consisting of the name(s) of the original developer(s) and evaluator(s), a description of the program intervention and program materials, a description of the evaluation methods and findings, and a list of references. Each summary was accompanied by one or more key journal articles providing further detail on the program and its evaluation.
5. Select programs. Panel members were asked to review the briefing materials and, based on the selection criteria described in Step 2 above, assign each candidate program an overall priority score ranging from 1 (weakest evidence of effectiveness, lowest priority for HAPPA) to 10 (strongest evidence of effectiveness; highest priority). Panel members were informed that HAPPA staff would interpret an average priority score of 7 or higher as their consensus judgment that the program was an EBI approved for inclusion in the HAPPA collection. In order to avoid any conflicts of interest, Panel members were asked to abstain from voting on their own programs. For a complete list of the 20 EBIs approved for inclusion in HAPPA in Round 1 (1997-2001) and 35 EBIs approved for inclusion in Round 2 (2007-2011) please refer to the table titled EBIs Selected by Scientist Expert Panel for Inclusion in HAPPA provided on the Sociometrics website (
6. Acquire selected programs. HAPPA staff worked with developers or current holders of selected programs to acquire their programs and evaluation materials for archiving and public dissemination. The table referenced above notes the 24 (of the 55 approved EBIs) we were able to acquire for HAPPA. We gave acquisition priority to programs selected by the HAPPA Panel that were not yet publicly available from other sources such as the CDC DEBI program.
7. Prepare HAPPA program packages in both print and digitized formats. Each HAPPA program package includes a comprehensive User’s Guide containing detailed instructions and tips for how to implement the intervention; all relevant implementation materials (e.g., facilitator’s manuals, videos, posters, handouts), reformatted as needed to increase their visual appeal and usability; and evaluation instruments that were used in the original investigation to demonstrate the program’s efficacy in changing HIV-related risk behavior. In addition, each program package contains a research-based program adaptation toolkit, a new tool to help users tailor EBIs for new target populations and contexts, while maintaining fidelity to the intervention’s theory of change and core components. Preparation of the HAPPA program packages was done in collaboration with the original developer(s), who received full authorship credit for the program package, with HAPPA staff receiving secondary credit for work on documentation and production.
Since HAPPA’s inception, 88 programs have been submitted to the Scientist Expert Panel as candidates for inclusion in the collection. Of these, 55 have been selected by the Panel as EBIs for inclusion in HAPPA, based on the strength of their evidence for effectiveness per the objective criteria previously described. Sample sizes for the outcome studies documenting the efficaciousness of these 55 programs ranged from 43 to 38,635 participants. The outcome studies were done in all parts of the U.S., often in multiple locations, on diverse samples of young, middle-age, and older adults.
The 55 EBIs approved for inclusion in HAPPA target many different populations such as gay, bisexual, and non-gay identified MSM (N=10), women (N=13), substance users (N=12), sexually transmitted infection (STI) clinic patients (N=6), and PLH (N=10), among others (
Significant differences emerged between the 20 programs selected as EBIs in Round 1 versus the 35 programs selected a decade later, providing some data on trends in the field. As summarized in
|
|
||
---|---|---|---|
Round 1 EBIs, Selected form 1997-2001 (N=20) | Round 2 EBIs, Selected from 2007-2011 (N=35) | Total (N=55) | |
|
|||
Primary prevention program | 20 (100) | 25 (71) | 45 (82) |
Prevention with positives | 0 | 10 (29) | 10 (18) |
|
|||
Individual | 1 (5) | 13 (37) | 14 (25) |
Couples | 0 (0) | 2 (6) | 2 (4) |
Small group | 15 (75) | 20 (57) | 35 (64) |
Community | 4 (20) | 0 (0) | 4 (7) |
|
|||
Injection drug use | 2 (10) | 2 (6) | 4 (7) |
Sexual risk behaviors | 14 (70) | 25 (71) | 39 (71) |
Injection drug use and sexual risk behaviors | 4 (20) | 7 (20) | 11 (20) |
Antiretroviral Therapy (ART) Adherence | 0 (0) | 1 (3) | 1 (2) |
|
|||
Clinic/hospital/treatment facility | 9 (45) | 24 (69) | 33 (60) |
Community-based organization | 7 (35) | 11 (31) | 18 (33) |
Community-wide (e.g., neighborhood/city/county) | 4 (20) | 0 (0) | 4 (7) |
|
|||
Males | 6 (30) | 9 (26) | 15 (27) |
Females | 4 (20) | 9 (26) | 13 (24) |
Both males and females | 10 (50) | 17 (49) | 27 (49) |
|
|||
Primarily Black or African American | 5 (25) | 12 (34) | 17 (31) |
Primarily Hispanic/Latino (a) | 1 (5) | 2 (6) | 3 (5) |
Primarily Asian-American or Pacific Islander | 1 (5) | 0 (0) | 1 (2) |
Primarily White | 7 (35) | 4 (11) | 11 (20) |
No predominant race/ethnicity targeted | 6 (30) | 17 (49) | 23 (42) |
|
|||
Men who have sex with men (MSM)/gay/bisexual | 6 (30) | 5 (14) | 10 (20) |
Heterosexual or unknown/not reported | 14 (70) | 30 (86) | 45 (80) |
Original target population was comprised of more than two-thirds of one particular race/ethnicity.
The public availability of the 55 programs selected as EBIs by HAPPA’s Scientist Expert Panel is also provided in the EBIs Selected by Scientist Expert Panel for Inclusion in HAPPA table provided on the Sociometrics website (
Only 11 (20%) of HAPPA Panel-approved EBIs are not publicly available. Some developers deemed their EBI “out of date” or had concerns that their evaluation results were not sufficiently positive overall, despite the positive priority score assigned to their program by the HAPPA Scientist Expert Panel. Other developers, for their own reasons, preferred to keep their EBI private.
|
|
---|---|
Approach | |
Behavioral Skills Development | 22 (92) |
Community Outreach | 7 (29) |
Condom Education/Access | 19 (79) |
HIV/STI Education | 21 (88) |
Needle Cleaning/Exchange | 6 (25) |
Self-Efficacy/Self-Esteem | 19 (79) |
Pedagogical Techniques | |
Behavioral Skills Practice | 20 (83) |
Group Discussion | 21 (88) |
HIV Antibody Testing/Counseling | 6 (25) |
Lectures | 19 (79) |
Peer Counseling/Instruction | 8 (33) |
Role Play | 21 (88) |
Safer Sex Communication/Negotiation | 19 (79) |
Video/Slide Presentation | 14 (58) |
HAPPA has contributed significantly to the dissemination and implementation of EBIs. Since HAPPA’s inception, 4,488 HAPPA-related materials have been obtained by over 600 different organizations. A related study
All HAPPA program packages have now been converted to digitized PDF format, for 24/7 download from the Internet (
In addition, multimedia, computer-delivered versions of two HAPPA programs, SISTA for African American women ages 18-29 and WiLLOW for HIV+ African American women, have been developed and evaluated. The computer-delivered version of SISTA is known as SAHARA: Sistas Accessing HIV/AIDS Resources At-a-click (
An online HIV EBI Search Assistant has been created. The Search Assistant, freely available from the HAPPA website (
Global EBIs, developed for and evaluated in international settings, are currently being added to HAPPA. Global EBIs were selected for public dissemination by the Global HIV Archive’s Scientist Expert Panel consisting of seven
A compendium of effective structural interventions, Best Evidence Structural Interventions for HIV Prevention
Over the last 15 years HAPPA has selected, under the guidance of its Scientist Expert Panel and based on objective criteria of efficaciousness, 55 HIV prevention programs as EBIs, or evidence-based behavioral interventions. In collaboration with developers of these selected EBI programs, HAPPA has then gone on to create publicly available program packages (also known as replication kits) for 24 of the 55 EBIs. The HAPPA replication kits are the largest private sector collection of HIV-related EBIs. HAPPA’s pioneering work has foreshadowed several current scientific zeitgeists such as “evidence-based medicine”, “research to practice”, and “translation science”.
Over the last 12 years, over 4,000 of HAPPA’s replication kits and associated materials (evaluation instruments, facilitator’s manuals), in printed and digitized/downloaded formats, have been acquired by schools, clinics, local and State governments, and community-based organizations. The materials have been obtained for grant-writing, research, implementation, adaptation, and/or re-evaluation purposes
In addition to presenting an up-to-date list of current U.S. HIV-related EBIs, this paper has also presented up-to-date lists of effective global HIV-related interventions, whether behavioral or structural in their approach. HAPPA will shortly be facilitating access to these global EBIs and structural interventions, via printed and downloadable materials that will allow schools, clinics, or community-based organizations to replicate or adapt them.
HAPPA has been a major complement to the CDC’s REP and DEBI projects. Although the HAPPA and CDC EBI lists overlap, they are by no means identical, because of slightly different inclusion criteria used. Thus HAPPA offers an alternate, complementary supplementary lens through which HIV-related EBIs can be viewed
In the decade since the original paper describing HAPPA was published
This paper was funded by the National Center for Minority Health and Health Disparities (Grant R44 MD005177-02). The HIV/AIDS Prevention Program Archive (HAPPA) is funded by the National Institute of Allergy and Infectious Diseases and the National Center on Minority Health and Health Disparities. The Program Archive on Sexuality, Health and Adolescence (PASHA) is funded by the National Institute on Child Health and Human Development and the U.S. Office of Adolescent Health.
The authors wish to thank Dr. Carmela Lomonaco and Ms. Tamara Kuhn for their help with the manuscript.