Understanding social work sage publications

A Report to the Wisconsin Governor’s Juvenile Justice Commission and the Wisconsin Office of Justice Assistance

Stephen A. Small, Arthur J. Reynolds, Cailin O’Connor, and Siobhan M. Cooney

A joint initiative of the University of Wisconsin–Madison

Schools of Human Ecology and Social Work, and the University of Wisconsin–Extension, Cooperative Extension

June 2005

This project was supported by funds from the Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs.

What Works, Wisconsin What Science Tells Us about Cost-Effective Programs

for Juvenile Delinquency Prevention

What Works, Wisconsin What Science Tells Us about Cost-Effective Programs

for Juvenile Delinquency Prevention

Stephen Small, Arthur Reynolds, Cailin O’Connor, and Siobhan Cooney

University of Wisconsin–Madison

School of Human Ecology 1300 Linden Drive

Madison, WI 53706 sasmall@wisc.edu or areynolds@waisman.wisc.edu

Acknowledgements We are grateful to the Policy and Legislative Committee members of the Wisconsin Governor’s Juvenile Justice Commission for their initiative in making this project

possible, and to the staff of the Wisconsin Office of Justice Assistance for their support and collaboration throughout the project. We also appreciate the editorial assistance of

Mari Hansen of the University of Wisconsin–Madison/Extension.

The views expressed in this report are those of the authors and do not necessarily represent the official positions or policies of the U.S. Department of Justice,

the State of Wisconsin, or other funders.

This publication may be cited without permission providing the source is identified as: Small, S.A., Reynolds, A.J., O’Connor, C., & Cooney, S.M. (2005). What Works, Wisconsin: What science tells us about cost-effective programs for juvenile delinquency prevention. Madison, WI: University of Wisconsin–Madison.

Table of Contents

EXECUTIVE SUMMARY .............................................................................................................................5 I. PROJECT OVERVIEW ..................................................................................................................................1

PROJECT HISTORY ..................................................................................................................................1 KEY CONCEPTS .......................................................................................................................................2

Risk-Protection Framework ..............................................................................................................2 Positive Youth Development Approaches .........................................................................................4 Human Capital Perspective ..............................................................................................................4 Evidence-Based Programs and Practices ..........................................................................................4 Registries of Evidence-Based Programs ............................................................................................6 Classification of Prevention Programs..............................................................................................6 Cost-Benefit Analysis .......................................................................................................................7

INFLUENCE OF THE WASHINGTON STATE STUDY ON THE PRESENT REPORT .................8 II. COST-EFFECTIVE PROGRAMS TO PREVENT DELINQUENCY AND CRIME ............................9

PRINCIPLES OF EFFECTIVE PRIMARY AND SECONDARY PREVENTION PROGRAMS .......9 PRINCIPLES OF EFFECTIVE JUVENILE OFFENDER PROGRAMS ..............................................10 SUMMARY OF SELECTED COST-EFFECTIVE PROGRAMS.........................................................11

III. REVIEW OF PROGRAMS AND APPROACHES..................................................................................15 PRIMARY PREVENTION PROGRAMS ..............................................................................................15

Preschool Education........................................................................................................................15 Family Support Programs ..............................................................................................................17 Social-Emotional Learning Programs.............................................................................................21

SECONDARY PREVENTION PROGRAMS........................................................................................23 Family Training Programs .............................................................................................................23 Social Skills Training Programs .....................................................................................................23 Mentoring Programs ......................................................................................................................24 Vocational/Job Training Programs .................................................................................................25 Emerging and Unproven Programs................................................................................................26

JUVENILE OFFENDER PROGRAMS ..................................................................................................27 Juvenile Offender Program Categories and Considerations ...........................................................27 Diversion/Community Accountability Programs .........................................................................28 Therapeutic Interventions...............................................................................................................30 Case Management/Multimodal Interventions................................................................................31 Emerging and Unproven Programs and Approaches ....................................................................32

IV. CONSIDERATIONS FOR IMPLEMENTING EVIDENCE-BASED PROGRAMS .........................34 SELECTING AN APPROPRIATE EVIDENCE-BASED PREVENTION PROGRAM ....................34 STRATEGIES FOR SUCCESSFULLY IMPLEMENTING PROGRAMS ...........................................34 BARRIERS TO IMPLEMENTING EVIDENCE-BASED PROGRAMS .............................................36 CULTURAL/ETHNIC CONSIDERATIONS........................................................................................37

V. RECOMMENDATIONS .............................................................................................................................38 VI. CONCLUSIONS...........................................................................................................................................43 VII. REFERENCES ...............................................................................................................................................44

APPENDIX A: EVIDENCE-BASED PROGRAM DETAILS

APPENDIX B: EVIDENCE-BASED PROGRAM DESCRIPTIONS

APPENDIX C: LIST OF PROGRAM REGISTRIES

APPENDIX D: GUIDELINES FOR SELECTING EVIDENCE-BASED PROGRAMS

What Works, Wisconsin 5

EXECUTIVE SUMMARY

Significant advancements have been made in the science of juvenile delinquency preven- tion in recent years. There is considerable evidence that some approaches are more effec- tive than others in preventing crime and reducing recidivism among youth. Research has shown that implementing proven, scientifically sound programs and interventions can have a preventive effect, making it less likely that indi- viduals will engage in crime and equipping them to make positive contributions to society. Many of these programs have been shown to result in economic benefits to society far out- weighing their costs. Through the use of evidence-based programs, practices and policies, the state of Wisconsin can more effectively address the problem of juvenile delin- quency while making the best use of increasingly lim- ited financial resources.

This report builds on several recent efforts to analyze the growing evidence in the field of delinquency prevention. Most notably, the Washington State Institute for Public Policy’s cost-benefit analysis of dozens of prevention and intervention programs related to juvenile delinquency provided the impetus for this report. The Office of Juvenile Justice and Delinquency Prevention’s Model Programs Guide also informed our work. Drawing on those resources, we highlight a number of proven, effective programs and review the strongest evidence available in several catego- ries of interventions, from universal or primary prevention with children and families through community-based programs for juvenile offend- ers. Special attention is also given to several programs and approaches developed in Wiscon- sin.

KEY CONCEPTS

A number of principles and frameworks guide current thinking about the prevention and treatment of youth problems in general and de- linquency in particular. Most fundamental to current approaches is the risk-protection framework. Certain traits of individuals, fami- lies, and communities have been identified as risk factors for the development of anti-social behavior and negative outcomes; criminologists refer in particular to risk factors for delinquent or criminal activity as “criminogenic factors.” Other traits, called protective factors, make negative outcomes less likely and positive out-

comes more likely. Identify- ing risk and protective factors makes it possible to target relevant attitudes, needs, or behaviors among people at risk for negative outcomes, and provide ser- vices and resources that promote positive outcomes.

A related perspective that has informed many recent prevention and youth programming ini- tiatives identifies “developmental assets,” the building blocks necessary for healthy develop- ment.

The human capital perspective is central to understanding not only the reasoning behind offering programs to at-risk individuals, but also the concept of cost-benefit analysis. Human capital refers to the investment of resources to increase the social, emotional, and educational skills of children, parents, and families. The hu- man capital perspective emphasizes that investments in young people’s education and development can produce economic returns to the general public and personal returns to indi- viduals. All of these “returns” can be included when estimating the effect of a program or in- tervention.

Through the use of evidence- based programs, Wisconsin can

more effectively address the problem of juvenile delinquency

while making the best use of limited financial resources.

What Works, Wisconsin Page ii

Putting the human capital perspective to use, information on the effects of programs and interventions can be converted into monetary values and compared to the investment required to achieve those effects. The economic benefits of prevention programs cover a wide range, in- cluding, but not limited to, reduced delinquency and crime, increased economic well-being of participants (and associated tax revenues), and cost savings within major public service and rehabilitative systems. Cost-benefit analysis (CBA) offers a practical method for helping poli- cymakers consider alternative programs when faced with limited resources. CBA is a major departure from traditional measures of program effectiveness, which take into account only the strength of a program’s impact, while ignoring the costs. Using CBA, program options can be ranked according to their effectiveness per dol- lar of expenditure.

CBA necessarily builds on evidence gener- ated from high-quality program evaluations. The programs for which this type of information is available are called evidence-based programs. Evidence-based programs are those that have been shown through scientific research and evaluation to be effective and reliable. They have been subjected to rigorous evaluation us- ing comparison groups, often including long- term follow-up to track various outcomes for program participants and non-participants. A number of federal agencies and research organi- zations maintain registries of evidence-based programs to guide the selection of programs for prevention of problem behaviors. Evidence- based programs are somewhat rare, but growing in number.

REVIEW OF PROGRAMS

In this report, we review the available evi- dence in ten categories of programs, grouped into three broad areas:

PRIMARY PREVENTION • Preschool Education • Family Support Programs • Social-Emotional Learning Programs

SECONDARY PREVENTION • Family Training Programs • Social Skills Training Programs • Mentoring Programs • Vocational/Job Training Programs

JUVENILE OFFENDER PROGRAMS • Diversion or Community

Accountability Programs • Therapeutic Interventions • Case Management/Multimodal

Interventions

Within each category, we highlight one or two evidence-based programs, including cost- benefit information whenever it is available. We also discuss the practices and approaches that appear to increase program effectiveness within each category.

The most cost-effective prevention pro- grams reviewed in this report include preschool education, home visitation programs, and social and emotional learning programs for elemen- tary school children. In all of these programs, the quality and intensity of services are high, staff members are well trained, and the program has a well-articulated vision with a strong con- ceptual base. Although mentoring and job training programs were also found to have good evidence of effectiveness, their economic returns are lower.

Among juvenile offender programs, the strongest empirical evidence of cost- effectiveness is for diversion programs and therapeutic interventions that provide a range of intensive services over relatively long periods of time. Overall, there are fewer evidence-based juvenile offender programs than prevention programs. However, a number of principles of effective intervention have been identified that can increase the likelihood that a given program or approach will be effective.

Throughout the review of programs, we also identify emerging or unproven delinquency prevention programs that appear to follow key principles of effective programs but have not yet

What Works, Wisconsin Page iii

demonstrated reliable program impacts. These include after-school programs and enhanced probation and supervision programs. Addi- tional research on these and other programs is needed.

CONSIDERATIONS IN IMPLEMENTING EVIDENCE-BASED PROGRAMS

Knowing that a program has undergone rigorous evaluation and has strong evidence that it works is an essential first step in moving toward more effective, evidence-based practice. However, implementing a program that will have the desired effect involves a great deal more than just using an evidence-based pro- gram. Among other things, it requires that the program selected be appropriate to the audi- ence, that it is adequately funded and staffed, and that the selected program is implemented with fidelity. These less tangible matters are of- ten overlooked by program sponsors, but are as important as the program model itself if the program is to have a positive impact. In addition to these issues, there exist a number of practical considerations related to the realities of program administration, which are often barriers to the use of evidence-based programs. These consid- erations are articulated in the full report.

RECOMMENDATIONS

Based on our synthesis of the available evi- dence on the effectiveness of juvenile crime prevention programs, we make the following recommendations for consideration by the Wis- consin Governor’s Juvenile Justice Commission:

(1) Strongly support the use of evidence- based prevention and intervention pro- grams and practices.

(2) Educate policy-makers and practitioners about evidence-based programs and prac- tices and their practical and economic benefits.

(3) Use results of cost-benefit analysis to bet- ter prioritize funding of education and social programs.

(4) Adopt an appropriate and validated as- sessment tool in order to direct juvenile offenders to the level of intervention and supervision that is most likely to be effec- tive for them.

(5) Develop mechanisms for disseminating effective program models and good prac- tice guidelines to practitioners and decision-makers.

(6) Provide support for local-level delin- quency prevention initiatives.

(7) Increase investments in research and de- velopment (R & D) and in evaluation of emerging, innovative, and promising pre- vention programs.

(8) Provide a greater balance between preven- tion and intervention programs and strategies.

(9) Create new, state-level, operational poli- cies that encourage cross-agency collaboration and funding for prevention.

(10) Develop new state funding mechanisms that are equitable and consistent with the economic benefits of prevention pro- grams.

These recommendations have the potential to positively alter the future life chances of Wis- consin youth, reduce crime, and contribute to significant cost savings. However, putting into action most of these recommendations will re- quire both vision and courage – the vision to look beyond short-term solutions and the cour- age to challenge the status quo and adopt new ways of operating. We hope that this report will serve as an impetus for change and contribute to the emergence of Wisconsin as a national leader for innovative, scientific, and cost-effective poli- cies and programs on behalf of its youth.

What Works, Wisconsin Page 1

I. PROJECT OVERVIEW

Significant advancements have been made in the science of juvenile delinquency preven- tion in recent years. There is considerable evidence that some approaches are more effec- tive than others in preventing crime and reducing recidivism among youth. Research has shown that high-quality implementation of evi- dence-based programs and principles often results in reduced delinquency and future re- cidivism as well as economic benefits to society that outweigh expenditures. Through the use of evidence-based programs, practices and policies, the state of Wisconsin can more effec- tively address the problem of juvenile delinquency while making the best use of increas- ingly limited financial resources.

In response to a request from the Wisconsin Governor’s Juvenile Justice Commission and the Wisconsin Office of Justice Assistance, this report synthesizes the current research on scientific approaches to delinquency prevention and community-based intervention. In addition, this report provides an overview of the latest concepts, terms and models related to juvenile delinquency prevention and interven- tion. It concludes with recommendations regarding future directions for the state of Wis- consin that can lead to more effective and economically viable approaches for reducing juvenile crime, enhancing youth development and contributing to responsible citizenship.

Three questions guide this report:

• What does science tell us about effective approaches to preventing and treating delinquency?

• What are the economic returns of the most effective programs?

• What steps should the state take to de- velop a more effective and cost-

beneficial strategy for preventing juve- nile delinquency and future crime?

PROJECT HISTORY

Over the past five years, our knowledge about cost-effective programs, practices and principles in the field of juvenile justice and de- linquency prevention has grown at a phenomenal rate. The pace is so rapid that even in the process of developing the direction and scope of this report, new information became

available that led to modifica- tions in what we and the Wisconsin Governor’s Juvenile Justice Commission intended to accomplish through the present report.

Initially, we had planned to provide an update and ex-

tension of the Washington State Institute for Public Policy’s 2001 report on The Comparative Costs and Benefits of Programs to Reduce Crime [1]. However, within weeks of our beginning this project, Washington State released a new ver- sion of their report that essentially did just that (see Benefits and Costs of Prevention and Early In- tervention Programs for Youth [2]). The availability of this new report meant that much of the work on estimating costs and benefits for delinquency and recidivism prevention pro- grams had already been accomplished, freeing us up to address other important issues.

The current report builds on existing re- sources, including the Washington State report as well as the Office of Juvenile Justice and De- linquency Prevention’s (OJJDP’s) web-based Model Programs Guide [3]. Consequently, the cur- rent report includes a discussion of most of the key findings from these previously released documents but also goes well beyond them. We provide here a fairly broad and inclusive sum- mary of what is known about the prevention of delinquency and recidivism in community-

Our knowledge about cost- effective programs, practices and principles in the field of delinquency prevention has grown at a phenomenal rate.

What Works, Wisconsin Page 2

based programs. Drawing on available cost- benefit analyses, we also discuss what is known about the potential cost savings of various ap- proaches to prevention and community-based juvenile offender programs. In addition, build- ing on the OJJDP Model Programs Guide and the numerous other guides and registries that have been developed in recent years, we pro- vide an overview of what is known about proven, effective approaches to the reduction of delinquency, including a synthesis of the princi- ples that underlie successful programs. Finally, we provide a review of some of the strategies that have been found to be critical to the success- ful selection and implementation of prevention and early intervention programs.

KEY CONCEPTS

There are a number of principles and frameworks that guide current thinking about the prevention and treatment of youth problems in general and delinquency in particular. In this section, we provide an overview of some of the most relevant frameworks, concepts and princi- ples as a way to orient the reader to the current scientific thinking in this area and provide a foundation for our later discussion of what works in delinquency prevention.

Risk-Protection Framework

Most scholars and professionals in the field of prevention are guided by some variation of the risk-protection framework. This approach assumes that the best way to prevent delin- quency or other problematic outcomes is to reduce or eliminate risk factors and to increase or enhance protective factors [4]. Risk factors

typically are defined as individual or environ- mental markers that are related to an increased likelihood that a negative outcome will occur [5, 6]. Conversely, protective factors usually are defined as individual or environmental safe- guards that enhance a person’s ability to overcome stressful life events, risks or hazards and promote adaptation and competence [7, 8]. Criminologists use the term “criminogenic fac- tors,” which describes risk factors that (a) have been empirically linked to delinquent or crimi- nal behavior, and (b) are dynamic, or amenable to change through intervention in an individ- ual’s life [9].

Most prevention researchers and practitio- ners view prevention within an ecological framework [e.g., 5, 10, 11] which assumes that risk and protective factors can exist both within individuals and across the various settings in which they live such as the family, peer group, school, and community. Closely related is the idea that most problems are multiply deter- mined [11]. That is, there may be diverse paths to the development of a particular problem like delinquency, and efforts to address a single cause are likely to fail, because most problems have multiple causes. Similarly, the same risk factor can be related to a variety of different out- comes [8]. For example, many criminogenic factors, which put youth at risk for delinquency, also put them at risk for early parenthood or school failure. Thus, efforts to prevent youth problems must account for and target multiple settings and risk factors [11, 12]. Figure 1 pre- sents a list of common risk factors and protective factors for juvenile delinquency organized by ecological level.

What Works, Wisconsin Page 3

FIGURE 1. Common Risk and Protective Factors for Juvenile Delinquency

RISK FACTORS PROTECTIVE FACTORS INDIVIDUAL LEVEL

• Early initiation of problem behavior • Early and persistent antisocial behavior • Low IQ • Hyperactivity • Rebelliousness • Favorable attitudes toward deviant behavior • Involvement in other problematic or dangerous be-

havior

• High IQ • Intolerant attitudes toward deviant behavior • Positive social orientation • Ability to feel guilt • Trustworthiness

FAMILY LEVEL

• Family history of criminal or delinquent behavior • Family conflict or violence • Favorable parental attitudes and involvement in

problem behavior • History of maltreatment • Parental psychopathology • Teenage parenthood • Poverty

• Good relationships with parents • Good family communication • Parents/caregivers who possess strong parent-

ing skills

PEER LEVEL

• Friends who engage in delinquent behavior • Peer rejection

• Non-delinquent friends (or prosocially oriented friends)

SCHOOL LEVEL

• Academic failure or poor performance beginning in late elementary school

• Lack of commitment or bonding to school • Low academic aspirations

• Positive commitment to school • Academic achievement • Strong school motivation • Positive attitude toward school

COMMUNITY LEVEL

• Availability of drugs and weapons • Low neighborhood attachment and community dis-

organization • Media portrayals of violence • Extreme economic deprivation • Concentration of delinquent peer groups

• Non-disadvantaged neighborhood • Low neighborhood crime • Community norms and laws that condemn

drug use, crime and deviant behavior • High neighborhood stability and cohesion

SOURCE: Adapted from Preventing & Reducing Juvenile Delinquency (2003), p. 105. By J.C. Howell. Thousand Oaks, CA: Sage Publi- cations [13] and Child Delinquents: Development Intervention and Service Needs (2003), R. Loeber and D. Farrington, (Eds.). Thousand Oaks, CA: Sage Publications [14].

What Works, Wisconsin Page 4

Positive Youth Development Approaches

An increasingly popular approach to prob- lem prevention emphasizes the positive aspects of youth development and well-being. This ap- proach is especially common with traditional youth serving agencies that provide after school and non-formal educational programs, and with initiatives concerned with facilitating youth in- volvement in the community. In contrast to traditional prevention programs, positive youth development assumes that simply preventing problems is not enough to prepare youth for adulthood, and that the best way to prevent problems from occurring throughout the life- span is to promote the developmental potential of young people [8, 15].

While there exist a number of positive youth development models, in Wisconsin the most widely used approach is the Search Institute’s Developmental Assets framework [16]. This model is built around 40 developmental assets, defined as the building blocks that are crucial for promoting healthy youth development and well-being [17]. According to the Search Institute, assets center on the relationships, social envi- ronments, patterns of interactions, and norms that are central to promoting youth development. Correla- tional data from the Search Institute show that the more developmental assets a young person possesses, the fewer problems they exhibit (in- cluding delinquency) and the greater the likelihood that they will experience positive de- velopmental outcomes like school success and social responsibility [18].

Human Capital Perspective

Preventive interventions are increasingly conceptualized from a human capital perspec- tive, recognizing that social programs for children and youth are “investments” that pro- mote well-being among participants and for society at large. Human capital is a general iden-

tifier for investments of human and financial resources to increase the social, emotional, and educational skills of children, parents, and fami- lies. If they are substantial enough, these resources can improve learning and behavior in the short-term as well as economic and social well-being in the long-term. Generally, the ear- lier in the life course that an intervention occurs, the greater its capacity for creating enduring effects, with long-term consequences for indi- viduals, their families, and the communities in which they live [19, 20]. The human capital per- spective emphasizes that investments in young people’s education and development can have economic returns to the general public as well as personal returns to individuals, and that all of these “returns” can be included when estimating the effect of a program or intervention.

Evidence-Based Programs and Practices

The need for proven, effective, high quality prevention and intervention programs remains a high priority in Wisconsin and across the nation.

Unfortunately, the effective- ness of many current programs and practices re- mains unproven at best, while some are known to be ineffective or even harmful. Increasingly, state, federal and private funders are ask- ing that the programs they

fund have solid research evidence that they work. These programs are known as evidence- based programs.

Evidence-based programs incorporate strategies, activities and principles that have been shown through scientific research and evaluation to be effective and reliable. In order for programs to be considered evidence-based, they must go through a process that draws on scientifically gathered information and uses a commonly agreed upon criteria for rating re- search interventions, principles and strategies. Such programs:

The need for proven, effective, high quality prevention and

intervention programs remains a priority in Wisconsin and

across the nation.

What Works, Wisconsin Page 5

• Are based on a solid, scientific, theoreti- cal foundation

• Have been carefully implemented and evaluated using rigorous scientific methods that usually include:

o A control or comparison group o Well-established measures and

methods

• Have been evaluated in a variety of set- tings with a range of audiences

• Have evaluation findings that have been subjected to critical review by other re- searchers and published in respected scientific journals

• Have been certified as evidence-based by a federal agency or well-respected research organization based on the above criteria

There has been some confusion regarding the terms used to describe evidence-based pro- grams. Some agencies and professionals refer to them as research-based or science-based programs. In addition, the term model programs is often used interchangeably with exemplary programs. In most cases, they are based on similar princi- ples and criteria for scientific evidence and are therefore comparable.

There is, however, a distinction between evi- dence-based programs and evidence-based practices or principles of effectiveness. Evidence-based prac- tices and principles are elements that increase effectiveness across a variety of programs, rather than self-standing programs that can be ac- quired and implemented. Researchers try to identify these “active ingredients” to increase the likelihood that programs – whether evi- dence-based or not – will be effective.

It is important to keep in mind that the cur- rent scientific evidence is at the level of the program, not at the level of practices or princi- ples. Because each program is an integration of many elements, the contributions of individual program components (staffing, learning activi- ties, timing, or duration) to the total effect are

not estimated. Syntheses of the evidence across different programs lead to identification of a common set of principles, which are valuable, but not “evidence-based” in the same way that individual programs can be.

There are a number of reasons for the grow- ing interest in the use of evidence-based programs. The first is accountability. The public wants to know that tax dollars are being spent on programs and services that actually work. Similarly, public agencies and their funders in- creasingly want to invest their dollars in programs that have scientifically demonstrated their effectiveness. In fact, it is becoming more common for funders to mandate that evidence- based programs be used by the organizations they fund. Another reason for the growth of evidence-based programs is efficiency. Instead of having to “reinvent the wheel,” agencies can select from the growing number of programs that are known to be well designed and have undergone rigorous evaluation. Finally, and perhaps most importantly, when these pro- grams are implemented properly, there is good evidence that they will have the positive impact that they were designed to produce. In this way, evidence-based programs enable limited public resources to be used wisely.

The growth of evidence-based prevention programs is a relatively recent phenomenon. While there were few such programs prior to the 1990s, over the past decade the number of pro- grams has grown quickly and continues to increase at a rapid pace. Unfortunately, while there has been a remarkable growth in the num- ber of evidence-based prevention programs, their adoption and use by practitioners lags far behind. One recent study on the use of evidence- based programs by schools found that less than a third of the prevention programs used by pub- lic schools were evidence-based [21]. In the field of juvenile justice, the percentage of programs that are evidence-based may be even lower.

What Works, Wisconsin Page 6

Registries of Evidence-Based Programs

As the number of evidence-based programs has grown, various federal agencies and non- profit research and education organizations have developed web-based program registries that certify and list programs that have met the standards of being evidence-based. The most rigorous of these are listed in Appendix C of this report.

Many of these registries focus on a particu- lar area of interest such as drug abuse, violence or juvenile delinquency. Others are more gen- eral in that they attempt to cover programs that address a range of outcomes. In order for a pro- gram to be included in a registry, it must have met the sponsoring organization’s criteria of being evidence-based. Some registries also in- clude programs that are classified as promising. These are programs that are based on good re- search and theory and usually have some preliminary evidence of being effective, but have not been evaluated as rigorously, as often, or with as much long-term follow-up as the model or exemplary programs.

There is a great deal of overlap between evidence-based program registries. This is true even for registries that are fairly diverse in the outcomes or problems that they are concerned with. The reason for this is that many evidence- based programs have been found to have a posi- tive impact on addressing a range of problematic and positive outcomes. For exam- ple, some of the most effective family-based programs have been found to not only reduce drug abuse among youth, but also help prevent aggressive behavior and promote positive de- velopmental outcomes like school success.

Classification of Prevention Programs

Traditionally, three different types of ap- proaches to prevention have been identified: primary, secondary, and tertiary [22]. Primary prevention is concerned with preventing the initial occurrence of a problem within a normal population. Secondary prevention involves in- tervening with populations that show signs of

early problems, so that more serious problems can be prevented. Tertiary prevention involves the reduction of a problem among groups of people already experiencing it [23].

A more recent development in the classifica- tion of prevention programs comes from the Institute of Medicine [24]. In this system, pro- grams are classified based on the characteristics of the audience they target. The three categories of prevention programs are known as universal, selective and indicated. Universal prevention in- volves interventions directed at the general public or an entire population (e.g., all middle school students); selective prevention programs are directed at a subgroup of a population that is at risk of developing the problem but is not yet exhibiting any difficulties; indicated prevention involves interventions targeted at high risk in- dividuals who show some signs or symptoms of a problem. In terms of old and new terminology, primary and universal prevention often are used interchangeably, as are indicated and secondary prevention [4].

COMMON LABELS USED TO DESCRIBE

EVIDENCE-BASED PROGRAMS

Evidence-based programs and practices are often referred to by different names by various organizations. Though this can be confusing, these labels are for the most part, comparable. Some of the common labels used to describe programs that meet the criteria of evidence-based include:

• Exemplary Programs (U.S. De- partment of Education)

• Effective Programs (CDC, NIDA, DHHS)

• Model Programs (SAMHSA, OJJDP, Blueprints, Surgeon Gen- eral)

• Proven Programs (Promising Prac- tices Network)

• What Works (Child Trends)

What Works, Wisconsin Page 7

Cost-Benefit Analysis

Cost-benefit analysis (CBA) offers a practical method for helping policymakers consider al- ternative programs while facing limited resources. CBA is a major departure from tradi- tional measures of program effectiveness, which take into account only the strength of a pro- gram’s impact, while ignoring the costs. Using CBA, program options can be ranked according to their effectiveness per dollar of expenditure. In this report, we provide estimates of the eco- nomic returns in 2004 dollars for a wide range of crime prevention programs.

Levin and McEwan define CBA as the “evaluation of alternatives according to their costs and benefits when each is measured in monetary terms” [25, p. 11]. The use of cost- benefit analysis to document the payoffs of edu- cation, prevention, and human service programs has increased in recent years [2, 19]. When program outcomes cannot be easily converted to monetary terms, cost- effectiveness analysis (CEA) is recommended. In CEA, pro- gram costs are in monetary terms but benefits remain in the metric of the outcome measure, such as the per- cent decrease in a particular undesirable behavior. The major advantage of CBA over CEA, however, is that benefits for multiple out- comes can be summarized in dollar terms, expressed as either the net return (benefits mi- nus costs) or return per dollar invested (benefits divided by costs).

The ability to conduct a CBA depends on whether or not it is possible to capture program benefits and costs in dollar terms. Typically, it is more difficult to calculate the dollar value of benefits than costs. Calculating program costs is relatively straightforward given that most costs can be identified and monetized. However, be- cause the outcomes of many prevention and treatment programs are not limited to delin- quency and crime, the estimated economic returns must take into account all the effects of

program participation. Experiences that place children and youth at risk of delinquency should be taken into account (such as child mal- treatment and school underachievement) as well as later behaviors that are expected conse- quences of delinquency (such as lower educational attainment and economic hardship). As with nearly any assessment of program im- pact, another key requirement of CBA is that equivalent outcomes are available for a control group who received no program services (or received “the usual services”) and for an ex- perimental group whose members received the modified or new program or services of interest. Consequently, the economic benefits of a par- ticular program are derived from the average performance of program participants relative to control group participants.

The economic benefits of prevention pro- grams cover a wide range and include youth delinquency, adult criminal behavior and related outcomes that can be readily converted to economic benefits. These outcomes in- clude increased economic well- being of participants (and asso- ciated tax revenues) and cost

savings associated with the administration and treatment of children and youth in major service systems. The two categories that account for the largest share of potential economic returns are (a) cost savings to the criminal justice system and crime victims and (b) increased earning ca- pacity, sometimes measured indirectly by higher educational attainment. The first category is a benefit to the general public (i.e., taxpayer, gov- ernments, and crime victims). The second is primarily a benefit to program participants, but also has direct benefits for society when in- creased tax revenues are taken into consideration.

CBA is generally used to report on the eco- nomic benefits to society at large, which is the sum of estimated benefits to the general public and to program participants. The two major in- dicators of these benefits are the net program

Cost-benefit analysis (CBA) offers a practical

method for helping policymakers consider alternative programs.

What Works, Wisconsin Page 8

benefit and the benefit-cost ratio, which repre- sents the economic benefit to society per dollar invested in the program. Values above $1.00 indicate that economic returns exceed costs.

Note also that both actual and projected benefits are generally included in estimated benefits. Benefits can occur in the short- and long-run. The extent to which evaluations ad- dress long-term impacts depends on a range of factors including the goals of the program, whether sufficient time has elapsed to evaluate target outcomes for program participants, and whether investments have been made to evalu- ate the effects of the program over the long run. Primary prevention programs in the early child- hood area, for example, are more likely to be assessed for long-run effects a decade or more post-program, whereas programs for juvenile offenders usually have a much shorter time ho- rizon and are unlikely to track participants past the age of 18. It is worth noting that for many programs, the research findings are still limited to immediate program impacts or short-term follow-up studies. However, as longer-term fol- low-up becomes more widespread, evidence for the impacts of these programs – and better esti- mates of the long term cost savings – will become available.

INFLUENCE OF THE WASHINGTON STATE STUDY ON THE PRESENT REPORT

A major impetus for this report was the re- cent report commissioned and funded by the Washington State legislature. Aos and col- leagues at the Washington State Institute for Public Policy estimated the economic benefits and costs of a wide range of prevention and in- tervention programs to enhance the well-being of children and youth [2]. Unlike their earlier report [1], the researchers measured economic benefits associated with both crime prevention and non-crime outcomes such as substance abuse prevention and educational attainment. The study’s investigation of the effects of differ- ent programs was intended to advance public

policies on behalf of Washington’s families and, by implication, those across the country.

The Washington State report is notable be- cause it is the most complete account to date of the benefits and costs of social programs for children and youth. Spanning 1970 to 2003, evaluations of seven types of programs were synthesized including pre-kindergarten educa- tion, child welfare/home visitation, youth development, mentoring, teenage pregnancy prevention, youth substance abuse prevention, and juvenile offender programs. The assessed outcomes for estimating economic benefits were in seven areas: crime, substance abuse, educa- tion outcomes (e.g., achievement, school completion), teenage pregnancy and parent- hood, suicide attempts, child abuse, and domestic violence. Aos and colleagues found that programs and intervention approaches in four areas were particularly cost-effective. These included youth development programs with the goal of enhancing decision-making skills or par- ent-child relations; juvenile offender programs such as diversion interventions and family therapies; home visitation programs that pro- vide intensive services; and preschool education for 3- and 4-year-olds.

As with the Washington State report, our report includes information on cost-effectiveness as a major indicator of program impact. We de- scribe programs that were identified in their report as well as in other publications. Never- theless, our report differs in several important ways. First, we emphasize programs that have demonstrated relatively high levels of cost- effectiveness in achieving their goals. Thus, the programs we describe are a select list that we believe provide the most payoff per dollar in- vested. Second, we base our CBA information on the original (or in some cases, most recent) cost-benefit analyses conducted for each pro- gram whenever possible. When no such original analyses exist, we revert to the Washington State study estimates, converted to 2004 dollar values. Finally, the present report goes well beyond the provision of cost-benefit information. The suc- cess of any particular program involves many

What Works, Wisconsin Page 9

factors aside from cost-effectiveness, such as its appropriateness for the target audience and how well it is implemented. In this report we include a thorough review of such factors so that deci- sion makers and practitioners can have a more

complete picture of what it might take to select, fund and implement a cost-effective, evidence- based program directed at the prevention of ju- venile delinquency.

II. COST-EFFECTIVE PROGRAMS TO PREVENT DELINQUENCY AND CRIME

Based on a comprehensive review of the available evidence on crime prevention pro- grams, we describe effective programs for children and youth in three major classes: pri- mary prevention programs, secondary prevention programs, and juvenile offender programs. Primary prevention programs are interventions such as preschool programs that occur before children exhibit problems or anti- social behaviors. Secondary prevention pro- grams are implemented during the early cycle of delinquency when misbehavior and conduct problems are evident, but prior to full blown delinquency. Juvenile offender programs, which are often referred to as treatment programs or tertiary prevention programs, are interventions that occur after youth have been arrested. The goal of juvenile offender programs is to prevent recidivism. For the present report, we limit our discussion of juvenile offender programs to non- residential programs and approaches that are delivered to juvenile offenders in the commu- nity.

Before describing the most effective pro- grams from our review, we summarize broad principles that appear to make prevention and juvenile offender programs effective and, for many of them, exemplary.

PRINCIPLES OF EFFECTIVE PRIMARY AND SECONDARY PREVENTION PROGRAMS

The most effective primary and secondary prevention programs related to delinquency share a number of key principles. Though there

is great value in using established, proven pro- grams, that is not always possible. There will always be new and emerging programs that do not yet have the evaluation data to document their impact. Drawing on a range of studies and comprehensive reviews [26-28], these principles can serve as guidelines to those who are devel- oping new programs, wish to improve existing programs, or have the responsibility for making funding decisions about what is likely to work.

Primary and secondary prevention pro- grams have similar principles of effectiveness. Effective programs are:

• Delivered at a high dosage and inten- sity – Effective programs tend to have relatively greater amounts of contact time with participants whether it be number of sessions, hours, weeks, or years.

• Comprehensive – Multi-component programs that address a variety of risk and protective factors are usually more effective than single-component pro- grams.

• Appropriately timed – The most effec- tive programs address relevant factors or processes at specific times of need, and when participants are most recep- tive to change.

• Developmentally appropriate – Pro- grams should be age and develop- mentally appropriate for the target au- dience of children, youth, or adults.

What Works, Wisconsin Page 10

• Socio-culturally relevant – Tailoring the program to the cultural traditions of youth and their families enhances re- cruitment, retention and sometimes outcome attainment.

• Implemented by well-trained, effective staff – The effectiveness of a program is tied to the staff’s personal characteristics such as efficacy and confidence, and their level of training, either by educa- tion or experience.

• Supported by strong organizations – Effective programs receive administra- tive support, have low employee turnover rates, and have staff members who share the same vision.

• Implemented using varied, active methods – Interactive skills training methods are much more likely than di- dactic lecturing to increase program effectiveness and client satisfaction.

• Based on strong theory – High-quality programs have a strong theoretical justi- fication, are based on accurate information, and are supported by em- pirical research.

• Evaluated regularly – Staff members are able to make modifications and improve program effectiveness when they sys- tematically document and reflect on implementation, processes, and results.

PRINCIPLES OF EFFECTIVE JUVENILE OFFENDER PROGRAMS

Criminologists studying the effectiveness of approaches and programs for juvenile offenders [e.g., 9, 29, 30] have identified several principles of effective intervention. These principles are based on meta-analyses1 of hundreds of pro- 1 Meta-analysis is an increasingly common statistical technique for analyzing and comparing the findings of a body of previously conducted empirical studies. A meta-analysis is a study of existing studies that allows researchers to examine whether there are any systematic patterns or findings that emerge across studies.

gram evaluations and reviews of the records of thousands of adult and juvenile offenders [31]. These principles include:

• The human service principle – Punishment, control, and surveillance will be ineffective at changing offenders’ behavior if human services are not also provided.

• The risk principle – Offenders should be assigned to interventions and ser- vices based on level of risk of recidivism, and that it is possible to clas- sify offenders by risk of re-offending.

• The need principle – Interventions should be selected based on the crimi- nogenic needs (crime-related risk factors) of the individual juvenile of- fender, non-criminogenic needs are not worth targeting.

• The responsivity principle – This prin- ciple has two parts, based on responding to offenders’ learning styles and willingness to change: o Interventions should take a be-

havioral approach, based on cognitive and social learning theo- ries, so that offenders can learn new behaviors (“general respon- sivity”)

o Interventions should be “fine- tuned” to maximize response from individual offenders, based on their motivation, maturity, learning style, anxiety level, etc. (“specific responsivity”); and

• The fidelity principle – Programs should be implemented with integrity and fidelity to the original design, as re- lates to staff training and retention, duration and intensity of program con- tact, and caseloads for staff.

Application of the principles of effective in- tervention requires the use of an assessment tool that provides the information needed to draw conclusions about level of risk, criminogenic needs, and the appropriateness of available dis-

What Works, Wisconsin Page 11

positions or diversion programs for an individ- ual offender. Accurate assessment – above and beyond the screening typically used to deter- mine whether a juvenile should be detained or referred for mental health services – is crucial to effective intervention [32]. Several states have developed standard assessments and require their use at juvenile intake [33, 34]. Wisconsin’s Juvenile Classification System [35] is currently used to assess offenders placed in juvenile cor- rectional facilities, but could be used more widely with revision and re-validation.

SUMMARY OF SELECTED COST-EFFECTIVE PROGRAMS

Table 1 describes prevention and juvenile offender programs that have been demonstrated to be among the most successful and cost- effective programs in achieving their objectives, either specifically in reducing delinquency and criminal behavior, or for outcomes that place children at significant risk of criminal behavior. The programs featured are not the only ones known to be effective, but were selected because of their solid documentation of impact and, in many cases, the availability of good cost-benefit data. In addition, we felt that these programs served as strong examples for the particular category of program being highlighted.

Unlike the Washington State report, the em- phasis in our report is on a select set of programs we consider to be among the “best bets” for delinquency prevention in Wisconsin and in other states. Of course, new evidence on emerging or established programs will further inform policy decisions and complement the evidence we present.

The selected programs and the evidence for their effectiveness are described in the review of programs and approaches that follows, in a more detailed table in Appendix A, and in Ap- pendix B, which also includes contact information for individual programs. Other ef- fective, evidence-based programs can be found in the program registries that are listed in Ap- pendix C.

An important emphasis of this report is the economic benefits to society at large, as de- scribed above. The main categories of benefits for the programs we review are presented below in order of their typical contribution to economic benefits:

• Reduced costs for the administration and treatment of individuals in the jus- tice system as juveniles and adults. For most studies, cost savings are projected over adulthood based on the available evidence on the program.

• Savings to crime victims including tan- gible (e.g., hospitalization) and intangible (e.g., pain and suffering) costs associated with delinquency and crime.

• Increased earnings and compensation of program participants in adulthood based on actual earnings, employment, or educational attainment at particular ages. Estimates are for projected lifetime earnings up to age 65.

• Increased tax revenues to state and fed- eral governments based on estimated earnings and compensation.

• Reduced costs for K-12 remedial ser- vices including special education placement and grade retention.

• Reduced costs for the administration and treatment of children in the child welfare system due to reported child abuse and neglect. The tangible and in- tangible savings to maltreatment victims also are included.

• Reduced costs for the administration and provision of social services includ- ing welfare and food stamps.

• Reduced costs for substance abuse treatment and related services.

We note that for some programs there are unique benefit categories that are not listed above. In center-based early childhood interven- tions, for example, parents have increased time available to devote to educational, economic, or personal development. This is measured as a

What Works, Wisconsin Page 12

benefit to program participants. Savings associ- ated with reduced rates of teenage parenthood as well as health and mental health problems also are applicable. Seemingly non-economic outcomes such as achievement test scores and

conduct problem ratings can be converted to economic benefits on the basis of their docu- mented links to educational attainment and juvenile crime [2].

TABLE 1. Summary of Selected Effective Programs for Preventing Crime and Enhancing Well-Being

Per Participant Costs and Benefits in $2004

Program Target group/ Focus

Intensity and length

Major crime prevention impacts

Impacts linked to crime prevention

Benefits Costs

Net benefit (Benefits –

Costs)

Return per $1

invested

Primary Prevention Preschool Education

Child-Parent Centers

Child/ Enrichment, Parent involvement

Part day, 1-2 years

Arrests, Incarceration up to age 24

Ed attainment Achievement Abuse & neglect

78,732 7,755

70,977 10.15

High/Scope Perry Preschool

Child/ Enrichment, Home visits

Part day, 1-2 years

Arrests, Incarceration up to age 40

Achievement Ed attainment Income

145,414 16,648

283,995 16,648

128,766

267,347

8.74 (age 27)

17.07 (age 40)

Abecedarian Project

Child/ Enrichment

Up to 8 years, including full day for 5 years

None reported Ed attainment Achievement Mother employment

142,327 70,588

71,739 2.02

Family Support

Nurse-Family Partnership – High risk

Parent/ Home visits

2 hrs biweekly, 2-2 ½ years

Arrests up to age 15

Abuse & neglect Substance use Teen pregnancy

37,041 7,324

29,717 5.06

Strengthening Families Program 10-14

Parent, child/ Groups

2 hrs weekly, 7-14 weeks

Conduct problems

Substance use Peer pressure

6,833 874

5,959 7.82

What Works, Wisconsin Page 13

Per Participant Costs and Benefits in $2004

Program Target group/ Focus

Intensity and length

Major crime prevention impacts

Impacts linked to crime prevention

Benefits Costs

Net benefit (Benefits –

Costs)

Return per $1

invested

Social-Emotional Learning

Skills, Opportunities, and Recognition

School population/ Parent and teacher training

Consistent through primary grades

Violent behavior

School behavior Achievement Alcohol misuse

14,810 4,712

10,100 3.14

Olweus Bullying Prevention Program

School population/ Improving school environment

Consistent through school years

Vandalism, fighting, theft, and bullying

School behavior -- -- --

Secondary Prevention Family Training

Family Effective- ness Training

Child, Family/ Training and therapy

1-1.5 hrs weekly 13 weeks

Conduct problems

Peer association Family functioning

-- -- --

Social Skills Training

Positive Adolescent Choices Training

Child/ Social skills training

Unspecified Aggression Violent behavior Arrests

None reported -- -- --

Mentoring

Big Brothers Big Sisters

Youth/ Mentoring

2-4 times monthly 12 months

Violent behavior

School behavior Achievement

4,166 4,117

49 1.01

Vocational/Job Training

Job Corps Youth/ Residential training

Full day, 4-6 months

Arrests Incarceration

Income Ed attainment Welfare use

22,883 15,804

7,079 1.45

Juvenile Offender Programs Diversion

Adolescent Diver- sion Project

Youth/ Case mgmt, mentoring

6-8 hours per week, 18 weeks

Reduced recidivism

None reported 24,708 1,825

22,883 13.54

What Works, Wisconsin Page 14

Per Participant Costs and Benefits in $2004

Program Target group/ Focus

Intensity and length

Major crime prevention impacts

Impacts linked to crime prevention

Benefits Costs

Net benefit (Benefits –

Costs)

Return per $1

invested

Therapeutic Interventions

Multisystemic Therapy

Youth/ Family therapy

60 hours, 4 months

Re-arrests Out-of-home placement

Family function Mental health

15,395 5,832

9,563 2.64

Functional Family Therapy

Family/ Family therapy

8-12 hours Re-offending Sibling delinquency

Family communication

29,111 2,197

26,914 13.25

Multidimensional Treatment Foster Care*

Youth/ Foster care with therapeutic treatment

Residential, 6-9 months

Re-arrests Subsequent days incarcer- ated

Hard drug use 27,460* 2,524*

24,936* 10.88*

Case Management/Multimodal Interventions

Repeat Offender Prevention

Youth/ Case mgmt; center- and home-based services

Daily, 12-18 months

Sustained petitions for new offenses

Drug use Completion of court-ordered obligations

-- -- --

Standard Services**

Juvenile Court Youth/ Formal hearing, adjudication, & disposition

2-3 hearings; length of supervision varies

N/A N/A Average cost = 2,000

N/A N/A

Standard proba- tion

Youth/ Monitoring & supervision

Weekly contact, min. 12 months

N/A N/A Annual cost = 2,160

N/A N/A

Juvenile Correc- tional Institution

Youth/ Confinement & treatment

Residential; length varies

N/A N/A Annual cost =

68,255

N/A N/A

Note: The estimated economic benefits include actual and projected economic returns through adulthood. Estimates of both costs and benefits across studies are based on different sets of assumptions and different lengths of follow up. Estimates from the Washington State study are based on a different set of assumptions than those of the other reports.

* Costs and benefits for Multidimensional Treatment Foster Care (TFC) are given in comparison to regular group home treat- ment. One year of TFC costs $2,524 more than group home care for the average participant, and yields $27,460 in benefits per participant, as compared to group home residents. All other program costs and benefits are stated in comparison to “no treatment.”