It’s right up there with the search for the Holy Grail or the Loch Ness monster: no, not the hunt for weapons of mass destruction, but the hunt for evidence-driven programs that produce verifiable progress in children and youth services.
If designing, testing, evaluating and bringing proven program winners to scale were easy, doing so would be entrenched at the core of public and philanthropic grant making.
Now the possibility looms that positive program results will truly be linked to who gets funded to do what. Urging a results-based policy is none other than President George W. Bush. “We will start by funding only what works,” candidate Bush pledged when speaking before the Latin Business Association in September 1999. “My administration will require every federal program … to prove results. If it can’t, we will shift money to a program that is using it wisely.”
Whether Bush’s avowed commitment to sound evidence of success or failure will be applied evenhandedly remains to be seen.
Top-notch research leading to near universal adoption of proven interventions is a standard feature of medical research. In the early 1960s the U.S. Food and Drug Administration mandated randomized control trials for new medical interventions. The result of this felicitous policy has been an increase in the use of randomized trials from about 100 to 10,000 per year. The physical health benefits for Americans have been enormous.
But throughout the education, social welfare, criminal justice, job training and youth development fields, the building of the knowledge base for best practices (and best spending) has been slow. Fad often drives out fact; politics routinely trumps proven results.
Passing the Test
Despite formidable obstacles, a number of programs have passed the randomized control evaluation test. Among them is the High/Scope Perry Preschool for poor children, initiated in 1962. An evaluation that followed the children through the age of 27 showed that, compared with a control group, program participants were more likely to have graduated from high school or earned GEDs (by 71 percent to 54 percent), less likely to have been on welfare (59 percent vs. 80 percent for the control group) and less likely to have had five or more arrests (7 percent vs. 35 percent of the control group).
Another example is the low-cost Life Skills Training program for junior high school youth, which has been shown to reduce monthly use of tobacco, alcohol and marijuana by more than 40 percent by the end of high school.
Alas, these programs, as well as those endorsed by the “Blueprints for Violence Prevention” produced by the University of Colorado’s Center for the Study and Prevention of Violence, have yet to capture center stage with grant makers.
How much money effective social interventions would save has been the subject of constant research, as well as speculation tinged by world view. Vanderbilt University researcher Mark Cohen’s 1997 study, “Diverting Children From a Life of Crime: Measuring Costs and Benefits,” estimated that a typical six-year crime career produces associated social costs of $1.7 million.
Savings through prevention – or youth development, as we prefer to think of it – are by their nature elusive. A 1999 study, “A Matter of Money: The Cost and Financing of Youth Development,” by the Center for Youth Development and Policy Research at the Academy for Educational Development, estimates that “ensuring that developmental opportunities and support are available to all school-age youths” for 1,192 hours a year would cost $2.55 an hour (about $3,000 annually) per youth. That works out to $143 billion a year for about 47 million school-aged children.
But consider that federal spending in 1997 on addressing kids’ problems totaled $52 billion, according to a Pacific Research Institute review. Add to that figure state and local spending on kids for remedial services, and investing in comprehensive youth development through proven programming doesn’t look so expensive after all.
What if …?
A concerted campaign is being mounted to introduce randomized control groups as the norm in federally funded children and youth program evaluations. The nonpartisan, D.C.-based Coalition for Evidence-Based Policy, one of those behind that effort, has focused on U.S. Department of Education programs, including the much-lambasted $422 million Safe and Drug-Free Schools and Communities effort. In a report funded by the W.T. Grant Foundation, the coalition goes beyond that oft-heard favorite of the research class: recommending more investigations to “build the knowledge base.”
Rather, the coalition urges that once a government agency sets its priorities, that agency should provide strong incentives for the widespread use of interventions that have been proven effective by the kind of “scientifically based research” called for in the No Child Left Behind Act. That would mean, for instance, recasting the competitive grant-making process so that extra points are awarded in grant evaluations to applicants that implement proven approaches.
Let the good drive out the bad. If only it were so simple! The Bush administration is committed to further lowering taxes and cutting domestic discretionary spending. So when an evaluation of the first year of the 21st Century Community Learning Centers’ after-school programs hinted at mixed results, the White House was quick to concur.
But according to a recent report, “Politics and Science in the Bush Administration,” released by Rep. Henry Waxman (D-Calif.), the administration is cooking the books on science-driven information. Unpalatable findings on everything from lead paint poisoning to the effectiveness of condoms are being censored.
But what will happen if some of this administration’s cherished beliefs are challenged by evidence-based research? What if sectarian faith-based providers get worse results than secularized agencies? What if the abstinence-only approach to sex education and HIV/AIDS prevention programs is found ineffective? Will the White House and Congress cut them off in favor of, for example, the Children’s Aid Society Carrera Teen Pregnancy Prevention Program?
What if it’s proven that the better crime prevention policy is to keep convicted juveniles in juvenile facilities, not adult prisons? Will failed but politically powerful programs like DARE be shunned? Will the evidence prevail, or will political consideration triumph?
Evidently, we’re going to find out.
