The federal government’s increased support for after-school funding and its drive to boost academic achievement have collided head-on, leaving one of the biggest federal programs for youth services limping toward possible extinction.
President Bush’s 2004 budget, released last month, proposes to cut the 21st Century Community Learning Centers (CLC) program by 40 percent, saying that a new study shows that CLCs do not improve youths’ school performance or their behavior.
The CLC program, which serves 7,500 elementary and middle schools in 1,400 communities, would go from $1 billion this year to $600 million.
The move shocked and infuriated many after-school advocates, who blasted the study as flawed, and launched local and national campaigns to convince Congress and the public that after-school programs have provable benefits for kids.
“It’s just implausible to suggest that after-school programs should be held accountable for the academic achievement of students,” said a steaming Judy Samelson, executive director of the nationwide Afterschool Alliance. “I have real fears that the emphasis they’re putting on schools [to increase academic performance], that’s now seeping into after school, is going to totally change how programs are run, what they’re about and why they exist.”
The dispute has exposed a fundamental conundrum for after-school programs: Many of them have accepted more and more funding in recent years from benefactors (such as the federal government) who see the programs as helping to fulfill the national push to improve academic achievement, even though many after-school advocates see academic enrichment as only one small part of the benefits of such programs.
So now a federally funded study has shown little academic benefit from a sample of CLC programs, giving the Bush administration a rationale to slash funding – and prompting CLC supporters to charge that CLCs are being judged by a standard that youth development advocates never said should be the measure of success.
The dispute thus brings to the fore questions about how after-school programs should be evaluated and what benefits have been found by past evaluations.
‘Limited Impact’
Some people smelled trouble from the start.
The U.S. Department of Education (DoE) originally issued a request for proposals for a study of best practices of early CLC programs. However, Mathematica Policy Research, the Princeton, N.J.-based group that does many evaluations for federal and other government agencies, responded with a more ambitious proposal involving tracking youth in several programs to assess program impact. The DoE awarded Mathematica a $13 million contract to do just that in 1999, paying several times what it had originally planned to spend.
The Charles Stewart Mott Foundation, a major after-school supporter which partners with DoE on the CLC program, contributed $3 million more, as it had pledged to do before the study changed focus.
Mathematica had tapped right into a federal trend. Mike Dynarski, the co-principal investigator for Mathematica on the project, recently wrote in response to critics that the study “was designed to meet a growing demand to apply more rigorous, ‘scientifically based’ research” outlined in such federal actions as the No Child Left Behind Act.
But advocates like Samelson tried in vain to push the federal government back toward the original plan. “This was not the right kind of research for this program at this time in its life,” Samelson said.
Among the concerns: The study would measure programs early in the implementation of the CLC (for the 2000-2001 school year). The programs and DoE “were struggling with how to design quality after-school programs,” Afterschool Alliance Associate Director and former DoE staffer Jen Rinehart wrote in a recent critique of the Mathematica study. They hadn’t had time to make adjustments based on what they learned early in the process.
Also, she said, the goals of the CLC have shifted toward academic achievement since the programs in this study were funded.
The study – “When Schools Stay Open Late: The National Evaluation of the 21st-Century Community Learning Centers Program” – looked at more than 5,000 children from centers in 41 school districts. While the CLCs “changed where and with whom students spent some of their after-school time and increased parental involvement,” Mathematica said, “they had limited impact on academic performance, no influence on feelings of safety or on the number of ‘latchkey’ children, and some negative influences on behavior.”
There were, of course, differences among various groups – elementary students did better in social studies, for instance, while there was less absenteeism and tardiness among blacks and Hispanics. The study found few students attending the programs daily, with more than half the middle-school youth attending fewer than 25 days a year – a key impediment for programs trying to demonstrate impact.
The study was released at about the same time that Bush unveiled his 2004 budget proposal. The budget summary for the DoE said the proposed decrease for the CLCs “acknowledges that the program needs some time to address disappointing initial findings from a rigorous evaluation. … The centers funded in the program’s first three years are not providing substantial academic content and do not appear to have a positive impact on student behavior.”
Thus the Mott Foundation saw a study that it helped to fund used to justify cuts in the largest program it supports. “We were surprised they [the Bush administration] did that,” said Mott program officer An-Me Chung.
Showing Evidence
That one-two punch set off a flurry of activity among after-school advocates. Their first order of business: attempting to discredit the value and relevance of the Mathematica findings for judging the CLC program.
Samelson said the study criteria essentially amounted to a change in the rules, because the DoE “always pushed hard on the academic piece” of CLCs but never said it expected the programs to show academic improvements based on rigorous scientific studies. The CLCs “never set out to be programs that were going to change the academic landscape,” she said.
Rinehart’s critique, distributed on the Promising Practices in Afterschool Listserv, said the study sample was too small and the programs and youths not representative of CLC programs or the youths they serve.
She said that Mathematica’s summaries ignored or played down positive findings, such as higher math grades and improved class participation among girls, increased parental involvement (“a remarkable finding”) and an increase in self-confidence.
The alliance and other groups, such as the Washington, D.C.-based Forum for Youth Investment and the Harvard Family Research Project Out-of-School Time Program, have countered the findings by referring to studies that show positive impacts from after-school programs. (See Pittman, page 55.) Rinehart noted an upcoming study of 96 programs run by the After-School Corporation in New York City, which found that youths had reported less risky behaviors (such as alcohol use and sex) since participating in the program. She said a Boys & Girls Club study in Louisville, Ky., showed that drug use and vandalism increased significantly in housing projects without B&G Clubs compared with those that had new clubs.
In addition, Fight Crime: Invest in Kids was compiling a report that contends that the CLC cut would drop half a million youths from after-school programs, which “will increase crime significantly,” said spokesman Phil Evans.
Some of the studies “do not meet adequate scientific standards for measuring program effects,” Dynarski wrote in response.
Get Intense
Dynarski accused the Afterschool Alliance of bias in highlighting positive findings in the study while blasting the study’s methodology because of the negative findings. “The Alliance illustrates why prominent researchers have called for greater rigor,” he wrote.
One CLC provider said that while the funding cut is unjustified, some CLCs have hurt themselves by refusing to adopt practices that would increase academic outcomes without turning the programs into homework clubs.
“Objective research is the law of the land,” said Paul Ahrens-Gray, president of Global Learning, a Portland, Ore.-based after-school provider that is a partner in two CLC programs in Oregon and California.
Ahrens-Gray said after-school providers have repeatedly rejected his suggestions to adopt more intensive curricula that would employ a “high level of professional development and training” to involve youth five days a week, three hours a day, in structured activities using writing, drama, art and reading.
What he often saw instead was a patchwork of activities, many of which had already been provided in some form and which were stitched together as a CLC. He cited a California elementary program where the managers said, “ ‘We have these people from the dance program who are going to come on Wednesday, and on Thursday the people from the science project will do a half-hour.’ They described this program that one student might attend for one hour a week, another might attend for three hours. … [A youth] might have 22 days of service in the course of a year.”
After-school advocates said CLC programs have been working toward increasing participation rates and enriching their curricula, but the proposed federal cut is based on a study that looked at programs that by and large had not made such adjustments.
Valentine Cards
Do after-school supporters have enough clout and organization to fight the cuts? On the national level, the Afterschool Alliance is using the Internet and phones to push a “write your congressman” campaign, using its website to post summaries of the main issues, samples of letters and contact information for Congress. Samelson also expects to organize policy briefings for staff and lawmakers on Capitol Hill. The alliance has gathered testimonials from all corners, from former U.S. Secretary of Education Richard Riley to Lynn Sobolov, project director of the Kaleidoscope Community Learning Centers of Morgantown, W.Va.
In San Antonio, Texas, children and their parents in a CLC program have written some 1,500 letters to Bush urging him not to cut the funding, said Helen Kimsey, who organized the effort as project coordinator for after-school care in the North Side Independent School District. She said copies of the letters will be sent to the state’s senators and representatives in Washington.
In Iola, Kan., youths, parents and staff at Safe Base, also a CLC program, signed six poster-sized Valentine’s Day cards for the president and their congressional representatives. The cards say, “Don’t be a fool and take away a tool that helps us in school.”
But Richard Murphy, co-director of the Center for Youth Development and Policy Research at the Academy for Educational Development, cautions that a more effective tactic might be to focus on something more basic, something that parents know is true without getting confused by competing studies.
“Parents need this [after-school care] desperately just to manage their lives and their kids’ lives,” he said. “Forget about academic outcomes and the reduction of crime.
“You can’t leave kids alone. That may ultimately be the best argument.”
Resources
Judy Samelson, executive director
Afterschool Alliance
P.O. Box 65166
Washington, DC 20035-5166
(202) 296-9378
www.afterschoolalliance.org
Mike Dynarski
Mathematica Policy Research
P.O. Box 2393
Princeton, NJ 08543-2393
(609) 799-3535
www.mathematica-mpr.com
Research
Afterschool Alliance, Research Links
www.afterschoolalliance.org/research_links.cfm
Out-of-School-Time Program Evaluation Database
Harvard Family Research Project
www.gse.harvard.edu/hfrp/projects/afterschool/evaldatabase.html
(617) 495-9108
The Forum for Youth Investment
www.forumforyouthinvestment.org
(202) 207-3333
