There is a new mantra on Capitol Hill: scientifically based research. The term is used more than 100 times in the No Child Left Behind legislation. But …
What exactly is scientifically based research? Is there enough of it available to really guide policy development and implementation?
If so, will it really be used to ensure quality? If required, could it actually undermine quality? These are the kinds of questions that arise when policy and research are forced to occupy too-close quarters in the name of improving practice.
Case in point: after-school programming. The expansion has been helped by the fact that the prevention of almost every known youth problem can be linked loosely to the provision of extracurricular activities, organized youth programs, service opportunities or mentors. Giving young people things to do and people to talk to seems to cure a variety of ills, from delinquency to school failure, and to promote a variety of less-valued virtues, from volunteering to voting.
With polling conducted by Lake Snell Perry and Associates for the Afterschool Alliance showing almost nine out of 10 adults in favor of expanding out-of-school opportunities, it doesn’t take much research to convince policy-makers to take a stab at filling the void.
But think about the challenges. Once the demand is created, who monitors quality, equity or access? When the public call for programs is almost universal, research should help put the brakes on the bandwagon growth that strains capacity or targets places and populations for solely political reasons.
With state coffers emptying rapidly, research should help curtail the inevitable dilution of quality as per-child costs are reduced to keep overall numbers growing. And when the demand for more youth development opportunities intersects with the demand for higher academic achievement, research should help us find the right balance.
Unfortunately, research on after-school and out-of-school programs is not up to these tasks.
We have solid and varied evidence that after-school activities matter. The National Research Council’s report, “Community Programs to Promote Youth Development,” confirms this. We have spotty but growing evidence that specific programs make a difference (although some would question whether this evidence is scientific enough). But we have only begun to tackle the much more taxing questions of why, for whom, under what circumstances and with what outcomes.
It is not clear how or how much weight even definitive research findings bear on policy decisions. This is not because policy-makers have a natural disdain or distrust of research. It is because of the natural dynamics of politics. The certainties of votes, trade-offs and constituent interests outweigh the certainties of data.
But the findings are not in yet. It has only been within the past few years that public and private funders have invested in the development of large, “scientifically rigorous” studies of after-school programs that begin to categorize the variation across programs (e.g., in size, duration, focus, activities, staff/student ratios, staff turnover and training).
These studies, undertaken by major universities and large research organizations such as MDRC, Public/Private Ventures and Mathematica, are only beginning to yield results that can help to answer questions about quality.
Organizations like California Tomorrow have undertaken less rigorous but no less important work to answer the “for whom” question. It is documenting the similarities and differences in programs that serve predominantly African-American, Latino and white children and youth, and looking at why these differences exist and how much they affect outcomes.
But to my knowledge, there is no major research under way aimed at answering the politically charged questions of whether the heavy public investment in the elementary school years versus the secondary school years is warranted.
These research gaps make it easier for politicians to deliver Volkswagens and call them Volvos, leaving advocates looking for creative ways to argue for program quality, diversity and age coverage. But these research gaps leave after-school and youth programs vulnerable on another front.
Without sufficient research to create a codebook for building appropriate out-of-school time (OST) structures – ones that have enough space for those who need it, enough rooms to accommodate different activities, enough ceiling height and light to ensure quality space and enough floors to accommodate children and teens – OST builders may be forced to build to the education industry’s code and meet economists’ cost-benefit standards.
The first requirement constrains form and function – pressuring out-of-school programs to not only produce academic results but look like schools. The second requirement, given the absence of hard data on benefits, may keep per-child costs so low that we end up with an infrastructure that cannot be renovated to achieve quality – once the research studies tell us what quality is.
Karen Pittman is executive director of the Forum for Youth Investment. Contact: karen@iyfus.org. To read a policy brief on this topic, go to www.forumforyouthinvestment.org/whatsnew.htm.
