Bottom Line
After years of “siloed” school reform efforts focused on such issues as teacher quality, mentoring, class size, school size and curriculum, a stringent third-party evaluation is providing scientific evidence that the CIS model for integrated services is having a positive effect on keeping youths in school and helping them graduate.
Compared with other dropout prevention programs listed in the U.S. Department of Education’s best practice website (the “What Works Clearinghouse”), CIS is:
• One of a small number of dropout prevention organizations with scientifically based evidence of effectiveness.
• One of a handful of evidence-based programs proven to decrease dropout rates.
• The only such program proven to increase graduation rates.
“You can bring a stand-alone student services program into a school, but bringing it into an integrated services school will make it more effective than the program would be on its own,” said Susan Siegel, vice president of research, evaluation and learning management for the national office of Communities In Schools.
“You can have the best teacher in the country, but if the child’s missed 40 days, or is sitting there with a toothache, that child’s not going to be ready to learn,” Siegel said.
“Integrated student services – making sure that students are in their seats and ready to learn – should be a part of any school reform effort.”
Background
Founded in 1977, Communities In Schools has grown into the largest dropout prevention organization in the United States, offering services in more than 3,200 public elementary, middle and high schools. It serves nearly 1.2 million students each year through nearly 200 local affiliates in 27 states and Washington, D.C.
CIS stresses giving children access to five basic assets: a one-on-one relationship with a caring adult; a safe place to learn and grow; a healthy start and a healthy future; a marketable skill by graduation; and opportunities for community service.
Services include tutoring, mentoring, after-school programs, career development, financial literacy, community service opportunities and life skills development – all are coordinated through a single point of contact at the school and are linked to academic outcomes.
According to Siegel, any student attending a CIS school has access to “all of the enrichment and prevention activities available” at the school. Youths identified as having specific risk factors – such as poor health, absenteeism or a lack of school supplies – are referred to a CIS case manager, who assesses the youth’s needs and crafts an individual plan that might include tutoring, referring the family to social services, or attendance monitoring, among other things.
CIS sometimes provides those services, but “in most cases” they are provided by community partners like Big Brothers Big Sisters, Siegel said.
Once the plan is in place, “we don’t leave [the student] alone,” she said. “He has benchmarks. He has meetings. He has at least one adult who is making sure he is getting the services he needs.”
Methodology
The Phase 2 implementation study consisted of a survey assessment that awarded points for the degree to which a school provided evidence of implementation for the six core elements of the CIS model. Evidence of full implementation of each element added up to a total score of 100. The core elements are:
• A CIS school-based, on-site monitor.
• A comprehensive school- and student-level needs assessment.
• A community asset assessment and identification of potential partners.
• Annual plans for school-level prevention and individual intervention strategies.
• Prevention services and resources for the entire school, coupled with coordinated, targeted and sustained intervention services for specific youths.
• Data collection and evaluation, with monitoring and modification of services offered to individual students and/or the entire student population.
The most rigorous component of Phase 2 was the quasi-experimental assessment, in which multiple years of publicly available school data were compared for matched pairs of CIS schools and non-CIS schools in a control group.
The schools were matched on demographic and academic criteria that helped assure the results were due to the presence of the CIS model and not random factors. The control schools were selected from pools containing all of the non-CIS schools in a given state, according to Siegel.
Educational data were gathered for CIS schools beginning one year before model implementation, and for three years after implementation. Data on control schools were gathered for those years corresponding to their matched CIS schools.
The natural variation study consisted of a follow-up survey to the original implementation survey, with some questions related to quality assurance standards that focused on implementation fidelity of the model.
The case studies, conducted on site by ICF staff, helped CIS understand the “nonquantifiable things that went into the data,” such as teacher quality and the quality of the community partners offering services, Siegel said.
What They Were Looking For
The evaluation was intended to provide evidence of how community-based collaboration affects school retention and graduation rates. It was designed to correlate positive student and school outcomes with the CIS model of providing integrated student services in public schools, and to examine the relationship between those outcomes and various levels of fidelity in implementing the CIS model.
What They Found
According to Siegel, the three main findings of the mid-term evaluation are:
• CIS-schools perform better than non-CIS comparison schools in increasing the percentage of youths meeting or exceeding math and reading proficiency in fourth and eighth grades.
• When the model of integrated services is implemented with high fidelity, it correlates much more positively with school outcomes than just stand-alone service provision.
• Having a coordinator on site more than 50 percent of the time correlates with much stronger positive school outcomes than when a site coordinator is present less than 50 percent of the time.
When the outcomes of high-fidelity CIS schools were compared with closely matched non-CIS comparison schools, researchers found that:
• 3.6 percent fewer students dropped out of CIS schools.
• 4.8 percent more students graduated on time with a regular diploma from CIS schools.
• 5.3 percent and 6 percent more CIS school students reached proficiency in fourth- and eighth-grade math, respectively.
• 2 percent and 4.9 percent more CIS school students reached proficiency in fourth- and eighth-grade reading, respectively.
Based on those percentages, in a high school of 1,000 students, “36 more will remain in school and 48 more will graduate on time with a regular diploma,” in high-fidelity CIS schools, according to a CIS summary of the evaluation findings. Out of 1,000 middle school students, high-fidelity CIS schools will produce 60 more students proficient in math and 49 more students proficient in reading.
The findings of the study exceed the U.S. Department of Education effect sizes for showing a “substantial impact,” according to the What Works Clearinghouse.
