While efforts to expand the school day in communities around the country are pitched as a way to boost youth achievement and even open doors for after-school providers, a new study has found less than stellar results from one of the more ambitious initiatives.
The study of the Massachusetts Expanded Learning Time (ELT) initiative found “few differences in outcomes for schools, students and teachers between ELT” schools and other schools, according to a report by Abt Associates.
While ELT programs take various forms, and many look like academic-based after-school programs, they go beyond the typical strategy of coordinating activities with schools; they also employ tight structures and more requirements, including youth participation. As such efforts have expanded in recent years, some have been run by after-school programs, such as The After-School Corporation in New York City. (See “How To … Run Effective Extended Learning Programs” at www.youthtoday.org.)
Various reports have shown positive behavioral and educational benefits of providing academic enrichment after school, although not all such efforts are as stringent in structure and requirements as the more ambitious ELT programs. A 2009 study (The Evaluation of Enhanced Academic Instruction in After-School Programs) examined schools across the country that offered structured academic instruction in reading or math after school to randomly selected students in grades two through five and compared them with other students at their schools who received mainly homework help after school. After one year, the evaluation found significant improvement in the Stanford Achievement Test math scores of the students who received the structured academic help, but no significant gain on the Stanford Achievement Test reading scores.
Abt’s new study of the Massachusetts initiative stands out because it applies rigorous scientific standards, using a comparison group, to trace specific measureable impacts in a large system – apparently the first such study of an ELT program. Among the findings: no statistically significant differences among most of the groups of students in scores on a standardized state test; more out-of-school suspensions among ELT youth; and higher satisfaction among teachers with salaries in the first year, but less satisfaction with teaching at their particular schools.
ELT in Massachusetts
Through the Massachusetts ELT initiative, schools can receive grants of $1,300 per student to add at least 300 hours to their school year, in the form of longer and/or additional school days. The aim is to improve student performance in core academic subjects, broaden enrichment opportunities, and increase planning and professional development time for teachers.
The state awarded its first grants to 10 schools in the 2006-07 school year. The grants eventually expanded to 26 schools in 2008-09. (There are now 22.) Participating schools can get technical assistance from Massachusetts 2020, a nonprofit organization that works to expand educational opportunities for Massachusetts children and families and was a driving force behind the ELT initiative.
Schools often work with community groups, such as Boys and Girls Clubs, the YMCA or the Boston Ballet, that help provide the student enrichment offerings.
What was studied
Abt is conducting a multi-year study of the initiative. The first report, issued last year, looked at the early decision-making and implementation of the ELT programs in the schools and districts. The new report examines outcomes for students, schools and teachers. A future report will measure whether the approaches to implementing the initiative are related to the outcomes.
The evaluation aimed to measure the effects of the ELT initiative, after the first and second years of implementation, on student standardized test scores; attendance, suspensions and truancy; eighth-grade students’ use of computers for school work, time spent on homework and plans to attend college; and fifth-grade students’ participation in extracurricular and recreational activities, relationships with their teachers, perceptions of school, and frequency of being hungry in school.
The study also examined teachers’ attitudes toward school, the students and their school district and compared the characteristics of students and teachers at ELT schools and non-ELT schools.
Abt relied on information from the Massachusetts Department of Elementary and Secondary Education and surveys of students and teachers. Among data used were:
• School-level data on ELT and on 24 matched “comparison” schools that did not participate in ELT. The data included school-level student achievement, demographic characteristics, school enrollment and the percentage of teachers ranked as highly qualified.
• Data from the Massachusetts Comprehensive Assessment System (MCAS), including reading/English/language arts, math and science exams.
• Student demographic data and behavior information (such as attendance and suspension rates).
• Questionnaires that eighth-graders completed during MCAS testing on such topics as time spent on homework, computer use and college plans.
• A survey of fifth-, eighth- and 10th-graders that measured students’ participation in extracurricular activities, relationships with teachers, attitudes toward school and the degree to which they felt tired or hungry at school.
• A survey of teachers’ attitudes toward teaching in general and at their school, and their perceptions of school and district leadership, parental involvement and student attitudes.
Results
The findings are divided into three areas: those for students, teachers and schools. Here are some highlights:
• The initiative had a “significant, positive effect” on fifth-grade science MCAS scores in the second year of implementation, but no statistically significant effect on other MCAS outcomes in either year. The researchers note that some matched comparison schools had increased the instruction time for language arts and math, even without a longer school day, but not for science and social studies.
• ELT had a “statistically significant, negative effect” on attendance in the first and second years, but the small, estimated difference “may not be practically meaningful.”
• ELT schools had slightly higher rates of out-of-school suspensions in both years.
• Eighth-grade ELT students were significantly more likely than their counterparts at comparison schools to report using school computers for school work at least once a month in the first year, but there was no difference in the second year.
• Fifth-grade ELT students were less likely to participate in a non-academic club at school.
• ELT had no effect on fifth-graders’ perceptions about their relationships with teachers, including whether they spent more time with teachers.
• After the first year, ELT teachers were significantly more likely to report that they were satisfied with their salaries and that they would still become teachers if they were starting over, but they were less likely to say they were satisfied with teaching at their school. These differences did not show up in the second year.
• Significantly more teachers in ELT schools reported thinking about transferring to another district after the second year, compared with their non-ELT counterparts. The study does not say why. There was no difference in the first year.
Supporters of ELT admit the results are disheartening. “We were disappointed, for sure, that we didn’t see greater gains,” said Fran O’Reilly, vice president of research at Massachusetts 2020.
But she said that while the overall results seem flat, ELT made a big difference at some schools. At Clarence R. Edwards Middle School, for example, 68 percent of low-income students achieved proficiency in English/language arts in 2009, compared with 61 percent of low-income students statewide and 36 percent of low-income students at the school in 2006, according to Massachusetts 2020. In math, 57 percent of low-income students achieved proficiency in 2009, compared with 25 percent of low-income students statewide and 11 percent of low-income students at the school in 2006.
The school, where 89 percent of the students are low-income, provides increased instruction time in small classes tailored to students’ most pressing academic needs. Seventh- and eighth-graders choose from enrichment electives including swimming, Latin dance and fashion design.
JC Considine, spokesman for the Massachusetts Department of Elementary and Secondary Education, noted that the initiative is still in its early stages. Abt will conduct additional outcomes studies over the next few years. Considine also said the department has made changes to increase accountability, including a requirement that schools must work toward specific goals that they establish in performance agreements with the state.
