Eight U.S. cities worked to develop and manage data in their citywide after-school systems over four years.
A look back at their experience shows, among other things, that one size doesn’t fit all.
“[Initially] there was a sense you could take a management information system and plop it down in a city and you have a data system,” said Julie Spielberger, a research fellow at Chapin Hall at the University of Chicago.
But different cities have different goals for their after-school programs, she said.
“Some wanted to be able to show how after-school programs can support school success,” she said.
The Wallace Foundation provided funding and other assistance to Baltimore; Denver; Grand Rapids, Mich.; Jacksonville, Fla.; Louisville, Ky.; Nashville, Tenn.; Philadelphia, and St. Paul, Minn. from 2012 to 2016.
Some cities, such as St. Paul and Denver, were mainly focused on developing social and emotional strengths among kids. The trouble was finding adequate measures for social and emotional development — and that effort was more time-consuming, Spielberger said.
In implementing a data management system, it’s necessary to fully understand the local context and goals, according to the report “Using Data to Strengthen Afterschool Planning, Management and Strategy: Lessons from Eight Cities.” Spielberger was among the authors of the report.
Citywide after-school systems bring together a variety of organizations. In the eight cities studied, the players mostly included the city, the school district and a variety of community-based organizations. Each city had a coordinating entity, which in some cases was a nonprofit, in others a local government agency.
“Over the years we’ve learned that bringing after-school programs together into a system … provides many more benefits,” Spielberger said. Individual after-school programs are not necessarily connected to the resources they need, she said.
But the programs that came together into a citywide system often had no connections or communications with each other, she said.
With this diversity of players, a strong systems-level focus is necessary, the report said. It helps the overall system weather challenges.
It’s important to collaboratively agree on meaningful indicators of progress, the report said. The cities that were most effective had recognized indicators of early progress. For example, one indicator was the successful negotiation of a data-sharing agreement with a school district. Another was the development of a data-sharing dashboard.
In addition, “you want to share progress and keep people informed to keep them engaged,” Spielberger said. “City leaders are very busy.”
It’s important to realize that participating organizations share motivation but may have different priorities, according to the report.
Among after-school providers, “the interest is in working with children, not trying to document everything going on,” Spielberger said. “There needs to be a lot more time spent in teaching people how to use data.”
“There’s not always agreement on the goals of the data system,” she said.
IWhile the process can be difficult, it’s necessary to bring all key players together and talk through different views, she said.
The system needs to have flexibility to withstand the unexpected, the report said.
“One thing that wasn’t anticipated was how much it would take to train and engage staff,” Spielberger said.
Turnover among frontline staff sometimes made data training and collection difficult. But turnover also occurred at the highest level of city leadership.
