Children fan out across the tennis courts at the MaliVai Washington Youth Center in Jacksonville, Fla. Founded by tennis star MaliVai Washington, the center offers an after-school program called TnT (Tennis-n-Tutoring) where kids learn the sport. Not only do they hone their athletic skills, they also gain life skills and get academic support.

On the court, kids toss the ball upward and take a swing. Then they look to see if their serve landed where they intended. They’re measuring their actual performance against what they’d like it to be.

Something similar is happening among after-school providers. They’re honing their skills, too, and measuring their programs against carefully devised standards to be as effective as possible. Tennis-n-Tutoring, for example, adheres to the Florida Afterschool Network’s quality standards, a requirement for gaining funding from the city of Jacksonville.

[module type=”aside” align=”right”]

Find articles, research and resources on Program Quality at the Youth Today Out-of-School Time Research and Resource Hub.

[/module]

Pushed by statewide after-school networks and some funders, the creation of standards is part of a broad movement toward making the quality of after-school programs consistently high. Some cities are also adopting standards.

The Wallace Foundation has supported 14 cities in developing citywide after-school systems, starting in 2003 with Boston, New York, Chicago, Washington and Providence, R.I. In 2012, Wallace provided grants to nine more: Jacksonville; Baltimore; Denver; Fort Worth, Texas; Nashville, Tenn.; Philadelphia; St. Paul, Minn.; Grand Rapids, Mich., and Louisville, Ky.

“All nine have now adopted citywide quality standards, and eight are assessing program quality using a common citywide assessment tool,” said Priscilla Little, initiative manager for learning and enrichment at The Wallace Foundation.

“We believe after-school programs are the perfect opportunity to practice and refine social and emotional learning skills, and that only will happen when you have really deep program quality in place,” Little said.

Research led the way

As early as the 1980s, the National AfterSchool Association and the National Institute on Out-of-School Time encouraged research and assessment of programs.

Now a body of research has shown after-school programs can have a number of positive outcomes — and it has tied those outcomes to specific programmatic elements. For example, research by Deborah Lowe Vandell and colleagues at University of California-Irvine School of Education points to key ingredients of effective programs. A 2013 publication edited by Terry Peterson, “Expanding Minds and Opportunities: Leveraging the Power of Afterschool and Summer Learning for Student Success,” draws on a range of research.

Based on the research, standards are meant to ensure program practices line up with desired outcomes.

Some states have modified and adopted standards created by the National AfterSchool Association. Others have developed their own. Standards for safety, staff qualification and instructional quality are core elements, said Gina McGovern, performance information unit manager at the David P. Weikart Center for Youth Program Quality. Standards often call for nurturing strong relationships among youth and staff, and maintaining connections to families and schools, she said. The Weikart Center has helped some state networks and providers develop their standards.The process requires them to figure out how they define quality, McGovern said.

Washington state, for example, includes cultural competency and respect for diversity as one of its nine major elements. California includes program sustainability — the building of supportive partnerships — as one of its 12 core elements.

Georgia is one of the states that will adopt standards this summer. They are written as a guide to “what best practices and what quality can and should look like,” said Katie Landis, of the Georgia Statewide Afterschool Network.

The standards “are not saying you need to be excelling at each and every [standard] right from the get-go,” she explained.

“It’s really more of a tool to help providers improve their quality rather than a monitoring tool,” she said.

While some providers may already meet most of the standards, small or start-up providers may focus on two or so areas initially. “Then the next year, work on another two,” Landis said.

More than standards

“Standards are only one part of the whole quality discussion,” said Jen Rinehart, vice president for research and policy at the Afterschool Alliance.

After establishing standards, the next steps are to provide professional development for staff to implement them and to adopt a tool for measuring adherence.

Most of the nine cities aided by The Wallace Foundation have adopted an assessment tool developed by the Weikart Center or by the National Institute on Out-of-School Time.

In addition to providing staff training and program assessment, some states have gone on to develop a credentialing system for staff. Some have also adopted a system for rating providers.

For many after-school providers, securing funding through state agencies or foundations will now depend on having quality standards in place.

“You can only assure consistent quality by a set of standards,” asserted Bill Hodges, director of government relations and strategic initiatives at the Jacksonville Children’s Commission.

A public report card

Some cities and states have gone a step further and have created a public rating system.
Pennsylvania, for example, has a list of early-learning and school-age programs that participate in the rating system known as Keystone STARS. Programs are rated with one to four stars in the areas of staff, environment and operations. Ratings are assigned by state administrators based on a program’s assessment of itself, an independent assessment and documentation that it meets state requirements.

For more information on ratings, see QRIS National Learning Network at qrisnetwork.org

---

A Provider’s Experience

Jacksonville’s citywide standards help providers home in on best practices

Cheryl Wilder went to the Jacksonville Children’s Commission several months ago for training. Her small faith-based nonprofit was one of more than 60 after-school providers that would soon be covered by citywide standards.

More than half of all states have adopted quality standards for out-of-school time programs, and at least 10 more are in the process, said Jen Rinehart, vice president for research and policy for the Afterschool Alliance.

“This is an area the after-school field has been committed to,” she said. It’s “a way to show parents and funders that you are a quality program,” she said.

Wilder and her husband, Clint, founded and served as pastors at the Carpenter’s Shop Church in Arlington, an area of Jacksonville, Fla., where she grew up.

The church’s outreach to children in their low-income neighborhood grew quickly, and the Wilders shifted their focus to after-school work and dissolved the church.

Called the Carpenter’s Shop Center, their after-school program now serves 80 children from kindergarten through age 15 each year. Clint developed Survival Kids, an outdoor skills program, for use in the Carpenter’s Shop Center. A summer camp for about 100 kids is also offered.

The Wilders anticipated little problem meeting the new standards.

“The city already had a strict monitoring program,” Wilder said. And she was a member of the Florida Afterschool Network, whose standards Jacksonville was using.

Wilder went home from the training and looked at her program.

“It focused me a little bit more to make sure my goals were aligned each quarter,” she said.

One of her goals was to have 75 percent of the children improve reading and math skills by 50 percent. The children used the online skills program Stride Academy, and Wilder began documenting their progress more frequently.

In addition, she had always kept an eye on how the staff communicated with the children, and she found the communications standards a helpful tool.

“It’s like walking around with a checklist in hand,” she said.

Uniform standards are a good foundation for all after-school providers, she said, “so you know kids across the city and state are getting the same quality they deserve.”

Wallace Foundation support

Jacksonville has had a long commitment to after-school care, and its city-funded programs currently serve 1,200 children. The process of adopting citywide standards began with a $765,000 grant from The Wallace Foundation in 2012. Programs are funded through a city agency, the Jacksonville Children’s Commission, and grantees must adhere to the quality monitoring program.

After the initial training earlier this year, providers did a self-assessment of their programs, said Jon Heymann, chief executive officer of the commission. Later this year, each will be independently assessed.

In May, staff from The Wallace Foundation and the National League of Cities visited Jacksonville to look at the city’s progress.

The initiative has had an impact on school performance, Heymann told a local radio station.

“We’ve seen some schools go from F to A [in the Florida Department of Education school accountability report] and they credit, in part, the after-school programs,” he told local news station WJCT.

---

Why have standards?

It’s “a way to show parents and funders that you are a quality program.”
— Jen Rinehart, vice president for research and policy at the Afterschool Alliance

“Standards actually help start-up programs understand what they need to do.”
— Gina McGovern, performance information unit manager, David P. Weikart Center for Youth Program Quality

“High quality early-learning and after-school programs can help close the achievement gap and show significant promise for improving long-term outcomes such as high school graduation.”
— “Why Afterschool Quality Matters,” National AfterSchool Association brief

Many funders expect a provider to have standards in place: “Right now it’s not a requirement, but grantees must attend training.”
— Tameyer Evans, program manager, Afterschool Care Program, Georgia Division of Family and Children Services, which provides $15.5 million per year to 40 after-school contractors

The elements of a high-quality program

According to the National Network of Statewide Afterschool Networks, a high-quality program has:

• Positive relationships between students.
• A mix of academic and non-academic skill-building activities.
• High levels of student engagement.
• An orientation toward mastery of skills.
• Appropriate levels of structure.
• Positive relationships between students and staff.

New York state and other models

In 2010, the New York State Afterschool Network launched a set of standards along with a method of assessing them. Called the Quality Self Assessment Tool, it has served as a model for other states and is widely used in New York. Unlike other standards, it calls for a method of self-study that focuses on continued improvement. An online user’s guide leads users through the steps of assessing their program. It provides ways that programs can improve themselves immediately and in the future.

Several other models were examined by the Forum for Youth Investment in its publication “Measuring Program Quality: A Guide to Assessment Tools, Second Edition”

Assessing your program

So you’ve got standards in place. How do you tell if they’re being met? Many cities and states have developed their own assessment tools.

The steps to create an assessment are, according to the National League of Cities:

• Review and select an assessment tool.
• Align it with local standards.
• Encourage or require programs to use the assessment tool.
• Use the tool to inform programmatic improvements.
• Create a quality rating system or use an existing one.
• Connect the rating system to funding eligibility.

Ready-made assessment tools

Some commercial tools for assessing program quality:

• SACERS stands for the School-Age Care Environment Rating Scale. It is for programs serving ages K-8 and focuses on the social interactions in the program. It also is concerned with the space, materials and scheduling that support the interactions.
• The National Institute on Out-of-School Time developed a method that helps after-school providers look at specific observable practices that support students’ learning and development. These are practices that have been shown to lead to outcomes measured by the Survey of Afterschool Youth Outcomes questionnaire. The tool is called APT, the Assessment of Afterschool Program Practices Tool.
• The David P. Weikart Center for Youth Program Quality focused on key developmental experiences for young people and on assessing the quality of the learning environment and the training needs of staff. It’s known as the YPQA or Youth Program Quality Assessment Tool.

Source: “Identifying and Improving Quality Programs,” a National AfterSchool Association brief.

A sampler of standards

A set of standards often consists of seven to 12 broad categories. Each category has a group of substandards. A sampling from New York’s 10 categories includes:

Environment and climate

• Provides healthy and nutritious snacks and/or supper.
• Promotes psychological and emotional safety through a culture of support, inclusion and mutual respect.

Administration and organization

• Has a clear salary structure for program staff.

Relationships

• Teaches participants to interact with each other in positive ways.
• Has scheduled meetings with its major stakeholders.

Staffing and professional development

• Fosters understanding and appreciation for established program quality standards, evaluation and program improvement strategies among staff.

Programming and activities

• Offers high-quality academic support.
• Provides regular opportunities to be outdoors.
• Includes activities that take into account the language and culture of participants.

Linkages between day and afterschool

• Secures commitment of resources from school principal, when possible.
• Establishes strong links to the school day.
• Is represented on the school’s curriculum planning committee.

Youth participation and engagement

• Has participants who take ownership of program selection and development.
• Involves participants in the development of disciplinary practices.

Parent, family and community partnerships

• Builds relationships with arts, cultural and other community institutions to expand and enhance program offerings.

Program sustainability and growth

• Develops a long-term plan for sustaining the after-school program.
• Measuring outcomes and evaluation
• Creates an internal method for assessing program activities.
• Includes feedback from stakeholders in the program evaluation.