 |
|
RICHMAN: “We as academics are quick to say, ‘You should do this,’ when we don’t understand” what nonprofits are “up against.” |
What could be better than a free program evaluation? How about a free program evaluation and free training at a major university so that your staff can conduct more evaluations?
If it sounds too good to be true, that’s because it is – for all but a few community-based organizations (CBOs) in Los Angeles that have tapped into an innovative collaboration offered by UCLA’s Center for Community Learning (CCL).
The center offers a semester-long honors course, “Client-based Program Evaluation,” that teaches evaluation skills to undergraduates, with a refreshing twist: It focuses on reality.
The course is offered free to staff members from CBOs that become “community partners” and that will let the class design and carry out an evaluation of them. UCLA pays for the staffers’ books and parking, and offers them college credit through the university’s extension office.
Too bad all CBOs don’t have the same opportunity. “I don’t know of anyone who does this kind of community-based research within a service-learning course,” Kathy O’Byrne, CCL’s director and one of the instructors for the course, said via e-mail.
Why not? “That’s a very good question,” said Neal Richman, associate director of UCLA’s Advanced Policy Institute and another of the course’s instructors.
Richman surmised that the course’s unconventional design has something to do with its rarity on campuses. “It is pragmatic learning: You hop into problems, no one gives you the background and training; no one spends weeks for you outlining all the theory and rules. You learn by doing. … I think that’s very threatening to academics,” Richman said.
So, how do you get your free lunch?
“You need to find an ally” in a local university, he said. “You need to find someone on the inside that understands this [hands-on] perspective. And very often, it’s a nontraditional academic.”
Meeting Needs
The idea for the program emerged about five years ago, when several UCLA faculty members discussed the number of requests they’d received from community-based nonprofits for assistance with program evaluations, according to a recent article by O’Byrne in the online publication Youth Media Reporter (an initiative of the Academy for Educational Development).
The course was launched in 2003 through CCL, which used a $40,000 Community Partnership Grant from UCLA’s Center for Community Partnerships.
The class has been offered for the past four spring semesters. According to O’Byrne, 10 university students and 10 CBO staffers were enrolled in each of the first two years, while in each of the past two years there were 15 of each.
The class spends the first four weeks of the 10-week course reading a series of texts that teach basic methodologies, theories and concepts. Then teams from the class spend four weeks collecting and analyzing data at the CBOs. They use the final two weeks to prepare evaluation presentations. At the end of the semester, teams from the class present their findings to an assembly of academicians, policymakers and community nonprofit leaders.
CCL sets three goals for the course: to provide the partner CBOs with a usable program evaluation report; to connect undergraduates interested in public service careers with CBOs; and to train CBO staffers so they can continue and expand their evaluation efforts at the agencies.
The Need for Evaluation
For the CBOs, the evaluations fill an important but expensive need.
“I can’t raise funding for evaluation, but everybody wants evidence-based outcomes,” said Jeff Farber, executive director of Helpline Youth Counseling, which enrolled its deputy director in the course in the spring of 2006. “If there are tools that can be brought to our organization that allow us to better assess and evaluate our service delivery … that’s positive. And if we can do more training to get more people aware of how to do that, that’s ideal.”
Agreeing wholeheartedly is Gloria Davis, executive director of the Girls Club of Los Angeles.
“We’re always trying to find collaborations for which we don’t have to pay, because evaluation can be quite an expensive part of your budget,” Davis said. “So when we see workshops like [UCLA’s], we take advantage of it.”
The course is designed to have real impact. The CBOs spend months with the course instructors planning the research questions for the class, and each team of researchers must connect its work to a policy issue. O’Byrne believes that groundwork is one reason that the evaluations have been used for advocacy, policy work, needs assessments, program decisions and grant proposals.
Perhaps more important than a final evaluation is the need to connect evaluation and community service in the minds of a new generation of student researchers. By the end of the course, O’Byrne wrote for Youth Media Reporter, the participants “exhibit a high level of dedication and responsibility to a ‘real world’ audience, especially when working to collect and analyze data that can be given back to the community.”
Part of that real world experience includes overcoming evaluation obstacles. “It was a learning experience for [the students] to understand when you’re dealing with either youth mental health or substance abuse programs, there are restrictions in place regarding confidentiality,” said Eva Estrada, director of clinical services at Helpline. “There were some major hoops they had to jump through.”
UCLA’s Richman says the real-world learning flows both ways: “There are things [academicians] can learn from practitioners as well: managing the complexities, particularly the lack of resources in the field. We as academics are quick to say, ‘You should do this,’ when we don’t understand what they’re up against.”
The Need for Training
Having a free source of ongoing staff training is proving invaluable to local nonprofits. “It’s easier to recruit community partners every year” for the course, O’Byrne said by e-mail. “We have good ‘word of mouth’ in the community, and some nonprofits even send new staff in consecutive years.”
Davis, at the Girls Club, says CBOs – although often painfully pressed by funders for evidence-based outcomes – typically don’t set aside either the staff time or money for the evaluation training.
“Our staff is really overworked, and a lot of times we don’t see [training] as necessary, but I found it to be very helpful,” she said. “We want to show outcomes. We want to show that our program is making an impact in the community.”
After taking the course in 2006, Girls Club staffer Jessica Hutcheson became the club’s evaluation coordinator. Among other duties, she monitors and manages the work of a research firm hired by the club to evaluate its teen pregnancy prevention program.
“It’s not ‘we drop in and do something for you and then we split,’ ” said instructor Richman. “The staff there is getting trained in evaluation research that they can then use as an integral part of their ongoing program. …
“I love that it leaves capacity [to do more evaluation] with these organizations.”
Contact: O’Byrne (310) 835-7867, kobyrne@college.ucla.edu.
