The Court Appointed Special Advocate (CASA) program couldn’t sound more apple pie, more thousand points of light.
CASAs are a cadre of 74,000 volunteers trained for dozens of hours, then dispatched to conduct independent investigations of child abuse and to represent the children’s interests in courts around the nation. What could be wrong with that?
Virtually nothing, according to past evaluations. A qualitative consumer satisfaction survey of 23 CASA programs last year, commissioned by the National CASA Association, gave glowing reviews.
So, a second, more ambitious evaluation, a national one including a control group, seemed without risk.
That’s not how it turned out.
While containing some information for national CASA to brag about – such as judges assigning CASAs to the most difficult cases, then frequently doing exactly what the advocates recommend – the report commissioned by the association delivers some surprisingly damning numbers.
It says CASAs, an overwhelmingly white and female group, spend little time on cases, and even less on those of black children. It says youngsters with CASAs are associated with more removal from parents, less kinship care and less reunification with parents.
CASA critics, including social workers who say problems in child welfare should be addressed by hiring more professionals rather than relying on volunteers, seized on those numbers, contending that they prove CASAs might actually harm children and families.
CASA officials focus on the positive findings and argue that the negative ones are questionable or need more study. Even the evaluation’s author, Caliber Associates, has taken the unusual step of responding to CASA critics by stressing that some of the numbers may not prove anything because the controls in the study may have been faulty.
Whatever the truth about CASAs, the organization’s experience with the study illustrates an increasingly important point for the youth field: the risks that groups take when complying with mounting demands from government and foundation funders to prove that what they do works.
Initial Findings
Michael Piraino, national CASA’s chief executive officer for the past decade, says he sought funding for a thorough CASA evaluation long before accountability became the demand du jour. He recalls that in the mid-1990s, when he was searching for $1 million to pay for two evaluations and a data collection system to support them, funders wondered if they were worth doing. (See “Court Advocate Program Grows, But How Much Does It Help?” June 2000.)
Beginning in 1997 and ending in 2000, Piraino received three grants totaling $1 million from the David and Lucile Packard Foundation of Los Altos, Calif. The grants paid for software, called COMET, to help local CASA organizations track information, such as the hours that volunteers spent on cases and how many recommendations were accepted by judges. The grant also paid for two evaluations: the first, a qualitative study conducted by Pat Litzelfelner of the University of Kentucky College of Social Work, and the recently released quantitative study by Caliber.
In Litzelfelner’s satisfaction survey, which began in 2001 and examined responses from 742 judges, lawyers, parents, foster parents and social workers, every group gave CASA a positive ranking on every question. The questions ranged from whether the respondent understood the role of a CASA (which got the highest scores) to whether the CASA visited the children regularly (which got among the lowest).
Litzelfelner notes that her results can’t be generalized to all CASA programs because the sites surveyed were not randomly selected.
CASA critic Richard Wexler, executive director of the National Coalition for Child Protection Reform, points to comments from caseworkers that CASAs needed to spend more time on cases, and that many CASAs have middle-class values and do not appreciate the different cultural backgrounds of their clients. Wexler says these comments reinforce his contention that CASA is little more than a bunch of white women with matching shoes and purses telling poor black mothers how to run their households.
The Caliber study, which cost $317,000, was based on COMET data and on the U.S. Department of Health and Human Services’ (HHS) National Survey of Child and Adolescent Well-being, which contains statistics on 5,500 children from across the country. Twenty-five of the 915 CASA organizations around the country provided information from their COMET databases.
Among Caliber’s findings from COMET: Ninety percent of volunteers were white, and 79 percent were women. Only 8 percent were African-American, a sharp contrast to the children in the nation’s child welfare system, 40 percent of whom are African-American, according to HHS statistics. Piraino says that among all CASAs nationwide (as opposed to the 25 CASA organizations studied), 18 percent are African-American, a figure he notes has increased from 13 percent five years ago.
The COMET data also showed that after receiving an average of 43.9 hours of training, volunteers spent 3.2 hours a month working on the case to which they were assigned. When this figure was announced at the CASA national conference in Washington in June, a sucking noise of disbelief was audible, recalls Dennis S. Hockensmith, executive director of the Pennsylvania CASA Association.
CASA directors across the country say their volunteers give much more time than the report says. Caliber says the number in its report may be incorrect because CASAs may not have done a good job of logging their hours on COMET. Volunteers are asked but not required to log their hours, and it is unclear how many of them do it promptly or completely.
The data did show that volunteers spent most of their time with the children, much more than on other activities, such as writing reports for the court and interviewing parents and foster parents. But when the child was black, the amount of time spent per month dropped by more than an hour per volunteer.
On the positive side, the data also showed that in 61 percent of cases, judges accepted every one of the volunteers’ recommendations. But there was this oddity, considering the overwhelming number of white women volunteers: Judges were four times more likely to abide by advice from men than from women, and 2.5 times more likely to follow recommendations from African-American volunteers than those of other races.
From there, the findings get more controversial, and the research more tricky.
Out-of-Control Group
National CASA has long suspected that CASA volunteers are assigned the most serious and difficult child abuse and neglect cases that come before the judges. Using data from the federal well-being survey, Caliber confirmed that.
To isolate the impact of CASAs on the youth assigned to them, Caliber attempted to create a control group of children who did not have CASAs, but whose cases were as serious as those who did. Caliber considered factors such as age, sex, race, previous out-of-home placements, and the number of risk factors reported by caseworkers
The results: Children who were assigned CASAs, and their parents, received more services than those without CASAs. Children without CASAs weren’t any more likely that those with CASAs to be mistreated again, and they stayed in the child welfare system about the same amount of time.
But CASA kids were more likely to be removed from their parents: 89 percent of the time for CASA kids, compared with 18 percent for non-CASA kids. For children whose cases remained open at the end of the study period, CASA kids were less likely to be reunified with their parents and less likely to have been placed with kin.
Wexler jumped on the findings, saying the study showed that the CASA program “does nothing to actually improve the lives of children and may well make them worse.” Part of Wexler’s criticism of CASA stems from his conviction that judges, based on recommendations from child welfare caseworkers and CASAs, remove children from their parents far too often.
By contrast, for people like David W. Soukup, who founded CASA when he was a judge in Seattle, these removals usually signal that the system is working to protect abused children.
Policy differences aside, Caliber says Wexler misconstrued its report. But the consultant has also backed away from some of its own findings. Caliber says that when it used a mathematical model to test the control group findings, it discovered that it may have failed to establish comparable groups.
“It doesn’t look statistically plausible to me that having a CASA has more of an effect than the type of abuse, what is going on with parents, et cetera,” says Caliber senior associate Jennifer Brooks, who worked on the study.
So, the most that can be said about the findings is that there is a relationship between the CASA and other effects, such as more frequent removal from parents. But the study, Caliber researchers say, probably does not show that the CASA caused that effect.
Raymond Kirk, research professor at the school of social work at the University of North Carolina/Chapel Hill, agrees that the comparison group findings should be viewed with caution. “The CASA model should not be over applauded or criticized,” he says. “The science part of this report is too soft for that.”
The feelings of Susan Cammarata, a Pittsburgh attorney with some expertise in child welfare cases and several years of experience as a CASA volunteer, may represent that perspective. The Caliber evaluation confirmed her inklings. “CASA has its place,” she says, “but I would rather see the child welfare system improved.”
The Packard Foundation, the funder, declined to comment on the evaluation’s findings.
Picking and Choosing
One reason the findings threw CASA officials for a loop is that, although they were participating in a scientific evaluation, what they really wanted was validation. A statement on the National CASA Association website calls the Caliber study part of CASA’s commitment to “measuring success in our work.”
“We hear good stories about what we do for kids,” Piraino says. “But we wanted to document it.
It’s not surprising, then, that CASAs were taken aback when the report suggested that what they thought about themselves might not be true. The reaction of CASA organizations to the negative parts of the report has almost uniformly been denial. Kelly Warner, spokeswoman for the Oklahoma CASA Association, says, “I read the study and my reaction was: Who are they talking about? Because this cannot be Oklahoma. I do not think we are doing something wrong.”
National CASA has boasted about the parts of the study it liked, while saying the findings that could be considered critical are questionable and in need of further study.
This might be a natural organizational reaction, but it can border on duplicity.
Trudy Strewler, executive director of the Pikes Peak Region CASA, based in Colorado Springs, Colo., is among those who say CASAs should be proud of the results showing that CASA kids and families get more services. The National CASA Association has emphasized that result as well – even though it comes from the same part of the study in which the association rejects the negative findings because of potential control group problems.
As for those negative findings, CASA organizations are calling for more study and improved data collection. Hockensmith says of the racial disparity in the time CASAs spend on cases, “I am sure CASA will look long and hard at that. It needs to be analyzed more deeply.” And Warner of Oklahoma is among those who say volunteers must improve data collection by being more diligent about recording their time in the COMET database.
Potential Impact
Faltering on an accountability test can’t be all that bad. After all, every state in the nation has failed the federal Child and Family Service Reviews of their child welfare programs, and HHS, the agency holding their purse strings, hasn’t done a thing about it.
HHS says states now know what they must do better.
The negative findings in the Caliber study didn’t prompt CASA to promise changes in practices. “National CASA will not allow inconvenient facts to get in the way of its insistence that CASA is a success,” Wexler said in a news release. “This state of denial only compounds the harm done by the program itself.”
CASA’s attempts to stress the positives may be nothing more than instinctual self-preservation. Negative evaluation findings, followed by an outcry from critics, could hinder a youth group’s ability to raise money and expand.
Piraino concedes that the report could hurt CASA. “It depends on how people look at it,” he says. If they believe the criticisms sound too simplified, he says, the evaluation won’t be a problem.
It can’t be much help, however, in places like Cleveland, one of the last major CASA holdouts. Cuyahoga County Juvenile Division Senior Judge Peter J. Sikora says he prefers to have children represented by licensed attorneys who are trained for the work by the bar association and who may be held accountable for their performance in ways that volunteers cannot be, such as loss of license.
Because it would be costly to start and run a CASA program, Sikora says he’d have to be convinced of its value before moving in that direction. “Our court was taking a bit of criticism for not having a CASA program,” he says. “But unless you can show me children and families benefit, and can show me a cost benefit, I see no reason to switch from the system where children are represented by licensed attorneys.”
The Caliber evaluation won’t do that for him.
Maybe another study will. Caliber is recommending that one be done in the same form as a double-blind medical investigation, where some patients get an experimental drug and some get a sugar pill.
Piraino says that’s unlikely, because judges want CASAs appointed to all of the most serious cases and won’t stand for half not getting the volunteers
He agrees that it might be done in a place where there is no CASA program, so that judges will feel that they’re getting something they didn’t have before (CASAs), at least for half the serious cases.
Perhaps Piraino’s hometown of Cleveland would work.
CASA
The Court Appointed Special Advocates program was created in 1977 by a Seattle judge, David Soukup, who was desperate to get more information about abused and neglected children and their families so that he could make better decisions. He wanted trained volunteers to research family situations and provide crucial information to the court. As of last year, there were 73,860 volunteers in 915 CASA programs in 49 states, the Virgin Islands and Washington, D.C.
The National CASA Association has a budget of $14 million, $12 million of which comes from Victims of Crime Act through the U.S. Department of Justice.
CASA Findings
From the report by Caliber Associates:
• CASA volunteers were 90 percent white, 8 percent African-American and 79 percent female. They served a child population that was 48 percent white, 36 percent African-American or biracial and 49 percent female. Hispanic or Latino children made up 5 percent of the study group. That racial statistic was not given for CASAs. The study found that judges assigned CASAs to disproportionately fewer Hispanic and Latino children.
The mean number of hours per month that CASAs spent on African-American children was 2.67, vs. 4.30 for children of other races.
• Most CASAs (87 percent) had college degrees or had taken college courses. College-educated CASAs spent significantly less time, 3.12 hours a month, volunteering than did others, who gave 4.37 hours. Judges ordered implementation of all the recommendations of college-educated CASAs twice as often as they did recommendations from those without degrees.
• The average CASA received 43.9 hours of training, then spent an average of 3.2 hours a month working on cases. Caliber questioned the validity of the 3.2-hour finding, because it also found that one-third of the volunteers recorded spending no time with the children, which the researchers believe was unlikely.
• CASAs spent more time each month (about 45 minutes more) with children who had prior placements outside the home. Volunteers spent less time on cases each month those cases were open.
• Judges overwhelmingly take CASA advice. In 61.2 percent of cases, every one of a CASA’s recommendations were ordered by the court. Judges were four times as likely to accept all of the recommendations offered by the small number of male volunteers as from the female volunteers, and 2.5 times as likely to order all of the actions advised by African-American volunteers.
• Children assigned CASAs were more likely to be assessed by caseworkers as having experienced a severe level of harm and to be at severe risk for harm. They were significantly more likely to have been mistreated previously and to have received child welfare services in the past.
The Caliber researchers believe their attempt to level the playing field in comparing outcomes for children with a CASA to those without was not entirely successful. They matched children using nine factors, including previous out-of-home placements, abuse or neglect, child welfare involvement, and risk factors reported by the caseworkers. Despite that, they believe they missed a factor, so that the CASA cases were still more serious and difficult than non-CASA cases. So while there is a relationship between having a CASA and the following findings, the researchers do not believe the CASA caused the findings:
• Children with CASAs and their parents received significantly more services than those without CASAs, but there was no difference between the CASA and non-CASA groups in percentage of parents’ or children’s needs that were met.
• For children whose cases were closed by the end of the study, those with or without CASAs were no more likely to experience additional maltreatment. The two groups stayed in the child welfare system about the same amount of time. Children who had CASAs were more likely to have been removed from their parents.
• Among children whose cases remained open, there also was no difference in new reports of maltreatment. But all of those with a CASA were removed from their parents, while only 45 percent of those without a CASA were removed.
• Among the children who were removed and whose cases remained open, those with a CASA were more likely to remain in foster care and less likely to be assigned to live with kin.
• Among children who had been removed from their parents and whose cases had closed, there was no difference in placement with kin or reunification with parents.
• There was only one difference in the 16 measures of well-being for children who had a CASA and those who did not: Adolescents without a CASA reported slightly greater support in their relationships with adults.
Resources
Lisa Lunghofer, Managing Associate
Caliber Associates
10530 Rosehaven St., Suite 400
Fairfax, VA 22030-2840
(703) 385-3200
www.calib.com/home, Llunghofer@caliber.com
Michael Piraino, Chief Executive Officer
National CASA Association
100 W. Harrison St.
North Tower, Suite 500
Seattle WA 98119
(800) 628-3233
www.casanet.org, Michael@nationalcasa.org
Richard Wexler, Executive Director
National Coalition for Child Protection Reform
53 Skyhill Rd., Suite 202
Alexandria, VA 22314
(703) 212-2006
www.nccpr.org, info@nccpr.org
