When Michael Young attended the first Abstinence Education Evaluation Conference in 2005, it “was almost like a pep rally for abstinence,” he says. But the 2007 conference in March was more subdued and focused on scientific evaluations, which he and other abstinence supporters see as a sign of progress.
There is plenty of reason for the mood change.
While the general message at the conference in Baltimore was that good evaluation makes for good program quality, there’s a growing sense that abstinence proponents must address concerns about the scientific validity of their core philosophy: that abstinence-only education delays the initiation of sexual activity in adolescents.
Then last month, the U.S. Department of Health and Human Services (HHS) released its long-awaited scientific evaluation of federally funded abstinence-only programs – which found that the programs showed “no overall impact on teen sexual activity.” (See story, page 1.)
The conference, hosted jointly by several offices within the HHS, featured two days of discussion and training to help abstinence-only programs strengthen their evaluation designs in hopes of fostering the evidence to support the programs.
Some evaluations were highlighted at the conference, but they were hardly the stuff of scientific import, with such finding as: “intervention group reported increased agreement with the abstinence message,” and “a statistically significant change … generally supportive of an abstinent lifestyle” and changes “in the direction of more agreement with positive and healthy attitudes.”
Slow to Evaluate
Abstinence-only programs receive federal funding through at least three main sources: HHS’ Adolescent Family Life program (AFLA), the 1996 Welfare Reform Act (“Title V” block grants) and Community-Based Abstinence Education (CBAE) grants.
Grantees under all of these programs must comply with eight requirements set forth in Title V, including these teachings: that sexual activity outside marriage is likely to have harmful psychological and physical effects, that that abstinence from sexual activity outside marriage is expected among all school-aged children, how to reject sexual advances, and how alcohol and drug use increases vulnerability to sexual advances.
Even with the recent federal study, there remains a dearth of scientific evidence that the abstinence programs have a long-term positive impact on the sexual behavior of adolescents.
 |
|
Brown: People are asking, “Were these programs any good to start with?” |
“The abstinence community was kind of slow to organize itself to evaluate its programs,” said Sarah Brown, executive director of the National Campaign to Prevent Unwanted and Teen Pregnancy.
Among the reasons:
Ideology: Some experts say the mostly faith-based programs are guided more by ideology than by science, fueling unrealistic expectations about youths remaining abstinent until adulthood, even though statistics show that half of all youth in every industrialized country have sex by age 17. So during data gathering, the observers say, the programs don’t even ask youth about their sexual behavior.
“It behooves them not to ask the question” about when youths first had sex, said Debra Hauser, vice president of Advocates for Youth, a national nonprofit that promotes comprehensive sex education. “But it is definitely the question we all want answered.”
Young, a presenter at the conference and co-author of the abstinence-based curriculum “Sex Can Wait,” wrote in the American Journal of Health Studies in 2004 that the federal abstinence requirements impose unrealistic standards based on religious value judgments. The way the requirements “are carved out,” he said in an interview, “seems to be more important to at least some of the people than whether the program actually works or not.”
Time/Money: Programs were never given the time or the money to do evaluations right – with comparison or control groups, enough participants and follow-up testing, say reproductive health researchers.
“Funding and time constraints were a huge problem in the past, making it very difficult to do good work,” said Doug Kirby, co-author of several sex education publications and curricula and a senior research scientist at ETR, a nonprofit publisher of health promotion and education resources.
When no one is paying for evaluation and time is short, programs “end up doing what they can do,” Brown said. “They count the number of people. They have people fill out questionnaires. The real question is, ‘Did these programs succeed in raising the initiation age of first intercourse?’ ”
Design: Some experts say that because practitioners are usually not evaluators, they know little about how to build rigorous scientific evaluation into their programs. This can create several problems, including decisions to measure easily gathered outcomes – such as changes in knowledge and attitudes – rather than actual sexual behavior.
“Basically, the quality’s been very poor, so you cannot really learn much from them,” Kirby said of the evaluations. “Which doesn’t mean the programs don’t work. What it really means is we cannot tell if they work or do not work.”
The pre- and post-tests presented by abstinence-only groups at the conference typically looked like the evaluation instrument approved by the U.S. Office of Management and Budget for use through 2008 in the evaluation of Adolescent Family Life programs. It asks youth for their general beliefs about premarital sex, but asks nothing about sexual behavior.
Motivated to Evaluate
The federal government and federally funded abstinence-only programs have plenty of reason to begin focusing on getting their science right.
Hauser cited pressure from debates in Congress over the funding of abstinence-only programs. Also, more states are rejecting federal matching funds from the Title V State Abstinence Education Grant program. (See story, page 14.)
Numerous articles and studies published by scientists affiliated with major research institutions, such as the Institute of Medicine, and national public health-focused organizations have concluded that no evidence has shown that abstinence-only programs delay the initiation of sexual activity in adolescents. An October 2006 study by the U.S. Government Accountability Office concluded that “most of the efforts to evaluate the effectiveness of abstinence-until-marriage education programs … have not met certain minimum scientific criteria … that experts have concluded are necessary in order for assessments of program effectiveness to be scientifically valid.”
“The abstinence issue has been very contentious,” said Brown, a member of the technical work group for the recent federal study. She said people are wondering, “Were these programs any good to start with? Did they achieve anything? Could we have used that money in a better way?”
Moving Forward
The federal government has recently responded to pressure for more rigorous scientific evaluation. As of fiscal 2006, the project period for CBAE grants was extended from three years to five years, in part to “ensure that programs can adequately evaluate the effectiveness of their intervention,” according to CBAE grant application instructions from the U.S. Administration on Children, Youth and Families.
Beginning with fiscal 2007, CBAE grantees must certify that “all medical materials proposed in this application and funded during the project period of this grant are medically accurate.” They are also “expected” to measure two outcomes for six months after completion of the program: the number of youth who have never had sexual intercourse and remain abstinent following the abstinence education program, and the number of youth who had sexual intercourse in the past but discontinue having sex after the abstinence education program.
Young hopes last month’s conference got project directors thinking about the importance of evaluation and helped them network with professional evaluators. “If they really want to see if the program impacts adolescent sexual behavior, then that’s one thing they ought to be able to determine,” he said. “But if they’re not interested, well, it’s not going to happen, because doing evaluation right is a lot of trouble.”
