HARY/SHUTTERSTOCK
Child welfare systems are on the brink of a paradigm shift when it comes to technology use in their difficult work. A variety of federal legislation efforts are enabling states to receive long-needed financial support to modernize their data systems.
New initiatives such as the Comprehensive Child Welfare Information System (CCWIS) hold great potential to help ensure the utilization of high-quality data for evidence-based treatments (EBT) and fully integrating trauma-informed care into strategies. These efforts, when pursued correctly, can create a new landslide of information to enhance diagnosis, conceptualization of intervention strategies, service utilization and implementation of care for youth and families in distress.
To help states achieve such goals, the experts in technology, child welfare and research behind the VitalChild solution (I am director of child welfare there) have devoted years to the development of a transformational case management platform. VitalChild connects stakeholders throughout the child welfare systems of care, allowing all to take advantage of the opportunities provided by the latest in technological advancements.
Given that many in the field may feel the demands of this avalanche of accountability information are overwhelming, I would like to share how new technology can strategically improve support, safety and care efforts. States’ child welfare systems need more than just snazzy technology that provides an electronic format to manage existing outdated processes and protocols. I’ll explain why this is the case.
In addition to CCWIS, the development and launch of the Family First Prevention Services Act (FFPSA) provides a substantial opportunity for child welfare systems and the families they serve. Family First provides proactive access to evidence-based interventions that may indeed provide opportunities for children and families to remain intact, and avoid more complicated, protracted struggles with mental health and behavioral challenges.
Because rigorous data collection is such an integral part of FFPSA guidance, states are taking this opportunity to combine these efforts into their CCWIS modernization. However, a fragmented, compartmentalized approach to this modernization may create scenarios in which the integration of smaller companies’ modules is slow to reveal important data, as these smaller systems are not necessarily designed to “speak” to each other.
Data analysis needed too
As states move toward their modernization, the agility of technology systems has become paramount. Current events have made this even more difficult. In many ways, the COVID-19 pandemic brought the child welfare system to its knees. With the inability to do in-person visits, clinical service availability falling off a cliff and reports to hotlines plummeting, our most vulnerable youth and families have quickly become a secondary crisis. The multidimensional complexity of delivering care to troubled youth and families is not as simple as plugging in a technology solution with new bells and whistles.
States must confront the reality of the changing landscape of service delivery in child welfare. New technology will streamline many functions and increase efficiency. But the immense data must be made sense of. This is particularly true given that “standards of care” are being challenged as “teletherapy” and there are in-person restrictions, school closures (or modified schedules), unemployment, etc. that may be with us for the foreseeable future. As a result, states’ child welfare modernization must work shoulder-to-shoulder with experts from the field to guide modifications into practice.
We have all heard the statement: “Working with troubled youth and families is not ‘rocket science.’ It’s harder.” But administrators, clinicians and case workers can’t know everything. Wide-ranging resources are devoted to addressing these challenges, yet tens of thousands of youth enter the child welfare system each year. More and more youth qualify for psychiatric disorders, but too often our outcomes fall short of how we would define success.
Evidence-Based Treatment Clearinghouses
Support for the use of evidence-based treatments offers many opportunities to bolster outcomes. With access to funding, training and utilization of new models of care can renew enthusiasm for achieving even better outcomes and enhancing the quality of life of those we serve.
| Title IV-E Prevention Services Clearinghouse Endorsed Programs | |||
| Brief Strategic Family Therapy | Well Supported | Child-Parent Psychotherapy | Promising |
| Families Facing the Future | Supported | Functional Family Therapy | Well Supported |
| Healthy Families America | Well Supported | Homebuilders – Intensive Family Prevention & Unification Services | Well Supported |
| Methadone Maintenance Therapy | Promising | Motivational Interviewing | Well Supported |
| MultiSystemic Therapy | Well Supported | Nurse-Family Partnership | Well Supported |
| Parent-Child Interaction Therapy | Well Supported | Parents as Teachers | Well Supported |
| SafeCare | Supported | Trauma-Focused Cognitive Behavioral Therapy | Promising |
As there are just over a dozen endorsed programs from the Administration for Children and Families (ACF) at this time, many states will be crafting their own FFPSA plans to be approved by ACF by selecting programs offered within evidence-based treatment clearinghouse resources. Through all these EBTs, outcome expectations must be realistically understood by considering the population of children and families your state/county/community is trying to serve. And all the intervention strategies must begin with accurate assessment of the problems facing the youth and family. In our current system, when you identify the challenges (diagnose), you then apply an EBT for those problems (treat) and expect positive outcomes (measure).
This process hinges on an accurate assessment at the beginning and the ability to monitor outcomes in an efficient way. Moreover, using these service strategies, in the midst of our COVID-19 pandemic, and in the aftermath, brings new challenges that must be addressed.
Fidelity to models of care: All the endorsed programs include in-person provision of services in most respects, if not completely. To deviate from these methods will definitely jeopardize the fidelity to treatment protocols. However, where it is necessary, increasing our efforts to rigorously monitor/measure services and outcomes may in fact advance our understanding of what works and what doesn’t. This will create a “sea change” in terms of understanding outcomes. To capture this data, however, reliable and valid tools are essential.
A Treatment Effect Example
To demonstrate the challenges of modifying EBT’s delivery of services, let’s consider fidelity to treatment and the effectiveness of a particular treatment. In this example, let’s consider the ACF-endorsed intervention, Trauma-Focused Cognitive Behavioral Therapy (TF-CBT).
In Study 10047, TF-CBT effectiveness is evaluated in a sample of over 150 children. This study examined TF-CBT in a community sample and compared it to Treatment as Usual (TAU). In fact, the TF-CBT demonstrated superiority over TAU.
Robert Foltz
But there are interesting considerations when applying this to your population of traumatized youth. Consistent with other TF-CBT studies, the intervention is time limited. In this study, there were 15 sessions in the protocol. The study was conducted in Norway, with the average age of about 15. None of the participating youth were on psychiatric medications (they do things differently in Norway). So, will your results reflect what they achieved in this study? Maybe.
In the U.S., a concerning number of children in the child welfare system are medicated, often on multiple medications. For a child with post-traumatic stress disorder, the American Academy of Child and Adolescent Psychiatry recommends an SSRI antidepressant, if medications are to be used. Despite an FDA “Black Box Warning” in 2004 for SSRIs (indicating that they can increase the likelihood of suicidal ideation and behaviors in youth), these antidepressants are commonly prescribed for children and adolescents.
The research that solidified their popularity for youth was the Treatment for Adolescent with Depression Study (TADS). In this study, researchers demonstrated that the SSRI, fluoxetine (Prozac), combined with cognitive behavioral therapy, outperformed all other treatment options. However, over time, these benefits dissipated so much that CBT alone matched the outcome at 36 weeks.
Further, an effort to replicate these findings, comparing Prozac + CBT to placebo + CBT, was unable to show any benefit of the added Prozac. Finally, companies seeking FDA approval for different antidepressants often compare themselves to Prozac and placebo to demonstrate effectiveness. In these studies, with hundreds of youth, Prozac has not shown superiority over placebo.
Whie the use of medications is within practice guidelines, all medications come with potential side effects. For SSRI antidepressants, irritability, depression, suicidal ideation, anxiety, among many others, can emerge as a result of medication exposure. Medications are designed to change the way we think, feel and/or behave, and these changes are not always positive or reliable.
So now the question becomes: How do you know if the clinical presentation is failing to respond to treatment or if it’s caused by the treatment? But I digress. Suffice it to say, multidisciplinary interventions come with many challenges that are difficult to decipher without sufficient data and without sufficient conceptualizations of this complicated care.
Regarding a trauma-exposed youth, here is an example of why fidelity matters. When you are examining the effectiveness of an intervention, you will look at a statistical measure called the “Cohen’s d” to establish “effect size.” This measure allows you to compare the averages between two groups, such as a treatment group and a control group.
A moderate effect size of 0.5 reveals that 69% of the control group fell below the average of the treatment group. A large effect size of 0.8 reveals that 79% of the control group are below the average of the treatment group. A larger effect size increases your confidence that the treatment may achieve positive outcomes. But these studies utilize tightly controlled conditions, such as the clinicians often receiving regular supervision, adhering to treatment manuals, using worksheets and delivering services in person. Deviating from these steps may negatively impact the effect size, thus, reducing one’s confidence in the effectiveness.
Based on the above examples of effect size, it’s important to acknowledge that in a moderate effect size, nearly a third (31%) of the control group was not below the average of the treatment group; in the large effect size, 21% of the control group was not below the average of the treatment group. But keep in mind, the effect size is achieved on specific measures that are used to monitor symptom presentations.
In the above TF-CBT study discussed, it achieved an effect size of 0.44 on “Mood and Feelings Questionnaire (end of intervention)”, but a small effect size (0.07) on “Child PTSD Symptom Scale: Functional Impairment.” That’s right, even the tool they use to measure a particular symptom can influence the outcome of the study.
Achieving the Same Outcomes with Your Youth, Families?
All studies don’t apply to all people. Observing dramatic improvements for children and families can create excitement and renewed enthusiasm around the difficult work in child welfare. Of particular importance, across EBTs, many — but not all — participants in the study achieve some benefit. As a result, if a child/family does not respond to the chosen EBT one should recognize that it may be a failure of the treatment rather than a failure of the child/family. However, those in charge of coordinating the services for children and families must also consider the external validity of research. That is, to what extent can the conclusions of a study (or EBT) generalize to your particular population.
Researchers often look to determine the effectiveness of an intervention on a specific disorder. If the child has more than one disorder, cognitive impairments or other health implications, it changes the “profile” of who is being studied. Then, once their outcomes are measured, those are either representative of a singularly diagnosed group (homogeneous) or complicated group with multiple conditions/characteristics (heterogeneous). Unfortunately, complicated clinical conditions are more reflective of children in child welfare, which threatens the ability to generalize many studies to your particular children or families.
Location matters too. Consider, for example, all the resources available in a large metropolitan area like Chicago. There are experts in any discipline, offering a seemingly limitless array of services. But in, say, Bozeman, Montana, the clinical challenges aren’t less, but the available resources are dramatically different. These factors will inherently impact the decisions you make, the treatments you offer and the outcomes you achieve. Data collection, analysis and formulation will be invaluable, as states modernize, to customize their conceptualizations toward optimizing their strategies. A technology solution developed for one state may require considerably different analytic strategies compared to another.
Introducing New Technology
As a result of the recent health concerns nationally (and internationally), child welfare systems have begun utilizing technology components, such as Zoom or Skype, to maintain contact with the children and families in their care. Aside from the concerns with security and privacy in these systems (such as being HIPAA compliant across utilization), questions remain around intervention strategies being delivered in this format.
Basically, can you expect to achieve the same outcomes in “teletherapy” as you can within in-person services? There is, indeed, a lack of robust research to answer this question. The vast majority of research is based on in-person delivery of services. Evidence-based treatment protocols are quite strict in the effort to guide fidelity to the intervention and replicate the outcomes. However, there are some investigations examining the use of “teletherapy” or psychotherapy provided online.
Recently, an article suggested that this online format is comparable to in-session CBT. Great! But upon review, just one measure within this study showed that texting (on average 10 minutes per week) achieved the similar results to in-person CBT therapy. However, trying to extrapolate these findings to the current utilization of technology in child welfare requires several leaps of faith. Participants were well-educated adults with health anxiety (not the complex trauma often seen in the child welfare system) in Sweden, with long-standing symptom presentations.
Moreover, these results were achieved in one brief measure, but not consistently found across the other outcome measures. Finally, drawbacks included lower therapeutic alliance in the text-based therapy. As it often feels as if we’re “drinking from a firehose” when it comes to the release of new research, the highlighted article here has an attractive title, but when scrutinized, has little applicability to the use of technology in child welfare.
When specifically looking at resources to guide the use of online Trauma-Focused Cognitive Behavioral Therapy (TF-CBT, which is an endorsed treatment in the Title IV-E Prevention Services clearinghouse), this author found no controlled research to evaluate it.
Technology advancements will definitely improve our ability to understand what works and what doesn’t. Capturing data from all stakeholders is essential. The voices of youth and parents/caregivers is too often neglected in our analysis of outcomes — and often reveal a much different picture than the outcome measures completed by clinicians. Until these perspectives are integrated, our understanding will be incomplete. And the stakes are too high.
We are on the threshold of a major evolution through the child welfare system. No doubt, the technology will be impressive, but my concern is how this technology will lead to improved outcomes for our vulnerable youth and families. It must enhance how we understand and address the needs of these families. It must have the capacity and agility to guide decision-making — identifying what is not working and how to enrich what is. It must provide oversight for the home and facility environments to ensure safety. It must be child-centered, family-focused and trauma-informed. And it must be built upon a thorough understanding of child development, trauma, psychological disorder and treatment (child and adult), educational and functional outcomes and research analytics.
A CCWIS technology platform must have the dexterity to be quickly modified when unforeseen events disrupt the delivery of care. Moreover, standardized, reliable and valid measures — integrated into the system — will enhance the continuity of services, as well as the monitoring of outcomes regardless of disruptions to the standards of care. Indeed, the “new normal” may be the incorporation of new service strategies that were innovated during the recent health crisis. In the long run, these evolutions in care will also influence what works, and what doesn’t, in the view of funding opportunities like FFPSA.
As states consider their adoption of technology solutions to meet their needs of CCWIS, FFPSA or simply improving their infrastructure to monitor services and outcomes, aligning themselves with the vision of the companies making them available is worthy of contemplation. There are many overlapping features across the systems being offered — many exciting advancements in technology that will make this difficult work more efficient. The less obvious differences are often in the teams and their commitments to assisting states in substantially improving the lives of our most vulnerable children and families.
Resources, time and considerable finances are being devoted to this process of modernization. In their shadow, children and families are waiting for happier childhoods, less distress and more fulfilling futures. Just plugging in new technology is necessary, but not sufficient. This convergence of funding, legislation and need offers exceptional opportunity to make a measurable difference. Now is the time.
Robert Foltz, Psy.D., is associate professor of clinical psychology at the Chicago School of Professional Psychology, focusing on the diagnostics and multidisciplinary treatment of children & adolescents. He is also director of child welfare at Multi-Dimensional Education Inc., offering the VitalChild solution for CCWIS modernization and Family First data collection and analytics.
