Monday, July 1, 2019

With no evidence whatsoever that its multi-million dollar program works, a giant CASA program sinks to fearmongering

● A careful, in-depth comparison of outcomes for children who have CASAs in Florida to children who don’t shows no benefit, and possibly some harm – based on criteria for measuring “influence” set by the program itself.

● Instead of facing facts, the program sends out fear-and-smear emails that ratchet up a climate of foster-care panic in the state.

From Prof. Robert Latham's analysis of the Florida GAL (CASA) program: The map
shows the variation from region to region in the proportion of children who have a GAL.

First of two parts. Read Part Two here.

To read the full analysis discussed below, click here.

I have previously discussed how the largest, most-comprehensive study ever done of that most sacred cow in child welfare, Court-Appointed Special Advocates – a study commissioned by the National CASA Association itself with the specific intent of proving to the world how great CASA is -- found that the program actually does harm.  I’ve noted a second, much smaller study done by a CASA who has desperate to show success.  She couldn’t.  And I’ve noted case after case of racism – and sometimes worse, in individual CASA programs.

Well, now there’s a new analysis of a mass of data from what is probably the Cadillac of CASA programs – a huge program with a large support staff and exceptional lobbying clout. A program that has spent $600 million in taxpayer dollars over the past 15 years and now has a budget of nearly $55 million per year.  A program that’s been called the state’s “second DCF [Department of Children and Families].”  The analysis is based on the program’s own criteria for measuring success.

Here’s the (sort of) good news: Using those criteria, the study found no indication that the program is doing a lot of actual harm; though it may do some harm.

Here’s the bad news: The study found no evidence that this CASA program does any good for children at all – even as it burned through $600 million in taxpayer funds.  And that’s the best case scenario.

The analysis was done by Prof. Robert Latham. Latham is no CASA basher. He’s currently a clinical instructor and associate director of the University of Miami School of Law Children and Youth Law Clinic.  Before taking that job he was a senior program attorney in the state CASA program (which, in Florida is called a GAL program, but it’s the same thing). In 2016, he won the GAL program’s Excellence in Advocacy award. 

Latham emphasizes that correlation is not causation, and he is not claiming causation.  But the Florida GAL program does claim causation using these same criteria. So, in effect, Latham is calling their bluff.

The backstory

The backstory behind this research is as interesting as the findings.  Not only had Prof. Latham worked in the program and received an award from it, in addition, he writes: “I love the people there.  I got started blogging in child welfare through defending the program against unfair criticism…”

But he has become increasingly alarmed by the fear-mongering from the program as it flails about trying to fend off a better approach to representing children.  Or, as Latham writes:

I thought we were past the days when … program leadership and their friends accused children’s attorneys of wanting kids to die.  And yet here we are.

That fear-mongering is especially damaging in Florida, where it echoes the demagoguery of the Miami Herald and the Tampa Bay Times and throws a little more gasoline on the fires of foster-care panic in that state.

Obviously, the program didn’t literally say children’s attorneys want children to die – and Latham doesn’t claim they did. He reprints exactly what the program said, and I discuss all that in part two of this post.  But for now, know that the Florida GAL program’s descent into fear and smear made Latham wonder: Is there really any evidence that this program works?

Testing the program using its own criteria

The Florida GAL (again, that means CASA) program has a bizarre way of “proving” its effectiveness.  They post a list of five child welfare outcomes under the heading “GAL Influence on Child Welfare Outcomes.” At no point to they provide evidence that the GAL program actually influenced the outcomes. (In one of the fear-mongering emails, they take a stab at this by saying that there are other states where the outcomes are even worse, something I’ll discuss in detail tomorrow.) 

In any event, as Latham notes, “the GAL Program holds these numbers out as evidence of its impact.”

So Latham decided to try to find out if there really is any such evidence.  He’s particularly well-suited to the task.  He has a Bachelor of Science degree in computer science.  He knows his way around numbers and the law.

Screenshot of one of the maps created by Prof. Latham
to track the placement history of thousands of foster children.
He knows how to analyze data and how to visualize it. Take a look at the amazing database he created that allows anyone to trace the placement history of more than 17,000 Florida foster children.  Suddenly the abstract concept of children moving from home to home to home to home becomes heartbreakingly visible.

So when he began to question how the GAL program measures it’s alleged effectiveness, he knew exactly how to find the answers.  Those answers can be found in this post to his blog.  It runs 9,300 words (and you thought my posts were long). At times, Latham goes pretty deep into the weeds.  But it’s well worth the trip.

The post is long because it covers not only his own research but the controversy that led him to   It’s long because he goes into great detail concerning methodology.  He does this in the interests of full transparency and to allow anyone who wants an intellectually honest debate to join in.  And it’s worth reading in full so you can assess his findings unfiltered by anyone’s interpretation, including mine.
undertake it and some arcane, but important, matters of legal ethics.

But – spoiler alert – here’s his final paragraph:

So, after all this, the question of whether the GAL Program is ethical, effective, or even good for children is all the same: we don’t know. And before we spend another $600 million, we should probably find out.

Prof. Latham’s method

Florida is a big state.  The court system is divided into 20 circuits.  Though Florida law says every child should have a GAL (again, in Florida a GAL is the name for a CASA) that doesn’t happen. There is wide variation among the circuits in the proportion of children who have a GAL.  So if the GAL program is right about how wonderful it is, then, by the logic the program itself uses to measure success, circuits in which a higher proportion of children have GALs should have better average outcomes that circuits where the proportion is lower.

Of course in any natural experiment there are what researchers call “confounding variables.” One reason Latham’s post is so long is that he goes into great detail explaining how he tried to account for the variables.

As Latham writes:

The plan is to look for significant correlations between GAL appointment rates and the “influence” measures [cited by the GAL Program itself]. This would normally be the time to say that correlation does not prove causation, and in most circumstances we would need to be very careful with that. But here, the causation has already been declared [that is, the GAL program already is claiming causation] and we’re working backwards to test whether it is justified given the existing data. The main question is: Does the public data support a claim of causation, correlation, or “influence”?

Prof. Latham’s results

Latham began with the particular outcomes that the GAL program says it influences for the better, including time to “permanency,” in all forms, time to adoption (but not, notably, time to reunification – that, apparently is not an important goal to the program) number of placements, and a crucial safety measure “foster-care recidivism” – the proportion of children sent home from foster care who are placed again within 12 months.

The result: the GALs had no demonstrable effect on any of the variables except one. Foster-care recidivism actually was worse when a GAL was on the case.  Did the GALs cause this? We don’t know. But again, it’s the GAL program that points to this measure and claims responsibility for “influencing” it.

Latham went on to look at a whole series of outcomes that the Florida Department of Children and Families uses to measure progress – or the lack of it.  Writes Latham:

What struck me most is the long list of circuit measures that had no correlation with GAL appointment rates. … I thought for sure that circuit-level GAL appointment rates would correlate with [improved] medical and dental care rates, or with siblings being placed together, or with minimizing the number of moves. They didn’t. …
What’s worse, if you want to measure “influence” instead of correlation, a higher GAL Appointment rate in a circuit was more strongly correlated with worse results in five measures. That was shocking to me: GALs correspond to worse outcomes. …
There were no significant direct correlations between GAL representation rates and any of the DCF Dashboard measures that survived controlling for circuit size. …
Furthermore, if non-significant “influence” is how we’re measuring and we’re assuming causation (two things you SHOULD NOT DO in a normal situation), then the GAL Program is failing and possibly even bad for kids on four additional measures that appear to be core to its mission. [Emphasis in original.]

There’s one more outcome that suggests cause for concern:

GAL representation rates do not seem to have any significant correlation to the percent of kids adopted or reunified in a circuit. To the extent they’re suggestive, GALs may depress reunification and accelerate adoption. [Emphasis added.]

And there was still another alarming finding in Prof. Latham’s research: One of the biggest problems in foster care is racial disparity – the overrepresentation of children of color in foster care.  Latham found that in circuits where more children have GALs, that racial disparity is greater.

Latham writes:

Racial disparity is the sort of thing GALs have been accused of having very unfortunate effects on, and a fact that National CASA takes very seriously. Seeing a correlation here is not good.

On one point here I disagree with Latham: I’ve seen no evidence that National CASA takes the issue of racial bias seriously – except as a p.r. problem.  For example: I am aware of no action taken against the Snohomish County, Washington program – the one that one had this guy as a volunteer.

But guess what MIGHT improve outcomes

Latham went on to look at factors that might influence these outcomes, factors that have nothing to do with the GAL program, and nothing to do with child protective services – at least not as it operates now.  He found – surprise! – that on multiple measures outcomes were better in communities with things like less poverty and better mental and physical health care.

Latham writes:

If it turns out that the number of community doctors and therapists who take Medicaid has a stronger and broader impact on foster care outcomes than GALs do, then we could have spent the $600 million GAL Program budget in very different ways.

In Part Two, available here: Desperate to preserve the program in the face of a push for a better alternative, the Florida GAL program starts fighting dirty.