Los Angeles County quietly drops its first child welfare predictive analytics experiment

● Apparently, a 95 percent false positive rate was considered a tad high

● Head of county’s Office of Child Protection urges slow, cautious approach to any use of predictive analytics

In Los Angeles County, they called it Project AURA (Approach to Understanding Risk Assessment).

It was among the most highly-touted experiments in the burgeoning fad for using predictive analytics in child welfare – that dystopian sci-fi nightmare-come-true in which computer algorithms predict who will abuse a child (but, we are assured, child protective services agencies would never ever actually use that information to tear apart families).

Project AURA was the subject of gushy news stories, and even gushier stories promoting the gushy news stories.  It was an experiment particularly beloved by those who are most extreme in their desire to see more children taken from their parents.

And now, thankfully, it is reportedly dead.

Buried on page 10 of a report to the Los Angeles County Board of supervisors by Michael Nash, executive director of the county’s Office of Child Protection, is word that the county Department of Children and Family Services (DCFS) “is no longer pursuing Project AURA.”

AURA stood for Approach to Understanding Risk Assessment. It was developed by software firm SAS.  Exactly what’s in it is a secret. No one outside SAS knows exactly how the algorithm works.

AURA was never used on any actual cases. Rather it was tested on past reports alleging child abuse or neglect. Then SAS looked to see what actually happened to those families.

As Nash’s report revealing the death of Project AURA explains:

While the tool correctly detected a high number of children (171 cases) at the highest risk forabuse, it also incorrectly identified an extremely high number (3,829 cases) of falsepositives (i.e., children who received high risk scores who were not at risk for a negative outcome). [Emphasis added.]

In other words, AURA identified a staggering number of innocent families. Had AURA actually been in use, an astounding number of children would have been placed at risk of needlessly being torn from their homes and consigned to the chaos of foster care.

 What finally killed AURA?

The results of the AURA experiment – including the false positive rate -  have been known for nearly two years. But that didn’t stop the county from pushing ahead – and it didn’t stop the gushy news coverage. It’s not clear what finally prompted DCFS to pull the plug. 

Perhaps it’s because, as Nash points out, all those false positives would further overload the system. More likely, it was an initiative by the State of California to try to come up with a “better” predictive analytics model.

Unlike AURA, developers of the new model are promising a completely open process, including consultation with various “stakeholders” and transparency about exactly what risk factors are used and how they are weighed - allowing anyone to “interrogate the algorithm.”

Also encouraging, Nash’s report, commissioned by the Supervisors themselves, is filled with warnings about the need to proceed “cautiously and responsibly.” He says a set of strict standards “to address the important operational legal and ethical considerations…” should be adopted “before considering the use of predictive-analytics models.”  Those standards should include “understanding how racism and other biases may be embedded in systemic data and addressing these within the model.”

Nash even noted that the independent journalism nonprofit ProPublica found exactly that bias in predictive analytics tools already in use in criminal justice.

All this means that, if nothing else, the nightmare of “Minority Report”- style policing in Los Angeles child welfare is at least another year or two away.

The bad news is that Nash’s report accepts the naïve view that once a good algorithm is created it can be properly controlled and limited. 

He writes:

Determining [predictive analytics’] “right” use – to identify families most in need of supports, rather than to trigger any negative consequences for them – will be fundamental.

But Nash, himself a former juvenile court judge, must know that’s now how child welfare works in the real world.

Whatever controls are in place at the outset will disappear the moment a child “known to the system” dies and the caseworker handling the case says “DCFS had all this information about the family, and they knew it was ‘high risk’ but they didn’t tell me.” 

Philip Browning

Then, all bets - and all restrictions - are off, and it will be take-the-child-and-run in every family where the computer spits out a high "risk score."

One more bit of bad news: One of the strongest boosters of predictive analytics in Los Angeles, former DCFS director Philip Browning, has been hired as a consultant to “help” New York City’s child welfare agency.

SDM is let off the hook

The other bad news concerns the other model of risk and safety assessment that the Supervisors asked Nash to study – the one currently used in Los Angeles - Structured Decision-Making.

Like predictive analytics, SDM also has been found to raise issues of racial and class bias. Nash acknowledges those issues in passing:

Users of the tool, in particular, fault it for not incorporating into its assessments theentire story of what is happening within a family, but instead focusing on a few broadstrokes without giving weight to important nuances. Users additionally state that the toolis too narrowly focused on the caregiver and does not take into account the strengths ofthe family as a whole.

But immediately he adds this parenthetical aside:

(The latest version of SDM has been revised to try to be more strength-based in its approach.)

But in my own experience, some version of  “Yes, but the new version is different” is what developers of SDM have said for more than a decade, each time similar concerns are raised.  That can only leave one wondering about all the “risk assessments” and “safety assessments” performed with old, unimproved versions of SDM.

The defeat of AURA shows that, contrary to what some predictive analytics proponents say in their worst moments of hubris, it is not inevitable that every legislative body and child welfare agency will embrace this latest fad in child welfare.

At a minimum, opponents in Los Angeles have more time to organize. And using predictive analytics in child welfare no longer has an AURA of inevitability.

New Column: You can’t fix child welfare spending with distorted data and doublethink

Listen closely. That giant sucking sound you hear is the foster care-industrial complex grasping for every dollar it can swipe from every possible “funding stream.”

George Orwell gave us the concept of  doublethink.
Foster care advocates perfected it.

 In 1984, George Orwell defined “doublethink” as holding two contradictory beliefs in one’s mind simultaneously, and accepting them both.

In child welfare, for example, we have been told for decades that child welfare systems don’t take away children because their families are poor. ... But now we also are told in a column by too advocates of taking away more children, that every single federal program designed to ease poverty – including housing assistance, food stamps, even the Supplemental Security Income program for the aged, blind and disabled – is a foster care prevention program, and every dime from every one of them should be counted as child welfare spending.

In other words, great gobs of money are going to prevent something – removal of children from their parents because they are poor – that child welfare agencies say they don’t do anyway.

Orwell would have recognized the technique. 

Read the full column here.

Philadelphia RTC is the latest in a long line of rotten barrels

It wasn’t the repeated rapes that finally forced the state of Pennsylvania to shut down the  Wordsworth “residential treatment center” in Philadelphia.  It wasn’t the assaults by staff against children and children against each other.  It wasn’t the fact that over ten years, police were summoned to the place more than 800 times.

It wasn’t even the enormous cost to taxpayers - $119,000 per year per child for all this tender loving care – that prompted the state finally to act.

No, the Philadelphia Inquirer and Philadelphia Daily News report, a 17-year-old, David Hess, had to die first, during a struggle with staff. Authorities ruled the death a homicide.

Through all of this, year after year after year, the Philadelphia Department of Human Services kept warehousing children at Wordsworth – children as young as age 10.  Some were delinquent, others were said to have been abused or neglected.

It’s not as if nobody knew what was going on.  As the newspapers report:

“Interviews, court records, state inspection reports, and police records reveal a trail of injuries to children, from broken bones to assaults to the suffocation death of Hess. Along the way, lawyers, licensing inspectors, and others found conditions there appalling and sounded the alarm with little success.”

Why wouldn’t the City or the State do more? They didn’t dare.  In Philadelphia substitute “care,” in all its forms, is a sellers’ market. As Joan Erney, director of Community Behavioral Health, the agency that oversees publicly funded mental-health services for Philadelphia told the newspaper:

“Our approach to agencies generally is that we need them, and if there are opportunities to improve, we work with them. … We did rely on Wordsworth extensively. Places outside of Philadelphia don’t want to take our kids. They tell us our kids are too complicated. They tell us our kids are too hard. We have kids with some really difficult problems.”

In other words, they were begging for beds, and beggars can’t be choosers.

But that tells only part of the story. The real reason Philadelphia turned a blind eye to the horrors at Wordsworth is because of Philadelphia’s long, ugly history of embracing worst practice in child welfare.

● Among America’s ten largest cities and their surrounding counties, Philadelphia tears apart families at the second highest rate when rates of child poverty are factored in. (When you don’t factor in poverty, Philadelphia is #1.) The rate of removal in Philadelphia is 60 percent above the big-city average, more than triple the rate in New York City and more than quadruple the rate in Chicago.

Were Philadelphia taking children at the rate of New York or Chicago it would have plenty of room in good therapeutic foster homes for children who really needed them – and no need to warehouse children in places like Wordsworth.

● Philadelphia needs something else, too: The guts and imagination to embrace safe, proven alternatives to residential treatment.  One of the striking revelations in the Inquirer / Daily News story is the fact that the RTC at Wordsworth wasn’t some hundred-year-old orphanage that rebranded itself to stay in business and then deteriorated. This facility was brand new in 2006 – and apparently it was abusive almost from day one.

In other words, at a time when most of the rest of the country was trying to shut down institutions, city officials in Philadelphia and their state counterparts in Harrisburg thought it would be a great idea to send children to a brand new one.

● And no, the almost universal cry of those who institutionalize children and their apologists – the claim that the children are just too difficult to handle in families – is not true.  There is nothing a “residential treatment center” can do that can’t be done better (and at lower cost) through Wraparound programs.

As they name implies, such programs do whatever it takes – bringing the help a child needs into her or his own home or a foster home.  In this video, Wraparound pioneer Karl Dennis describes how it worked on the kind of case that usually lands a child in a place like Wordsworth.

Not only does Philadelphia overuse institutionalization; it institutionalizes children for whom the harm is greatest: younger children.  This is such horrific practice that in his original version of the proposed Family First Act, Sen. Orrin Hatch (R-Utah) proposed to simply eliminate all federal aid for any placement in any institution for any child under age 13.

That never passed, of course.  So all American taxpayers continue to subsidize places like Wordsworth.

● Worst of all, there’s no guarantee that the children are any better off now that Wordsworth is closed. Because the children were simply shipped to other institutions, often out-of-state – so it will be even harder to keep track of what happens to them.

Even when institutions don’t become hellholes, rife with physical and sexual abuse, a mountain of research shows that they are inherently bad for children, and there are better alternatives.  And there is nothing unusual about the kind of abuse that was rife at Wordsworth.   The Wordsworth story is repeated in America over and over, year after year. When the topic is institutionalization, we’re not talking rotten apples. We’re talking rotten barrels.

New columns on race and class bias in child welfare from the 19th Century to today

NCCPR has two new columns on racial bias in child welfare.  One deals with how the same newspaper can expose racial bias in policing while remaining blind to it in child welfare.
Read the column here.

Another deals with the the desperate lengths to which some will go to deny there's a problem.
Read the column here.

This column deals with another example of how the biases in child welfare are magnified by the latest fad in the field, "predictive analytics."
Read the column here.

None of this is new. In fact, American child welfare has its very roots not in benevolence but in bigotry. That's the topic of this column for The Daily Progress in Charlottesville, Va. It sets the record straight about Charles Loring Brace and his "orphan trains."
Read the column here.

A lesson for child welfare from the Hillary Clinton campaign: Don’t rely on predictive analytics

Hillary Clinton

I’ve written before about how one of the biggest losers in the 2016 elections was “predictive analytics.”  All those algorithms kept assuring us that Hillary Clinton was all but certain to win.  The media were suckered.

It turns out the media were not alone.  In her review of a new book, Shattered: Inside Hillary Clinton’s Doomed Campaign, Michiko Kakutani of The New York Times writes that the campaign itself made the same disastrous error:

As described in “Shattered,” Clinton’s campaign manager, Robby Mook — who centered the Clinton operation on data analytics (information about voters, given to him by number crunchers) as opposed to more old-fashioned methods of polling, knocking on doors and trying to persuade undecideds — made one strategic mistake after another, but was kept on by Clinton, despite her own misgivings.

Yet “predictive analytics” continues to be sold, literally and figuratively, to child welfare systems as a way to target which parents should have their children taken away.  In fact, as is discussed indetail here, predictive analytics magnifies the racial and class biases that are built into the child welfare.

It will work every bit as well in child welfare as it did in the Clinton campaign.

New columns on state-sanctioned ransom, child welfare's addiction to bad science, and a tragedy in Pennsylvania

In Youth Today, I’ve written about the payments some parents are forced to make to get their children back from foster care. The euphemism used by child welfare systems for such payments is “child support.” But when someone takes away a child and makes parents pay money to get the child back, the proper term is “ransom.”  

Read the column here.

In the Chronicle of Social Change, I have columns about how Arizona’s plan to secretly tape record parents and then run the tapes through a “Computer Voice Stress Analyzer” is another example of child welfare’s 163 year addiction to bad science ...

 Read the column here 

... and on the latest twist from the latest example of that addiction, "predictive analytics": a plan that specifically targets poor people - and penalizes those among them who reach out for help.

Read the column here

In the Morning Call in Allentown, Pa., I look at how child welfare's double standards are at the heart of the tragic death of a child, allegedly at the hands of the foster parents who adopted her.

Read the column here.

NYC’s new child welfare chief is looking for help in all the wrong places

● The consultants he’s bringing in have one thing in common: a fondness for computerized racial profiling.

● Mayor Bill de Blasio, who has crusaded against racial bias in law enforcement, now seems to find it acceptable it in child welfare.

The depressing script is being followed to the letter in New York City.

Newspapers discover that children “known to the system” sometimes die. Though there is no evidence that these tragedies are any more common than before, now the press is paying attention.  That turns the deaths into a “series” or a “spate” or a “rash.” Then the child welfare agency, in this case the Administration for Children’s Services (ACS), officially is christened “embattled” and/or “beleaguered.”

New York City Mayor Bill de Blasio

As is discussed in detail in a series of previous posts to this blog, a slew of mayor wanna-bes rushes in exploit the tragedy by announcing investigations and issuing reports. Gov. Andrew Cuomo does the same to gain leverage in his feud with Mayor Bill de Blasio. De Blasio fails to stand up for an agency that has, in fact, made remarkable progress, taking away far fewer children with no compromise of child safety.

That should come as no surprise. Back when he chaired the City Council’s Human Services Committee de Blasio proved adept at grabbing headlines for himself by exploiting an earlier tragedy, the death of Nixzmary Brown.

And of course there is the Ritual Sacrifice of the Agency Chief, in this case Gladys Carrion. (Officially she retired. If so, it was because the mayor wouldn’t stand behind her.)  Her replacement, David Hansell, has no experience specific to child welfare. So he’s seeking advice. 

There are several outstanding reformers in the field to whom he could have turned. But chances are he doesn’t know about them. Instead Hensell is looking for help in all the wrong places.

The people/organizations he’s bringing have done nothing to distinguish themselves in the field. And they have one thing in common: a fondness for computerized racial profiling, or to use the child welfare field’s preferred euphemism, “predictive analytics.”

Consultant #1

David Hansell

Of all the choices Hansell has made the most difficult to explain is his choice of Philip Browning, who

recently resigned as director of the Los Angeles County Department of Children and Family Services.

In the 40 years I’ve followed child welfare, when I’ve read stories in which people are asked to name systems that function relatively well, I’ve never heard anyone mention Los Angeles.

There’s a reason for that. The Los Angeles child welfare system is almost always embattled and/or beleaguered. It has the same sorts of high-profile horror stories as New York City, and they provoke the same sort of response: Foster-care panic.  But unlike New York City, L.A. tends not to recover from those panics. It just lurches from crisis to crisis.

Browning is beloved by those who embrace a take-the-child-and-run approach to child welfare. That’s because while Los Angeles has all the same problems keeping children safe as New York City, Los Angeles tears apart families at among the highest rates of America’s big cities. In fact the rate-of-removal in Los Angeles is well over double the rate of New York City – in fact, it’s more than 150 percent higher.

Or to put it another way, if New York City tore apart families at the rate Los Angeles tears apart families, instead of taking away 3,702 children in fiscal year 2016, the city would have had to take away more than 9,200 – a number that would be higher than all but four of the past 24 years.

Philip Browning

Browning did not make Los Angeles’ dreadful record of removals any worse. But he didn’t make it any better. The already high rate-of-removal in Los Angeles increased further at the beginning of his tenure, which started at the end of 2011, then returned to about where they were when he got there.  And the number of children trapped in foster care on any given day increased by more than 15 percent during his tenure.

And, by the way, Chicago does better than both New York and Los Angeles – and in Chicago independent court monitors have found that the emphasis on family preservation has improved child safety.

Los Angeles outperforms New York City in just one area: the percentage of children placed in kinship foster care – that is, with relatives instead of strangers. That improvement took place during Browning’s tenure.  Other than that, however, the only thing Philip Browning can teach New York City is what not to do.

There was one other distinguishing feature of Browning’s tenure in Los Angeles. He was a huge cheerleader for “predictive analytics” in which computer algorithms use various “risk factors” to tell caseworkers who is supposedly most likely to abuse a child.

As is outlined in detail in our publication Big Data is Watching You, predictive analytics has  proven itself permeated with the same kinds of racial and class biases that already plague child welfare. Yet in Los Angeles, Browning brought in a private for-profit software firm to experiment with predictive analytics using secret, proprietary software.

They didn’t use it on any actual cases. Rather, past cases were fed into the computer and then it predicted risk. The algorithm predicted many of the cases that, in fact, lead to deaths, near fatalities or “critical incidents.” There was just one problem: It predicted vastly more cases where there none of those things actually happened. In fact the rate of “false positives” was over 95 percent. If you predict that a vast number of cases will result in catastrophe, it’s no wonder you’ll often be right, even as you are wrong far more often.

This means that, were this kind of system actually implemented, vast numbers of innocent families would come under additional scrutiny and their children would suffer the enormous trauma of needless foster care placement – because no caseworker is going to risk being on the front page as the worker who defied the algorithm and left a child in an unsafe home. So all those children would face the high risk of abuse in foster care itself.

And in the real world, all the time and resources squandered pursuing these cases would be stolen from finding children in real danger – potentially undermining any alleged gains the  algorithm produced in finding such children.

Consultant #2

Consultant #2 is a private child welfare agency based in Florida known as Eckerd Kids. In Florida, everything after the initial removal of a child is handled by private “lead agencies.”  Eckerd, which had the contract for the St. Petersburg area was brought in to take over nearby metropolitan Tampa after, yes, a “series” or “spate” or “rash” of deaths of children “known to the system.” 

They implemented a predictive analytics algorithm called Rapid Safety Feedback. Eckerd then implied that this caused the deaths to stop – they brag about this on their website.  As the same time, they piously proclaim that they don’t really know if the two are related – and they really aren’t claiming any such thing.

In fact, the picture in Tampa is far murkier than Eckerd and proponents of predictive analytics claim.  Full details are in Big Data is Watching You.  (Scroll down or search for “What Really Happened in Tampa.”)

And while backers of a take-the-child-and-run approach across the country have been fawning over Eckerd’s methods, the agency seems to be having trouble keeping its own house in order. A foster child in the care of one of Eckerd’s subcontractors died late last year and the foster mother has been charged with first degree murder and aggravated child abuse. The foster mother worked as a marketing manager for another Eckerd subcontractor.

There is no indication that Eckerd uses Rapid Safety Feedback to screen foster parents. But if they did, this one probably would have gotten a low risk score. Why? Because as the Tampa Bay Times put it in an editorial that revealed a lot about bias in child welfare, analytics – and journalism – the accused

seemed in many ways an ideal foster mother. College-educated with a $70,000 income, she lived in a nice Riverview neighborhood …

Of course, every agency has such failures. But it appears that Eckerd is being sought out by child welfare systems across the county largely on the basis of hype about how it supposedly stops such tragedies with predictive analytics.

Consultant #3

Consultant #3 is Casey Family Programs. This is one of several separate but similar foundations all endowed through the fortune of UPS founder Jim Casey (the Annie E. Casey Foundation, which, long ago, funded NCCPR, is another). Casey Family Programs is run by William Bell who joined Casey after an undistinguished tenure running ACS.  But I think the reason Casey is being brought in is because of the recent work of its Executive Vice President of Systems Improvement, David Sanders.

Sanders also ran the Los Angeles child welfare agency – and he did a good job there. But more recently he’s been far less successful. Sanders chaired a wretched mess known as the Commission to Eliminate Child Abuse and Neglect Fatalities. 

The commission was chaotic, it was angry, it was dysfunctional, it was secretive and it made its decisions based on newspaper horror stories.  In other words, a commission tasked with studying the child protective services system devolved into a microcosm of that system.  Details are in a series of previous posts to this blog, in NCCPR’s report critiquing the commission’s work, and in a scathing dissenting report from one of the commissioners.

And what was the key recommendation from this commission? Take the racially biased, class biased approach of predictive analytics and make it even worse.  And what was the basis for this recommendation?  Eckerd’s supposed success in Tampa.

From all this, an ugly picture emerges. New York City appears poised to reverse decades of progress, albeit inconsistent progress, in safely reducing the number of children consigned to the chaos of foster care.

And in another classic example of liberals who forget everything they claim to believe in when someone whispers the words child abuse in their ears, we have Mayor de Blasio. He has campaigned against racial bias in policing, but apparently he’s ready to accept it in child welfare.

 Good reason to be scared

During a previous foster-care panic in New York a 14-year-old wrote an essay called “I am scared of ACS.” 

Today, New York City’s vulnerable children have good reason to be scared of ACS once again.