argues that “Guns don’t kill people, people do,” Joshua New the use of predictive analytics in child welfare by telling us, in effect, that computers don’t remove children, caseworkers do.
But there’s a corollary: Just as we should be doing more to keep big, powerful guns out of the hands of people who don’t know how to use them, the human beings who run child welfare systems can’t be trusted with the nuclear weapon of big data.
This is illustrated by the way New himself handles data. In the second sentence of his column, the champion of big data misunderstands the first statistic he cites. He writes: “Consider that in 2014, 702,000 children were abused or neglected in the United States …”
But that’s not true. Rather, represents the number of children involved in cases where a caseworker, typically acting on her or his own authority, decided there is slightly more evidence than not that maltreatment took place and checked a box on a form to that effect.
For purposes of this particular statistic, there is no court hearing beforehand, no judge weighing all sides, no chance for the accused to defend themselves.
I am aware of only one study that attempted to second-guess these caseworker decisions. It was done as part of the federal government’s second “National Incidence Study” of child abuse and neglect. Those data show that caseworkers were two to six times more likely to wrongly substantiate maltreatment than to wrongly label a case “unfounded.”
I don’t blame the federal government for compiling the data. I don’t blame the computers that crunched the numbers. My problem is with how the human being – New – misinterpreted the numbers in a way favorable to his point of view.
I’m not saying he did it on purpose (after all, he’s only human); I use the example only to illustrate why, when he says that predictive analytics systems are merely “decision support systems,” that’s not reassuring.
Nor is it reassuring to find that while New tells us how the predictive analytics experiment in Los Angeles, called AURA, allegedly pinpointed a large proportion of cases that led to severe abuse (according to a study done by the same company that developed the software), he leaves out the fact that more than 95 percent of cases flagged by AURA .
New also accuses those of us who disagree with him not simply of opposing predictive analytics, but “sabotaging” it, a word that conjures up images of luddites from the Vast Family Preservation Conspiracy sneaking into offices to destroy computers. He offers no data to support his claim of sabotage.
Then, he concludes by dredging up the latest version of the classic canard: If you don’t support doing exactly what I want to do in exactly the way I want to do it, then you don’t care about child abuse! His version is to allege that those of us who disagree with him are “more fearful of data than they are concerned about the welfare of children.”
Human fallibility intrudes in other ways as well.
Human beings decide which “risk factors” the computers seek out. So, for example, in AURA, the computer looks at things like whether a child has been taken to an emergency room often. But impoverished parents rely more on E.Rs, whether they also happen to abuse their children or not.
Another alleged risk factor: changing schools a lot. But that happens to impoverished families who are homeless or get evicted because they can’t afford the rent – and to families of color whose children were victimized by the well-known racial bias in school discipline – whether they also happen to abuse their children or not.
Instead of compensating for human biases, AURA is likely to magnify them.
And let’s not kid ourselves. How many child protective services caseworkers will dare to leave a child in his or her own home — notwithstanding the 95 percent false positive rate – when it means that, in the event of a tragedy, that caseworker will be on the front page for ignoring the “sophisticated computerized risk assessment protocol” and “allowing” a child to die? New’s own rhetoric makes clear what such a caseworker will face.
Right now, AURA is going to be used to analyze reports received by the Los Angeles child protective services “hotline” and passed on for investigation. But why wait for reports? Once we have all that “big data” about where the “high risk” cases are, will CPS workers simply be empowered to barge in the door of every home that gives off an “aura” of child abuse? That federal on child abuse fatalities may recommend something similar.
So no, we should not give child protective services agencies the power to data-nuke our most impoverished families. Even if the data are up to the challenge, the human beings are not.