If the Orwellian predictive analytics model in Allegheny County is really so great, why does the county’s human services director have to obfuscate about what it is and how it works?
Read between the lines in some of the comments of those supporting Allegheny County’s Orwellian use of “predictive analytics” in child welfare – now including a plan to try to label every baby with a “risk score” at birth -- and you find something that boils down to this: Well, yes, it’s potentially dangerous, but we trust Marc Cherna. Cherna is the longtime director of the Allegheny County Department of Human Services.
The sentiment is understandable. Cherna has held the job since 1997. I served on a search committee that unanimously recommended him. For a long time I was proud of that. Cherna built an impressive record as a reformer, curbing the number of children in foster care and becoming a leader in the use of kinship foster care, among other accomplishments. That built him a reservoir of goodwill.
But there are two principal problems with the “trust Marc” argument. First, Marc won’t be there forever. And second, the way he is selling the program undermines trust.
It’s not the embrace of predictive analytics that raises questions about trust. There are other good, honorable child welfare system leaders who support it (though it appears that most of the leading proponents also are the most extreme supporters of a take-the-child-and-run approach to child welfare). The trust questions arise because of how the county has sought to sell the first predictive analytics program, the Allegheny Family Screening Tool (AFST), which is used to help decide when to investigate reports alleging child neglect, and the even worse “Hello Baby” program – the one that seeks to slap that risk score on every newborn in the county.
The problems with the sales job are apparent in an article Cherna wrote for the Department of Health and Human Services publication Children’s Bureau Express.
Compare the rhetoric to reality
Cherna writes that the program is “supported by the use of integrated data and a predictive risk model…” That’s it; there’s no further explanation. In fact, the “predictive risk model” is the lynchpin of Hello Baby – the whole thing collapses without it.
Under that model, vast amounts of data are collected, most of it from systems that enmesh primarily poor people. Then the system coughs up a “risk score” for the newborn based on how their parents have interacted with those systems. Cherna promises that the risk score will be used only to target prevention programs. Those with the highest risk scores will get the most intensive “help.” But there is nothing to stop Cherna, or his successor, from using the data in any other way they choose. We just have to trust them.
Cherna writes that “Hello Baby is a program of DHS's Office of Community Services and is not connected with child protective services.” He neglects two key points:
● The helpers – who are being sent into a home precisely because an algorithm says the baby is at high risk of child abuse -- also are mandated reporters of child abuse.
● The person in charge of both the Office of Community Services and child protective services is – Marc Cherna. He, or a successor or a successor’s successor, are free to tear down that wall of separation.
Cherna writes that the program is “voluntary.” Technically, yes. But when most people think of voluntary, they think of something that is offered to them and they then choose whether to take part. Hello Baby doesn’t work that way. Hello Baby assumes that you have agreed to surrender your data and have your baby labeled with a risk score at birth unless you affirmatively opt-out. And it is unclear how much chance you have to do that.
So ask yourself: If Hello Baby is so great, why do they have to, in effect, sneak it past the very people its proponents say are most likely to benefit, instead of being open and aboveboard about exactly what it is, and letting people opt in if they really want it? If it’s as wonderful as Cherna says, people should be lining up to take part.
Cherna goes on to tout the “ethical analyses” done of Hello Baby. But he doesn’t mention how the county has stacked the deck for such analyses from the start.
The first “ethics review” done for the AFST program, was a once-over-lightly review of a few papers, all of which were written by one of the designers of AFST or by one of the “reviewers” himself. That reviewer is a faculty colleague of one of the AFST designers – in fact, they’ve co-authored papers together.
Even that review said AFST was ethical, in part, because it was triggered only if someone called the child abuse hotline and alleged maltreatment. They wrote:
[The issue of informed consent] is one of a number of points at which we think that it is ethically significant that the AFST will provide risk assessment in response to a call to the call center, rather than at the birth of every child. [Emphasis added.]
So, now that Cherna is rolling out a use of analytics that does just that, what to do? First, call it “voluntary” by, as noted above, stretching the meaning of that word almost to the breaking point. Then, commission more ethics reviews! But make sure they’re from people strongly predisposed to support Hello Baby or apparently unfamiliar with child welfare.
All of which leaves one question: Should we be comfortable giving so much data and so much power to use that data to an agency that sells the program this way?