Our annual holiday reminder

 

Food or foster care? Places that say they never take children because of poverty get ready to take more children because of poverty

Pittsburgh, Pa. and the State of Georgia are prime examples

Georgia makes its child welfare priorities clear

Got time for a one-question pop quiz? 

With SNAP benefits cut off, poor families are under even more stress. Faced with a family going hungry and desperate to find food for the children, the stress causes a parent to “maltreat” the children. Do you: 

A) Put the family under investigation, interrogate everyone, rifle through cabinets, drawers, cupboards and refrigerators, stripsearch the children (often standard practice regardless of the investigation) and leave the entire family traumatized.

B) Do all of the above and tear the children from their parents, consigning them to the chaos of foster care, from which only one in five is likely to do well in later life: a system that leaves emotional scars so great it may make it more than four times more likely they’ll die by age 20, while putting them in places where there’s at least a one in four chance, and probably more, they will be abused.

C) Get the family some food.

The answer should be easy, of course. But in Allegheny County, Pa. (metropolitan Pittsburgh), they seem to prefer A or B.  Because this is how a story from the Pittsburgh Media Partnership Newsroom begins: 

Fewer families are stepping up to foster, just as experts warn a surge of children could soon flood the system if the state and federal budget standoffs drag on. 

Allegheny County is now preparing for an increase in referrals to the Office of Children, Youth, and Families. 

“Parental stress is a big predictor of child maltreatment,” said Julia Reuben, an administrator for Allegheny County’s Department of Human Services. “And so, when there’s not enough money to feed your children … I think we can surmise that that will lead to increased maltreatment.”

Well, it certainly will lead to increased maltreatment if more children are forced into foster care, which often is an act of maltreatment itself, and, as noted above, a place where independent studies keep finding high rates of abuse. 

It’s another indication of how much child welfare in Pittsburgh has deteriorated.  Once a model for reducing needless foster care, the county turned its back on its own success. Now it’s a leader in targeting families using an algorithm that its own co-designer described as “Big Brother,” an algorithm that reportedly was* investigated for bias by the Biden Justice Department.  

The county also is a leader in tearing apart families. The most recent comparative data for Pittsburgh are from 2022, but they show a rate of removal more than 50% higher than the national average, even when rates of child poverty are factored in. 

Pittsburgh does continue to be a leader in using the least harmful form of foster care, kinship foster care, but for how long? Kin are also likely to be poor and so also likely to face additional stress due to the cut-off of SNAP benefits. 

As it happens, Pennsylvania is in the midst of a state budget impasse as well. That threatens payments to foster parents.  So Allegheny County has been dipping into reserve funds to pay the foster parents.  Maybe they should try using more of those funds to get food to birth parents so their children aren’t “maltreated” due to all that extra stress. 

It’s not just Pittsburgh 

Notice how in Pittsburgh, they use a rhetorical fig leaf to cover the naked truth about taking children due to poverty.  Oh, no, it’s not the poverty, they say, it’s the maltreatment caused by the stress caused by the poverty. 

In Georgia, they’re not even pretending.   

Kate Blair, director of the Savannah Court-Appointed Special Advocates Program (which expanded into additional services and now calls itself “Brightside”), offered up exactly the sort of comments one would expect – if you know CASA. She told WTOC-TV: 

“For a lot of our parents working towards reunification, they need to have a safe place for their children to return to, as well as food in those cupboards,” Blair said. “And without SNAP benefits, they may not have both.” 

The organization is anticipating families needing to be separated longer, and even more children possibly entering the foster care system. 

When a child is removed, parents need to come up with a case plan for the court to determine that their home is safe. Those factors include having warm shelter and food. 

“We’re taking away one really important piece,” Blair said. “And in the end, who’s going to suffer the most are not the politicians, are not adults like myself. It’s the children.” 

It’s not that Blair wants this to happen, in fact, she was appealing to people to donate food. But there’s something else her agency could do: Those CASA volunteers could oppose delaying reunification because of poverty and demand that the Georgia family police agency (a more accurate term than child welfare agency) find a way to put food in all those cupboards. 

DFCS makes it worse 

Instead, that agency is making things even worse. 

The Imprint reports that 

In emails sent late Friday to child welfare agencies in Georgia, state officials announced that the federal government shutdown has required them to suspend new services that protect children from entering foster care and ensure family reunification — unless approved in writing by the state. 

Under the directive, without that level of approval, caseworkers cannot initiate contracted services for child and family assessments, aides for parents, wraparound care, measures to prevent foster care, including “unnecessary out of home placement,” and “early intervention” services. [Emphasis added.] 

The only exceptions are “court-ordered or emergency services.” 

Local offices of the state Division of Family and Children’s Services remain free to shovel children into foster care and even ultra-expensive ultra-useless institutions,  with no new restrictions. 

Of course, this makes explicit that Georgia is willfully violating federal laws. One law – almost universally ignored – requires states to make “reasonable efforts” to keep families together. Another requires states to make “active efforts” in cases involving Native American children. Neither law has an exception for government shutdowns. 

Yet now DFCS is explicitly cutting back on services to prevent “unnecessary out-of-home placement.” 

As for where Georgia might come up with the money to pay for all those services it’s cutting back, there’s actually an untapped gold mine.  

The most expensive form of “care” for foster youth is institutionalization. It can cost hundreds of dollars per day per child. It’s also the least effective – in fact, it’s not effective at all. As a U.S. Senate Committee then chaired by Georgia’s own Sen. John Ossoff recently reminded us, the entire model is a failure. 

And that’s before we get to the fact that institutions are the placements where children are most likely to be abused, as can be seen in scandals that have engulfed institutions in Arizona, Kentucky, Tennessee, Indiana, Utah, Iowa, Oklahoma, Rhode Island, Washington state , Arkansas, New York, Connecticut, Idaho -- and Georgia. 

Here’s what else is so significant about places that institutionalize children: They cost a fortune, often hundreds of dollars per day per child. 

Which brings us to the goldmine. 

Nationwide, two percent of foster children are institutionalized.  But in Georgia, it’s 12%. As of Sept. 30, 2024, Georgia was institutionalizing 1,331 children! And while Georgia institutionalizes children at what is probably one of the highest rates in the nation, it uses the least harmful form of foster care, kinship foster care, at one of the lowest rates, 22% - barely over half the national average. 

So what DFCS should be doing is reexamining the case of every institutionalized child. Figure out which could go straight home (yeah, I know, these are supposedly the most difficult children, but that’s often according to those who run the institutions), which can go to the homes of extended families as kinship foster care placements, and which really do need more intensive help – the kind that can be provided far more effectively and at far less cost through the very wraparound services DFCS is now cutting back. 

The children would be far better off, and, though the federal government would reap some of the savings, Georgia still would have millions in additional funds to provide the help needed to keep children out of the system in the first place. 

*-If there was such an investigation, as the Associated Press reports, it may still be going on, but given who’s in charge of the Justice Department now, that seems less likely.

 Image designed with deepai.org

Our annual holiday reminder

 

Predictive analytics: The Project 2025 of child welfare

“Yes, it’s Big Brother.  But we have to have our eyes open to the potential of this model.”

-- Rhema Vaithianathan, co-designer of child welfare “predictive analytics” algorithms, discussing the idea of assessing children’s risk of abuse – while they’re still in the womb.

The Lincoln Project is a group of disaffected Republicans appalled by what has happened to their party. They’re probably best known for their ads.  One that’s gotten a lot of attention is their vision of what would happen if the near-total ban on abortion envisaged by the Heritage Foundation’s Project 2025 becomes reality.  Among the proposals: a massive increase in “abortion surveillance.” 

In the Lincoln Project ad, a father and daughter are pulled over by a police officer and ultimately arrested, the father for trying to take the daughter across the state line for an abortion, the daughter for “evading motherhood.”  (And just to be clear, some states already are trying to do things like this.) 

Have a look at the ad and then I’ll explain what all this has to do with child welfare: 

Among the striking features of the ad: the creators’ understanding of how the data we routinely surrender every day can be turned into an Orwellian nightmare of omnipresent surveillance.  Fortunately, something like this would only be supported by certain elements on the far right.  Liberals would never stand for it, right? 

But that brings us to the Great Exception of the Left: the fact that some of my fellow liberals (though many fewer than in decades past) will discard everything they claim to believe about civil liberties as soon as somebody whispers the words “child abuse” in their ears.  It’s one reason there is so little due process in what should be called family policing. 

Consider the quote at the start of this post.  It’s from a Boston Globe story in 2015, the early, heady days of the movement to bring Orwellian hyper-surveillance, what amounts to computerized racial profiling, to child welfare in the form of “predictive analytics” algorithms.  

We were told algorithms would indicate who is most likely to abuse a child so the family police agency (a more accurate term than “child welfare” agency) could swoop right in!  Big, centrist media – news organizations such as The New York Times, which now are appalled by Project 2025 -- swooned over it in 2015 and beyond.  (Almost the lone exception: a prescient warning from Prof. Virginia Eubanks in her book, Automating Inequality, excerpted in Wired. Many more, on the left and right, are opposed now.) 

At the time, the focus was on models like the Allegheny Family Screening Tool (AFST), used by  screeners in metropolitan Pittsburgh when they receive reports alleging “neglect.” Without even a pretense of informed consent, AFST harvests vast amounts of data originally surrendered, voluntarily or unknowingly, for entirely different purposes.  Then it coughs up a “risk score” – an invisible “scarlet number” -- to determine the urgency of sending out an investigator.  The questionable means by which it was sold, the questionable claims about how it would work, and the many problems that have arisen are outlined in detail, with sources, in our publication Big Data is Watching You. 

In the years since, some reliably blue states and localities rushed to adopt algorithms. Illinois, Los Angeles and Oregon all later backed away, in the first two cases after spectacular failures.  But blue-leaning metropolitan Pittsburgh presses on, even after an independent evaluation found problems with racial bias and, the Associated Press reports, the U.S. Department of Justice is investigating whether AFST is biased against the disabled. 

But the designers of AFST are proposing ever more dangerous algorithms. Even in 2015 and before, they had bigger plans, plans that sound remarkably like something out of that Lincoln Project ad. 

The Pittsburgh algorithms stamp an invisible "scarlet number
"risk score" on children. Once there, it can never be erased,

That Boston Globe story included a reminder that the co-designer of AFST, and prominent advocate of taking away more children, Emily Putnam-Hornstein had written as far back as 2011 that “prenatal risk assessments could be used to identify children at risk of maltreatment while still in the womb.” (Recall the officer in the video asking the daughter: “What are you, about eight weeks pregnant?”) 

“Prenatal risk assessments” are what Putnam-Hornstein’s co-designer, Vaithianathan, was talking about when she said: “Yes, it’s Big Brother.  But we have to have our eyes open to the potential of this model.” 

Yes, we do, but not in the way Vaithianathan has in mind. 

Pittsburgh has moved ahead with a model, called “Hello, Baby,” that stamps that invisible scarlet number risk score on every child, if not quite in the womb then immediately upon birth. Proponents say it’s only to target prevention, not policing.  That is the case – for now. But, just as  Vaithianathan says, and as is discussed in detail below, “We have to have our eyes open to the potential of this model.” 

Technically “Hello, Baby” is voluntary, but good luck taking advantage of the extremely limited opportunity to opt out – right in those first days after your child is born when, after all, it’s not like you have anything else to think about.  (Ever notice how often, when a corporation forces you to opt out of something it’s because they know you won’t want to opt in.) 

Once infants are branded with those scarlet numbers, they can never be erased.  If they’re labeled as being at high-risk of abuse as infants, it increases the chances that their own children will be labeled at high risk of abuse. If ever these “high-risk” children, as adults, are accused of child abuse, the “Hello, Baby” risk score could raise his AFST score, making it that much more likely their own children could be torn from their arms. (Again, authorities in Pittsburgh say they’ll never, ever do such a thing – but, as Vaithianathan says …) 

Among those most at risk: anyone who actually has been in foster care – since, no matter what the algorithm, having once been in foster care is likely to increase the “risk score” as a potentially abusive parent. 

But even this isn’t enough for Putnan-Hornstein and Vaithianathan. They’re pushing still another algorithm, the  Cross Jurisdiction Model Replication project.  

Like “Hello, Baby,” CJMR generates a risk score for every child at birth. Unlike “Hello, Baby” there is no opt-out at all.  And while with AFST the developers bragged about not explicitly using race (while using poverty), and with “Hello Baby” they claimed (falsely) that they weren’t using poverty, this time there’s no more let’s pretend.  The use of race and poverty as risk factors is out in the open. 

But fear not, say Putnam-Hornstein and Vaithianathan, as noted above, unlike AFST these algorithms are only to target prevention. The places that adopt them would never, ever use them as a tool to investigate families.  They promise!  

Right.  After all, to think these data would be abused is like imagining that a big company like Facebook would sell data without consent.  Oh, yeah, that.  Well, OK, but that’s private industry.  Let’s try again: 

The notion that all that “Hello, Baby” and CJMR data would be misused is like imagining a police force would misuse juvenile records that are supposed to be sealed.  Oh, yeah – that. 

And when a child “known to the system” dies and demagogic politicians condemn the family police agency for not using all that data, you can be sure that agency will stand firm and never, ever change its policy and use algorithms like “Hello, Baby” for investigations! 

The people who want us to believe that are the same people who repeatedly used questionable claims to sell their algorithms in the first place.  And that’s not the only problem. 

The co-designer of the Pittsburgh algorithms and the CJMR project, algorithms that supposedly have no racial bias problem, Putnam-Hornstein, denies the field has a racism problem in the first place. She demeans Black activists and takes pride in being part of a group that defends a self-described "race realist" law professor who hangs out with Tucker Carlson. She has said "I think it is possible we don’t place enough children in foster care or early enough" and signed onto an extremist agenda that proposes requiring anyone reapplying for public benefits, and not otherwise seen by "mandatory reporters," to produce their children for a child abuse inspection.  

She’d even taken to spewing weird personal attacks like this one against her betters on LinkedIn(!) – before deleting and replacing her profile.  (LinkedIn has rules about this, by the way.)

 

Would you trust an enormously powerful algorithm in these hands?  In fact, it’s too powerful for anyone’s hands. 

To see why, take another look at that Lincoln Project video.  But instead of a police officer pulling over a car, imagine a caseworker at the door of a newborn and her family: “The algorithm says you’re high risk,” she says to the mother.  “So we’re taking your baby.” 

Here’s the moral of the story: When you begin a sentence with “Yes, it’s Big Brother …” the only ethical way to end that sentence is: “… so we’re not going to use it.”

Our annual holiday reminder

 

From the people who brought you AFST: The most dangerous "child welfare" algorithm yet

 

 ● It aims to stamp a scarlet number risk score on EVERY child at birth.

● There is no opportunity for any family to opt-out or deny the use of their personal data.

● It’s literally computerized racial profiling: race and ethnicity are explicitly used to rate the risk that a child will be harmed.

● Developers say: Don’t worry, the data will be anonymized – but the promise is unenforceable.

First there was the Allegheny Family Screening Tool (AFST), an algorithm that uses a mass of data, mostly about poor people and disproportionately about nonwhite people, to cough up what amounts to a “scarlet number” risk score.  When the county’s child abuse hotline receives a report alleging neglect, the score is used to guide screeners in deciding which children will have to endure the trauma of an investigation. 

Key features of its implementation include: 

● Ethically challenged ethics reviews.

● No informed consent – or any consent - for the use of the data as a policing tool.

● Deceptive marketing.

● Evaluations almost always done by the developers or commissioned by Allegheny County.

 When, finally, independent researchers got to evaluate it they found racial bias.  That same algorithm now is reportedly under investigation by the U.S. Department of Justice for possible discrimination against disabled families. An Associated Press story and an analysis by the American Civil Liberties Union make clear just how much havoc the Pittsburgh algorithm can cause.  

Hello, Baby 

None of that stopped the county from moving ahead with the even more dangerous “Hello, Baby” algorithm.  This algorithm stamps the scarlet number on every child at birth, regardless of whether the parents have been accused of anything – unless the family, in the days immediately after birth, takes advantage of one obscure chance to opt out.  One of the ethics reviews of AFST said it was OK in part because it was not applied to every child at birth.  When they created, Hello, Baby, an algorithm that does just that, the county got around this by commissioning a friendlier ethics review.  It’s no wonder the county resorted to an even more deceptive marketing campaign to sell it. 

The county promises that Hello, Baby will be used only to target “preventive services.”  Even if they keep this unenforceable promise, the services are likely to be delivered by mandated reporters of child abuse and neglect, who go in knowing an algorithm has labeled the family high-risk. 

CJMR 

Some of the data variables in the Cross Jurisdiction Model Replication Project

And now comes the next all-too-logical progression.  The same team that brought us AFST and Hello Baby, are part of the team offering up the most Orwellian algorithm yet: the Cross Jurisdiction Model Replication project. 

Like “Hello, Baby,” CJMR generates a risk score for every child at birth. (The developers claim this is ok because, they say, in effect, the number is invisible. More about that below.) Unlike Hello, Baby there is no opt-out at all.  And while with AFST the developers bragged about not explicitly using race (while using poverty), and with Hello Baby they claimed (falsely) that they weren’t using poverty, this time there’s no more let’s pretend.  The use of race and poverty is out in the open. 

Data variables include: 

● Paternal multi-race code

● Material multi-race code

● Paternal ethnicity Hispanic/Latino

● Maternal ethnicity Hispanic/Latino 

As for poverty, data variables include: 

● “WIC food” (that is, use of the Supplemental Nutrition Program for Women, Infants and Children).

 ● “Birth payment method for delivery” (in other words, whether it was paid for by Medicaid, the health insurance program for poor people).

And sure enough, the developers say, it works!  How do they know?  If this all weren’t so dangerous the answer would be laugh-out-loud funny: They know it works, they say, not because the algorithm was good at predicting actual child abuse, but because, in many cases, it was good at predicting whether a child would wind up in foster care! 

But like everything else in family policing, the reasons children wind up in foster care are arbitrary, capricious, cruel – and subject to racial and class bias.  As is so often the case with these algorithms, they are less prediction than self-fulfilling prophecy. 

So really guys, you didn’t need to bother!  We already know that if you’re poor and especially if your poor and Black or poor and Native American your kids are more likely to wind up in foster care; in fact, we’ve been trying to tell you why that’s a huge problem for years. 

With CJMR there’s no way to protect your family from government agencies taking your most basic,
and in some cases, most private data, and turning it against you.  There is no chance for informed consent or any kind of consent.  And, of course, there is nothing to show this actually prevents child abuse. 

But don’t worry, the developers say.  All of this is OK because the data are anonymized.  That scarlet number risk score on your child is invisible, so no problem!  The purpose of the algorithm, the developers say, is not to target individuals, but to target communities.  The documents touting the algorithm are pretty vague, but they appear to be suggesting that if enough “high-risk” babies are from the same neighborhood that would be a good neighborhood for concentrating preventive services. 

But since poverty itself is commonly confused with “neglect” we already know where the “high-risk” neighborhoods are – they’re the neighborhoods where poor people live.  So all you need to do is find those neighborhoods – and send money.  Not a lot of money either, since we know that remarkably small amounts of cash are enough to significantly reduce what family police agencies call “neglect.” 

But for America’s giant child welfare industry of helping professionals, that spoils all the fun.  

This was best explained decades ago by Malcolm Bush in his book Families in Distress: 

“The recognition that the troubled family inhabits a context that is relevant to its problems suggests the possibility that the solution involves some humble tasks … This possibility is at odds with professional status. Professional status is not necessary for humble tasks … Changing the psyche was a grand task, and while the elaboration of theories past their practical benefit would not help families in trouble, it would allow social workers to hold up their heads in the professional meeting or the academic seminar.” 

So it should come as no surprise that some of the biggest enthusiasts for scarlet number algorithms also are push a narrative of false complexity on a problem that really isn’t all that complex. 

That’s best case. Worst case, they just want a rationale for taking away more kids. 

The co-designer 

And that brings us to the co-designer of all these algorithm projects, Emily Putnam-Hornstein.  An algorithm can be only as unbiased as its designer, and Putnam-Hornstein has revealed increasing devotion both to tearing apart more families and to minimizing the racial bias problem in family policing has a racism problem.  She has: 

● Declared that "it is possible we don’t place enough children in foster care or early enough.” 

● Called for forcing every parent reapplying for “public benefits” who has a child not otherwise seen by a mandated reporter to produce the child for a child abuse inspection in exchange for the financial aid. 

● Not merely disagreed, but demeaned the work of Black activists. 

­● Taken pride in being part of a group that rushed to defend what it says is the right of a self-proclaimed “race realist” beloved by Tucker Carlson to rant about the inferiority of Black people – without one word condemning the content of those rants. 

● Abandoned even the pretense of scholarly discourse on her Twitter feed, devoting it almost exclusively to lurid accounts of child abuse deaths. 

So when Putnam-Hornstein helps design and promote an algorithm that explicitly uses race and income to determine “high risk” for child abuse, and when developers or users of that algorithm say, in effect, “Trust us, we’ll always keep the data anonymized,” that’s not good enough. 

In fact, given that we already know how to target communities for prevention, the CJMR can serve no real purpose unless agencies using the algorithm stop anonymizing the data and use it to target individual families – and that’s what’s likely to happen as soon as there is a highly-publicized death of a child “known to the system” (like the ones in Putnam-Hornstein’s Twitter feed) in a state or locality that adopts the CJMR.  

The only thing stopping that is a family police agency’s equivalent of a pinky-swear.

Our annual holiday reminder ...

 

The violence of family policing, in analog and digital form

J. Khadijah Abdurahman speaking at a virtual panel on predictive analytics
in “child welfare” in New York City.  Organizers had to be pressured to include her.

t

In this powerful story from Logic Magazine, J. Khadijah Abdurahman, who is both a parent with lived experience dealing with New York City’s family policing agency, the Administration for Children’s Services – and a Tech Research Fellow at the UCLA Center for Critical Internet Inquiry, ties it all together: her own experience of retaliation after she complained about one of the private foster care agencies with which ACS contracts, an overview of how “predictive analytics” makes things worse, and a call for everyone to dig deeper into how ACS is using it. 

In a couple of ways, Abdurahman was fortunate: She has the education and the experience with the system – she was a kinship foster parent for her brother’s children – to know how to fight back, how to document everything and how to go on offense.  Most parents facing the kind of assault her family faced wouldn’t stand a chance on their own.  (ACS is doing its best to keep it that way, successfully lobbying against legislation to simply inform families of the few rights they already have.  So JMac For Families has launched a campaign to tell them – including ads on city buses.) 

Oh, and Abdurahman had one other thing going for her: When the family police pounded on the door and demanded entry at 2 a.m., the home was tidy.  

I’ve often written that there is no line of work I know of that takes more seriously than family policing the idea that cleanliness is next to godliness – and none where the consequences can be so awful.  Sure enough, Abdurahman writes, one of the family police caseworkers said: “We don’t take kids from apartments that look nice like yours …”  (So remember, if you really are one of the very few parents who brutalizes their children, just be sure to keep the home spotless!) 

But even so, Abdurahman writes, the caseworkers 

insisted on completing the most dreaded aspect of an investigation: waking up the kids for strip searches to check them for bruises. I marched each of them out one at a time into the bathroom, where they had to remove all of their clothes down to their underwear, including the baby. 

I mention this because, for some reason, the single element of trauma inflicted on children by the family police that reporters find hardest to believe really happens is the widespread practice of stripsearching.  (That’s why I so often cite a story from The New Yorker, which has an outstanding record for fact-checking.) 

Perhaps reporters find it hard to believe because they are mostly white and middle-class. And, as Abdurahman writes: 

While what happened to us might seem shocking to middle-class readers, for [us,] family policing … is the weather. 

Abdurahman doesn’t stop with her own case. Rather, she uses it to show how much worse things would have been had she lived in, say, Pittsburgh, which has the most highly developed system of computerized racial profiling.  No, that’s not what they call it.  But it’s now well-documented that Pittsburgh’s highly-touted Allegheny Family Screening Tool predictive analytics algorithm bakes-in racial bias.   

Abdurahman writes: 

What AFST presents as the objective determinations of a de-biased system operating above the lowly prejudices of human caseworkers are just technical translations of long-standing convictions about Black pathology. 

In New York, one leader of ACS during the administration of former Mayor Bill de Blasio, Gladys Carrion, understood that.  Abdurahman reminds us that when asked about predictive analytics in child welfare, Carrion replied: 

“It scares the hell out of me… I think about how we are impacting and infringing on people’s civil liberties,” she replied. She added that she ran an agency “that exclusively serves black and brown children and families” and expressed her concern about “widening the net under the guise that we are going to help them.” 

But Carrion misunderstood her job.  She thought her first responsibility was to protect children. But in the de Blasio Administration the first responsibility of any commissioner was to protect de Blasio.  So after the death of a child “known to the system” made headlines, Carrion was out and David Hansell – who perfectly understood de Blasio’s priorities -- was in.  

He started the drive to bring predictive analytics to ACS.  As Abdurahman notes, ACS is notoriously “opaque,” so it is unclear how it is being used.  A 2019 PowerPoint presentation offers some hints, suggesting that, at least at that time, they were not using it the way Pittsburgh uses AFST – or its even more Orwellian “hello baby” algorithm.  But almost from the start, ACS officials have made clear they’re open to that. 

And as Abdurahman makes clear, had it been applied in her case, everything could have been far worse.  Maybe it’s time for reporters in New York to take a close look at exactly what ACS is doing with predictive analytics, what it plans to do and whether the current commissioner, Jess Dannhauser, is willing to put a stop to it. 

After reading the article in Logic Magazine, you’ll see why predictive analytics proponents were so desperate to try to keep Abdurahman off a panel that was billed as a virtual “examination” of the topic.  Family advocates and defenders had to fight to get her included.  Start the event video here and you’ll see why they were so afraid – and why the rest of us have reason to be grateful that J. Khadijah Abdurahman and others like her are fighting back.  Because, as Abdurahman writes: 

Data and predictive risk modeling is not something that exists outside obscene forms of analog violence; it is an inextricable part of it.

Cutting through the spin about predictive analytics in child welfare

The Scarlet Number: Allegheny County (metropolitan Pittsburgh) has been
trying to slap a "risk score" on every child at birth. The score could haunt
them their entire lives.

In Allegheny County, Pa., even the county’s hand-picked ethics reviewers had reservations about the county’s Orwellian “Hello Baby” algorithm.  A key feature of the program flunked one reviewer’s ethics test. 

Second of two parts.  Read part one here.

Yesterday’s post to this blog discussed the amazing good fortune of Emily Putnam-Hornstein, America’s foremost evangelist for using “predictive analytics” to advise family policing agencies concerning everything from who should get “preventive services” to which children should be torn from their parents’ arms. (Another term for this is “predictive risk modeling” (PRM), but a better term than either is computerized racial profiling.) 

It seems that whenever Putnam-Hornstein co-authors an algorithm, the people chosen to do an “independent” ethics review are predisposed to favor it.  At a minimum, they seem to be ideological soulmates.  Sometimes they’ve co-authored papers with Putnam-Hornstein herself or with someone who wrote an algorithm with her. 

But even with the deck so stacked, in one case the ethics reviews offered some strong cautions–including suggesting that a key part of the program for which the algorithm would be used is unethical.  Though generally the reviews were favorable, the reviewers’ concerns were so serious that the agency that commissioned the reviews, the Allegheny County, Pa., Department of Human Services, went to great lengths to spin the results and direct readers toward the spin instead of the reviews themselves.  

The algorithm in question is the second of two in use in Allegheny County. 

The first, the Allegheny Family Screening Tool (AFST) stamps an invisible “scarlet number” risk score on every child who is the subject of a neglect allegation screened by the county’s child abuse hotline.  The higher the score, the greater the supposed risk.  Even though the ethics review for that one was co-authored by a faculty colleague of one of the creators of the algorithm, it cautioned that one reason AFST is ethical is that it does not attempt to stamp the scarlet number on every child at birth – something known as “universal-level risk stratification.” 

This is so Orwellian that even other family policing agencies can’t stomach it.  As noted in yesterday’s post, about Putnam-Hornstein’s work in California, the California Department of Social Services declared that 

The Department believes that “universal-level risk stratification” is unethical and has no intention to use it now or in the future. Identifying and proactively targeting services to families with no [child welfare services] involvement is a violation of families’ privacy and their rights to parent as they see fit. This would be an overreach in the roles and responsibilities of a government agency. 

So when Allegheny County decided that, ethics-be-damned, it wanted an algorithm to do exactly what appalled their counterparts in California, and exactly what their own prior ethics review implied would be unethical, the solution was obvious: Commission another ethics review! 

In fact, they commissioned two (or maybe three) – one of them from an ideological soulmate of the co-author of both Allegheny County algorithms -- Putnam-Hornstein.  

Sure enough, the county got much of what it wanted.  But the reviews displayed far more nuance than the county apparently expected, going into detail about serious problems with this approach, even as they claimed these obstacles could be overcome. 

So the county went into full spin mode.  In 2019, its first publication about the new algorithm, part of a program called “Hello Baby” merely declared that the ethics reviews existed, implying that Hello Baby got a seal of approval – but with no link to the documents themselves. 

A year later, the county put out its own summary of the ethics reviews. Although at last the actual reviews were posted online, there were no links from the county’s summary – and the reviews remain harder to find.  As we noted in our previous post, it’s sort of like the way Donald Trump’s attorney general, Willam Barr, handled the Mueller report.  In the case of the Allegheny County algorithm, the gap between the actual documents and the spin isn’t as wide – but it still tells an interesting story. 

So let’s look closely at the parts of those reviews that Allegheny County, and Putnam-Hornstein, probably least want you to notice. 

● The first thing to notice is that one of the two published reviews may never have been completed. It’s labeled a draft. 

● The second thing to notice is that the draft refers to itself as “one out of three perspectives from cross-disciplinary researchers looking different aspects of the risk-scoring system that Allegheny County plans to deploy.” [Emphasis added.]  But the county has only published two, and only ever refers to two.  What happened to the third? UPDATE, FEB. 22: Responding to an email query from NCCPR, Erin Dalton, director of the Allegheny County Department of Human Services, says there were only two ethics reviews. She said the draft may have been referring to a separate review of methodology and data science.

The other published ethics review strongly suggests a key feature of  Hello Baby – the fact that you’re in it unless you remember to opt-out – is unethical.  The review sets criteria for such a feature to be ethical. Hello Baby doesn’t meet the criteria. 

Selling Hello Baby 

There are two key selling points for Hello Baby: One, it’s supposedly a purely voluntary program, two, the vast troves of data will be used only for targeting prevention.  We’ll start with the second. 

Child abuse investigations are run by another division of the same agency that oversees Hello Baby.  Both divisions of this same agency are ultimately overseen by Erin Dalton, who is as nonchalant about the harm of foster care as she is fanatical in her desire to vacuum up data about poor people.  Nevertheless, Dalton’s agency publicly promises that child abuse investigators won’t see the Hello Baby risk scores or other data from that program. 

One of the ethics reviewers, Prof. Michael Veale of University College, London, saw the problem. It turns out, there’s even a name for it: Function Creep.  He writes: 

One underlying anxiety concerning predictive systems in the public sector is that by virtue of being created for one task, they establish an infrastructure consisting of many aspects—including data, technology, expertise and culture—which might expand beyond its original scope into areas its original democratic and societal mandate did not permit. …

Some will be concerned that while [using the Hello Baby risk score only for prevention] might be the policy today, it might not be robust to change in the future. Similarly, those who might have lost trust in a public service more generally might not trust assurances that this inferred data is deleted or not passed onto other actors in the system. 

Veale suggests that the county come up with 

some legally binding declaration … delimiting the purposes of this system in advance to a sufficiently narrow scope and set of actors. This agreement would then serve as a mechanism that could be used to hold future uses of this model to account—at least insofar as it would have to be actively and ideally publicly removed before the purposes of a score or a model could change. 

This appears based on the naïve assumption that, were Allegheny County to want to use Hello Baby for child abuse investigations, the shame of having to go public might be a deterrent. 

On the contrary, when – not if, because it’s going to happen – the data are used to decide who to investigate as a potential child abuser and when to take their children it will be done with pride and fanfare.  Because here’s how it will happen: 

A three-year-old boy, call him Jason, is killed by his father.  Jason was “known to the system,” a previous allegation had been deemed unfounded.  Somebody leaks the fact that Jason’s father had a high risk score using Hello Baby.  The caseworker who investigated the father gives a tearful television interview in which she says: “If only I’d known that Hello Baby thought he was high risk, I never would have left the child in that home.” 

At that point three things happen: 

● A member of the Pennsylvania Legislature introduces “Jason’s Law,” a bill requiring that information from Hello Baby and anything else in the state like it be fully shared with child protective services.  He calls it “Jason’s Law” of course. 

● Erin Dalton or her successor calls a news conference and declares that the Allegheny County Department of Human Services isn’t about to wait for the legislature – they’re ordering full information sharing right now!  

● There are warnings that algorithms that predict terrible harm will come to a child, including AFST, have a record of being wrong more than 95% of the time  - potentially flooding the system with “false positives” that do enormous harm to innocent families and make it harder to find the few children in real danger.  The warnings are ignored. 

A pinky swear is not enough. 

Having raised an urgent concern, Veale comes up with a solution that has all the enforceability of a pinky swear – or maybe something more like this: 

 

There’s still another danger.  Anyone Hello Baby labels high-risk will be offered a series of services not offered to anyone else.  At the highest alleged level of risk, the program calls it “relentless engagement.”  Therefore, the service provider, who will be regularly coming into the home to engage relentlessly will know from day one that a high-tech algorithm has branded these parents high risk for abusing their children.  That service provider almost always will be a mandated reporter, required to report any suspicion of child abuse and neglect (and in Pennsylvania, the training curriculum is fanatical about urging reporters to report! Report! Report!) 

So even the other ethics reviewer, Deborah Daro, a Senior Research Fellow at Chapin Hall, and an ideological soulmate of Putnam-Hornstein expressed concern about this.  She writes: 

All home visitors report a proportion of their participants to child protective services. … The [Predictive Risk Model] gives service providers additional information on a family’s history that may alter the way workers interpret the conditions they do observe. Even if the exact details regarding a family’s history is [sic] not provided to program staff or other providers, the fact parents have been identified through the PRM as being at high-risk will convey a general profile of concerns. As such, key  implementation questions for the county to address include: 

• How might knowledge of a family’s prior history with the child welfare and justice systems impact a provider’s judgment regarding current relationships in the home and the ability of other caretakers (particularly the father) to appropriately care for the infant? 

• How does this knowledge impact how providers might interpret a mother’s actions – will they be less forgiving of minor concerns they observe? 

• Will knowledge of a family’s history increase the likelihood a provider will report the family to child welfare as a potential risk for maltreatment if the family drops out or refuses additional program services? … 

Heightened awareness of a family’s circumstances may create surveillance bias, resulting in a higher probability of a family being reported. Providers will know more about a family and will need to weigh this knowledge against a family’s willingness or reluctance to remain in the program. 

Notice Daro’s own bias here.  Allegheny County brags that all services provided under
Hello Baby are purely voluntary and families are free to drop out at any time.  But Daro seems to think exercising that right is still another reason for heightened suspicion. 

Having raised the surveillance bias issue, Daro then cops out, suggesting the same failed solution that proponents of the child welfare surveillance state fall back on whenever the harm they do comes to light: We’ll fix it with more “training.” 

Defining “voluntary” 

Another key element of the selling of Hello Baby is the claim that it’s purely voluntary.  Technically yes, but you’d better be very sharp and wide awake during the first days and hours of your baby’s life to avoid being forced into the program – and isn’t everyone wide awake and able to absorb everything during that time? 

Because Hello Baby forces you in, unless you affirmatively opt out.  And you get only two chances to opt out.  The first chance is while you and your newborn are still in the hospital.  Amidst everyone else coming and going and handing you forms and discharge papers and God-knows-what else, you are given an information packet selling Hello Baby that also tells you how to opt out.  The second, and last, chance comes in the form of a postcard sent to your home – it’s not clear when, but presumably very soon after coming home with your baby.  You have to mail it back.  Miss those chances and Allegheny County has free reign to dig up all the electronic dirt on you that is called for in the algorithm and slap a risk score on you and your baby.  The score can follow you, and your child, forever. 

Oh, you can drop out of any services offered under the program at any time – though, as noted above that might prompt the service “provider” to call the child abuse hotline on you – but you never again get a chance to opt out of data collection or make them delete the data they’ve already gathered. 

Here’s what Daro writes about when this approach, called “passive consent,” is ethical and when it is not: 

This approach is considered appropriate only if the intervention or strategy involves minimal risk to the participant and if obtaining written approval for the procedure is not practical or feasible. [Emphasis added.] It is not clear if this approach has already been approved by the county’s Institutional Review Board. If it has, then the approach has been judged appropriate in this instance. If it has not, the county will need to make the case as to why it is not asking parents to “opt in” for the screen. 

It is just as “practical and feasible” to presume someone is not in the program until they check a box saying they’re in, as it is to presume they’re in until they check a box that says they’re out.  So by Daro’s own criteria, this key aspect of Hello Baby is unethical. 

And the county’s response illustrates perfectly why putting all this data power in their hands is so dangerous. They respond that: 

The “passive consent” is only for running the PRM, which commits clients to nothing. 

After all, the county continues, families still don’t have to accept the “services.”  But, of course, allowing the county to run the PRM commits the family to surrendering vast amounts of personal data that can then be turned against them at any time. That’s hardly nothing. 

As for Daro’s stipulation that this aspect of Hello Baby should be approved by the county’s Institutional Review Board, the Allegheny County Department of Human Services replied: 

Allegheny County does not have an institutional review board.