“The machine has labeled you as high-risk”

Is Pittsburgh’s “child welfare” predictive analytics algorithm running amok? Inquiring minds (at the US Dept. of Justice) want to know! 

The Allegheny Family Screening Tool slaps an invisible scarlet number "risk score"
on every child whose parents or other caretakers have been accused of neglect.

(For a comprehensive examination of the dangers of this kind of computerized racial profiling, see our publication Big Data Is Watching You.)

 The most highly touted, most far-reaching example of computerized racial profiling in family policing (more accurate terms than “predictive analytics” in “child welfare”) is the one in Pittsburgh.  Called the Allegheny Family Screening Tool (AFST) it slaps an invisible scarlet number “risk score” between 1 and 20 on any child who is the subject of a hotline call alleging neglect.*

 In response to allegations of bias, the designers and supporters of AFST offer three defenses:  1. The algorithm is used only at the screening stage, to decide which allegations are most in need of investigation.  2. The investigators can’t be biased by this because they don’t even know the risk score.  3. The algorithm doesn’t tell investigators when to tear children from the arms of their families and consign them to foster care; that’s left to humans. 

We’ve always known that the claim about workers not knowing the risk score was b.s.  They may not know the precise risk number but if they’re told to rush out and investigate a particular family, they know the predictive analytics algorithm, known as the Allegheny Family Screening Tool (AFST), has labeled the family “high risk.”  So of course that’s going to bias the investigator.

But now we know about a case in Pittsburgh that is calling into question all the claims about AFST’s supposed lack of bias, and about who -- or what -- makes the decisions.  And the U.S. Department of Justice is doing some of the questioning.  These revelations come in a story from the American Bar Association’s ABA Journal.  That story builds on outstanding reporting by the Associated Press. 

Both stories center on Andrew and Lauren Hackney, who followed their doctor’s advice and took their infant to the ER when the baby wasn’t eating enough.  They believe the hospital called the Allegheny County family police agency, which immediately took away the child.  

Here’s where we find out who’s really calling the shots, the humans or the algorithm.   According to the Hackneys’ lawyer 

when the Hackneys asked their caseworker what they could do to get their daughter back, “the caseworker said, ‘I’m sorry. Your case is high-risk. The machine has labeled you as high-risk.’” [Emphasis added.] 

That was the first the Hackneys knew about AFST.  

As for why the machine labeled them high-risk – nobody knows.  Allegheny County brags that the data elements in AFST are public, but they don’t say how much weight they give to each factor or even if it counts for or against a family.  And they don’t say what led to the risk score in a given case. 

The Hackneys and their lawyer suspect that the algorithm, and the humans who may be slavishly following it, discriminated against them because of disabilities.  As the ABA Journal story explains: 

Lauren Hackney has attention-deficit/hyperactivity disorder that can cause memory loss, and Andrew Hackney has some resultant damage from a stroke. 

That’s why the Justice Department is involved. In this case, and at least two others, they are investigating whether, in Pittsburgh, humans and machines alike violated the federal Americans with Disabilities Act. 

The ABA Journal article quotes two experts on disability law and family policing, Prof. Robyn Powell of the University of Oklahoma School of Law and Prof. Sarah Lorr, co-director of the Disability and Civil Rights Clinic at Brooklyn Law School.  They say that 

Not only are some of the criteria [used in the AFST algorithm] explicitly disallowed … pointing to disability-related factors, but long-standing racial biases also are implicitly included. 

“The ADA explicitly says that state and local government entities cannot discriminate based on disability, and within that requirement is the idea that you cannot use screening tools or eligibility criteria that would [point to] people with disabilities,” Powell says. 

And in Colorado, where at least one county is using a similar algorithm, family defender Sarah Morris says 

“All this does is launder human biases through the mirage of some kind of transparent nonbiased machine calculation.” 

Notwithstanding the specific comments allegedly made by the caseworker in the Hackney case, a co-designer of AFST, Emily Putnam-Hornstein still claims that the risk score is just one of many tools and it’s “advisory only.” 

But even if that’s true, should we trust the “advice” of an algorithm designed by someone who denies the field has a racism problem, demeans Black activists, and takes pride in being part of a group that defends a self-described "race realist" law professor who hangs out with Tucker Carlson. 

She has said: "I think it is possible we don’t place enough children in foster care or early enough" and signed onto an extremist agenda that proposes requiring anyone reapplying for public benefits, and not otherwise seen by "mandatory reporters," to produce their children for a child abuse inspection.  And she helped design another algorithm that explicitly uses race as a risk factor.  (Designers of that one say it’s only supposed to be used to target “prevention.”) 

It's been almost two years since the machine said the Hackneys’ infant was “high risk.” She’s still in foster care.

*-In Pennsylvania, the state runs the child abuse hotline but counties do everything else.  Counties are allowed to screen most “neglect” allegations but not allegations of abuse or “severe neglect.”

UPDATED: A Pennsylvania case illustrates again why, for children, “best interests of the child” is among the most dangerous phrases in the “child welfare” lexicon

         

UPDATE, DECEMBER 15, 2021: The York Daily Record has an excellent story about this case.

  Consider a recent case involving the family policing system in a county in Pennsylvania.  Everything I am about to recount is true except for one detail: 

            Two young children are taken from their mother.  Their father is eager to take them in, but at the time of the removal he’s in the hospital.  That’s because after he was hit by a drunk driver and confined to a wheelchair it led to medical complications that sometimes require hospitalization. 

            But once home from the hospital, the children still are left in foster care – with foster parents who are eager to adopt.  Dad must jump through the usual hoops, but, as is so often the case, it’s never enough.  

            Sure, the usual “bonding evaluation” says, the children are excited to see Dad during visits.  But the court focuses on the part where the evaluator complains that the children have to be the ones to initiate contact with Dad because of Dad’s “compromised mobility.” 

            Then there’s the matter of how long it would take Dad to get upstairs, even with a wheelchair lift in the home.  Three or four minutes – that could be too late in an emergency.  And while it’s true Dad’s aunt is living in the home, the court is worried because she is, to use the judge’s word “elderly.”  She’s 68. 

           

            So the county wants the children taken from their father forever.  They move to change the case goal for the children from reunification to adoption.  The judge rubber-stamps the change with enthusiasm.  An appellate court agrees.  Next stop, presumably: Termination of parental rights, after which the children, now ages five and three may never see their loving father again. 

            But even in a system as riven with every bias under the sun this wouldn’t really happen, right?  Well no* – and yes.  Because, except for that one small detail, it did just happen, in Jefferson County, Pennsylvania. 

            Here’s the detail: Dad’s mobility isn’t compromised because he’s confined to a wheelchair.  It’s compromised because of his weight.  At least partly because of hypothyroidism, he weighs over 400 pounds.  It was complications from his obesity that landed him in the hospital, there was no auto accident.  And, of course, there’s no wheelchair. It’s the obesity that slows him down going up stairs. 

            Everything else, though is straight from the appellate court decision – including the children’s love for their father, and the fact that the trial court judge wrote off the 68-year-old aunt as “elderly.”  I changed the one detail because so many people view obesity as both repugnant and a moral failing – apparently including the trial court judge, whose decision includes finger-wagging comments suggesting the problem is that dad lacked the willpower to embrace a healthy diet. 

            Even were that so, however, that still wouldn’t be grounds to keep this man’s children from him forever – or even for one day.  

            ● For starters, there’s another caretaker in the home.  Apparently, as noted above, she’s ruled out because at age 68 she’s “elderly.”  Presumably this also would rule out a large proportion of those providing kinship foster care, since they tend to be grandparents. 

            The appellate court had no problem with this. In fact, in one way the appellate court decision may be even worse.  In a footnote, the judges write that 

 “While Father complains that the ‘Agency offered no evidence that the aunt is a threat to the children and nothing more than a helping hand[,]” Father has not offered any evidence that his aunt was an appropriate, substitute caregiver.” 

            But why should it be up to a father who never has himself been accused of abusing or neglecting the children, and an aunt who faces no such accusation either, to prove the aunt is suitable?  Shouldn’t it be up to the family policing agency to prove she isn’t suitable?  By this court’s logic, every family that decides to have grandma babysit while they go on vacation should have to go to court and get approval first – with the burden of proof on them to show grandma is “suitable.” 

            ● Even if there were no aunt available, the decision still is wrong.  Federal law
requires that states make “reasonable efforts” to keep families together.  This case is one more illustration of the fact that the law is routinely ignored.  In this case, home health care should have been enough to alleviate all of the family policing agency’s concerns.  And if they were still obsessing over Dad’s ability to climb stairs, how about finding the family a place to live that doesn’t have stairs?  This is what affluent families do in such situations.  That’s one reason why, you may be sure, rich parents don’t lose their children because of obesity. 

            The case also illustrates bigger problems. 

  

          ● The fundamental fact of American “child welfare” is that if you’re not white and affluent the system will discriminate against you. (That’s not unique to “child welfare” of course, but the field seems to be “in denial” about it to an extraordinary degree.) Usually we see this in the form of discrimination based on race and class.  But discrimination against families with physical or mental disabilities gets far less attention, even though it is just as widespread. 

            As Prof. Robyn Powell of Stetson University College of Law has written: 

The disproportionate rate of child welfare system involvement in families headed by parents with disabilities is striking. Although children of parents with disabilities comprise only 9 percent of the nation’s youth, they make up 19 percent of the children in foster care. 

The National Council on Disability—an independent federal agency that advises the President and U.S. Congress on disability policy—described the child welfare system’s bias against parents with disabilities as “persistent, systemic, and pervasive.” 

Similarly, the U.S. Department of Justice and the U.S. Department of Health and Human Services have stated that the child welfare system’s discriminatory policies and practices toward parents with disabilities are “long-standing and widespread.”     

Indeed, those two departments demanded reform from Massachusetts family policing officials for violating the federal Americans with Disabilities Act.  They eventually reached a settlement.  Similar issues have arisen in Oregon.  

But the ADA rarely is raised in family policing cases. UPDATE, DEC 15, 2021: Though it is not always clear when it applies in cases involving obesity, Prof. Powell told the York Daily Record it does apply in this case.  According to the story:

Under the Americans with Disability Act, the father in this case would likely be considered a person with a disability, Powell said after reading the court document.

"It seems to me that they’re discriminating against him because of the disability and not recognizing the in-home support he has," she said. 

Why “best interests of the child” isn’t in the best interests of the child 

This case also illustrates why one of the most common and most revered phrases in the child welfare lexicon is, for children, one of the most dangerous.  The phrase, of course, is “best interests of the child.”  It is an invitation for family policing agencies and courts to inflict their own biases, whether based on race, religion, ethnicity, sexual preference, disability – or weight, on a family and then confiscate and transfer children at will. 

You can’t say you’re doing that, of course, but that is the practical effect or such broad, vague language in a system with no real checks and balances. 

That’s always been true in American family policing, but it was made worse with the passage of the so-called Adoption and Safe Families Act.   The appellate court appears almost gleeful when it cites ASFA, even putting the last sentence in this paragraph in bold: 

Placement of and custody issues pertaining to dependent children are controlled by the [Pennsylvania] Juvenile Act, which was amended in 1998 to conform to the federal Adoption and Safe Families Act (“ASFA”) The policy underlying these statutes is to prevent children from languishing indefinitely in foster care, with its inherent lack of permanency, normalcy, and long-term parental commitment. Consistent with this underlying policy, the 1998 amendments to the Juvenile Act, as required by the ASFA, place the focus of dependency proceedings, including change of goal proceedings, on the child. Safety, permanency, and well-being of the child must take precedence over all other considerations, including the rights of the parents.  [Emphasis in original, citations omitted.]

 By that standard, of course, family policing agencies should be empowered to walk into any and every home, regardless of whether there is even an accusation of abuse or neglect and if they decide the child would be in any way better off with someone else take them away on the spot. 

            That is why, in theory, courts are not supposed to reach the issue of “best interests” until after determining birth parents are unfit.  The one good thing about this decision is that it reveals what judges really do. 

Even if we were to accept the judges’ reasoning, and even if we were to accept that ASFA was necessary for any of these goals – and, on the contrary, it impedes them -- in this case there was no safety issue that couldn’t be remedied with “reasonable efforts” – and possibly no issue at all.  There should have been no issue of the children languishing in foster care because there was no need for them to be in foster care once the father was out of the hospital. That takes care of “permanence.”  And there is nothing to indicate that tearing these children from their father forever would improve their “well-being.” 

So this case perfectly illustrates how ASFA is simply a license to take children from people we don’t happen to like and hand them over to people we do. 

            Strip away the legalese and the decision of the trial court judge in this case, in its entirely can be boiled down to: “Eeewwwww, he’s so fat.  I’ll just give the kids to the people I like better.”  The decision of the appellate court boils down to “We have no problem with that.” 

            Come to think of it, substitute “Black,” “Brown,” “Native American” “LGBTQ” or “poor” for “fat” and you have a huge body of “child welfare” caselaw. 

*-That “no” might be overly optimistic.  I’m not aware of a case in which children were taken from a parent forever because the parent was confined to a wheelchair, but sadly, it wouldn’t surprise me if it’s happened.  This post may yet be updated.