Cutting through the spin about predictive analytics in child welfare

The Scarlet Number: Allegheny County (metropolitan Pittsburgh) has been
trying to slap a "risk score" on every child at birth. The score could haunt
them their entire lives.

In Allegheny County, Pa., even the county’s hand-picked ethics reviewers had reservations about the county’s Orwellian “Hello Baby” algorithm.  A key feature of the program flunked one reviewer’s ethics test. 

Second of two parts.  Read part one here.

Yesterday’s post to this blog discussed the amazing good fortune of Emily Putnam-Hornstein, America’s foremost evangelist for using “predictive analytics” to advise family policing agencies concerning everything from who should get “preventive services” to which children should be torn from their parents’ arms. (Another term for this is “predictive risk modeling” (PRM), but a better term than either is computerized racial profiling.) 

It seems that whenever Putnam-Hornstein co-authors an algorithm, the people chosen to do an “independent” ethics review are predisposed to favor it.  At a minimum, they seem to be ideological soulmates.  Sometimes they’ve co-authored papers with Putnam-Hornstein herself or with someone who wrote an algorithm with her. 

But even with the deck so stacked, in one case the ethics reviews offered some strong cautions–including suggesting that a key part of the program for which the algorithm would be used is unethical.  Though generally the reviews were favorable, the reviewers’ concerns were so serious that the agency that commissioned the reviews, the Allegheny County, Pa., Department of Human Services, went to great lengths to spin the results and direct readers toward the spin instead of the reviews themselves.  

The algorithm in question is the second of two in use in Allegheny County. 

The first, the Allegheny Family Screening Tool (AFST) stamps an invisible “scarlet number” risk score on every child who is the subject of a neglect allegation screened by the county’s child abuse hotline.  The higher the score, the greater the supposed risk.  Even though the ethics review for that one was co-authored by a faculty colleague of one of the creators of the algorithm, it cautioned that one reason AFST is ethical is that it does not attempt to stamp the scarlet number on every child at birth – something known as “universal-level risk stratification.” 

This is so Orwellian that even other family policing agencies can’t stomach it.  As noted in yesterday’s post, about Putnam-Hornstein’s work in California, the California Department of Social Services declared that 

The Department believes that “universal-level risk stratification” is unethical and has no intention to use it now or in the future. Identifying and proactively targeting services to families with no [child welfare services] involvement is a violation of families’ privacy and their rights to parent as they see fit. This would be an overreach in the roles and responsibilities of a government agency. 

So when Allegheny County decided that, ethics-be-damned, it wanted an algorithm to do exactly what appalled their counterparts in California, and exactly what their own prior ethics review implied would be unethical, the solution was obvious: Commission another ethics review! 

In fact, they commissioned two (or maybe three) – one of them from an ideological soulmate of the co-author of both Allegheny County algorithms -- Putnam-Hornstein.  

Sure enough, the county got much of what it wanted.  But the reviews displayed far more nuance than the county apparently expected, going into detail about serious problems with this approach, even as they claimed these obstacles could be overcome. 

So the county went into full spin mode.  In 2019, its first publication about the new algorithm, part of a program called “Hello Baby” merely declared that the ethics reviews existed, implying that Hello Baby got a seal of approval – but with no link to the documents themselves. 

A year later, the county put out its own summary of the ethics reviews. Although at last the actual reviews were posted online, there were no links from the county’s summary – and the reviews remain harder to find.  As we noted in our previous post, it’s sort of like the way Donald Trump’s attorney general, Willam Barr, handled the Mueller report.  In the case of the Allegheny County algorithm, the gap between the actual documents and the spin isn’t as wide – but it still tells an interesting story. 

So let’s look closely at the parts of those reviews that Allegheny County, and Putnam-Hornstein, probably least want you to notice. 

● The first thing to notice is that one of the two published reviews may never have been completed. It’s labeled a draft. 

● The second thing to notice is that the draft refers to itself as “one out of three perspectives from cross-disciplinary researchers looking different aspects of the risk-scoring system that Allegheny County plans to deploy.” [Emphasis added.]  But the county has only published two, and only ever refers to two.  What happened to the third? UPDATE, FEB. 22: Responding to an email query from NCCPR, Erin Dalton, director of the Allegheny County Department of Human Services, says there were only two ethics reviews. She said the draft may have been referring to a separate review of methodology and data science.

The other published ethics review strongly suggests a key feature of  Hello Baby – the fact that you’re in it unless you remember to opt-out – is unethical.  The review sets criteria for such a feature to be ethical. Hello Baby doesn’t meet the criteria. 

Selling Hello Baby 

There are two key selling points for Hello Baby: One, it’s supposedly a purely voluntary program, two, the vast troves of data will be used only for targeting prevention.  We’ll start with the second. 

Child abuse investigations are run by another division of the same agency that oversees Hello Baby.  Both divisions of this same agency are ultimately overseen by Erin Dalton, who is as nonchalant about the harm of foster care as she is fanatical in her desire to vacuum up data about poor people.  Nevertheless, Dalton’s agency publicly promises that child abuse investigators won’t see the Hello Baby risk scores or other data from that program. 

One of the ethics reviewers, Prof. Michael Veale of University College, London, saw the problem. It turns out, there’s even a name for it: Function Creep.  He writes: 

One underlying anxiety concerning predictive systems in the public sector is that by virtue of being created for one task, they establish an infrastructure consisting of many aspects—including data, technology, expertise and culture—which might expand beyond its original scope into areas its original democratic and societal mandate did not permit. …

Some will be concerned that while [using the Hello Baby risk score only for prevention] might be the policy today, it might not be robust to change in the future. Similarly, those who might have lost trust in a public service more generally might not trust assurances that this inferred data is deleted or not passed onto other actors in the system. 

Veale suggests that the county come up with 

some legally binding declaration … delimiting the purposes of this system in advance to a sufficiently narrow scope and set of actors. This agreement would then serve as a mechanism that could be used to hold future uses of this model to account—at least insofar as it would have to be actively and ideally publicly removed before the purposes of a score or a model could change. 

This appears based on the naïve assumption that, were Allegheny County to want to use Hello Baby for child abuse investigations, the shame of having to go public might be a deterrent. 

On the contrary, when – not if, because it’s going to happen – the data are used to decide who to investigate as a potential child abuser and when to take their children it will be done with pride and fanfare.  Because here’s how it will happen: 

A three-year-old boy, call him Jason, is killed by his father.  Jason was “known to the system,” a previous allegation had been deemed unfounded.  Somebody leaks the fact that Jason’s father had a high risk score using Hello Baby.  The caseworker who investigated the father gives a tearful television interview in which she says: “If only I’d known that Hello Baby thought he was high risk, I never would have left the child in that home.” 

At that point three things happen: 

● A member of the Pennsylvania Legislature introduces “Jason’s Law,” a bill requiring that information from Hello Baby and anything else in the state like it be fully shared with child protective services.  He calls it “Jason’s Law” of course. 

● Erin Dalton or her successor calls a news conference and declares that the Allegheny County Department of Human Services isn’t about to wait for the legislature – they’re ordering full information sharing right now!  

● There are warnings that algorithms that predict terrible harm will come to a child, including AFST, have a record of being wrong more than 95% of the time  - potentially flooding the system with “false positives” that do enormous harm to innocent families and make it harder to find the few children in real danger.  The warnings are ignored. 

A pinky swear is not enough. 

Having raised an urgent concern, Veale comes up with a solution that has all the enforceability of a pinky swear – or maybe something more like this: 

 

There’s still another danger.  Anyone Hello Baby labels high-risk will be offered a series of services not offered to anyone else.  At the highest alleged level of risk, the program calls it “relentless engagement.”  Therefore, the service provider, who will be regularly coming into the home to engage relentlessly will know from day one that a high-tech algorithm has branded these parents high risk for abusing their children.  That service provider almost always will be a mandated reporter, required to report any suspicion of child abuse and neglect (and in Pennsylvania, the training curriculum is fanatical about urging reporters to report! Report! Report!) 

So even the other ethics reviewer, Deborah Daro, a Senior Research Fellow at Chapin Hall, and an ideological soulmate of Putnam-Hornstein expressed concern about this.  She writes: 

All home visitors report a proportion of their participants to child protective services. … The [Predictive Risk Model] gives service providers additional information on a family’s history that may alter the way workers interpret the conditions they do observe. Even if the exact details regarding a family’s history is [sic] not provided to program staff or other providers, the fact parents have been identified through the PRM as being at high-risk will convey a general profile of concerns. As such, key  implementation questions for the county to address include: 

• How might knowledge of a family’s prior history with the child welfare and justice systems impact a provider’s judgment regarding current relationships in the home and the ability of other caretakers (particularly the father) to appropriately care for the infant? 

• How does this knowledge impact how providers might interpret a mother’s actions – will they be less forgiving of minor concerns they observe? 

• Will knowledge of a family’s history increase the likelihood a provider will report the family to child welfare as a potential risk for maltreatment if the family drops out or refuses additional program services? … 

Heightened awareness of a family’s circumstances may create surveillance bias, resulting in a higher probability of a family being reported. Providers will know more about a family and will need to weigh this knowledge against a family’s willingness or reluctance to remain in the program. 

Notice Daro’s own bias here.  Allegheny County brags that all services provided under
Hello Baby are purely voluntary and families are free to drop out at any time.  But Daro seems to think exercising that right is still another reason for heightened suspicion. 

Having raised the surveillance bias issue, Daro then cops out, suggesting the same failed solution that proponents of the child welfare surveillance state fall back on whenever the harm they do comes to light: We’ll fix it with more “training.” 

Defining “voluntary” 

Another key element of the selling of Hello Baby is the claim that it’s purely voluntary.  Technically yes, but you’d better be very sharp and wide awake during the first days and hours of your baby’s life to avoid being forced into the program – and isn’t everyone wide awake and able to absorb everything during that time? 

Because Hello Baby forces you in, unless you affirmatively opt out.  And you get only two chances to opt out.  The first chance is while you and your newborn are still in the hospital.  Amidst everyone else coming and going and handing you forms and discharge papers and God-knows-what else, you are given an information packet selling Hello Baby that also tells you how to opt out.  The second, and last, chance comes in the form of a postcard sent to your home – it’s not clear when, but presumably very soon after coming home with your baby.  You have to mail it back.  Miss those chances and Allegheny County has free reign to dig up all the electronic dirt on you that is called for in the algorithm and slap a risk score on you and your baby.  The score can follow you, and your child, forever. 

Oh, you can drop out of any services offered under the program at any time – though, as noted above that might prompt the service “provider” to call the child abuse hotline on you – but you never again get a chance to opt out of data collection or make them delete the data they’ve already gathered. 

Here’s what Daro writes about when this approach, called “passive consent,” is ethical and when it is not: 

This approach is considered appropriate only if the intervention or strategy involves minimal risk to the participant and if obtaining written approval for the procedure is not practical or feasible. [Emphasis added.] It is not clear if this approach has already been approved by the county’s Institutional Review Board. If it has, then the approach has been judged appropriate in this instance. If it has not, the county will need to make the case as to why it is not asking parents to “opt in” for the screen. 

It is just as “practical and feasible” to presume someone is not in the program until they check a box saying they’re in, as it is to presume they’re in until they check a box that says they’re out.  So by Daro’s own criteria, this key aspect of Hello Baby is unethical. 

And the county’s response illustrates perfectly why putting all this data power in their hands is so dangerous. They respond that: 

The “passive consent” is only for running the PRM, which commits clients to nothing. 

After all, the county continues, families still don’t have to accept the “services.”  But, of course, allowing the county to run the PRM commits the family to surrendering vast amounts of personal data that can then be turned against them at any time. That’s hardly nothing. 

As for Daro’s stipulation that this aspect of Hello Baby should be approved by the county’s Institutional Review Board, the Allegheny County Department of Human Services replied: 

Allegheny County does not have an institutional review board.

Pittsburgh’s child welfare agency goes full Orwell

Starting in 2020, Allegheny County, Pa. will attempt to, in effect, stamp EVERY
child born in the county with a "scarlet number" risk score that could haunt
the child and her or his family for life.

KEY POINTS 

● They’re moving ahead with plans to try to label EVERY child born in the county with a predictive analytics risk score that could haunt the child for life.

● To avoid the stigma, parents have to affirmatively opt out. If they opt out, they lose out on help for their newborns. But there may be even bigger risks if they stay in.

● The county’s first “ethics review” found that its use of algorithms was ethical in part because it wouldn’t be applied to all children at birth.  The county solved this little problem by commissioning another ethics review.

● County officials promise this label-every-child-at-birth algorithm will be used only to target prevention. That’s absolutely true – until it isn’t.  Because this promise relies exclusively on self-policing by the same county officials who created this nightmare in the first place.

● And an algorithm isn’t needed to target prevention programs.

It is perhaps the ultimate Orwellian nightmare: From the moment your child is born, the child and family are labeled with a “risk score” – a number that supposedly tells authorities how likely you are to abuse your newborn.  The big government agency that slaps this invisible scarlet number on you and your newborn promises it will be used only to decide if you need extra help to raise your child, and the help will be voluntary. 

But once you’re in the database, that score stays there forever. And if, someday, the same big government agency wants to use the score to help decide you’re too much of a risk to be allowed to keep your child, there is nothing to stop them.  The scarlet number may haunt your family for generations. The fact that your child was supposedly born into a “high risk” family may be used against the child when s/he has children.

Welcome to the dystopian future of child welfare – and childbirth – in metropolitan Pittsburgh, Pa.

For a couple of years now, Allegheny County,  which includes Pittsburgh and surrounding suburbs, has been using something called the Allegheny Family Screening Tool (AFST), a predictive analytics algorithm, to help decide which families should be investigated as alleged child abusers.

Back when Facebook was fined, we pointed out the similarities
to how Allegheny County's child protective services agency
uses data.

The algorithm coughs up a “risk score” – an invisible scarlet number. The higher the number the greater the supposed risk.  It’s all made possible by a massive trove of data on families that Allegheny County has gathered in a way reminiscent of the Cambridge Analytica scandal. Though Allegheny County’s behavior is perfectly legal, it has amassed the without the informed consent of the poor people in the database to have the data turned against them.

The algorithm is weighted heavily toward punishing parents for being poor. In her brilliant book, Automating Inequality, Prof. Virginia Eubanks calls it “poverty profiling.”   In her review of Automating Inequality, Prof. Dorothy Roberts (a member of NCCPR’s Board of Directors) extends the analysis to show how predictive analytics reinforces racial bias.

To justify all this, the county submitted its plans to a couple of scholars for an “ethics review.”  But one of the reviewers is a faculty colleague and co-author of papers with one of the creators of the algorithm.  Even this ethically-challenged ethics review gave a seal of approval to AFST in part based on the premise that it would not be applied to every child at birth.

But getting the chance to slap a scarlet number on every child at birth is the Holy Grail for some predictive analytics proponents.  And now it appears that was the goal of the Allegheny County Department of Human Services all along. 

The birth of “Hello Baby”

In her book, Eubanks reports that the county was, at a minimum, considering introducing “‘a second predictive model … [that] would be run on a daily or weekly basis on all babies born in Allegheny County the prior day or week,’ according to a September 2017 email” from a deputy director of Allegheny County DHS, Erin Dalton.  (Dalton is also disturbingly sanguine about the harm of foster care.) As I noted in a 2018 column for Youth Today, such a model already exists — indeed it’s one of the models the designers of AFST proposed to the county in the first place.

The county apparently turned it down initially because they didn’t think they could sell it politically.  But clearly, with a couple of tweaks to the algorithm, now they think they can – and, sadly, they may be right.

And so, starting in January, 2020, the county plans to phase in a “prevention” program it calls “Hello Baby.”

Here’s how the county says it will work.

During some of the most chaotic hours of a family’s life, those hours in the hospital after a baby is born, when one medical professional, volunteer or other hospital-affiliated person after another is traipsing in and out of the room, the family will be handed a packet of information about the help available through “Hello Baby.”  A nurse may also discuss the program with the family.

The program offers three tiers of services.  Tier 1 is automatically available to everyone without having to surrender their data.  That tier is simply information about help that’s already out there.  Tiers two and three provide more intensive help to individual families. But to get that help you must accept having the child labeled by an algorithm as at moderate or high risk of abuse.

You have to opt out

The program automatically assumes you have given permission for this massive invasion of family privacy – it’s the equivalent of a “default setting” on an app you may download without realizing how much data you surrender in return. (Or just think of all the data you may have given to Facebook to share at will because you didn’t find the right button among the settings.)

The “Hello Baby” document is vague about the whole opt-out process.  But it appears you get very little chance to actually opt out. You get one notice – in the form of a postcard mailed to your home a few days after the child is born. Along with a reminder of the benefits of “Hello Baby” somewhere on that postcard will be a notification that you must specifically opt out of being run through the database – otherwise you and your child are slapped with that risk score whether you really wanted to participate or not. 

The material made available by Allegheny County does not mention how much time you have to opt out before your name is run through the database.  Nor does it say anything about expunging a risk score if you choose to opt out after the county has already done it.

And what, exactly, are you deemed at risk of doing?

According to the county:

The model was built to stratify families based on the likelihood that there may be future safety issues so significant that the courts require the County to remove the child from the home before the child has reached their 5^th birthday.

Think about that.  From the moment your child is born, you risk having that child labeled at high risk for being taken away and consigned to foster care. From the moment you say “hello, baby” you may be at greater risk of someday having to say “goodbye, baby.”  In effect, “Hello Baby” creates a ticking time bomb in the form of an electronic record that might go off if, say, an angry neighbor calls a child abuse hotline, or if you’re caught pot smoking while Black.

To avoid that risk you have to be alert to the chance to opt out, and if you opt out you risk losing out on what might be genuinely useful assistance.

We’ll never, ever misuse all that data we have on you – we promise!

County officials solemnly promise not to use the data that way – they say they’ll use it only to target help, and won’t make it a part of child abuse investigations. But even the promise has a loophole:

As the county’s “Hello Baby” overview puts it:

The County pledges that this Hello Baby analytic model will only be used to provide voluntary supportive services as described here and updated over time. [Emphasis added.]

Indeed, they will issue a signed document to that effect.  What could possibly go wrong?

I think Allegheny County really means it when they say they won’t pull away the football – sorry, misuse the algorithm – for now.  But there is no institutional safeguard in place. There is nothing to stop the leaders of the agency that created “Hello Baby” and crave having data on every child from birth from changing their minds whenever they damn well feel like it. 

When might that be? How about the first time there’s a child abuse tragedy and word leaks out that the family had been labeled “high risk” at the time of the child’s birth? That’s when the demands will come to make this information available immediately to child protective services and to use it to immediately trigger a CPS investigation – or worse.

That’s not the only problem.  The extra help families will get is likely to be provided by people who are “mandated reporters” of alleged child abuse and neglect.  There are penalties for failing to report and no penalty for mistakenly calling in a false report. So mandated reporters always are under pressure to make “CYA” referrals. Now, these mandated reporters will enter the home already knowing that a “scientific” algorithm has determined the family is “high risk” for abusing and/or neglecting their child. That’s bound to color the judgment of the helpers when deciding whether or not to phone in a report alleging child abuse or neglect.

It’s still poverty profiling

In order to counter the charge of poverty profiling, the county has tweaked the algorithm – slightly. But their claims are disingenuous at best.  Thus, they claim: “Unlike the Allegheny Family Screening Tool model, the Hello Baby model only relies on data where the County has the potential to have records for every family it only uses universal (rather than means tested) data sources.”

But the key weasel word there is potential.

Because right before making this claim, the county acknowledges that they probably will use “child protective services, homeless services and justice system data.” 

So unless Allegheny County’s jails are filled with wealthy white-collar corporate criminals, and its homeless shelters are filled with people spending the night because they misplaced the keys to their mansions in Sewickley and other wealthy Allegheny County suburbs, this is still poverty profiling.  And, of course, they include data from any previous encounters with child protective services – and CPS intervenes to a vastly disproportionate degree in the lives of poor people.  (As noted in many previous posts, CPS agencies often confuse poverty with neglect.  So if you use a previous “substantiated” allegation of child neglect to raise a risk score you are not countering bias, you are simply automating it.)

And, of course, both the justice system and the child welfare system are notorious for their racial bias – raising the risk that “Hello Baby” amounts to racial profiling as well.

 

Another ethically-challenged ethics review

As noted earlier, even the “ethics review” for AFST commissioned by the county itself – the one co-authored by a faculty colleague of one of the designers of AFST – emphasized that one reason AFST was ethical is that it was not triggered until someone actually phoned in a call alleging child abuse and neglect.  It was deemed ethical in part precisely because it did not seek to slap a risk score onto every child at birth.

How do you get around this little detail? Simple. Commission another ethics review from someone who is likely to tell you what you want to hear. 

So Allegheny County turned to Deborah Daro.  Like most people in child welfare, Daro really wants to help children, and she’s devoted her life to the cause.  But Daro spent much of her time at the group that now calls itself Prevent Child Abuse America – and she did so at a time when PCAA was fomenting hype and hysteria about child abuse, and taking data out of context.  They were particularly keen on minimizing the role of poverty in what we label abuse and neglect. I discuss this in detail in the section of this 2010 blog post called “PCAA’s record of extremism.” But don’t take my word for it – back in 2003, PCAA came startlingly close to admitting as much, declaring:

While the establishment of a certain degree of public horror relative to the issue of child abuse and neglect was probably necessary in the early years to create public awareness of the issue, the resulting conceptual model adopted by the public has almost certainly become one of the largest barriers to advancing the issue further in terms of individual behavior change, societal solutions and policy priorities.

Then Daro moved to the Chapin Hall at the University of Chicago.  The same 2010 blog post documents Chapin Hall’s bias, and some of Daro’s work there. 

More recently, Chapin Hall has been a leader in minimizing the role of racial bias in child welfare, and in fueling foster-care panic in Illinois.

And nearly a decade ago, Daro herself wrote a paper advocating for something very much like “Hello Baby.”  She called for:

Universal assessments of all new parents that carry the dual mission of assessing parental capacity to provide for a child's safety, and linking families with services commensurate with their needs.

So, in effect, Allegheny County asked Deborah Daro to offer an opinion as to whether using an algorithm for the kind of intervention she herself has been promoting for decades is ethical.  Apparently, she said yes. 

I say “apparently” because the actual document does not appear to be available on the Allegheny County DHS website.  Neither is a second ethics review done by Michael Veale a “Digital Center Fellow” at the Alan Turing Institute in London.  In fairness, I am aware of no biases on Veale’s part concerning child welfare. But his biography reveals no knowledge of or experience in the field.  So he was at the mercy of those who commissioned him to understand how child protective services agencies really work.

An intellectually honest ethics review would require bringing together a panel of experts who have strongly divergent views on child welfare and predictive analytics and seeing if they could formulate an ethical framework for using such an algorithm in child welfare.  But of course if Allegheny County tried that they would risk getting answers they don’t want to hear.

 
You don’t need an algorithm to target help

A crucial false premise behind efforts such as “Hello Baby” goes like this: Funds are limited, so we need this kind of algorithm to target help to the families who need it most.  But no such algorithm is necessary.  That’s because the families that need the most help have one thing in common: They’re poor.  So all you have to do is offer the high-end “Hello Baby” services to families of infants born in hospitals that serve the county’s poorest communities.  And, while you’re at it, make sure the help addresses concrete needs of poor families instead of just forcing them to run a gauntlet of counseling sessions and parent education classes.

The “Hello Baby” overview paper claims this won’t work because it’s “based on the incorrect assumption that poverty is the singular driver for abuse.”  But that is setting up a straw man.  No one says poverty is the singular driver for abuse.  But poverty is, by far, the most important driver of what we deem to be abuse and, especially, neglect. 

The “Hello Baby” document goes on to claim that other causes are “untreated mental illness, substance use disorder and intimate partner violence.”  But if you’re middle class your mental illness probably won’t go untreated – because you have the money to treat it.  Your substance use won’t be deemed a disorder, because middle-class parents can use substances pretty much with impunity. And an algorithm that checks criminal justice and homelessness records to determine risk isn’t likely to catch wealthy drug users, now is it?)

Most important, there is now a wealth of research documenting the simple fact that what we deem to be child maltreatment can be fixed primarily by transferring just a little more wealth to poor people.

So why do we need a giant Orwellian child welfare surveillance state to “help” these families? We don’t.  We only need it to target them, control them, and quite possibly, take away their children.

Does Chapin Hall want YOU (investigated for child abuse)?

In the previous post to this Blog, I said that home visiting programs like the Nurse Family Partnership or Healthy Families America have such strong support in the child welfare community that they should be called "goody-two-shoes prevention." The people whose 19th Century counterparts proudly called themselves "child savers" like these programs because they enormously expand the scope of intervention into families. Advocates of family preservation like them because the intervention is voluntary, and such programs really can be helpful.

Almost the only dissenters are a few on the political right who see it as a way to impose government child-rearing standards and take away children when the parents don't comply. Absurd, of course – except for the fact that every once in awhile, someone from the mainstream of American child saving says something to fuel those very fears.

A SPY IN EVERY LIVING ROOM

A little over a decade ago, for example, one of the most zealous crusaders against family preservation, Elizabeth Bartholet, suggested making home visiting programs like Healthy Families America mandatory – specifically for what she described as their "surveillance" value. Under Bartholet's spy-in-every-living-room scheme, parents would be required to admit the visitors and the visitors required to turn in any parent who falls short of Bartholet's standards for child rearing. Bartholet, a self-proclaimed liberal, explains that this "would simply provide society with a realistic means of enforcing" child maltreatment laws.

Which brings me to Dr. Deborah Daro, advocate-turned-"scholar."

In the previous post to this Blog, I discussed the extremist agenda of the group now called Prevent Child Abuse America when Daro was the group's "research director." Daro now is a "research fellow" at the Chapin Hall Center for Children, a place which claims to be a center of objective scholarship but I would argue has a checkered record in that regard.

Last February, Daro wrote an Issue Brief discussing the most recent National Incidence Study of child maltreatment (NIS-4). In addition to taking the findings of the study out of context (our own analysis is available here), toward the end of the Brief, Daro offers her own four-point plan for exactly what government should do in response to the data. Point three calls for "Universal assessments of all new parents that carry the dual mission of assessing parental capacity to provide for a child's safety, and linking families with services commensurate with their needs."

But Daro is remarkably vague about what this means. Does "universal" mean every new parent must be assessed, or has the opportunity to be assessed? If it's the latter, what happens to the parent who says no – does that make her automatically suspect? Who does these assessments? Are they also mandated reporters of child abuse? Before "volunteering," how much is the parent told about the risk of letting someone who may be a mandated reporter of child abuse into the home to "assess" her "capacity to provide for a child's safety"? The brief article doesn't say.

Daro's other writing, including a much longer article about a variety of prevention strategies, doesn't answer these questions either. That article suggests that what Daro has in mind is what they're trying to implement in Durham, North Carolina, as part of something called the Durham Family Initiative. The goal is to get a home visitor into the home of every newborn, apparently within 48 hours of the mother's discharge from the hospital. The visitor would complete "the standard risk assessment protocol" and do a "comprehensive psychosocial assessment." The goal is to "expand coverage to the families of newborns that are not now offered or do not accept these visits" [emphasis added].

THE MEANING OF "VOLUNTARY"

Once again this remarkably vague description (especially for "scholarly" writing) is all the reader gets about how the process works. So we don't know if the visitor is a mandated reporter of child abuse and what happens if, after knocking on the door, the new parent tells the visitor to go away.

And even if this is purely voluntary, there is a huge difference between stopping by to offer help and stopping by with "the standard risk assessment protocol" to assess "parental capacity to provide for a child's safety" – both of which sound remarkably like what Child Protective Services workers do after they've received a report alleging maltreatment.

Of course Daro couches all this in the language of help. The point of the process, she writes is to "ensure that the family is linked to a medical provider and that any immediate needs identified though the risk assessment are addressed through an appropriate service referral."

And I'm sure Daro would argue that family preservation advocates should favor this approach, since, if it gets families "services" and prevents maltreatment, no one will report them and their children won't be taken away. But that assumes the assessment instrument is strictly objective and all these friendly "home visitors" can check their prejudices at the door. We've learned through the failure of Structured Decision Making, a similar set of assessment tools used by CPS workers that was found by this excellent study to be permeated with racial and class bias, just how dangerous "help" can be when the prerequisite for getting it is submitting to a "comprehensive psychosocial assessment." And the failure of the CASA program illustrates the danger of letting overwhelmingly white middle-class strangers pass judgment on overwhelmingly poor disproportionately minority families.

In short, what Daro is talking about is sending what are likely to be overwhelmingly white, middle-class professionals into homes they can identify with, and a whole lot of homes they can't – the homes of poor, minority families, in a way that may be voluntary in name only, even when these families have not even been accused of maltreating their children.

In other words, a spy in every living room.

And, indeed, if point three of Daro's agenda was the velvet glove, point four is the iron fist: "Creating child welfare systems that have the capacity to work with parents who require mandatory intervention to insure their child's safety and the willingness to remove children in those cases in which parents are either unwilling or unable to change." As if somehow we don't have that already.

WERE THE RIGHT-WINGERS RIGHT?

So does this mean the critics on the right were right about home visiting programs?

I don't think so. These programs have been around for a long time. I'm aware of no evidence linking them to increased removals of children from their homes – and there is evidence, with varying degrees of strength depending on the program – linking them to better outcomes for children. I don't think an entire program should be condemned because a few extremists want to pervert it.

But there also is an obligation on the part of mainstream supporters of these programs, like the Children's Defense Fund, the Child Welfare League of America and, especially, Prevent Child Abuse America, to take the initiative and speak out, loudly, against the extremists in their midst. They need to say that they won't tolerate having the program distorted the way people like Bartholet and Daro suggest.

And I haven't heard that yet.

"SCHOLARS" WITH AN AGENDA

The other question here is why does a "scholar" have an advocacy agenda? According to the recent Youth Today story on evidenced-based evaluation of youth service programs:

Mark Dynarski, vice president and director of the Center for Improving Research Evidence at the research company Mathematica, said a researcher must be neutral in carrying out any study, "very transparent" about the methodology of the evaluation, and then should step out of the way and "let the debate ensue." Even if there are significant findings, researchers "don't want to be over strong" in reporting them, Dynarski said. "Over-strong discussions are a kind of advocacy." Researchers must "stay in the line of science and let the policymakers pick up from there."

But at Chapin Hall, it seems, the line between advocate and scholar can be blurry – sometimes it's even indistinguishable. And that makes the "scholarship" suspect.

Evaluating alternatives to foster care: The advocate in scholar’s clothing

The lead story in the current issue of Youth Today is about how to measure the success or failure of programs serving youth in an "evidence based" world. Should programs be funded only if they can prove their success based on the same kind of "Randomized Controlled Trials" (RCTs) used to test medicines? Or are there so many more variables in human services programs than in pills that a lot of very successful programs, and those who benefit from them, would lose out?

It's an important debate, and one which probably calls for a "middle ground" answer, in which there is a reasonable standard of proof of effectiveness short of RCTs. But that's not the main topic here today. Today's primary topics are double standards, hypocrisy and failure to disclose at least the appearance of conflict-of-interest.

The issues arise in a letter cited in a sidebar at the end of the Youth Today story. The letter, available here, was sent to President Obama by four "scholars." The issue is funding for "home visiting" programs, in which someone from a government or private agency visits new mothers to offer help with child rearing. These are, in fact, useful programs, though exactly how useful is open to debate. And they are purely voluntary – we really are talking about help here, not investigations. (Though as I'll discuss in a future post, at least one "scholar" may want to change that.)

Because these programs widen the net of intervention into families – in some cases, potentially, to every new mother in the country - and do absolutely nothing to threaten the child welfare status quo, they are enormously popular in America's child welfare establishment. Since almost no one is opposed, I call them "goody-two-shoes prevention."

But not all of these programs have been proven equally effective. The strongest evidence, by far, supports one particular model, the Nurse Family Partnership program in which, as the name implies, the home visitors are nurses.

There is less evidence behind other programs, including what is probably the best known, "Healthy Families America," a project of a group called Prevent Child Abuse America.

President Obama originally proposed a big increase in funding for home visiting – but only for the Nurse Home Visitor model because it has the strongest evidence base.

The four scholars urged him to fund programs like Healthy Families America as well. Their position has some merit (and the Obama Administration ultimately came up with what appears to be a reasonable compromise). It is, in fact, far harder to apply the Randomized Controlled Trial model to evaluating social programs, and relying only on that model can be a serious mistake, something that Lisbeth Schorr, senior Fellow at the Center for the Study of Social Policy and Lecturer in Social Medicine at Harvard, discusses in several articles on her website. Her absence from the Youth Today story is an unfortunate omission. (About 14 years ago, I assisted Prof. Schorr with the editing of one of her books, and was paid for the work.)


In addition, though there are many excellent scholars studying child welfare, the field is so loaded with bias in favor of taking away children, and so poorly self-policed, that a lot of good programs probably would be strangled if a pure medical model were applied now.

PCAA'S RECORD OF EXTREMISM

The letter from the four "scholars" proves the point – though not in the way they had intended.

One of the "scholars" is one Dr. Deborah Daro, a "research fellow" at the Chapin Hall Center for Children at the University of Chicago.

But before she was Deborah Daro, scholar, she was Deborah Daro, advocate – serving as "research director" of the group now called Prevent Child Abuse America (PCAA) at a time when the group took a variety of extremist positions. These included everything from fanning the flames of the 1980s "mass molestation" and satanic ritual abuse witchhunts (relying in part on a psychiatrist who later would have his license suspended for two years on grounds of "dishonorable, unethical and unprofessional conduct") to publishing a special Spider Man comic book – still in print - that can't help but leave young children with the impression that they should turn in their parents if they get a spanking.

At Daro's PCAA, poverty had nothing to do with neglect – ever. A PCAA pamphlet declared that "the heart of the problem always is an emotional lacking in the parents" for which the solution is "re-education." [Emphasis added.]

Indeed, Daro's mentor, PCAA's first executive director, Ann Cohn Donnelly, seemingly relished rubbing salt in the wounds of parents who lost their children to foster care because they couldn't afford adequate housing or had to leave their children home alone to keep their jobs.

Said Donnelly: "There are a tremendously large number of people in this country who have little or no money who do not neglect their children. When parents neglect their children and are of low income, it is not sufficient to say they are excused because they have no money."

The sheer cruelty of the comment aside, it also is the equivalent of saying that, because there are a tremendously large number of smokers who don't get lung cancer, smoking has nothing to do with lung cancer.

Daro's own work at this time included a 1988 letter she wrote to The Wall Street Journal hyping the numbers in a federal report on child abuse issued at the time, the second "National Incidence Study" known as NIS-2.

She cited a part of the study which found that, in Daro's words: "Nine percent of the cases determined to be unfounded by Child Protective Service workers did indeed involve mistreatment that involved significant harm to the child."

But Daro left out the fact that, using the same criterion, the same study found that from 27 percent to 61 percent of the cases CPS workers "substantiated" should have been unfounded. In other words, caseworkers were two to six times more likely to mistakenly "substantiate" a case than to mistakenly label one "unfounded."

That information is in the same document, on the same page, in the same table as the information Daro chose to include in her letter – yet she left that part out.

And she didn't do it just once. She took the same information out of context in the same way in a report issued in 1992.

In short, the PCAA of Donnelly and Daro was the epitome of 19^th Century "child saving" 100 years after it should have been extinct.

And Daro is still at it. When the fourth National Incidence Study (NIS-4) was released earlier this year and showed a significant decline in child abuse, Daro tried as hard to minimize the findings as she tried to hype NIS-2 more than two decades ago. (Our own analysis of NIS-4 is available here; see especially how our analysis of the study's findings on "neglect" differs from Daro's.).

WHAT THE "SCHOLAR" DIDN'T TELL THE PRESIDENT

But not everything PCCA did was harmful. They came up with a promising strategy for helping new mothers, thereby reducing child abuse. They called it "Healthy Families America." In the years since, HFA has become a huge part of PCCA's agenda. One could argue that PCCA, and those who created the program, have staked their reputations on it.

One of those people was Deborah Daro. In fact, when interviewed for a book, Daro declared that "In cooperation with my colleagues at Prevent Child Abuse America, we developed Healthy Families America."

But that little detail isn't in the letter to President Obama. Nowhere in the letter is there any disclosure of the fact that what Deborah Daro really is saying boils down to: Mr. President, please make an exception and lower the standards of evidence for the program I helped to invent.

This doesn't make the request invalid. Nor was it wrong for Daro to make the request. The problem is the absence of the kind of basic disclosure expected in other fields. For example, leading medical journals expect that if a study purporting to prove the effectiveness of a drug was paid for by the company that invented the drug, this fact will be disclosed.

The other problem with Daro's position is that she appears to be selective about which programs deserve this kind of consideration.

Daro led a Chapin Hall team that evaluated a promising child welfare innovation called Community Partnerships for Child Protection. This innovation long has been listed among NCCPR's Ways to do Child Welfare Right.
(The evaluation was published in 2005. Five years earlier, NCCPR received general operating funds from the Edna McConnell Clark Foundation, which created the Community Partnerships initiative.)

Did I say Chapin Hall evaluated the initiative? Savaged is more like it. The evaluation set an impossibly high standard for what the program sites had to do to prove success.

The Chapin Hall team essentially wrote off Community Partnerships despite the fact that, at three of four sites, there was significantly greater improvement in reducing child abuse at the partnership sites than in the surrounding communities, using the measure chosen by Chapin Hall itself. But these data were buried in charts in an appendix. The overall message was "community partnerships don't work" and that's the message picked up, understandably, by reporters writing about the study.

At one of the Community Partnership sites, St. Louis, there also were very large reductions in entries into foster care. But for Community Partnerships, Daro decided one site was not enough. In contrast, in her letter, Daro cites evidence of healthier births from just one of scores of Healthy Families America sites as sufficient reason for a big federal investment in the program.

In short, the evidence for the success of Community Partnerships is as least as strong as the evidence for Healthy Families America. And given that the whole approach of Community Partnerships – including working to cushion the blows of poverty and extending a helping hand instead of a wagging finger - is the opposite of that advocated by PCAA when Daro was their research director, it's fair to ask if Daro should have recused herself from any involvement in the Community Partnership evaluation.

THE RECORD OF CHAPIN HALL

This kind of issue is nothing new for Chapin Hall. They similarly stacked the deck against Intensive Family Preservation Services (IFPS) – but in that instance, other researchers did more careful studies proving its effectiveness. Indeed, in its comprehensive survey of what passes the "evidence-based" test and what doesn't, the Washington State Institute for Public Policy rates IFPS programs that follow the model of the original such program, Homebuilders, among the relatively few interventions that is truly an evidence-based success. Their detailed evaluation is available here. But Chapin Hall's hatchet job set back the field for years. (For details see NCCPR Issue Paper #11).

And it was a leading "scholar" at Chapin Hall who led an effort to punish a researcher for disclosing the very fact that child welfare scholarship is riddled with bias in favor of "child saving" interventions, something discussed in detail in this previous post to the Blog.

Now, don't get me wrong. I can't wait for the field to be fully ready for evidence-based evaluation, and for funders to act on the results. We already know that, when that day comes the government will cut off all funding for residential treatment centers (RTCs) and parking place "shelters," the CASA program will disappear, and there will be vastly less foster care, since none of these programs has real evidence of success, and they've all got significant evidence of failure.

But that's the whole problem with the debate over evidence-based funding. What evidence-based really means is: If you want to challenge the child welfare status-quo you'd better be able to dot every i and cross every t on a huge pile of randomized controlled trials. If, on the other hand, you just want to conduct business as usual and shovel children into substitute care – no evidence is required.

Or, to put it another way, the findings from the RCTs are no match for the clout of the RTCs.

LATER THIS WEEK: Does Chapin Hall want YOU (investigated for child abuse)?