Thursday, March 29, 2018

Predictive analytics in Pittsburgh child welfare: Was the “ethics review” of Allegheny County’s “scarlet number” algorithm ethical?

Allegheny County's use of predictive analytics in child welfare
generates the equivalent of a "scarlet number" that can mark
a child, and even that child's children.

Imagine you’re on a baseball team. The game is about to start when you find out that the home plate umpire moonlights as a volunteer coach for the other team.  He even co-authored a handbook with the manager of the other team.

That umpire still may be perfectly capable of objectively calling balls and strikes.  But would you be comfortable relying on him to make those calls?

Anyone who would say no should be deeply uncomfortable about one of the key elements in how Allegheny County (Pittsburgh) Pa. sold its use of “predictive analytics” to decide which families are investigated as alleged child abusers – and possibly, in the near future, to decide much more.

The Allegheny County model is known as the Allegheny Family Screening Tool (AFST). In effect, it stamps children who are the subject of reports alleging child maltreatment with an invisible “scarlet number” that supposedly measures how likely they are to be abused or neglected. I discussed the dangers of AFST in this column for Youth Today.  Here I’d like to focus on one element that has been crucial to selling AFST.  Over and over stories proclaiming how wonderful it is cite a so-called “ethics review.”  An op-ed in the Pittsburgh Post Gazette declares:
 The ethical assessment determined that the tool was so much more accurate than relying solely on human analysis that declining to use it would be unethical.

Similarly, a New York Times Magazine story, which I’ve discussed before on this blog, declares:
 Marc Cherna, who as director of Allegheny County’s Department of Human Services … had an independent ethics review conducted of the predictive-analytics program before it began. It concluded not only that implementing the program was ethical, but also that not using it might be unethical.

But “independent” is a stretch.

The review was conducted by Prof. Eileen Gambrill of the University of California – Berkeley and and Prof. Tim Dare of the University of Auckland in New Zealand. New Zealand? Seems like a long way to go to find an ethics reviewer.  Unless, of course, his selection has something to do with this: One of the designers of the predictive analytics model used in Allegheny County is Prof. Rhema Vaithianathan of the University of Auckland in New Zealand.

And these two don’t just pass each other in the hallway.  Much like that hypothetical manager and umpire, Prof. Dare and Prof. Vaithianathan co-authored papers.

That doesn’t mean Prof. Dare can’t be objective.  But if Allegheny County is going to commission an ethics review and tout it as independent, surely among all the world’s academicians the county could have found two with  no ties to the authors of the model being reviewed, thereby ensuring there would not be even the appearance of conflict-of-interest.

Unless, of course, the county was afraid that such a review wouldn’t tell county officials what they wanted to hear.

That’s probably why, while looking for scholars in New Zealand, they decided not to ask Prof. Emily Keddell of the University of Otago.  She’s already done a review of some of Prof. Vaithianathan’s work – and it’s not nearly as favorable as the one written by Vaithianathan’s co-author.

As for the review itself, it’s a startlingly superficial document, a nine-page once-over-lightly that starts on page 44 of this document.  Citations are few and far between – and they are limited to papers written by either the designers of AFST or Prof. Dare himself.

A key caveat

But even so, as I noted in the Youth Today column, the review includes one important caveat:
AFST is said to be ethical in part because it is used only after a call to a child protective services hotline alleging abuse and neglect.  The review specifically questioned whether AFST would be ethical if applied to all children at birth.  As the review states:

[The issue of informed consent] is one of a number of points at which we think that it is ethically significant that the AFST will provide risk assessment in response to a call to the call center, rather than at the birth of every child. In the latter case there is no independent reason to think there are grounds to override default assumptions around consent. The fact there has been a call, however, provides at least some grounds to think that further inquiry is warranted in a particular case. [Emphasis added.]

But that might not be the case for long. As I noted in my column for Youth Today, in her brilliant book, Automating Inequality, Prof. Virginia Eubanks reports

that the county is, at a minimum, considering introducing “’a second predictive model …[that] would be run on a daily or weekly basis on all babies born in Allegheny County the prior day or week,’” according to a September 2017 email from [Cherna’s deputy, Erin] Dalton.”  Such a model already exists – indeed it’s one of the models the designers of AFST proposed to the county in the first place.
So if the county starts branding every infant with a scarlet number at birth, a number that will even affect the number assigned to their children and grandchildren, is that model inherently unethical? 

I’ve known Marc Cherna for more than 20 years. When I lived in Allegheny County I served on a  screening committee that unanimously recommended him, and one other candidate, as finalists for the job of running the county child welfare system. We were right. He has an outstanding record for safely keeping families together. 

I’ve included Pittsburgh on NCCPR’s list of ways to do child welfare right, and referred journalists from everywhere from CNN, in 2002, to the Arizona Daily Star, just this year, to Pittsburgh to examine the county’s success. (Cherna still regularly quotes what I told CNN all those years ago.)  When Cherna says he wants to use analytics in the right ways for the right reasons, I believe him. 

But that’s not enough. And touting a stacked-deck ethics review is particularly disappointing. Again, as I wrote in Youth Today:

Cherna promises that the scarlet numbers under any such system will be used only to find the families most in need of help. … But  what about Cherna’s successor, and his successor’s successor?  Any system that depends for success on the benevolence of a single leader with near absolute power is too dangerous for a free society.

NCCPR’s full analysis of the role of predictive analytics in child welfare is available here. 

Wednesday, March 28, 2018

NCCPR in Youth Today on Pittsburgh child welfare's "Scarlet Number"

The way child welfare misuses big data would make Cambridge Analytica blush

One massive leak of middle-class Americans’ data seems to have the whole world in an uproar.

A firm known as Cambridge Analytica allegedly improperly obtained personal information given to Facebook by 50 million people. Then, according to The New York Times, “The firm, which was tied to President Trump’s 2016 campaign, used the data to target messages to voters.”
The data were offered to Facebook voluntarily. Now that Facebook apparently failed to keep it secure, some Americans are exercising their right to delete their Facebook accounts. Others are exercising their right to share less information on Facebook.
But if this one leak causes so much outrage — outrage that is entirely justified — can you imagine what would happen if, say, people were forced to surrender vast amounts of personal data and a big government agency could use those data to investigate them and even take away their children?
Why if that ever happened the outrage would be — well, actually, it would be almost nonexistent.
Because it’s already happening in Allegheny County (Pittsburgh), Pennsylvania.

Sunday, March 25, 2018

Child welfare in Arizona: Good journalism, and a bad bill

I have previously noted the outstanding reporting done by the Arizona Daily Star on child welfare.  A comprehensive three-part series looked at how to fix child welfare in that state – and what they found turns out to be how to fix child welfare everywhere.

The Star is promising to continue following the issue, starting with this excellent editorial.

But not everyone has gotten the message. There is a bill pending in the Arizona Legislature that would set back reform efforts that have barely begun.  The latest version of the bill is not as bad as the original, but it’s still harmful.  NCCPR has an op-ed column in the Star, and an in-depth analysis of both bills.

Monday, March 19, 2018

Child welfare, foster care and civil liberties: When CPS resorts to blackmail

Reporters in North Carolina exposed the practice of child welfare agencies blackmailing families into giving up all their rights and letting them take away children with no court review at all. 

The only thing unusual about this is that, in North Carolina, it’s illegal. Elsewhere it’s standard operating procedure.

What Associated Press reporters found in
one small county in North Carolina actually
is common all over the United states
I have often written about the lack of due process in proceedings involving child protective services.  CPS agencies can and often do take away children entirely on their own authority. Parents then have to go to court days later to try to get their children back.

At that point, while the CPS agency has had days to make its case, an impoverished parent – and it’s almost always an impoverished parent – if she has a lawyer at all probably met her overwhelmed public defender for the first time five minutes before the hearing.

The standard of proof is not beyond a reasonable doubt as in a criminal proceeding, or even the middle standard, “clear and convincing.” Instead, CPS need merely persuade the judge that it is slightly more likely than not that the child needs to remain in foster care – the same standard used to determine which insurance company pays for a fender-bender.

The judge usually is easy to convince. That’s because the judge knows that he can hold hundreds of children in foster care needlessly and while this will do enormous harm to the children, the judge is safe. Return one child to a home and have something go wrong and the judge’s career could be over.

Resorting to blackmail

But in many cases across the country, even this doesn’t stack the deck enough to suit CPS agencies.  So they’ve resorted to something else: blackmail, typically using a parent’s own extended family as bait.

They don’t call it that, of course. Usually it goes by a term such as “safety plan” or “parental child safety placement” or “custody and visitation agreements.”

Here’s how it works: The child protective services worker says to a parent: Sign this document allowing us to place the child with an extended family member or we’ll take the child on the spot and place the child with total strangers. (If that’s not enough, they might ratchet up the threat, saying they will proceed immediately to termination of parental rights.)

CPS then argues, with a straight face, that the placement was voluntary – the family chose to give up any rights they may have to a lawyer and court review.  In fact, these placements are about as voluntary as a mugger sticking a gun in your face, saying “give me your money” and then telling the judge “I didn’t mug the guy, he gave me the money.”

What the AP reporters found

All of this brings me to what two enterprising Associated Press reporters, Mitch Weiss and Holbrook Mohr found in Cherokee County, N.C.  But let them tell you:

When Brian Hogan got a call that his wife had suffered a massive heart attack, he knew he had to get to the hospital fast. So Hogan asked his neighbor to take care of his 10-year-old daughter, then headed 60 miles east to the intensive care unit in Asheville, North Carolina.
What happened next would eventually expose a practice by a child welfare agency that illegally removed potentially hundreds of children from their homes in this poverty-stricken mountain community …
Hogan said the Cherokee County Department of Social Services threatened to throw him in jail, place his child in foster care or give his daughter to another family for adoption if he didn’t sign a “custody and visitation agreement,” known as a CVA.
“They gave me no choice,” said Hogan, 38, who told AP that child-welfare workers wanted to remove his daughter because they believed he placed the girl in an “unclean” home while he was caring for his hospitalized wife.

AP found that the county “did the same thing with dozens, possibly hundreds, of other parents…”

The extent of the blackmail

But that’s just the tip of a very large iceberg.

In North Carolina, individual counties run child welfare with supervision by the state. To its great credit, the state Department of Health and Human Services says what Cherokee County did was illegal and ordered counties not to do it.  A judge also ruled the practice illegal.  And, in a follow-up story, AP reports that the state is going to take over the entire Cherokee County child welfare system.

But it’s a different story in much of the rest of the country.  One reason we don’t know how many blackmail placements exist is because states often don’t report them to the federal government as entries into foster care – even though federal regulations make clear they should be counted.

But by one estimate, on any given day, there probably are 300,000 children trapped in a foster-care Twilight Zone because of blackmail placements across the country – that’s over and above the more than 400,000 children states admit to holding in foster care.

In Texas nearly two-thirds of entries into foster care probably are blackmail placements.

In Illinois, blackmail placements have been the subject of repeated lawsuits that lead to settlements. Then the child welfare agency violates the settlements.

So to really understand the harm done to children by blackmail placements, take the pain inflicted on Brian Hogan’s daughter and multiply it – hundreds of thousands of times.

Sunday, March 18, 2018

Wealth, power, and child welfare: NCCPR in Youth Today on lifestyles of the rich and - neglectful?

If you're poor it's "endangering the welfare of a child." If you're rich it's a story for a glossy travel magazine!  In Youth Today I discuss why we need to apply the "Conde Nast standard" to poor people.

Read the column here.

Monday, March 12, 2018

Alabama child welfare reforms hold lessons for everyone - but especially, at the moment, Oregon, Iowa and Indiana

The Arizona Daily Star is in the middle of publishing the results from a year-long investigation into how to fix the child welfare system in that state.  As I’ve noted before, the answers are relevant to every state.  Right at the moment, the answers may be especially relevant for three states (besides Arizona, of course): Oregon, Iowa, and Indiana.

For part two in the series, the Star spent a lot of time on the reforms that have turned one of the last   
places most people might expect - Alabama - into, relatively speaking, a national model. (See especially the first three stories.)

The Star isn’t the first newspaper outside Alabama to take a close look at the Alabama reforms.  The first that I know of was the Springfield (Missouri) News-Leader. They did outstanding work in 2003 – but, unfortunately, like most Gannett papers, when they switched to a common website format they lost or hid their older stories so they’re no longer available online.

But a New York Times examination of those reforms is still available.

Now the Star stories bring things up to date.

It started with a lawsuit

The reforms began with a lawsuit brought by the Bazelon Center for Mental Health Law and the Alabama Disabilities Advocacy Program.  (The Bazelon Center’s Legal Director, Ira Burnim, also is a member of NCCPR’s volunteer Board of Directors.)  The Star stories show that, even though the system is no longer under court supervision, and though there has been some backsliding, the reforms remain in place. And, precisely because Alabama is a leader in family preservation, it’s also a leader in keeping children safe.

The stories are especially relevant to Oregon, Indiana and Iowa for several reasons:

● Oregon has been an extreme outlier for decades when it comes to taking away children. Iowa is a more extreme outlier. Indiana is an even more extreme outlier.

● Oregon remains deep in denial about the harm that does to children. Iowa is in even deeper denial. Indiana remains to be seen.

Perhaps most important for Indiana and Iowa:

● The Alabama lawsuit actually was welcomed by the leader of that state’s child welfare agency at the time, Paul Vincent. Vincent’s leadership was crucial to getting the reform effort underway and keeping it on the right track.  He’s got one of the best track records in the field.

Now Vincent runs the Child Welfare Policy and Practice Group, which helps other states and localities fix their child welfare systems. Two of the states CWG is working with right now are Iowa and Indiana.  They’ve already had some things to say about Iowa - and it's high rate of taking away children.  And they’ve already zeroed in on Indiana’s high rate of removal as one of the key “challenges” facing the system there.

Perhaps most important for Oregon:

● Oregon was, at one point, the state most likely to follow Alabama’s lead.  A lawsuit settlement in the 1990s called on Oregon to institute similar reforms. But unlike the Alabama suit, there was no real enforcement mechanism, no independent court-appointed monitor. And, unlike in Alabama, the child welfare agency in Oregon was not receptive to change.

As usual, the Oregon Department of Human Services proved to be the place where child welfare reform goes to die.  (Another NCCPR Board Member, clinical Psychologist Marty Beyer, who worked on the Alabama effort also was involved in the attempt to fix child welfare in Oregon. She discusses it in the epilogue to this story from the Salem Statesman Journal and Oregon Public Broadcasting.)

Spending smarter beats just spending more

The Alabama reforms also teach us something else. You can’t fix a child welfare system just by throwing money – or caseworkers – at it.

Yes, as a result of the lawsuit, Alabama spent more – but it also spent smarter.  Here’s how the Star explains it:

Child welfare leaders spread the word, through community meetings, that the state was shifting contracting dollars away from expensive services like group homes and residential treatment centers, in favor of less costly and more effective programs like therapeutic foster care and in-home services. 
         Providers had a choice: Evolve or risk going out of business.

In other words, while the Indiana child welfare agency was cozying up to private providers, Alabama was standing up to them.

As of 2014, the most recent year for which data are available, Alabama was spending on child welfare at one of the lowest rates in the country, while Oregon, Indiana and Iowa all were  spending at among the highest.

That doesn’t mean the answer is to cut spending. On the contrary, Alabama is good, compared to most of the rest of the country, but it could be better if it were willing to spend more.  But at least Alabama spends smart.

Oregon, Indiana and Iowa spend dumb, throwing away money on needless foster care and institutionalization – “services” that are both worse for children and more expensive than safe, proven alternatives to foster care.

As I’ve said many times before, I’m a tax-and-spend liberal and proud of it. But anyone who says the answer to these states’ child welfare problems is just to spend more is ignoring reality.

Instead, let’s imagine for a moment what kind of child welfare system you would get if you combined the amount that Oregon or Indiana or Iowa spends with Alabama’s priorities for how to spend it.

You’d probably get the best child welfare system in America.

Friday, March 2, 2018

Arizona Daily Star takes on how to fix foster care

It's unusual to see two large newspapers in one state take on massive projects about child welfare. It's even more unusual to see both newspapers do it really well.  Last year, the Arizona Republic did excellent stories, such as this one and this one. about decades of failure in Arizona child welfare.

Now the Arizona Daily Star has published the first parts of a three-week series on how to fix child welfare in Arizona (which turns out to be how to fix child welfare everywhere).  The stories are an excellent guide to reform.  This video offers an overview: