Inside Story

Machine learning

Does the federal government’s heavily qualified apology for the robodebt fiasco suggest that more trouble is on the way?

Mike Steketee 19 June 2020 1472 words

Repentant? Prime minister Scott Morrison in parliament on the day he apologised to some victims of robodebt. Lukas Coch/AAP Image

Back in 2015 it was billed as “one of the world’s largest transformations of a social welfare system.” Tony Abbott’s social services minister, Scott Morrison, declared that the replacement of the thirty-year-old computer system responsible for $100 billion in payments to 7.3 million people would “ensure more government systems are talking to each other, lessening the compliance burden on individuals, employers and service providers.”

The simplified system, said Morrison, “will make it easier for people to comply with requirements and spend more time searching for jobs, which is the key element of welfare reform.” Moreover, “this investment will also help us stop the rorts by giving our welfare cops the tools they need on the beat to collar those who are stealing from taxpayers by seeking to defraud the system.”

Those were the days. The government was in its first term and Morrison was an ambitious, gung-ho minister with a fondness for the police and military analogies that had stood him in such good stead in party circles when he was overseeing Operation Sovereign Borders.

Robodebt started the same year — though it didn’t acquire its pejorative nickname until later — and it reflected Morrison’s punitive approach. The increasing potential of automated data-matching — in Morrison’s words, the fact that more government systems were talking to each other — was for the first time making it cost-effective to pursue overpayments of welfare benefits.

The problem was that the systems were talking different languages. Crucially, the tax office was supplying income figures that couldn’t be matched to the benefits people were receiving. The government went ahead anyway, putting the onus on welfare recipients to prove the figures wrong, and causing real hardship and trauma, including reported suicides, among vulnerable people.

A program that had previously reviewed 20,000 cases a year conducted more than 900,000 reviews in the four years to the end of August last year, with 734,000 identified as having been overpaid. Except that many of them hadn’t been. Most of the reviews looked at benefits paid under Newstart and Youth Allowance, though they were eventually extended to other payments, including the age pension, the disability support pension, Austudy and the parenting payment.

Subsequent events have culminated in the government’s promise to pay back $721 million to 373,000 Australians for 470,000 illegally recovered and often non-existing debts. With almost two-thirds of the debts having been reversed, the government’s early depiction of the sunlit uplands looks particularly ironic.

Speaking on the same day as Morrison in 2015, human services minister Marise Payne was just as effusive about how data analytics would inform policy decisions. “Improvements to real-time data sharing between agencies will mean that, with customer consent, their information won’t have to be provided twice,” she said. “Improved data sharing will also significantly increase the government’s ability to detect and prevent fraud and non-compliance. This means customers who [simply] fail to update their details with us will be less likely to have to repay large debts and those who wilfully act to defraud taxpayers will be caught much more quickly.”

When applied to robodebt, this enumeration of the new system’s benefits turned out to be wrong or misleading in every detail. The data shared was not real-time: annual income figures from the tax office were averaged out to compare them with fortnightly benefit payments, producing many wrong assessments. Customers were not asked for their consent: they were pursued to provide information the government already had or was responsible for obtaining.

On top of all that, the relatively few perpetrators of welfare fraud are also being repaid their robodebt money because the government finally had to concede that the whole scheme broke the law. That admission came more than two years after Terry Carney made exactly that point as a member of the Administrative Appeals Tribunal. Carney, now a professor of law, has since described robodebt as “illegal, immoral and ill-constructed.”

The government is continuing with “online compliance intervention” — robodebt’s official title — but it will no longer use income averaging and it has promised other “refinements.” It has yet to give a clear commitment not to try to recover some of the same debts by different means. As Morrison put it earlier this month, the decision to refund the money “doesn’t mean those debts don’t exist. It just means that they cannot be raised solely on the basis of using income averaging.”

The government is also forging ahead with upgrading and increasingly automating its welfare payments system. Properly designed — and that’s a big caveat in the light of recent experience — the modernisation should make it easier for people to claim their correct benefits and easier for the government to make sure that they are paid the correct amounts.

The Welfare Payments Infrastructure Transformation project — the one described as among the biggest in the world five years ago — has another two years to run. Services Australia, formerly the Department of Human Services, claims that it has made practical improvements, including introducing prefilled claim forms using information already available to the government, enabling claims via mobile devices and verbally, and speeding up claims processing for some students and the unemployed. It boasts that the number of questions on the online claim form for students and trainees has been reduced from 117 to thirty-seven.

But, as the robodebt experience demonstrates, many of the new system’s claimed advantages are double-edged. The “digital assistants” introduced to answer customer questions, for example, mean less human interaction, which is reflected in staffing reductions that have already taken place. But most of us already know just how frustrating it can be dealing with digital assistants.

Similarly, analytics will be used to “proactively provide support to those who need it.” And also to take it away? A new “payment utility platform” promises same-day payments but also “simpler debt repayment processes.” In the wake of robodebt, how many people will be keen to use it?

A new “entitlement calculation engine” will determine payment levels. And if a person wants to challenge the calculation? Presumably they will be expected to sort it out with one of the new digital assistants.

It is one thing to increase automation for people well versed in the ways of the digital economy, but it is entirely another to impose it on vulnerable people who may or may not be familiar with online processing. As Australian Council of Social Service chief executive Cassandra Goldie told Inside Story this week, “Robodebt fundamentally failed because we stripped out the ‘human’ in human services. Instead it was up to individuals to try and prove their innocence in a David versus Goliath battle with automation.” For Goldie, humans must have a role in decisions about essential services like income support and “we must build in ways to enable people to easily correct decisions where mistakes have been made.”

In the end, it is how the system is designed that will determine the nature of the experience for its users and how much emphasis is placed on ferreting out suspected wrong claims.

Judged by the guidance from the top, the bias will be towards limiting entitlements. In 2018 the government introduced ParentsNext to add another layer of obligations to those already imposed on parents on low incomes who receive parenting payments. According to a Senate committee report, one in five parents had their payments suspended for missing appointments or failing to participate in “pre-employment” programs under this new scheme.

The social services minister who promised in 2015 to use the modernised welfare system to sool the “welfare cops” on to beneficiaries is now the prime minister who says the refunded debts still exist.

Having initially refused to apologise for robodebt for fear of legal liability, Morrison thought better of it and, in response to a pointed question from Bill Shorten, assured parliament of his deep regret for any hardship caused. But government services minister Stuart Robert immediately added that 939,000 Australians had $5 billion worth of debt “that the government lawfully has to collect across a whole range of programs.” Message? We’re still coming after you.

In her 2017 book Automating Inequality, American political scientist Virginia Eubanks describes how digital eligibility systems, matching algorithms and other tools have been used in the United States to drastically cut the welfare rolls. “At their worst these systems act as empathy overrides, allowing us to turn away from the most pressing problem of our age: the life- and soul-threatening legacy of institutional racism, classism and sexism in America,” she said in a speech last year. “They allow us to ignore our moral responsibility by replacing the messiness of human relationships with the predictable charms of systems engineering.”

It doesn’t need to be that way. But governments will have to resist the temptation to succumb to the convenience of allowing machines to make decisions that require judgement, compassion and humanity. •