State claims there’s zero high-risk AI in California government—despite ample evidence to the contrary

by Khari Johnson

This story was originally published by CalMatters. Sign up for their newsletters.

California uses algorithms to predict whether incarcerated people will commit crimes again. It has used predictive technology to deny 600,000 people unemployment benefits. Nonetheless, state administrators have concluded that not a single agency uses high-risk forms of automated decisionmaking technology.

Click Here to Subscribe to the California Political Review

 That’s according to a report the California Department of Technology provided to CalMatters after surveying nearly 200 state entities. The agencies are required by  legislation signed into law in 2023 to report annually if they use high-risk automated systems that can make decisions about people’s lives. “High-risk” means any system that can assist or replace human decisionmakers when it comes to encounters with the criminal justice system or whether people get access to housing, education, employment, credit and health care. 

The California Department of Technology doesn’t know which algorithms state agencies use today and only reported what agencies told them, state Chief Technology Officer Jonathan Porat told CalMatters. When asked if the employment or corrections department algorithms qualify, Porat said it’s up to agencies to interpret the law.

“I only know what they report back up to us, because even if they have the contract… we don’t know how or if they’re using it, so we rely on those departments to accurately report that information up,” he said.

The agencies, which were required to submit answers by the end of August 2024, reported high-risk automated systems used within the past year. Had any found any high-risk systems, they were required to report the kind of personal data these systems use to make decisions about people, and the steps they take to reduce the likelihood of that usage resulting in discrimination, bias, or unfair outcome. 

Some automated systems used by state agencies raise questions about how risk is being defined. The California Department of Corrections and Rehabilitation, for example, assigns recidivism scores to the vast majority of inmates to determine their needs when they enter and leave prison. One algorithm it uses, COMPAS, has a documented history of racial bias, but the corrections department told the Department of Technology it uses no high-risk automation. 

Click here to read the full article in CalMatters