The State Senate this week passed Senate Bill 1220, by Senator Monique Limón (D-Santa Barbara), aimed at regulating the use of artificial intelligence (AI) and automated decision systems (ADS) in public benefit program call centers. The bill seeks to ensure that AI does not replace core job functions of human workers in state and local agencies.

SB 1220 prohibits agencies from contracting with out-of-state call centers and restricts the use of AI that automates essential job functions. The bill mandates that agencies provide access to human call center workers upon request and develop impact assessment reports detailing the use of AI or ADS.

The proposed legislation responds to growing concerns about the potential negative impacts of AI on the workforce. The author’s office highlighted instances where AI replacement of human workers led to detrimental outcomes, such as the National Eating Disorder Association’s chatbot giving harmful advice and Michigan’s flawed unemployment insurance AI system wrongly charging recipients with fraud.

According to the California Labor Federation, the bill is crucial to maintaining the quality of public services. “With the high volume of callers in need of support, public benefit programs such as Medi-Cal and unemployment insurance assistance require a support staff in California that understands the issues facing the state’s most vulnerable populations,” the Federation said. TechEquity Collaborative echoed this sentiment, emphasizing that AI cannot match the empathy and effectiveness provided by human workers.

Meanwhile, the California Department of Tax and Fee Administration (CDTFA) has launched a Request for Innovative Ideas (RFI2) to modernize its call center operations with GenAI. This initiative aims to leverage GenAI to swiftly search reference materials and provide accurate responses to taxpayers. CDTFA Director Nick Maduros emphasized the complexity of tax regulations and the department’s commitment to assisting taxpayers. “Our goal is to use GenAI to make it easier for taxpayers to pay their taxes on time, thereby efficiently funding essential services for all Californians,” Maduros said.

The CDTFA initiative is part of a broader effort under Governor Newsom’s executive order from last fall.

The California Government Operations Agency (GovOps) announced partnerships with five vendors from its initial round of RFI2 solicitations. These vendors will develop proofs of concept using GenAI technologies on platforms like Amazon (AWS) Cloud, Google Cloud, Microsoft Azure, and ServiceNow.