Racist robots? NYC now requires 'bias audit' when AI used for job recruiting

City requirement for a third-party 'bias audit' aims to prevent AI-related discrimination against minority candidates

A new law in New York City is mandating that businesses using artificial intelligence programs to find employees must demonstrate the process is free from racism and sexism.

Under legislation that went into effect Wednesday, a third party is required to conduct a "bias audit" on any automatic employment decision tool (AEDT), which increasing numbers of companies are using to search for potential hires and eliminate candidates.

The bill also requires that candidates or employees who live in New York City be notified about the use of AEDT for hiring or promotions and be notified about the job qualifications and characteristics the tool is using. 

Companies are also required to publish the results of the bias audit if they use AI software for such purposes.

MINORITY GROUPS SOUND ALARM ON AI, URGE FEDS TO PROTECT ‘EQUITY AND CIVIL RIGHTS’

AI Italy illusttration

An artificial intelligence illustration from May 4, 2023. (REUTERS/Dado Ruvic / Reuters Photos)

Julia Stoyanovich, a computer science professor at New York University who serves on the city's Automatic Decisions Systems Task Force, lauded the new rules but expressed concern that more needs to be done, according to NBC News.

"It’s an important start but still very limited," she said. "First of all, I’m really glad the law is on the books, that there are rules now and we’re going to start enforcing them.

"But there are also lots of gaps. So, for example, the bias audit is very limited in terms of categories. We don’t look at age-based discrimination, for example, which in hiring is a huge deal, or disabilities."

BALLOONING AI-DRIVEN FACIAL RECOGNITION INDUSTRY SPARKS CONCERN OVER BIAS, PRIVACY: ‘YOU ARE BEING IDENTIFIED'

Empire State Building at center of NYC skyline in this photo

The New York City law governing AI hiring practices is considered the first of its kind in the world. (Gary Hershorn/Getty Images / Getty Images)

AI's potential to discriminate has been a growing concern among some watchdog groups.

The National Artificial intelligence Advisory Committee, an interagency group led by the Commerce Department, held a public hearing online last month aimed at informing lawmakers about how the government can best manage the use of AI.

Witnesses told panelists that bias and discrimination are the biggest fears among those they represent.

CLICK HERE TO GET THE FOX NEWS APP

Sam Altman in congressional hearing

Sam Altman, CEO and co-founder of OpenAI, speaks during a Senate Judiciary Subcommittee hearing in Washington, D.C., May 16, 2023. (Eric Lee/Bloomberg via Getty Images / Getty Images)

The advisory committee held multiple hearings in recent weeks in an effort to gain input for the Biden administration's goal of discerning how best to regulate AI. 

The White House promised last month that federal rules for AI are coming soon, and Congress has also looked into potential regulation of the burgeoning technology.

Fox News Digital's Peter Kasperowicz contributed to this report.