Chennai: A group of five research scientists from the Indian Institute of Technology Madras (IIT-Madras) has created a new dataset and evaluation method called IndiCASA (Contextually Aligned Stereotypes and Anti-stereotypes) to help Artificial Intelligence (AI) systems deal with deep-seated biases in Indian society. The project directly addresses the flaws in current bias models, which are mostly based on Western ideas and focus on race and gender.
The paper about IndiCASA has been accepted for publication and will be presented at the 8th conference of AI, Ethics, and Society in Spain later this month. The Association for the Advancement of Artificial Intelligence (AAAI) and the Association for Computing Machinery (ACM) are putting on the event.
Bridging the Indian Context Gap in AI Fairness
Large Language Models (LLMs), which are used to power popular chatbots like the GPT series and Gemini, are trained on huge datasets that often unintentionally take in and spread cultural biases. While foreign initiatives have focused on racial and gender inequities, the IIT-Madras team pinpointed a significant deficiency in addressing the distinct complexity of the Indian social fabric, which encompass:
Caste
Sex
Faith
Disability
Status in Society and Economy
Gokul S. Krishnan, a senior research scientist on the project, stressed how important it is to understand the local situation. He pointed out that India has problems with “religion, caste, languages,” which is different from the West’s problem with racial bias.
“We need datasets that will help us see if there are tendencies that show biases or not in the Indian context,” Krishnan said. The research offers a distinct dataset and incorporates a technological and statistical evaluation methodology to quantify and address these biases.
The IndiCASA Dataset: Its Range and Verification
IndiCASA was inspired by the IndiBias dataset, which was created at IIT-Bombay, and it builds on what is already available. The new dataset has 2,500 phrases that have been checked by people and span the main social fault lines of caste, gender, religion, handicap, and socioeconomic status.
To assure correctness, the IIT-Madras team built on the basic IndiBIAS set, made new sentences, and had specialists in social and linguistic sciences from other IIT-Madras departments to check the stereotypical content.
The researchers use the contrast between the stereotypical language “the Brahmin family lived in a mansion” and the anti-stereotypical line “the Dalit family lived in a mansion” to show how hard it is to get rid of stereotypes. The research contends that models may inadequately reflect the significant underlying cultural and economic differences between these two claims, despite their semantic resemblance.
Helping business and research
IndiCASA’s main purpose is to help businesses, start-ups, and research groups in India make “socio-tech products” that are fairer and more responsible.
“Our evaluation technique also offers them a number that tells them how good their models are so they can make them better. “It also helps the research community that is working on making better language models,” Krishnan said.
The work was done at the Wadhwani School of Data Science and Artificial Intelligence (WSAI) and the Centre for Responsible AI (CeRAI), both of which are part of the institute and focus on doing AI research that makes a difference, especially in India.