Unveiling Bias in AI Insights

When I started using generative AI early on with tools like ChatGPT, I noticed biases in the generated content. At first, I thought it was just me, but then I kept encountering more instances of bias. For example, I put in a prompt for a radio script aimed at a Black audience, and it responded with stereotypes like “Yo fam, we’re gonna get some grub.” It was like something out of an old movie. I even generated an image of Maya Angelou, and it produced an image of an elderly white woman. These experiences led us to conduct a study on bias in generative AI, which I believe is one of the first studies by an agency on this topic.

**(03:13)** To share a bit more about my background, I was born in New Orleans and grew up in Alabama, where people rarely talked openly about issues like racism. I later moved to Boston and lived in an area where Black people rarely live, and I experienced subtle racism even within progressive circles. All these experiences heightened my awareness of biases in society and technology.

On a personal note, last year, while waiting in a long line at the Atlanta airport, I read a McKinsey study on the racial wealth gap and how generative AI might widen it. The study resonated with me; it highlighted that if we don’t address this, the racial wealth gap could increase by $43 billion a year. This inspired our generative AI survey. I’m going to show a video by Joy Buolamwini, who has spoken on this topic before AI became mainstream. Her work addresses biases in AI, and this video from 2018 underscores how long this issue has existed.

[Video plays with Joy Buolamwini’s poem on AI bias]

**(09:33)** Katherine: Any thoughts on the video?

David: It’s shocking to see such blatant issues, especially given this was six years ago. Have you tested current AI models to see if this has improved?

Katherine: I haven’t tested it recently, but I do see similar issues. For instance, with Google Gemini, we’ve seen over-corrections, like depicting Black men as the founding fathers or showing an Asian woman as the Pope. It feels like they’re trying to diversify representation, but it misses the mark by altering historical accuracy. 

Attendee 1: Isn’t this largely due to a lack of diverse training data?

Katherine: Exactly. The data input is still very limited in terms of diversity. The majority of the data comes from people with similar backgrounds, often white men, so their biases can seep into the models unconsciously.

David: Have you seen any models that are intentionally trained with a more diverse data set?

Katherine: I’ve found a few that focus on specific communities, like a ChatGPT model geared toward Black users, but they’re rare. It shouldn’t be limited to one group—we need all models to be inclusive.

David: I’d be curious if AI models in countries like China or India, with more diverse populations, are handling this any differently.

Katherine: That’s a great point. However, without diverse teams creating these models, even in different regions, it’s likely they’ll still reflect biases.

**(15:36)** Katherine: In our survey, we found that awareness of generative AI often correlates with awareness of bias. Most respondents who are familiar with AI expressed concerns about racism, sexism, and classism. On classism, AI requires internet access, technical knowledge, and tools that not everyone has. Generative AI can inadvertently widen the gap between those who have these resources and those who don’t.

Younger people, particularly those aged 18-29, expressed strong concerns about discrimination and bias in generative AI, which aligns with how younger generations are more engaged with digital content and social justice issues. Interestingly, respondents over 60 also expressed concerns, although likely for different reasons, like fairness and ethics.

**(21:07)** We also asked respondents if they thought tech companies prioritize diversity, equity, and inclusion (DEI) when creating generative AI tools. Opinions were mixed, but a significant portion believed companies don’t focus enough on this. Respondents indicated that if companies were more transparent about addressing biases, it would make them more likely to use generative AI tools.

David: What dangers do respondents see in generative AI?

Katherine: The primary concerns were reinforcement of stereotypes, discriminatory content generation, and perpetuation of existing biases. Legal and ethical liability were also concerns, reflecting the increasing visibility of privacy and data misuse in AI discussions.

Respondents want transparency from tech companies. For example, after George Floyd’s death, many companies made donations to social causes, but there’s been little follow-through. Consumers are demanding more accountability and transparency from companies on issues like bias and diversity.

**(24:53)** Tech companies need to establish minimum standards and undergo regular bias audits, similar to how they approach privacy and security. Without a clear framework, there’s a risk that companies will continue to do the bare minimum. Education is another key piece, as many users might not even recognize when AI-generated content is biased.

David: What do you recommend for our AI Marketers Guild members, many of whom influence brands and technologies?

Katherine: Start by educating those in your networks about AI biases. Raising awareness can lead to more demand for unbiased models. Also, respond to issues when you see them. For example, if an AI tool only shows one perspective on a controversial topic, bring it up with the developers.

Attendee 2: Generative AI often relies on existing data, which means it can’t easily generate something outside the patterns it knows. For example, it struggles to create novel historical scenarios because it’s limited to what’s already been recorded or depicted.

Katherine: That’s a great observation. AI’s current limitations make it even more important to have diverse data sources and people guiding its development.

Lisa: As a researcher, I’ve noticed it takes significant effort to find sources that reflect true diversity, especially for historical events. I think tech companies could help by encouraging users to request unbiased or diverse perspectives in their prompts.

Katherine: Absolutely. Companies could suggest prompts that encourage users to seek unbiased answers.

**(33:10)** Attendee 3: Many issues arise because of a lack of representation at the table when these tools are developed. Diverse perspectives are critical in developing models that better reflect different communities.

Katherine: Definitely. If you don’t have a range of voices involved in developing the technology, the tools won’t reflect the diversity of users. Even tech giants like Amazon rely on internal affinity groups for DEI input, but they’re often unpaid or added on top of regular roles.

David: Are there initiatives for adding tax incentives for tech companies addressing biases in AI?

Katherine: I haven’t seen that yet, but it’s a great idea. Tax credits could encourage companies to put resources into reducing bias, which would benefit both the companies and the public.

Jay-Z: When recruiting speakers, I found it hard to find recordings of women experts. Where’s the incentive to gather diverse data for AI?

Katherine: Incentivizing data collection from underrepresented groups could help, and tech companies could fund these initiatives. Building a certification or board for bias auditing would also be a step forward.

Lisa: Perhaps we could have reputational ratings for AI engines based on their bias levels, similar to a trust pilot. If AI companies know their bias ratings are public, they may be motivated to improve.

Katherine: That’s a great idea—rating models for bias transparency could drive accountability. With DEI roles shrinking, public accountability could be one of the most effective ways to enforce change.

Attendee 4: Is there any progress with educational institutions? AI literacy could be a critical subject, especially given the current lack of algorithmic literacy.

Katherine: Some universities are exploring certifications on AI ethics and DEI, but support is often limited. Professors are taking the lead, but institutional backing remains weak. Expanding AI literacy in schools could make a significant difference in raising awareness.

David: Thank you, Katherine, for your insights. This discussion underscores how much work lies ahead in improving AI, particularly in addressing and mitigating biases. Thank you to everyone who joined and shared insights today.

Katherine: Thank you for having me, and I’m always happy to engage in these conversations. I appreciate the thoughtful questions and hope to continue discussing ways we can make AI more inclusive.