Whats Real in AI Marketing Ethics Trust and Transparency

(00:06) Hello everyone. I'm David Berkowitz and welcome to another edition of AI Insiders by AI Marketers Guild. And we've got an exciting guest today, someone I'm really eager to hear from, Dr. Cecilia Dones. Uh just a a tremendous background as a chief data officer among uh uh other esteemed roles. and we were geeking out on what's going on in the AI space. We met through uh initially another community.

### Community background and welcome to Cecilia

(00:33) I've been a longtime member of research wonks, a big fan of that for for folks who are are deep in the data and analytics space. And about 90% of the posts there are over my head, which is why it's fun being in some rooms where where where there are such smart people so deep in the field that uh uh I can just try to learn a few things from. But uh uh Dr. Don, Cecilia, welcome.

### Appreciation and audience context

(00:58) Good to have you here. Thank you. Thank you. And thank you for everyone for joining. I know it's the middle of the week. It's also lunchtime for some of us who are on the East Coast and so the dedication and attention is much much appreciated. Yeah. So would it be helpful unless everyone LinkedIn stalked me which could be a thing. You're definitely okay to do that. Um so a little bit about me.

### Cecilia’s background: qualitative research and storytelling with data

(01:25) Um, so I am a qual researcher who spent my entire career being fascinated in trying to tell stories about people using data and so everything I've ever done is following that curiosity. Um, the good part of my industry experience was focused in marketing and ad. So I have done my tour duty with uh, WPP and Pubis and um, I've also done a tour duty on the brand side.

### Brand-side experience and doctoral research focus on trust

(01:58) Um so some brands that may be familiar L'Oreal, LVMH and more recently um as being part of the data game that we all do. Um I finished uh my doctoral dissertation um and what I was focusing on was trust in authenticity signals in technology mediated interpersonal communications which is the longest way of saying I have trust issues with the internet um and I don't know what to believe anymore and so I figure maybe some research in that area might be helpful for others as well. Mhm.

### Teaching and AI ethics work (including K–12)

(02:29) Um what I'm doing recently uh I'm I do teach uh so I teach primarily AI ethics. Uh more recently I have been teaching K through 12 AI ethics which is very interesting. So trying to speak about AI ethics and responsible AI to middle schoolers. Oh so many things I've learned.

### AI literacy in youth and upcoming NYU role

(02:52) Um but it's very interesting to see how organizations are really investing in that kind of AI literacy uh especially in the youth. Um I teach that uh I will be placed uh in NYU uh in the spring uh teaching emerging technologies there. Uh so I will be continuing that and I hang out in the responsible AI AI ethics space um writing uh doing a little bit of research and consulting organizations that are trying to do this AI thing a little bit better.

### Topic pivot: fake news, artificial intimacy, and marketing lens

(03:24) So in transparency, David and I uh collaborated or conspired to have this conversation after he was back from holiday. Um because I had mentioned AI fake oh yes fake well first of all fake news and then AI fake news that makes it a little bit more complicated. Uh but David and I actually wanted to postpone this conversation till after he was back from holiday because I had mentioned my secondary research area which is AI um artificial intimacy.

### Agentic ecosystems and collapsing marketing funnel

(03:53) So when we start to have relationships that are a little bit interesting uh with machines and what does that mean from a psychological standpoint at the individual level and more broadly at the societal level. Um, but keeping it in marketing land, I was like, "Okay, I totally have a talk. I'll have slides, all the things, all the things." And then the news.

### OpenAI x Etsy and the rise of AI-native influencers (Tilly Norwood)

(04:16) And when the news happened um about Open AI and what they're doing with Etsy, collapsing the marketing funnel, and I said, "Oh dear, okay, this is interesting. This could be good for consumers or it could be interesting how marketers deal with it." um because there's almost no more any um the journey collapses, intervention points uh are removed from the process and then like within the last 24 hours which totally blew up my whole talk.

### Implications: from human-centric to agent-centric marketing

(04:51) Um does anyone know who Tilly Norwood is? Okay. Yes. Um, some would argue that this is our first uh um worldwide AI actor and apparently um she's looking for an agent. Um and so these two things represent what's happening in the latest wave of technology. Um and so I was thinking about it this morning actually as my presentation was blown up.

### Proposed idea: “AI marketing for AI” when machines sell to machines

(05:22) Us as marketers, our discipline began with oh gosh, we have to listen to our consumers. We have to understand our customers. We we are very human first. We have to be um making sure that there's always an alignment between product and the consumer and their consumer needs. And so we spend a whole bunch of energy trying to understand consumer psychology.

### Technology as actor vs facilitator and near-term outlook

(05:47) However, with these latest developments in technology that's uh changing our ecosystem, we're moving from what I would argue is human centric campaigns where we have personas, we focus on people, hopefully people um to more of an agent centric ecosystem meaning the machines are mediating so much of the process between the brand and the consumer.

### Will “agentic” experiences become ubiquitous?

(06:12) um it forces us to have to think about this ecosystem in a different way. And I will put this idea out there. If anyone wants to write this textbook, I am more than happy to be second author, third author, even an honorable mention. I'm very happy to.

### Cultural context and virtual influencers blurring lines

(06:29) Uh something I was thinking about that we should kind of write and maybe as practitioners it would totally make sense. AI marketing for AI when machines are selling to machines. I think that's it's a little bit clickbaity the title we can work on it. Um but I think that's where the industry is moving forward. Meaning more and more of the interactions especially in virtual and digital spaces will be mediated by these technologies that yes convey help us.

### Audience question: SAG, virtual characters, and what battles to fight

(07:01) It is the medium uh to convey our messages but they also act and that's the slight distinction from maybe previous iterations of technology. um previous iterations of technology was more like facilitating facilitating some of the consumption and so e-commerce that totally changed things for us. We all had to think about our e-commerce websites.

### Distinctions between human and AI in media consumption

(07:19) Um but this one is a little bit different in the sense that there is more agency and so do I think agentic is going to be a thing in 2026? We'll talk about it a lot. Maybe some brands will make mistakes that we can talk about even more. Um is it going to be a ubiquitous consumer experience? Um I'm cautious.

### Trust signals and marketer’s role in preserving trust

(07:38) Uh however, it is something for those of us who are marketing leaders and marketing strategists and have been in the game for a little bit longer. I think we have to actually really do properly think about how do we think of this ecosystem when the consumer journey has collapsed and that when machines on act on behalf of uh consumers itself.

### BIK (Benevolence, Integrity, Competence) trust framework

(08:03) Um, and and I mean the you bringing up Tilly Norwood right now, we have the Screen Actors Guild coming out very strongly against this and and making a stance for human creativity. Um, but it's also funny cuz like uh we also obviously like so much of what we consume, right? They're they're characters. Yeah. uh we're going and yeah we're watching the Marvel movies because of the characters in them.

### Transparency risks with platform-mediated recommendations

(08:36) Uh it's uh it's less important I think for most folks who's actually playing them. Although it's really fun to see Robert Daddy Jr. in his various roles and and whatnot. Um but then uh also you know we have this point where like like if we look at the rise of some of these virtual influencers like Lil Michaela um um that BMW actually did a sponsorship with and uh then like there like Lila for instance getting gigs anyway right like like there there are people paying her it you know its team to go and and run branded campaign. So, it's like like how do we even draw lines? I It's one thing to

### Why transparency matters: source, bias, and paid placement

(09:23) draw a line between like what's human and what's a AI, but when like so much of what we consume anyway is some like Yeah. Uh some CGI created Yeah. character to begin with when consumers are already like following these Yeah. characters that don't exist in real life. Um, yeah. I I mean like like how do we even make these distinctions anymore? And is it like like and I'm even curious your take cuz like I love like human first everything over tech, but like are these actual battles worth fighting? Like like is this like like is SAG going to be our last round of defense here, the Screen Actors Guild,

### Holding trust: benevolence, integrity, competence for marketers

(10:08) or is this just like some lost cause and they're going to be seen as Yeah. fighting some battle that that like that will that ship sailed, right? Like I'm I'm so curious where where your think of this as you see this controversy around Tilly Norwood come up. Yeah, no worries.

### Cultural norms, virtual influencers, and disclosure

(10:34) Um well, one compensation models have changed because business models are changing. That's just a given. uh as dynamic ecosystems change obviously the mechanisms and incentives that we have um to incentivize different behaviors will also change so that's going to happen do I have a crystal ball no um how it will change I don't know what I will say is that um maybe and I could be more clear about this is the two examples the open AI with Etsy and and this Tilly situation what makes it a little bit gives me the is that these platforms and these technologies are removing some of the social signals

### Defining trust in marketing relationships

(11:14) that we would normally depend on when we are trying to build trust. And as marketers, part of our agreement, part of our um the promise or the value we we create for the organization is we hold that trust with the consumer. we ensure that as best as we can um we are continuing to reinforce that trust between the consumer and the brand.

### Source transparency questions in AI-powered shopping

(11:43) Are there uh cases where there are cultures for example this is very common in China where you have virtual influencers that actually do live streaming and people still consume and it doesn't bother them and it's part of the cultural norms and values perfectly fine they are transparent that these are virtual influencers so in the case of chat GPT um and uh Etsy when a consumer starts to say okay I need a new charm for an or a gift or something like that and something pops up as a recommendation. Is it coming from

### Bias, paid placement, and missing signals

(12:20) OpenAI? Is it actually coming is it reading from u um the reviews from Etsy? Where is this recommendation coming from? Is it biased? Is it actually is the brand actually paying for it? We have no more signals for this and and that's where it gets a little bit tricky in the Tilly situation. I I get it. Good for PR.

### Framework recap: benevolence, integrity, competence in practice

(12:43) Um but the lack of transparency that hey she's AI and they were running all these adverts and uh messaging around her. Um yes it does get her in the news but again it's one of those things where it's one of the fundamental components uh regarding trust uh in specific that there needs to be some level of um one's perception and belief that the other person is going to act in such a way that is one beneficial um two um I would argue uh helpful And then three kind of consistent and I can give everyone a little bit of a framework here. Um so uh researchers from many many moons ago

### Applying BIK to AI marketing: what to prove

(13:28) it's called the Bick framework. I'm I'm quite certain some of us are already familiar with it. Um so in building trust and this was done with organizational psychology uh original research. So how do organizations trust each other and then more importantly how do people inside those organizations trust each other? We had to prove benevolence.

### Benevolence vs manipulation and the leader’s role

(13:47) Is this actually going to help the consumer? What is the consumer benefit in AI land? Um, common violations is that what's the distinction between manipulation versus a nudge? And so, as us marketers, when we think about utilizing AI responsibly, there isn't going to be a regulation, sorry.

### Integrity: delivering promises and avoiding AI-washing

(14:12) Uh, or let's say let's not hold our breaths for a federal regulation at least here in the US. Um, so that means at the leadership level, at the organization, we're going to have to make these decisions. So, one, benevolence, two, integrity. Are you going to deliver on your promise? Are you going to surprise? Are you going to uh surprise me in a not so good way? Or are you going to be transparent? So when we use AI for hidden automations, if we have exaggerated claims, um there has been already um uh cases where firms have been fined for exaggerating claims and in the data space we tend to call it AI washing um but in other contexts they

### Competence: reliable execution and guardrails

(14:50) use different words but the idea of uh going beyond u puffery uh this is something that's critically important to make sure that we are reinforcing with our consumers that we our brands have integrity even when we're using technology. And then uh the last component uh competence. So can we consistently show up? Can we execute reliably? And so if we use AI um technologies to help facilitate a conversation, okay, fine. We we have a chatbot. Excellent. Good idea.

### Audience Q&A: purpose-first and ontology of AI

(15:20) Is it actually making relevant recommendations? Um when a consumer tries to find out a little bit more information about a product, does it start to hallucinate? Did we actually check for that corner case, use case, all of those things? Um, are we actually reinforcing that we um we know what we're doing with this technology? And so when we're trying to reinforce trust between the consumer and the brand, benevolence, integrity, uh compliance, bick, um these are the questions I ask uh marketing leaders all the time. help me understand how are you actually showing this and demonstrating

### Market reality: many AI initiatives fail without clear purpose

(15:58) this to your customer as opposed to let's see all the shiny things uh that we can do with AI and technology and with that I'm going to pause for a second u because I see one hand raised Karen I see you you like to add or challenge I I would like to add AI certainly has a lot of capabilities so the First question that comes to mind is all of these ideas do they actually work right so agents talking to agents and eliminating people you see so many reports out there that 80% of all these AI initiatives in corporations are failing they're not producing the results that they are intended to

### “Philosophy beats AI”: purpose, ontology, and human nature

(16:45) produce there was a talk that u resonated with me and uh the title was philosophy beats AI and they broke down uh the philosophy of why you want to use AI and they said that the first most important thing is your purpose. Why are you wanting to use it? For example, in the example that you said you gave for having uh AI actors replace humans.

### Profit motives, business models, and consumer experience risk

(17:25) What's the purpose of that? Is the purpose to cut cost? Wh why are you doing what you're doing? And then the next one and you talked about that is theontology. understanding the nature of AI of what it can and it cannot do uh whether it's giving you the right recommendations and u so that is critical I think a lot of people kind of freak out about AI because they really don't step back for a second to say okay all of these ideas are there but will they actually work and uh a lot of that also requires a lot more understanding of the human nature

### Case study: airline “surveillance pricing” risk

(18:18) to be able to um assess what the benefits and the dangers of AI are. Agree agree very um very very much so. I think there has to be a radical clarity around the why or what is the um particular consumer benefit. I mean we can you can pull it back into economics right so all these big firms that are having multi-billion valuations they're not particularly profitable at the moment and if they want to have subscription models at 200 or 200 plus a pop uh they can keep trying but it's not going to be particularly scalable. So they are going to have to shift their business model such that they can be a

### Urgency signals and dynamic pricing harms

(19:02) bit more revenue positive and then eventually profitable. So it's not going away. Whether or not it degrades the consumer experience, oh I agree. I agree. The using technology for technologies sake. Um yes, most likely we're going to have all sorts of poor experiences. I'll give you an example where I put on my AI ethics hat recently.

### Responsible AI vs corporate ethics: semantics vs behavior change

(19:28) Um, and and this is where I get very curious about the the PR team or the comm's team that was putting this out. Uh, I believe it was uh an airline and I I want to say it was Delta Airlines. They got very excited. They wanted to communicate. They were utilizing AI and the way they were utilizing AI was oh well, you know, uh, we can use AI to like figure out this pricing challenge.

### “Clippy” as a metaphor for transparent assistance

(19:57) So for individuals, maybe large enterprise businesses, maybe they can pay more for a ticket versus somebody who's a regular person, mid-range, whatever, and and they can maybe pay less. And so, you know, people with deeper pockets, fantastic. We can like AI this problem and suddenly we'll we'll make sure that the flights are full and and we'll um be able to extract a bit more value um from different consumer cohorts. And so they were quite excited about this.

### Practical transparency, consent, and understandability

(20:26) And then it became very quickly for for those of us who hang out in AI ethics circles, oh, okay, this is nice. This sounds like surveillance pricing. And so now they're utilizing data signals, maybe your zip code, maybe your previous purchasing ex uh um behaviors, maybe the previous places you went to um to determine, oh, whether or not you should pay an extra 200, 300, 400.

### Measurement: confidence, comprehension, and ongoing consent

(21:00) Okay, maybe that's a little bit icky, maybe a little bit. Where it gets really icky is what if in the data marketplace and this does exist, we start to have these data science get really smart and they start figuring out, oh wait, the behavior of this individual, we don't need to know who they are, but the behavior of this individual, it seems like they need to purchase a ticket in urgency.

### Moderating tech talk to refocus on relationships

(21:23) Somebody passed away in the family, a new birth in the family. I need to go from here to there very very quickly. I don't have time. it's time sensitive. What if an airline jacked up the price as a result of that? Because that's what the data signal said. And and this is so those are the use cases where AI ethics and responsible AI is so critical because it's not a first order effect that we're worried about.

### RAG, fine-tuning, and bespoke models for value

(21:54) It's a second and third order effects that I would argue marketers we are in many ways we may be the only voices in our organizations that represent the consumer. I I can I say something? I would argue argue with you that this is really not AI ethics. It's human and corporate ethics. You're using tools to do something that's unethical. Data and science does what data and science does. It's not it's objective in its own way.

### Engineering reality: structure, lineage, and residual hallucinations

(22:18) It's the human beings who are using that data to extract you know do whatever it is that they do. So it has not it's not I would I argue against AI ethics. This has nothing to do with AI ethics. This is about corporate ethics, capitalist ethics and uh really the core problem with human nature and culture, not technology. Fair. Um technology is a tool.

### Marketer’s remit: protect relationships, not just tech

(22:50) I don't argue semantics because I think most disagreements are a result of slight variations of definitions. um if it changes behavior that's when I say okay we need to come to common ground. So if I say corporate ethics to a board will that change their behavior. If I say AI ethics or responsible AI will that change their behavior I we are saying um our values are aligned.

### Nostalgia lesson: Clippy’s transparency and limits

(23:18) I think the words we use in different forums are the ways that we kind of persuade the argument to move things forward. Okay. I mean we are in a marketing forum, right? So you do have to talk a little bit about marketing and I did promise to talk about technology. Um but I appreciate the point that maybe technology is we can be philosophical too.

### Practical guidance: disclose limits and provide human-out

(23:43) Um, I appreciate the point that maybe the technology is just a further extension of all the tools that we've ever made. So, fire included. So, um, something that I, uh, I recently came across in the internet that I got really excited about because it came back. How many of us remember Clippy? Of course. Okay. Yes. uh that that weird uh uh the the weird paper clip um that used to try to be ever so helpful.

### Adapt messaging by audience sophistication

(24:18) Oh gosh, I can't remember what decade it was at this point, and I don't want to age myself, so we're we're just going to say it was a while ago. Um and it was really annoying at the time. Um because it would constantly be popping up um in your word uh while you were using Word or while you were using Excel to try to help you.

### What to measure instead of “engagement”

(24:36) And I argue that as our as brands continue to experiment in this space, experiment with trying to use technologies to deliver different kinds of value to consumers in their experience. given that we don't have necessarily broad-based regulations, given that maybe we don't necessarily agree um on what values and rules to uphold that brands may want to consider how do we show ourselves to be a bit more like a Clippy.

### Q&A: motivation and doing more than the law demands

(25:16) And why I argue that is that um Clippy was not always uh the most helpful, but it was very transparent at what it could do and what it could not do. Um and also it was um always being very clear that it was there. And so I I really want to focus on kind of the transparency and understandably understandability components uh regarding technology use.

### Where responsible practices show up today (mental health tech)

(25:42) I don't have to use the words AI anymore if we don't want. Um, and so as marketers, as we continue to use these technologies, I ask the question always, are we being clear to the average consumer? Are we disclosing in a way that they understand? Are we owning the limits around these uh technologies? Meaning, you know what? This chatbot is only good for um uh I don't know um uh figuring out how to return a product.

### Guardrails: consent as a continuous process

(26:14) Maybe it's not good at um extrapolating out like different use cases of the product. Okay, be transparent about that. Um being clear that users should have some form of consent mechanism. Okay, you don't want to talk to the chatbot anymore. Um you want to speak to a real person. Okay. Um making things are understandable um to the end user. So showing the reasoning as appropriate to the end user.

### Re-centering on the marketer’s role

(26:42) So if it is a B2B firm, B2B SAS and um you're talking to devs. So it's um developer evangelism, the way you communicate and speak will sound quite different uh from a marketing standpoint as opposed to if you're speaking to the average consumer who may not be in the tech space.

### Prompt engineering reality: RAG + fine-tuning in the wild

(27:00) Um, and then from a measurement standpoint, I get asked this quite often. Okay, if we're not supposed to be measuring engagement, what are we supposed to be um, measuring? I would argue um, we need to measure the components related to trust. So like how confident is our consumer? Meaning do they understand what technologies are we using? Do they understand how the output came about? um the comprehension.

### Engineering details: structure in RAG and link accuracy

(27:32) Do they understand what they can do or what they can opt in and opt out of? And then I tend to say this quite often um consent, consent, consent. And consent isn't a one-time thing. It's a continuous thing uh in the sense that it is um it happens throughout the life cycle, throughout the process of the relationship of the consumer, with the brand.

### Residual hallucination even with citations

(27:52) And so I figure that doing those types of things can help to mitigate any potential risks um when it comes to utilizing these technologies when we don't necessarily know what the second or third order outcomes will be. David, I didn't realize your the chat is on fire. Yeah, there's a lot going on in the chat, so I was just checking on that. Any questions? Are we in mid group? We are. We are today.

### Incentives for responsibility without regulation

(28:18) So, I'm okay if people want to raise hands or throw out a throw out a question or throw out a provocation. I mean, um I'm curious as we talk about ethics and you mentioned that like uh regulation hasn't come here anytime soon in any meaningful national way, right? Um, so it it so what's going to motivate like marketers in particular to do more than the law demands in that sense and you know it's like uh and and uh you know in terms of uh in terms of ethics around AI like like if acting in you know if there's not going to be any uh like legal penalty for acting

### Accountability over capability: a marketer’s stance

(29:13) unethically in a lot of cases or there's going to be a lot of that gray area. um like for like for those that you're seeing that trying to do things right like what's motivating them or and is there anything that can like help yeah arm markers that want to do the right thing but might face an uphill battle when it's yeah not necessarily being required of them very true uh it can be quite challenging this is why in practice I tend to stay responsible as opposed to ethical um because one we have to agree on ethics that's not always true um and

### Examples of firms trying to do right and legal exposure

(29:53) ethics is usually reinforced through policy which in the US not likely to happen so from a responsible standpoint um again I I continue to reiterate transparency understandability and then accountability with consent um if marketers can do those things you mitigate some of the risks um you asked me if there are any firms that are thinking about doing these things and how are they kind of addressing it.

### Marketing’s proactive role: build seatbelts

(30:26) Uh I see a lot of this already happening inside of the mental health care tech space. Um there are firms that are trying to do right and yes it could be for the greater good or it could be oh wait who just got did Sam Alton just get sued because a very unfortunate irreversible poor outcome has occurred. Yes.

### Agents, consent mechanisms, and marketplace corrections

(30:50) um a teenager had decided to end their life as a result of a relationship a parasocial relationship with Chachi BT, uh that's open AI. Um if you are in a firm that maybe is not as deeply resourced or is not necessarily as favored in the marketplace in the moment and you're just in the middle of the road, do you want to take those risks? And so that's where I feel like marketing can take a proactive stance and uh take a proactive stance in terms of building these guardrails.

### Debate: AI ethics vs corporate ethics

(31:22) I I tend to um make the analogy um we have choices um we have a very very fastmoving car and the cars are only getting faster. Um and so that is what the platforms are doing and they have very much every right to do so. They're well resourced to do so. However, those of us who are facing consumers as part of our roles, um we have a choice to say, "Hey, can we build some seat belts? Hey, can we tell people about seat belts?" There's no um no requirement to force people to wear seat belts and it's going to take a long time if that's something that we as a

### Who is accountable when things go wrong?

(31:59) society choose to do. But the fact to raise the idea that hey, we should probably have seat belts. We want to mitigate any kind of uh litigation risk or hey or we want to talk about seat belts because you know reputational risk is a problem um and this is something we want to manage against.

### Car analogy continued: capability vs accountability

(32:20) So imagine a world very soon where we use only the LLMs to do our exploratory search. Help me um I don't know Perplexity help me book a vacation and per uh Perplexity gives me the hotels give me all those things. Um the brand will no longer have intermediary control.

### Why brand trust may matter more, not less

(32:48) So if um in that scenario it offers me a Hilton versus a Marriott and in my mind I'm like ooh Hilton I'm not sure if I'm okay with all of their business practices. I may choose differently. And so the role of the marketer I think slightly changes uh when it comes to all of these more recent technologies meaning trust becomes even more important. Brand becomes even more important and those are not performance marketer type of uh ideas. Um it's more arguably more traditional brand marketing.

### Pushback: will brands matter less if agents decide?

(33:23) That's where I think marketers can take a proactive stance. What do you mean by brands will be more important? Seems like it'd be less important if you have AI agents making the purchasing decisions as opposed to the consumer themselves. The there will be a consent mechanism, right? What what happens if we have a whole bunch of random agents like just booking um vacations and buying um how do you say buying goods and services? Okay, me as the consumer, I say, "No, no, my agent made a mistake. Credit card company, undo that. Undo that." That creates all sorts of second order and

### Learning curve effects and consent as safety valve

(34:01) third order effects of ignoring transaction this and that. And so the marketplace will have to figure out what are those mechanisms to minimize that. But there will be some version of consent because many people as they learn this new behavior of having agents purchase things on their behalf um will make mistakes.

### Market correction and bottom-line realities

(34:21) And so this is where I I'm I'm always very a little bit cautious because there is the future state that may happen. There's the current state and then there's the actual harms that are occurring now um that we could actually take action uh against. I would agree that we are in a very much in a new field in terms of the jump from search to social, social to mobile, now it's to AI and we're going to see where what brands are doing are doing moving fast and breaking things and I think there'll be a market correction. I'm not with Karan when he talks about the philosophical debate about that's debatable that's subjective I think in a

### Back-and-forth: bias vs limitations and training

(34:58) lot of ways but at the end of the day it's going to be the bottom line and brands are going to respond to how their implementation and configuration of AI and how they use it for their consumers or impact their bottom line. So you can get philosophical if you want and talk about brand values from a branding standpoint, but at the end of the day, it's about the money.

### Limits of black-box models and implications for critical use

(35:18) And if that's why Pepsi pulled the ad, Jenner, that's why we had the conversation with Jimmy in the last couple weeks. At the end of the day, how we choose to use AI or how brands choose to use it will be impacted by how it affects their bottom initiatives. That's just my argument. I hear you. Uh so it sounds like this crew is very much of the And please feel free to correct me.

### Synthesis attempt and automotive analogy

(35:41) Um, I'm not suggesting every brand has to use AI. Um, not all problems are solved by AI. Agreed. Um, but my bias is I am very much focused on trying to understand what to trust, who to trust, and what is the role of marketing in helping me understand who to trust, what to trust. So, I'm I'm actually quite curious um outside of generative AI capabilities, I'm curious if anyone has any um current experience utilizing a AI technologies to try to enhance the consumer experience if not or just just do an efficiency play. Uh I'm curious about people's experiences with that.

### Practitioner experience: RAG solutions and clarity of purpose

(36:34) Well, I have developed a rag solution and I think in in my experience what's critical is to understand the nature of AI and when you have a rag solution and you add your own content and uh then let AI act on that content it gives you a much deeper understanding about the value of it but also the shortcomings on how you can fix the problems.

### Black-box limits and realistic expectations

(37:12) Um you know the the key thing for using AI in anything is right now you you should have a clear purpose of why you want it. If your if your idea and I think that's where the market has failed a lot is a lot of the corporate heads the first thought is AI is going to help save me a lot of money because I don't have to hire people and I can shortcut things.

### Emerging consensus: purpose, engineering, and limits

(37:43) If that's your thinking I think that that is a recipe for failure. But if you have a very clear idea and objective for what you want to achieve and then have a very good understanding of the strengths and limitations of AI, what is hallucination, interference, how the and you can really get a good understanding of it.

### Conference takeaways: combining RAG and fine-tuning

(38:07) when you know the content that is being used to generate those answers then you can certainly have ideas you know and I think a lot of great solutions for customer support you know I think rag solutions have a big play in how AI will be used in the future certainly content generation and content marketing is also good but you have to have a very good understanding of the nature of AI and also understand that a lot of it is a black box.

### Engineering overhead beyond the LLM

(38:41) I mean the interpretability of AI, the fact that even the people who built it don't really understand how some answers are inferred uh how it actually makes decisions and if you have those black boxes then uh applying AI to some really serious critical applications will become very challenging.

### Example issue: correct answer, wrong citation link

(39:09) But if you have a very clear purpose of why you want to use it, I think there's a lot of great great understandings and I think that AI agents are also again if the focus is clearly defined then I think you can have success but to think that they can just do the job of a human is not realistic in in HR circles. Yes, that is definitely the sentiment.

### Residual hallucination rates under RAG

(39:37) I loved how you were bringing up um a little bit more technical uh conversation. Uh so I had the fortunate experience to actually be able to attend the Advertising Research Foundation's um marketing science institute conference earlier this week uh at Columbia University um around analytics and forecasting. And without getting too too technical, I'm not sure everyone's background.

### Refocusing on marketer’s responsibility

(40:04) Um what was very promising in terms of how do we make this AI actually useful um most of the presentations uh similar underlying thread. You cannot take anything off the shelf. Don't bad idea. Um however you can utilize uh rag methods um to constrain um and and constrain the the information related to the context of your business.

### Structured content and content engineering in RAG

(40:32) So the externalities with the the your competitors for example things like that while utilizing also fine-tuning of those models. So information about your particular consumers to actually create a more bespoke uh AI model. I is there going to be issues with explanability and interpretability? Yes.

### Closing the loop: relationships over tech details

(40:52) Yes. Yes. However, when it comes to making things a bit more bespoke um in in such a way that it actually delivers some value that makes some kind of sense in the context of your firm and your customers. um these are methods that combined together can be um quite promising. More research has to be done and obviously more implementation has to be done. In my experience, there's a lot of engineering required.

### Practitioner note: post-processing questions and answers

(41:17) LLM does a job, but you also have to then be able to process the questions and then on the back end you also have to then process um the answers as well. And the more structured content you have in a rag solution and then you can engineer your content to continuously provide relevant answers like recently I had an experience where you know the LLMs will always want to create an answer.

### Engineering fix: tagging content to improve citations

(41:54) So somebody asked a question uh and the answer was accurate but the reference link that was generated with that answer was not the right reference link. So the content was still there. It was able to create the right answer, but it Yeah. And then when I researched that a little bit, they said that well, you need to maybe tag your content differently in order to be able to So what I'm saying is that there's an engineering involved in creating systems that work and and that's that seems to be the reality of how all of this works right now.

### Even with lineage, some hallucinations persist

(42:34) Yes. Um and even in those cases uh there has been some empirical work done in the space um even in those cases when we're trying to create lineage and provenence as an output um we're still getting a little bit of hallucination so it can still be up to 10% of those citations even in a rag architecture and fine-tuning architecture is still a hallucination and so yes there are significant challenges um I always caution against getting into too deep of a technical conversation.

### Re-centering: marketers own the relationship

(43:07) Um because sometimes focusing on the technical conversation abstracts away from again I continue to argue the role of the marketer inside of the organization. The marketer is in charge of the relationships I thought or at least I was educated. They are okay good.

### Build-measure-learn remains, now with AI

(43:31) They are and sales marketer starts with leadership tape and sales closes the relationship in terms of bringing them on board usually. Okay. So if we start only focusing on the ones and zeros and we only start focusing on the platforms, sometimes we forget those things are actually meant to be people. Yeah. Relationships can hurt if if the answers are not trustworthy, right? So that's that's well yeah Karen, but you're overstating the point.

### Training vs black box: an ongoing debate

(43:54) The point of any campaign, any initiative, whether it's the Pepsi example, etc. or the example that uh CO mentioned before. It's a build, measure, learn loop. The whole point is to put something out in the market, get the feedback from your marketplace, from your consumers especially, and then adjust accordingly.

### Limits compared to deterministic software

(44:10) In the same way, we've been doing it for decades in this new internet industry. Keep in mind, the internet is only about 30 years old. We're doing the same thing now with AI. And we're going to teach AI in the same way we're teaching ourselves. My job as an analyst early in my career is to take a look at the data, analyze it, provide insights, and optimize campaigns. That's what we're teaching AI to do.

### Accountability concerns and seatbelts analogy

(44:28) So there's no difference from what I've been doing in my career to what we're teaching AI to do now. We just have to be more explicit like you talk about from the technical l technical standpoint creating structured data. There's no difference. I don't understand this anti- AI bias that people seem to have. There is no anti-AI.

### Who bears responsibility: users vs firms

(44:46) this understanding that like you said teaching AI what to do has some limitations because it doesn't always do what you tell it to do because we don't really understand the interpretability part of the AI so that's what we talking that hard training but that's fine and that's called training in the same way we would train an it's not just training it's it's not just training training has its limitations that's what I'm trying to tell you can train it all you want.

### Mixed liability: driver vs manufacturer

(45:18) But if if the if the engine is a black box, if interpretability there are engineers who built it are working on trying to understand how it interprets, there are limitations to the results and that's the the reality of this AI. That's with any technology. There are always limitations. We work around them.

### Current discomfort: not enough accountability

(45:41) No, in the old days when you programmed something, you always knew what you got out of it. In AI, you don't understand. It's a black box, right? You train it with a lot of parameters. There are too many parameters in there and they don't always know how whether the answer is accurate or not.

### Marketer as consumer advocate inside the org

(46:06) That's what hallucination is and that you have to put that into the equation. I'm going to I'm curious to try to synthesize and then provoke the conversation a bit further. Um let's let's talk about the automotive industry. And so now we're talking about a long time ago. Um, so there was no s we were the cars were competing with horses and buggies and there were no roads, so no infrastructure and we didn't have laws that said, "Hey, maybe you don't want the wheel to fall off.

### Handling harms: redress and repair

(46:38) Maybe you don't want the engine to explode. Maybe you want seat belts." Um, and so it was very, very new. We were learning into the space. I think the discomfort and now I'm going to speak for myself. The discomfort I I tend to feel about the whole thing uh is related to I get it technology there's always going to be limitations but what I get a sense of as a consumer as a person that is participating in the in this not just an outsider looking in there's not enough accountability so if something goes wrong what happens that's what we're learning now

### Shared accountability: platforms and users

(47:16) and that's where it gets a little bit tricky and this is where I continue to plead and advocate for marketers because again in many ways sometimes we're the only voice of the consumer inside of our organizations. Okay. How do we at least hold ourselves accountable that if something bad happens um we have a way of address redress uh we have a way of repairing maybe a damaged relationship with the consumer.

### Clarification and wrap-up

(47:48) And so I'm curious if the challenge we're facing now has to do more with accountability for negative outcomes as opposed to just things don't work. I'm curious if uh anyone agrees, disagrees, maybe I'm off topic. I I'm more than happy. This is a really smart group. So I'm again said as opposed to just finish what you said at the end. Say it again, please.

### Capability vs accountability, revisited

(48:18) You said opposed to and then I missed it. Oh, sorry. Um, I'm curious if we're if it's more important for us to think about accountability as opposed to capability. Ah, technology always changes. The capabilities are always limited in some dimension. So is it more that we should be thinking about how do we hold ourselves accountable to negative outcomes that maybe faces our consumers and then how do we protect our firms? Oh h I you know number one I think AI technology is incredible and I think that when you understand what you want to use it for and you have a clear purpose it has a lot of applications.

### Ethics is human; AI is a tool — but responsibility remains

(49:04) uh your ethics are your ethics you know if if you're an ethical person you look to use it in the right way if you're unethical you look to use it in ways you know for whatever purposes that you have so that's that's a human problem not an AI problem and we need to be able to distinguish that but AI is a very powerful technology it has a lot of great applications it solves a lot of problems and some but Sometimes people have the wrong ideas of why they want to use AI and that's again goes back to the person and not to the technology. The

### Consumer responsibility vs firm accountability

(49:43) technology is what it is. It has tremendous capabilities and it has some shortcomings and if you want to use it, you have to understand it. I don't understand how the insides of a car work. But I'm allowed to have a driver's license. Exactly. and and but but when you get into a car and you drive it, you have you you understand what the rules are for that.

### Use AI outputs carefully; do not treat as gospel

(50:19) When you use AI to solve problems, you also have to understand, you know, and not take whatever comes out of it and understand what the results are. That's your responsibility to take the data and do something with it. Right? You don't have to understand interpretability and inference and hallucination. But you do have to understand that everything that comes out of it is not gospel and you should use it, you know, carefully.

### Clarifying locus of responsibility

(50:45) How you use it, that's your responsibility. That's not AI's responsibility. Oh, so the responsibility of any negative outcome, the accountability for any negative outcome is at the individual user, not at the firm. You you're resp you're responsible for your own actions and your own thoughts. Well, I mean this and and this I mean it would be actually a fascinating debate for another section because we're almost a time here. But but even I mean I think the car example is a good one cuz we see examples where most of the time uh

### Liability examples: driver vs recall scenarios

(51:18) someone goes and and kills or hurts another person with a car, it's going to be the driver's fault. Um but there are times when an airbag doesn't deploy, the brakes don't work as planned. Um, and sometimes that's actually, you know, requires uh a whole a systemwide recall uh of tons of vehicles because there was something done at the manufacturer level that was either an oversight or came up after the fact and uh and and any harm that's done to that winds up clearly their fault. they have to also prevent uh any harm from happening even if it hasn't happened yet. So having this

### Platforms as manufacturers; users still learning to drive

(52:05) dichotomy of of when harm comes from the manufacturers in this case the open AIs and metas and Googles uh of the world um that that seem to have cart blanch to have some cars with some pretty wobbly wheels come and out there and some very fast engines and maybe not not enough treads on their tires.

### Closing thanks and community notes

(52:30) Uh but uh uh but also like uh to the other point like uh most folks out there we don't know how to drive yet, right? Like like you know uh we're just you know we're still like and if we're driving we're like you driving this 500 power horsepower car on like a dirt or gravel road that was not designed for the car was designed for the horses that came before it. So So we've got a lot to navigate here.

### Event wrap-up and next steps

(52:58) like I'd love to dive into that further. We don't have as much time here. I hope we can continue this in Slack and in future conversations. Uh the participation's fascinating as uh always here. So appreciate everyone who's been chiming in and also the chat threads been tremendous. So thank you uh all. Dr. Cecilia Dones, I mean just amazing. Look forward to following more of your work. hopefully having you back here.

### Gratitude and sign-off

(53:25) Uh and and you're bringing up so many important issues for us to follow. Thank you so much. And this is a very special group. Uh you've created a community and the fact that everyone shows up for themselves and each other. It's something special. Well, appreciate you getting to join us for all that and you sharing that. Um yeah, thanks everyone. Have a a wonderful rest of your week.

### Reminder: next in-person and speaker slate

(53:49) We'll keep seeing you next week. Next week, if you're in New York, we've got first Wednesday in person next Wednesday. Uh uh so be sure to check the event page for that. Um and uh yeah, just tremendous slate of speakers ahead. So thanks everyone and uh see you soon.