The Playbook for AI Personalization at Scale IAB The Weather Company Acxiom

(00:00) We're super excited to introduce Innovative Orchestrator, the first AI super agent for omni channel advertising. We're bringing humans and AI agents together so that marketers can conduct a symphony across channels to reach their most valuable customers. AI is not a matter of if, it's a matter of how and when. And we will help you solve that.

### Welcome and Host Introductions

(00:32) Welcome everyone to another edition of AI Insiders from AI Marketers Guild here from the downtown studio at the Institute of Culinary Education who I can thank for hosting today as I actually do some AI training here.

And apparently it's cookie day. Um, so if you've never had a a sesame seed cookie from a culinary school trained chef, I highly recommend that and I will be nibbling on this and anything else that comes my way while we have a very special host who's making a return appearance here. We've got Carolyn from

### Guest Panelists from IAB

(01:11) IAB who's just one of my favorite uh uh folks who who's not just thinking a lot about AI but putting it into practice within the advertising community. Uh and she's invited some special guests. So, I'm I I love when we get to bring in these heavy hitters and I get to sit back and learn.

### Community Conversation Format

(01:36) And as as most of you know, most of you have been here before, but in case you haven't, these are community conversations. We love interactivity. We would love questions and thoughts here and so um so glad to have you all involved.

Well, thank you David as always for having us and I feel like you should be like sending one of those cookies to all of us so we can have that while we do this conversation. It feels unfair.

### Introducing the IAB AI Personalization Playbook

(01:55) Sorry, the one-sided cookie situation. It is. Um but my name is Caroline Giegerich. I'm VP of AI at the Interactive Advertising Bureau. And a few weeks ago, we had an exciting release of the IAB AI personalization playbook. And these two incredible gentlemen that I will be joined by were in the working group, provided lots of thought leadership to go into this playbook. And before I introduce them, let me just give you a quick overview.

### From DCO to AI Personalization: Promise and Risks

(02:29) We've gone from DCO which is using AI to stitch together a bunch of assets using AI to AI personalization which is capable of generating many many many personalized assets right and this is an incredible innovation at the on the one side we have the promise of personalization increased ROI which these incredibly smart people will speak very astutely too.

### Brand Quality and Interoperability Challenges

(03:03) On the other hand, we have some of the risks in the challenges in organizations and I'm betting that many of you in the audience are experiencing some of these challenges. It could be brand quality if you're having AI generate all these assets. What about the risk there? What about I know Graham is going to talk a lot about this, the interoperability challenges of dealing with all of the technology and many more.

### Playbook Scope and Table of Contents

(03:21) So what we wanted to do with this playbook is basically create a in between best practices guide and I'm just going to share just the um table of contents. So you can see this here of how do we actually go about what is the opportunity? What's the challenge? How would you go about briefing this? How are you building all these assets? If you are at a media agency, a creative agency, if you're internal, how are all these departments going to work together? Where's the human in the loop? How do you assess the risk from different use cases from a simple social asset to a full television commercial?

### Meet the Panelists

(04:00) That's what we set out to do, and I'll also put it in chat in a in a brief moment. But first, I want to introduce our lovely panelists. Graham Wilkinson is the EVP chief innovation officer and global head of AI at Axiom. Welcome. And Brian Hall, head global creative labs at the weather company.

### The Weather Company’s Creative Labs

(04:29) I'd love to just kick it off, turn it to you, and have you give us a little bit more about what you're doing. Like Brian, let's start with you. Tell us in, you know, words that we can all understand that brings makes it clear for us what you do with the weather company. I'll do my best. Thanks, Carolyn. Hey, it's a pleasure to be here.

### Moment-Based Relevance with Contextual Data

(04:48) So, at the weather company, um my team, the global creative labs, we really focus on the unique intersection of real-time contextual data and worldclass creative execution. Our core mission is to help brands move beyond basic targeting to achieve momentbased relevance by leveraging our specialized data, things like weather, location, time of day, etc. So basically we're essentially applying our company's deep expertise in AI innovation and human machine collaboration to marketing.

### Operationalizing Personalization

(05:15) Uh this means we build the systems for creative automation and operationalizing personalization. And we're really looking forward to discussing how the IAB playbook frames these challenges. And Graham, I'll also kick it over to you because you clearly have a very large and broad very impressive um title.

### Graham’s Role at Axiom

(05:32) Tell us what are you doing day in day out at Axiom? What does your job actually look like? And I will say for everyone here, um, Graham is a total legend because he actually had his wisdom teeth out yesterday and he looks that good. Thanks, Caroline. Yeah. So, that kind of explains why I'm not fully opening my mouth when I speak. So, I apologize if it looks and sounds weird. Um, yeah.

### Organizational Update: IPG to Omnicom

(05:58) So, I actually just noticed as well that my name on here has IPG next to it, which RIP IPG. I uh I probably need to swap that out for Omnicom now. Um but I still work in Axiom and um and so, you know, my role is I suppose kind of twofold.

### Innovation Across Data, Tech, and Clients

(06:19) One, as chief innovation officer, I get involved in um you know, most parts of the business with regards to innovation. And you know, I'm given a fairly um free remit to go in and and and look at the things that we do, whether that be with the the data that we that we work with or whether it be our tech stack or even just working directly with clients and and and taking new innovations to them or or kind of collaboratively doing that.

### Global AI Leadership and Responsibilities

(06:47) And then from an AI perspective, you know, I've been responsible for AI across all of IPG media brands. for about, you know, just under 30,000 people for the for the last 3 years. And um and that means I'm the throat choke for AI everywhere around the world. Um as you can imagine, that means lots of questions every single day. Um multiple client meetings.

### From C-Suite Workshops to R&D

(07:11) I I will talk with CEOs, CMOs about their ambitions, their challenges with with AI. Um we'll I'll do workshops, presentations, demonstrations, build blueprints, road maps, all these kind of things. So it's basically um everything you can possibly think of. And I also run R&D teams. So that means that I'm managing development teams at the at the same time. I think I think that's a good thing.

### Closing the Loop Between Client Needs and R&D

(07:37) It means that I can direct I'm directly connected into our clients and I can take their needs and uh their challenges and and work with my R&D teams to to try and solve them. Incredible. And before I I'm going to stay with you, Graham, I will say for anyone that has a question as we're going along, feel free to just pop it in the chat and we'll sort of address it as we go along.

### Join the Conversation

(08:01) Don't I I know David kind of encourages this in all of these conversations, but don't be shy. We'll we'll kind of get to it along the way. Um Graham, I just want to stick with you to start. You're making the case for AI personalization with CMOs and CFOs. What is the actual revenue story here? Like what are we looking at in an actual ROI upside? Yeah, I mean it's kind of so I think the first thing is it's it's a step back from that, right? I think most people come with the efficiency ask, right? They want to know how can I drive efficiency or how is my agency going to

### Efficiency vs. Effectiveness: Two-Part Equation

(08:41) be more efficient? Um, I think that I mean, I certainly try to steer people down the road of this is a two-part equation, right? There's efficiency and effectiveness. One of them drives revenue, one of them drives margin. And and I think that you have to be very careful of how you balance that.

### Limitations of Generative Models

(08:59) I think efficiency is a short-term play and we have to be very careful not to um you know discard talent and people um and and and also recognize you know part of the problem is is all the hype around AI it it it doesn't really make it very clear to the to the general public that when we're talking about models that essentially are building semantic you know work off semantic relationship ships between words.

### Risks of Over-Reliance on Automation

(09:32) They're, you know, even if they're multimodal, they still operate in singular modalities at a given moment in time. And that isn't intelligence. That's not human intelligence. And so, we have to be very careful. Like, we can imitate that. You know, you can you can make a mechanical robot and and dress it up like a human being and say it's a human being, but it's not.

### Where ROI Comes From

(09:58) And so coming back to what I was saying, when you go down that efficiency road and you are putting all your eggs in the basket of a machine that actually can't do the job of a human being, it's a risky it's a risky path to go down, right? So then you get into the effectiveness play which is really where I think you know you get you get the multiplying effect and you get ROI.

### Reported Performance Uplift

(10:17) So you know I think typically I would see anything from a 20 to 40% um increase in let's say increase in performance and it obviously depends how you measure ROI and how you measure performance and all these key key metrics um and I mean certain instances you get I mean I we've run tests where we we'll get 110% uplift of a an asset that was uh AI augmented in its creation versus something that was kind of created in a in a standard way.

### KPI Example: CTV Eyeballs

(10:48) Say uplift what uplift in what KPI? It depends. I mean, you know, that particular metric comes from uplifting eyeballs. So, so you know, an ad for um a major sports league and their Christmas um their Christmas games. Um so that that's essentially measuring eyeballs across kind of connected TV. Um, yeah.

### Beyond Performance: CX and Brand

(11:15) Gotcha. And then Brian, from your standpoint on the creative side, we talked about effectiveness. Um, but is there something bigger that you're also looking at? Like are you looking at um customer experience or brand perception? The things that are about like overall how customers are thinking about the brand in mass. Yeah, great question.

### Personalization Impact: Conversion, CLV, Trust

(11:45) So like the playbook notes that personalization drives not just a conversion lift up 16% but also customer lifetime value CLV up to 20% increase and and consumer trust metrics right so trust fuels more data sharing which in turn fuels better personalization creating a self reinforcing cycle so my direct answer is it's definitely both but there's something bigger Carol is is crucial.

### Building Loyalty Through Relevance

(12:13) While performance lift meaningful lift and conversion and engagement is the immediate ROI, the true long-term value lies in brand perception and customer experience. Like for example, our data shows that when we deliver timely, relevant, contextually relevant, uh creative, we're not just making a sale, we're building loyalty. And also from a corporate standpoint, we view this as a natural evolution.

### Personalization as Natural Evolution

(12:38) We've been doing this for decades. NI is simply the natural progression of our product story and increased relevance to our fans, our brands, and our advertisers. That makes a lot of sense. I'm going to stay with you also because we're we're talking about the benefits, but I think we would be remiss to not also talk about some of the challenges.

### The Operational Bottleneck

(13:01) I think in talking to you several times about the weather company, you have these unique advantages because you have all this contextual data. Talk about data. You've got weather, location, time of day. It's all super impressive and can add a really interesting layer of personalization. When you're thinking about scaling all of this personalization, what's the biggest operational bottleneck you're hitting? Is it production? Is it QA? Is it measurement? Is it something I haven't mentioned entirely? That's a that's a it's a rabbit hole.

### Governance and Value Stream Mapping

(13:35) But the the challenge is the operational infrastructure, the crossf functional integration is probably the biggest the biggest situation and the governance frameworks or the lack thereof like linear human only workflows for QA for instance simply cannot handle thousands of AI generated variants and I'll talk about this a little bit later in the session about how we address it with a VSSM a value stream mapping exercise um later in the session. Do you want to talk about I mean you just you gave a cliffhanger. You want to give a little bit a little bit more?

### Breaking Fragmentation with Shared Workflows

(14:09) Yeah. Yeah. Well, it's um I think you know it all just again again it comes down to breaking apart the fragmentation and getting crossunctional integration and like I said governance frameworks but that's a lot of effort and it's a lot of work.

### Relay-Race Handoffs and Workflow Mapping

(14:25) Uh it's can only go so far if only one department or discipline within your organization is really AI ready, right? They can only do so much. You need to be able to do a baton handoff. Hey, you need to be able to pick up the baton. I use this analogy a lot of a relay race. Um, when it comes to organizational efficiencies and creating frameworks like this, you really do need to collectively come together uh from every single department and discipline from brief to benchmarking map out what are those steps and especially with something it's good to do anyway annually no matter what take AI out of the picture. It's

### Operational Hygiene for AI Workflows

(14:57) something it's good hygiene operational hygiene to do but especially now with the advent of the need to create architectural frameworks and and like workflows that incorporate AI. So getting every single department and discipline together to again map out from briefing to benchmarking what does it take what are those steps how long does each step take what is each what's the time between each step n 99% of the time uh when we do these exercises and we do them about twice a year we find bottlenecks we find opportunities for

### Finding Bottlenecks with VSM

(15:28) operational efficiencies and then our ability to streamline and to say this and to speak the same language did you mean tomato or did you mean tomato these little things make such a huge critical difference uh when you're working with datadriven processes like this.

### Using AI for Brand Compliance

(15:46) So uh the value stream mapping exercise is one really key critical way that we're approaching to to solve that challenge of operational infrastructure and gain that that cross functional integration together. I'm going to ask a question from Derek and this is for you both Graham and Brian. uh so whoever wants to pop in on this I love this question. Is AI used to check whether AI created content is on brand? Yeah, absolutely.

### Codifying Brand for AI

(16:09) I think you know it is but you but it still comes back to the fundamental question which is is talked about extensively in the in the playbook which is do does any given brand understand what its brand is and means? Like can you clearly articulate can you codify it? uh, you know, I over the last 3 years, I I don't think I've met any two brands that that articulate or structure the articulation of their brand in the same way. You know, some go, "Oh, that guy knows our brand.

### Clarity Precedes Automation

(16:47) He writes everything like he is our brand." Some go, "Yeah, we got loads of of content and there's duplicative documents that that has slight variation in them. there's there's just so much difference in it. uh, and so the real question is yes, you can you can get AI to do most things, but you know, this comes back to the fundamentals of like machine learning really.

### The Hallucination Gap

(17:13) If you can't explain a process in clear instructions and and language, then you can't expect a machine to do it. And the the exacerbating problem with gener generative AI, you know, that used to be a make or break with a machine learning process, right? It if there was a void in the process, the machine can't jump across the void.

### Separate Models for Create vs. Check

(17:38) But with generative AI, it will jump across it because it will just make its own bridge and that's that's more dangerous. And so I think yes, it is. you know, we we have a a system where, you know, I suppose you think about two systems, right? And you got to be careful that you're not having the say the same AI mark its own homework, right? So, you want AI that's trained on the brand to create content and then you want AI that's trained on the brand to check the content and it shouldn't be the same. And why? Say why.

### Start by Auditing Model Knowledge

(18:14) Oh, well, I mean, first of all, there's a fundamental step that most people miss, and it's really simple. You don't have to be a developer to do this. Most people don't actually ask the model what it knows about their brand first. They just start telling it things. And so, if you don't ask it what it knows about you, you don't know what you have to correct. Mhm.

### Value of Model Diversity

(18:33) Like there might be some fundamental things that are not addressed in your explanation and your the code of your brand that if you miss them then you're not correcting them and it is going to be fundamentally wrong. So I think that's an important thing.

### Are Brand Guidelines Fit for Personalization?

(18:51) So if you're then working across the same model and those things are missing they won't get picked up in either the creation or the checking of it. uh but secondly it kind of you know I'm a strong believer in diversity of models. There's a reason we we evangelize adversity in teams and people and all that sort of stuff.

### Rethinking Static Assets

(19:14) uh is because we're really good at picking holes in in things, right? And so it's just important to get a different perspective from just a a very fundamental level. uh and that's why we should constantly be testing testing new models, utilizing a real diverse ecosystem of models to do things. I like this a lot because in my former experience, most of my time has been marketing for brands, doing a lot of briefs, and I've seen exactly what you're talking about. I mean, even in a 30 person marketing team.

### Brand Guide Inconsistencies in Practice

(19:45) I've worked at Showtime, I've worked at As Music Group and Smashbox Cosmetics. I could have seen a different brand perspective from every single person on that team. we probably would each uniquely have created 30 different uh briefs if left to our own devices even in and I'm going to turn to you Brian because this question here is from Earl Richards Jr.

### Audit Before Automating

(20:08) asking about brand guides and of course even in the organizations I just mentioned we all we all had brand guides but I think what Graham brought up was a really interesting point if you're not really clear on what the brand is what the brand isn't AI is going to exacerbate that and it's going to turn into disaster so what are your thoughts on brand guides getting clear uh and this conversation that we're having here yeah I I think Grant really nailed it.

### From Static Docs to Adaptive Guardrails

(21:03) It's a core finding, too, that organizations often discover their brand guidelines are inconsistent or they have gaps and contradictions when they try to teach AI systems to follow them systematically. uh, auditing your current processes before automating. I've got to make that in a audit before automating. That's there's got to be a song title there is is critical first step.

### Model-Generated Creative Futures

(21:41) So there's a great deal of effort that needs to go into preparation to effectively not only collaborate but to train uh AI on how to effectively collaborate with you. I also can I can I just build on this Caroline because this is one of the things that I think is so fundamental to the AI transformation that's occurring in our industry is what this begs the question of is are brand guidelines even fit for purpose for the future of personalization because in in the past and and the current we you know we serve things in a very kind

### Evolving Ad Formats for GenAI

(21:59) of static manner and what I mean by that is an artifact act exists at a moment in time. This is an asset. We will serve this asset in the future. Maybe that won't. Maybe we'll serve a model and the model will just morph into whatever we want it to morph into.

### Designing Generative Ad Experiences

(22:41) And and then that begs the question, well, is a brand guideline a a static document or actually is it different for every person that looks at it from a different angle? And actually, how do you how how do you think about that? And and the reason I say it's super interesting because I think the most progressive people in the industry are thinking about these fundamental changes to the way that we've done and thought about marketing and and not thinking about I think the biggest trap with AI in advertising is solve old problems with new stuff like it maybe those old problems won't even exist in the future and maybe we should be thinking about the new stuff And

### From Concept to Execution

(23:05) yeah, you know, we'll we'll kind of sweep along with some of the older stuff, but that's the stuff I'm really interested in. You know, I love how your brain works and this is just another example. But then my question is, how does that work? I love this concept, right? I'm like 3,000 ft. I love this concept. It's amorphits.

### New Tech, Old Roads Problem

(23:32) It's like building with like everything that's happening into the ecosystem, but how exactly? uh, I don't know the answer to that. That's I mean, but that's why that's why it's actually important to bring it into forums like this so that people can can start to to think about it, right? I I think again part of there's a weirdness to the way that the and I'm sure everybody feels this, right, when you want to apply AI to what we do in in advertising and marketing.

### Working Within Legacy Ecosystems

(23:56) And I think part of this the weird feeling we've got is that we've got this kind of situation where we've got this new technology. You know, Ford has just invented the car and and people want to drive them, but there's no roads. Like we're we're like adapting the car to drive off-road instead of building highways that make like easy easier to drive the car.

### Defining the Gen-Ad Format

(24:19) And what I mean by the roads in this analogy is like I don't think ad formats have evolved yet. Mhm. And and so we're again we are using AI to apply to legacy advertising infrastructure and ecosystems and and that's but we again we have to do that that we have to keep businesses moving.

### From Cohorts to Fluid 1:1

(24:38) But I don't believe there's enough emphasis on what is a generative ad experience? What does that really even mean? because then I think everything will click into place for us and and the the weirdness and the kind of like I can't quite put my finger on the thing that's stopping me moving forward.

### Personalization That Adapts Over Time

(25:05) I think that will dissipate and we'll will all kind of move forward quite freely. Well, also this idea of real time, you said something at one point that equally blew my mind, which you said, okay, so for the record, in this playbook, we are thinking of personalization on a cohort level. So not on a oneto-one level, we decided mindfully that we feel like we're in a place where cohort-based makes sense for this discussion.

### Targeting Variables Today

(25:23) But if you look out into the future in some working group we had Graham said, "Well, not only is it going to be one onetoone personalization someday, it'll also change because Graham changes every day. For example, he just got his wisdom. He's wisdom teeth lighter as of today." Well, that's a change in Graham.

### Toward Brand-to-Individual

(26:13) Brian, I'm sure, has listened to 10,000 new songs from yesterday to today. And if Brian's a completely new person today, but you I'm I'm joking, but you you see what we're saying here. people grow and change and the personalization can do the same and and that is enlightening.

### Weather as Dynamic Context

(26:47) I uh Brian want to move to you on this idea of targeting. uh Earl Richards Jr. also asked a question what's the relationship for art audience targeting for your brand's audience and personalization with so many variables. I mean obviously what I just said is the Graham level future of like thousands of variables but in today we've got demo location contacts day parting life stages etc. Brian what are your what are your thoughts on this? I mean yeah that was a perfect setup.

### Forensic Weather Targeting

(27:06) I mean those variables and the variables like all those that are all mentioned demo location context day partying life stages looking at you know again the more information you have about somebody the more relevant contextually your message your product service or experience is going to be when it comes to and we're starting to capture more and more uh information on our product consumer product experiences so that we can give more you've heard of B2B you've heard of B See, but we're really working a lot on B to eye, like business to individual or brand to individual. And

### Empirical Targeting Rules

(27:25) as the weather brand, the variables for weather, you would not believe how they like you set it up perfectly, Caroline. Like every single day, the temperature is different and you're going to feel different in the morning versus the e afternoon versus the evening.

### Weather’s Broad Impact

(27:48) And it's not just about block and tackle scenarios or situations where I I say this a lot where you know people make assumptions about uh day partying or weather targeting like oh it's hot you know show a pair of wayfair sunglasses or it's cold show a Northace jacket in a contextually relevant environment.

### Utility Messaging Example

(28:21) What we like to do is take it completely and entirely forensic nature to another level of scenarios like if wind speed is X and D point is Y and cloud coverage is Z, we're going to show you this product instead of that product because we have empirical evidence and data that human behavior is affected differently based upon those combination of weather patterns. It can get it gets quite forensic and quite deep.

### Personalization Stakes in Real Life

(28:46) So, it's a it's a really fascinating place to play in with that on the combination of everything else that you get in terms of behavior and personal information to tie that into a weather story because weather affects every single thing you do, what you eat, what you wear, what kind of car you drive, where you go on vacation and give you one good example of like how can weather affect an individual on an individual basis because a lot of us feel the same way if we live in the same zip code and we get up. I mean, there are obviously variable differences on our individual makeup, but let's say I'm an allergy sufferer

### Human Judgment vs. AI Scale

(29:24) and I happen to let the Weather Channel app know that I'm allergic to uh, you know, dog ragweed pollen. We can give you a direct message letting you know in the morning before you get out the door and you get prepped, you know, to give you utility service to say, "Hey, you know, this is what's going to be like for you today, the minute you step outside your door.

### Human-in-the-Loop Philosophy

(29:51) So, how can we provide that real forensic personalized individual messaging to people? I love that. And I also uh am a pretty hardcore cyclist, both like a to work commuter and just performance cyclist. So, I live and die by the weather app for sure. And when it's off, it causes me quite the consternation. uh, so that's my personalization. uh Brian, I want to move to something we briefly talked about earlier and Graham also mentioned it is this idea of human and AI collaboration, right? And we got into this quite a bit in the the playbook and we've obviously mentioned that there is there are points where

### Augmenting Experts with AI

(30:14) humans just can't review every piece of creative, right? So, how in your day in day out of creative production at scale, where are you drawing the line? What what decisions should humans always make? And and where is AI engaged? That's a it's that's a big one. So, humans excel at strategy, cultural nuance, emotional resonance, and final judgment.

### OODA Loop in Creative Ops

(30:34) Uh and then AI excels at production efficiency to your point. sometimes too much uh to keep up with and data processing and all those variables. So it's a pretty critical challenge and it's one that we've been we've solved for on the consumer product side and are now applying to advertising. So for us the philosophy is simple.

### Human–AI as Trusted Copilots

(31:07) We've combined 20 plus years of data and AI innovation with 100 expert meteorologists. It's that human in the loop or human in control. And it's why we're 3x, you know, more accurate than our other competitors out there in the weather space. So, we're huge advocates of the power of AI to augment and empower human abilities and decisions.

(31:28) And we apply that same philosophy to our advertising businesses as well. But in creative production, we draw the line based on the playbook's framework. Uh, humans must decide on strategic direction, creative judgment, and final quality assurance. So AI uh must run on production efficiency and content generation and data processing. But the the the human loop it goes back to the UDA loop right generated that was owned by fighter pilots in the 50s in Korea which is observe orient decide and act.

(31:07) The human is in control. The AI gives us information to make decisions on. So I think I talk about this a lot like the relationship between AI and humans is very similar to the relationship for those of us that are old.

(31:28) It's the same relationship that Luke Skywalker and R2-D2 have C3PO or uh or even older it's enterprise. I prefer seasons Brian. Yeah. Yeah. or you know I'm going through all the generations here from Star Trek to computer to you know C3P or Luke Skywalker or even you know the younger generations really can understand maybe more of uh you know Iron Man and uh getting you know the Tony Stark and uh talking to Jarvis like again I just have to say they star I know did he cut off we lost somebody right right when he said Star Wars and then blank I'm like No. What happened? Justin, you put yourself on mute

(32:05) when you started talking. Justin, you put yourself on mute by accident. Oh, did I did I not get through? No, I got to do that again. Star Wars got excited and then he left us hanging. Oh no, he did it again. Oh god. You're doing it now. We can't hear you. Dark side doesn't want him to talk. Hey, you. Okay. You know, am I good? Can you hear me? Test.

(32:31) Don't touch anything. Okay. No, I did the thing where I tapped. Usually I go in. I just wanted to chime in because never worry about Star Wars going out of fashion. They have rebranded for everything. uh, whenever you mention C3PO or anything, it's like a it's like a time stone. You could worry about Star Trek. uh, that's kind of one that's too much.

(32:49) You know what I mean? That like the lore goes too deep. People don't get it. Star Wars is just a family story at the end of the day. So like anytime you mention Anakin or Darth Vader, they'll get it. and CPO's that, you know, the crew right there. I'm right there with you.

(33:06) I was recently in an Apple store and I made this exact same uh conversation with some younger employees there and neither one of them had ever watched the Star Wars movie. So, it completely blew my mind. So, you pulled out your phone and immediately showed them the trailer. That's what happened, right? Well, I got keep That's why I keep tricking tricking down like, "Okay, what about Tony Stark and Jarvis?" Like, okay, I know that one.

(33:23) But look, you get it. There's a there's a there's a symbiotic relationship here that needs to happen and it's again it's something that's been we've been working with for over a century now. A lot of people don't understand the agile formats were developed in 1896 by Sadachi Toyota. You know the the autonom and and we've got a lot of things that we need to watch out for in terms of of governance protection IP.

### Balancing Governance and Opportunity

(33:53) There are so many uh things but so many great so many concerns that there's so many incredible opportunities. Yeah. Well, and Graham on your end are you thinking about this human in the loop story differently whether it's say a social asset versus something that could be considered bigger like a television commercial or something of that nature? like are you thinking about how AI is engaged differently b verse b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b based on the breadth of the asset, I guess I would

### Risk-Based Governance by Asset

(34:27) say. Yeah. Yeah. And and look, I think I always go back to like right at the start of this explosion, I was uh working with uh well, Amazon were a big client of IPGs at the time. I was doing an an AI kind of workshop with them a long time ago and their creative teams were even back then putting asset putting AI generated assets live and I still think this rule of thumb that they developed was great which was basically the legal team said you can use AI for any creative asset that can be taken down within 24 hours. Yeah. And I was like, okay, that kind of if if a big brand like Amazon can can do

### The 24-Hour Takedown Rule

(35:12) that, that that that's a solid way of uh thinking about it. And it it fits with, you know, the way that we talk about it in the playbook, which is, you know, when assets are say low risk, say a a social post, uh maybe even, you know, one of the other things that we don't necessarily reference in the playbook, but it's kind of inferred is if you are if you're playing around with features and components of an asset, and they that could be a you know, it it could be a text ad in Google or or it could be a text ad And on a meta platform, right? If the platform itself

### Platform Guardrails Reduce Risk

(35:49) has ways to catch errors and violations, then the risk associated with your use of AI actually is is kind of mitigated because the system won't even publish your ad if you're violating something, right? So there there's that kind of compounding effect that is beneficial to you where in in these lower stake uh uh assets and using them on a platform that has its own checks and balances, you're you're going to kind of you're going to be safer in in that respect. But then as you move up the the tiers as we described them in the playbook, you

### High-Risk Assets and Regulated Sectors

(36:27) know, eventually you get to these higher risk scenarios that could be associated with the asset itself. So it could be like a product description which is obviously fundamental to to the thing that you're selling but it could equally be in a regulated industry.

### Universal Risk Aversion

(37:06) I do a lot has a ton of clients that are banks and the insurance companies and has done for decades. uh and you know they're generally risk averse but one thing I would say actually is everybody's risk averse with AI. It does not matter who I what brand I speak to whether it's Nintendo, Lego or banks. And I say the same thing to all of them.

### Scaling Within Human Constraints

(37:30) You all tell me you're risk averse because nobody wants to be the brand that makes a mistake, right? So it but but there obviously are some fundamental things with with regulation and that does obviously restrict what you can what you can do. uh and so I think there has to be humans in involved in that that component of it.

### Balancing Individual and Group Identity

(38:07) And you have to be realistic therefore about where you can really take scale with personalization with that. uh you know and and also even going back to what you said about what I said before Caroline around you know building assets for individuals you know although that might be something to try and do I do think that there will ultimately be a kind of diminishing return aspect to it as well right where you know it's not necessarily always beneficial to talk to an individual directly people like to associate themselves with groups you just said you're a keen cycle Cyclist. Cyclists are very tribal. uh, in fact,

### Privacy–Relevance Tradeoff

(39:10) they're very hard people to break into groups of because they I I cycled for many, many years and they're not particularly friendly always cyclists. But, you know, people think I'm very friendly. I I know you are, but many people are not as cyclists. And I do think that in situations, some people think in groups and some people and in different situations, people think as individuals.

### The Line Between Helpful and Creepy

(39:35) And I think that it's not a hey, we should all be aiming for individual personalization. It's thinking about scenarios too as well, right? And so I also thought you were going to go in the direction of don't we at some point hit a creepy factor to the user or is that not a concern? I mean I again I think it's with change there's always this kind of push back on on stuff, right? At the end of the day, you know, human beings, we want both sides of of everything, right? We want to go I want control of my data and privacy, but I also want to go on somewhere and I don't want to get an ad

### Is There an End-to-End Platform?

(40:08) that's super generic and you got to get somewhere into the middle and people have to be taken on a journey and bad actors will always make that more difficult and it'll make it difficult for good actors. So, I think that they You got an ad today that was like, "Hey, Graham, do you want these soft foods because I know you got some pain in the mouth going on right now." I mean, that would be terrible marketing copy.

### Fragmented Stacks Block Automation

(40:35) This is not how I write, but would that be cool? Yeah. I mean, I was literally going to ask Gemini before I came on this call, what would be something tasty for me to like uh actually drink rather rather than and and like like you know, I think that sort of stuff is very very important, but it's tough to get to that point because you have to know something deeply personal about somebody and there has to be a level of comfort about how that type of data and information is is dealt with. Well, and I'm going to open up a question in the chat, and this is for

### Walled Tools, Manual Glue

(41:12) either one of you. Derek asks, "What are the tools, platforms for marketers, agencies to use? Here's the brief. Here's what we want to communicate to whom. Now, turn that into personalized content." I mean, I don't believe there is a there is one there is one platform for that, right? And that's big part of the problem. And as you said, I I talk about it a lot.

### Incentives and Interop

(41:37) It's this idea that pe you know we as an industry and certain individuals have made entire careers out of the the fragmentation aspect of our industry. you know, we have departments that are dedicated to, oh, do you want to in implement a new CDP or let's let's instantiate a dam for you. But and and it all all happens is you just get all these different licenses and softwares and and now you know you want you want to part of the promise versus reality with AI is end to end uh automation and now you start to look at that you go well I've got five

### Beware Single-Platform Lock-In

(42:08) different platforms that run you know five very important aspects of my marketing uh flow but they don't talk to each other and actually if I want them to talk to each other I'm going have to do all the hard work to to make that happen because it's not in those in the interests of those platforms to to make that happen.

### Open Ecosystems vs Walled Gardens

(42:49) Right? So I do think that that that is that is really really tough and I don't think that there is that there is anyone solution now can I ask you a quick question before you move on because I thought a lot about this because we actually had someone from Adobe in our working group right so isn't it in the best interest of everyone and I mean those platforms included to lessen those challenges on let's call it end client you in this case to increase usage of all of the platform.

### Reconciling Platforms with AI

(43:31) Isn't it like all boats shall rise if this is easier for everyone? Yeah. I mean it maybe in certain instances, right? And and again I think you've got to be very careful. You know Google Google and Amazon they want to sell you data storage and processing, right? they they're giving you AI and they're letting you pay for it, but really what they want is is for your your entire stack to live in inside there and and that's okay, but also coming back to the diversity of models challenge, if you are wedded to a certain platform that say only offers a

### Will Walls Come Down?

(44:10) certain family of models or limited set of family of models, you are not being as progressive as you could be. uh when it comes to utilizing AI, I think that you know the the there there's a to me there's a fundamental kind of clash in this whole idea of platforms and and AI and it's and it's because in order to offer if we're really going to let AI do jobs that human beings have previously done and we want it to perform at this optimal level, it can't be restricted to to like thinking inside these these bubbles because that's the

### Orchestrating Across Teams

(44:43) mistake we've made for for the existence of humanity. And that is part of the promise of what AI does is it breaks silos down, right? And so the I I don't think tech has quite figured out yet what is it really what is really in its interest to to do like is it in its interest to be proprietary and make wall garden you know the walls around their gardens even higher or is it to actually maybe like lower those walls a little bit and and and create a uh a more fluid ecosystem. uh, I I don't have a good answer for you other than I don't think

### Media–Creative Divide

(45:09) I'm not entirely sure it is in the interest of them to do it yet because right now everybody's trying to make a play for your for for your consumption of of AI and that isn't in the best interests of the output of uh from AI. Yeah, I hear you. I also wanted to pick up on the the silos piece and you know breaking those silos down because we also spoke about the crossf functional issues that exist.

### Fixing Systemic Misalignment

(45:38) You've got internal organizations and an orchestration that has to go on between creative and ops and legal and data, right? And then you've got you've got the let's say the brand you've the creative agency the media agency. So, and this is a question for for either one of or or both of you, like what's the proper orchestration of all of that? Talk about silos.

### Shared IDs and Unified Dashboards

(45:56) I've talked a lot. I'll let Brian then sit. Yeah, it's a good one. So look at the the fragmented nature you just touched upon it of the agency roles is the big unspoken challenge Caroline that tension uh is resolved by actively building bridges that we talked about earlier requiring both agencies to use the same creative IDs and metadata and aligning dashboards so performance insights flow both ways.

### What’s Coming by 2026

(46:14) So the thing that nobody wants to talk about is the fragmented nature of those roles. The media agency owns the targeting data and activation. The creative agency owns the brand, voice, and production. So those two silos often are disconnected and don't use the same language or systems. Like I was saying earlier, tomato versus tomato.

### Let Agents Talk to Agents

(46:39) So when campaign stalled, it's usually because of this systematic misalignment. The handoffs are manual, the meta data is inconsistent, and performance insights don't flow back to the creative team. The playbook suggests you know that building those bridges by requiring shared ids and unified measurement.

### Policy-Aware Agent Interfaces

(47:08) It's a practical step we take is to start small with our advertising teams like I said earlier by internally aligning with a cross departmental value stream mapping exercise that again forces teams to visualize the entire workflow identify bottlenecks and define clear ownership. And our consumer product team is doing this as well but it's it's not a panacea to everything here. It's going to take it's going to take a while and it goes to the question that was asked earlier.

### Controlled Interop Between Giants

(47:54) There's I think 2026 is when you're really going to start seeing more inend objectent content creation platforms that you you can't set it and forget it. That's for sure. There's always going to have to be a human in control over the loop, but you're going to get more of that streamlined systemic kind of the content that you're looking for and the measurement, the outcomes that you're you're looking for. I also think Oh, go ahead. Sorry.

### Machine-Mediated Collaboration

(48:16) I again, I also don't think there's enough people point like focused down the these these gaps, these silos, the the gaps between silos, right? So, you know, a uh really uh simple use case that I talked about with Google back in in the summer was why would why would any of our agency teams in the future need to send an email to an industry head at Google to ask for a benchmark of a click-through rate, right? Why wouldn't our agents just be talking to your agents? And you know what? in terms of like the the proprietary nature of Google's data, that's way better for for them, right? Because that if their agents know the

### Don’t Lose the Big Picture

(49:02) rules of what data they can and can't share, then they're going to be much better at enforcing it in a very logical manner. And so, actually, that's a great example of a silo where you can start to open that up, that aperture up and create like a a bridge across the the two things.

### Transform to Realize Speed

(49:27) Even, you know, even when you think about wall gardens, you know, I I I did actually say, why don't uh, you know, why doesn't Sundar and Zuckerberg have their own agents that are just talking to uh each other and sharing the secrets they're willing to share? Because actually that would be beneficial to both organizations and you're then in control of the secrets that that that you know, the things you are willing and not willing to uh share to.

### Securing Agent Interactions

(49:52) Right now when you leave that in the realm of human beings we have all this other stuff that we carry with us like ego and pride and thieft and building and all this sort of stuff you leave it to to machines where you can give them rules and boundaries then actually there would be much more free flowing information across these wall gardens.

### The Human Communication Gap

(50:17) So I think there's an opportunity to build uh build build kind of much stronger communication and flows of information across these silos. But but but that's not what people are thinking about right now. People are just thinking about how can I do things faster? How can I save money? How can I reduce jobs? All that sort of stuff.

### Measuring What Matters

(50:50) Well, and I think what you're also talking about is this overall transformation in the industry that needs to go on. uh, and you're right, people are, you know, thinking about the faster cheaper aspect, but to truly optimize the faster treat, you know, faster and cheaper, you need to transform the entire organization. not just uh who who the players are, but how they all operate together.

### Granular Creative Element Testing

(51:26) As you were talking about the whole Sundara and Zuckerberg uh situation, it made me instantly think of how long it took me to guard rail in Sora on what my cameo was being allowed to do. And I can't even imagine the level of security necessary to have two agents of that caliber uh discussing potential IP with each other.

### PII and Model Training Risks

(52:05) Like my my my brain literally blew up as you were talking because it sounds so enticing, but I don't know who's a lawyer in here, if there are even lawyers in here. But I I immediately felt like I'm not even a lawyer and I felt proud. uh, but I I I like that a lot because I think you're exactly right that they're the silos around these things are ridiculous and sometimes it's not even an intended silo.

### Myth: Perfect Data Hygiene Needed

(52:37) Sometimes it's just that we as humans are moving so fast we forget to communicate the important details. Sometimes it's literally just that. uh, but I wanted to and we only have four minutes. I did want to get one uh question in for you both about measurement. uh because we've talked about all of the front end and the challenges and the opportunities and the human in the loop, but what are we actually measuring here? Like what metrics are you actually looking at beyond like click-through rates, Brian, when you're looking at the effectiveness of the stuff? Moving way beyond click-through rates, right? And I was talking about this earlier like to include conversion and

### Hybrid Deterministic + Probabilistic

(53:01) customer lifetime value impact the CLV and consumer trust metrics uh opt-in rates satisfaction with personalization your NPS scores. They should also use a creative element analysis to decompose winning assets and isolate which specific elements are driving you know engagement conversion lift and we need to look at it on a very kind of granular level right and we're so used to kind of spray and prey and some slight modifications with programmatic machine learning but we're going to be able to really take it to an entire new level to understand what combination

### Human Oversight Remains Essential

(53:28) of assets That's let's take for advertising headline, background, call to action. What combination of those assets? Let's say you have a hundred each and and you can, you know, come together with thousands of variables. uh, what combination of those visual assets are actually working for specific individuals or audiences not only in a national levels, which is what we're so used to, but getting down into national, regional, local, and then hyper local, and then again down to the BIO to the individual level. So that's what we're looking at uh right now. Taking it as

### Risk-Based Oversight

(53:52) far as we possibly can while also addressing things like personally identifiable information. Now everybody's so concerned about the IP of assets uh going up into the AI, but we also have to think very very seriously about the information or personally identifiable information also going up into training open AI models etc. For sure.

### Calibrating the Leash

(54:19) I'm gonna ask you two last questions and I want you to tell me if it's myth or reality. Okay. First, you need perfect data hygiene before you can start personalization. Myth or reality? Myth. uh best practice is a hybrid approach. uh you're using deterministic data to anchor strategy while probabilistic models expand reach.

### Careful Steps Toward Autonomy

(54:47) So, don't let the pursuit of perfect stop you from starting. Well done, Graham. Yeah, I myth. I mean, nobody has perfect data hygiene. I have not met the business that has that. So, if that was the case, then none of us would be doing anything. So true. Okay. Second, human in the loop is necessary for personalization. That's a spicy one.

### Thanks and Next Steps

(55:28) Myth or reality? Yeah, reality. Humans define strategy and judgment. AI handles scale. Human oversight remains essential for brand safety and accountability. Grant. Yeah, I'm I think miss in some instances reality in uh others.

(55:46) And I think this comes down to that kind of risk approach, right? I think that uh I I think in and and uh, Earl kind of wrote this in uh there. I think sometimes in our natural instincts to protect ourselves, we make ourselves way more important than we are. And and and and it's just a natural tendency. And so I do think that there are there are aspects that we have to be prepared that, you know, maybe we need to to kind of shorten the leash and and let some things happen.

(56:10) Otherwise, you run the risk of never being able to move at the speed of AI and therefore you'll never realize the benefit of it because all you're going to do is do something fast and then stop for a long period of time while while the human uh does the looping part of it.

(56:10) And so I think that it's about identifying the parts that can that that that maybe are less frequently reviewed by a human being and maybe they're not reviewing assets. They're reviewing restrictions, constraints on models, things like that. uh and then stuff that yes is a prerequisite. You just can't. We're just not willing to let that live without a human having reviewed it first. Yeah, for sure.

(55:46) where we're doing that with a couple things, Graham, to your point where we're sticking our toe carefully in the water with autonomous, you know, running of stuff, but uh it's really about, you know, human in the loop for right now. Well, Brian and Graham, I want to thank you so so much for joining us. And David, turning it back over to you and thank you for having us.

(55:46) Yeah, thanks everyone. This is amazing. We'll make sure to share the recording and uh and of course the personalization study. We'll make sure that goes out to everyone who registered along with the rest of the community. Brian Graham, so great to have you as part of this and you're uh uh now however you participate going forward, you're you're part of the community and and welcome back anytime. So, thanks everyone for the great questions and uh everything tonight.

(56:10) I hope to see some of you at the first Wednesday tonight if you're in New York. So, so uh so much going on and uh if you're local for these holidays, it be great to see you before the end of the year. Thank you so much. and and Carolyn's just keep doing these. These are amazing. Thank you.