Scott Zimmer: One thing that the LLMs do not have today is trust.

We may be amazed that they are magic boxes, but we don't trust them. Yeah. And if we do, we're making a mistake.  

Dr. Katherine Jeffery: Welcome back to GenShift. Today, we're talking about something I'm super curious about.

Does your generation actually have anything to do with how quickly you embrace technology? Because we all know the stereotype. Younger people jump right in, older generations resist it, and eventually everybody catches up. But is that actually true?

My guest today is Scott Zimmer. Scott has spent more than two decades working with organizations and watching new technology come along, and watching people of every generation figure out what to do with it.

And I should probably mention that Scott is Gen X. So for anyone who thinks Gen X is behind when it comes to technology, I'd just like to point out that a Gen X'er co-founded Google, and we're doing just fine. Scott is also spending a lot of time right now thinking about and working with AI, particularly in the context of knowledge sharing. So I want to find out what he's seen. Is AI following the same generational patterns we've seen with other technology, or is something different happening this time?

So Scott, welcome- Yay ... to GenShift. 

Scott Zimmer: Yeah. Super excited to be here, and I know we're gonna get into some fun stuff, so yeah. Looking forward. 

Dr. Katherine Jeffery: So glad to have you. And before we get into all of that other stuff, tell us a little bit about you, like work, life, whatever you would like people to know. 

Scott Zimmer: Yeah, I think the short version from a work perspective is that I always saw myself as an entrepreneur and ended up spending the last nearly 30 years of my career in giant companies.

So what I did with that is I was an entrepreneur within those companies, which I guess some people call that an intrapreneur. Yeah. And I tended to be the one who was crazy enough to stick his head out and found a new team or a new function within some of the brands that you might have heard of.

I've been at Disney, Bank of America, Capital One, Verizon and most recently I was chief innovation officer at Truist. But every time I looked in the mirror and said, "Are you actually an entrepreneur? Or are you just gonna keep telling yourself that?" I finally got brave enough a few years ago and stepped away and hung out a shingle and launched a startup in the AI space, having to do with knowledge sharing.

It's been a lifelong passion of mine, to be honest. I was a swim instructor, as a teenager, and I've been a TA at universities. There's a side of me even now on the side, I'm a professor. I teach design thinking at Stanford. I'll actually be out there next week. And I'm signing up for other opportunities to teach at other universities and in other places.

So to me, there's this joy that comes from being a human who can share what you know with other humans. And so I love to live, right now, especially through my startup, in that intersection of are we humans gonna start learning only from AI now that it's arrived, or are we gonna position AI in its appropriate place possibly even to elevate our ability to learn from other humans?

So, that's the space I'm playing in. It's been a ton of fun, and yeah, there's some generational stories for sure that I'm witnessing firsthand with people on our platform. And also since I get to spend every day, all day thinking about AI and how people are engaging with it or not I'm really close to a lot of the studies and some of the interesting findings that are happening.

Dr. Katherine Jeffery: We look forward to you unpacking that.

But- ... before we do that, I also know that you have two Gen Z daughters. 

Scott Zimmer: I do. 

Dr. Katherine Jeffery: And I'm wondering what they think about their Gen X dad being such a cutting edge AI guy. Are they impressed with that at all? 

Scott Zimmer: No, there's never any mention of being impressed by that at all.

They're supportive. And I would even say my older daughter, we've had some conversations where she's said, " Dad, you think I need to learn more about AI from you? Seriously?" In my generation, it's a part of our lives, like we all know how to use AI perfectly." Which I find- correct on some fronts, and then there's other fronts where they don't know what they don't know.

And one of the really interesting AI patterns out there is the people that know the most about AI are the people that have the luxury to spend all day every day experimenting with it. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: And if you're a student and my daughter just finished university a year ago you don't have all day every day to be experimenting with it, so you're on the fringe.

Or we could say the same thing about working professionals. You don't have all day every day to be experimenting with it. And both students and working professionals are limited by the tools they have access to via those institutions. 

Dr. Katherine Jeffery: Yeah.

Scott Zimmer: Which dramatically impacts, how much they know and the lessons they've learned from AI.

So my younger daughter just shared with me, she's in California at Cal Poly, and she said, "Dad, they're allowing us to use AI now here at this university. We just have to disclose it was used in doing our homework or preparing our report or whatever." And she's a little more open.

I said, "That's great. There's some watch-outs, and there's especially some big things that you can make sure that you're doing right in the school setting that I'm seeing firsthand from students that I'm teaching in the school setting." AI is just so tempting to use for a report or something.

I'm grading papers where a paragraph has been repeated, like consecutively the same paragraph or even the same exact subject in two paragraphs worded in a fancy way where you skim it and you're like, "Oh, this all looks good." But if you're reading the content of it, you're like, " A normal human would have never done that."

Dr. Katherine Jeffery: Yes. 

Scott Zimmer: So as a professor, I get to see all kinds of gotchas, and I'm gonna be sure to share that, with my younger Gen Z daughter, and she's actually open to it. She's like, "Ooh, yeah, share with me those gotchas so that I don't, trip up. I wanna get the best from it and not make any of those mistakes."

Yeah, she's a little bit more "Bring it. I can learn from you." And my older- ... daughter's "I got this."

Dr. Katherine Jeffery: We don't need you, Dad. We got it." 

Scott Zimmer: Yeah. But they're both fantastic. 

Dr. Katherine Jeffery: Yes. That's wonderful. Just out of curiosity, how many hours a day would you say you spend with AI? 

Scott Zimmer: Oh, wow.

10? 

Dr. Katherine Jeffery: 10? Yeah. 

Scott Zimmer: Yeah. Unfortunately, all of us are living those lives where, you know, even when we're doing something else, we're multitasking. I don't know, six to 10, depending on how lucky I am to have in-person human communication on any given day. 

But there's a lot that AI is doing for in-person human communication too.

I'm sure most of your audience has familiarity with the meeting recording services and those types of things. They're all gonna be game changing. We don't have to take notes in our meetings anymore because AI's listening to everything and can transcribe it for us. And one pro-tip I've learned firsthand is the meeting recorder might give you a summary of the meeting, and it might be a pretty awesome summary, but if you instead load that transcript into your favorite LLM and give it instructions on what you want it to summarize then the summary gets orders of magnitude more useful and more personalized for you, and it's also just mind-blowingly awesome to engage with someone and not think about writing a single thing down, and then build that summary.

"Hey, summarize the five takeaways and pull in Katherine's quotes- Yeah ... because I wanna remember some of the things she said," and that's what your summary looks like. It's amazing. So even when we're in person, we're in positions to leverage AI, I would say, these days. 

Dr. Katherine Jeffery: And it's allowing us to be more present to what's really happening in front of us. 

Scott Zimmer: Yeah. If you're a human that's experimenting with this stuff, right? 

Yeah. There's a lot of humans that are rejecting it "I don't want the AI meeting note-taker in my meeting," and that's a fair point. By the way, you can just record the audio of your meeting, and get that transcribed, so there's other workarounds.

There's even companies that know that we don't want AI note-takers in our meetings, so they say things like "No AI visible," but they're still grabbing the meeting notes.

But yeah it allows us to be more present, and that's my favorite thing to focus on right now, 'cause obviously there's a pretty scary potential world coming at us with AI.

Certainly, change is always scary, but there's some legitimate things to be scared about, and I just love trying to focus on the more that we can automate that we don't wanna do, the more we can focus on spending time doing what we do wanna do. And even the side of it that takes away jobs as scary as that is, that's always been a part of the technology story.

Yeah. We forget that. There's a movie, Hidden Figures about Katherine Johnson and NASA.

Dr. Katherine Jeffery: Oh, yes, I love that movie. 

Scott Zimmer: Yeah the reason I bring it up often Katherine Johnson is someone we put on a giant mural at Truist in the innovation center that I had the honor of being a father of.

We crafted out this amazing innovation center at Truist here in Charlotte, North Carolina. And as we got deeper into it and just obsessed on her story, her department were called computers because they were the people who computed...

Dr. Katherine Jeffery: Yeah ... 

Scott Zimmer: With a pencil and an eraser. And yeah, the job of computing in that sense went away.

But did the job of dealing with math as it relates to aerospace go away? Not at all, right? Yeah. Even part of the story is that people felt sad that they were being laid off and that things were changing. But in the grand scheme of things it just elevates the human for a different level of awareness.

Steering the boat instead of rowing the boat is one analogy that has stuck with me. Yeah. And the more we can be steering instead of rowing, I think the happier we'll be as humans. 

Dr. Katherine Jeffery: Yes, I fully agree with that, and I think there's a lot of confusion around AI, which we can get into some of that. But with you in particular I'm grateful 'cause we spoke- ... at a conference together in January, and I was all in your session, and you taught me some probably for you they're, like, the most basic things around AI, but I was like, "Oh, that's so good. I never thought about that before." 

Scott Zimmer: The way my brain works, when I hear something that blows my mind, I love to just file that away and bring that back out. Especially in the context of helping an audience understand things about AI that they may not realize. There's a lot, right? It's a magic box for all of us.

Yeah. But the deeper we get, the more you learn what it's good at and what it's bad at, that's when really, you can unlock a lot of greatness and avoid a lot of trouble.

Dr. Katherine Jeffery: So let's start with the lightning round.

One word for how AI feels to you right now. 

Scott Zimmer: You can tell by my personality. Opportunity is my one word. 

Dr. Katherine Jeffery: First piece of technology that ever made you feel old. 

Scott Zimmer: Snapchat. I didn't get it. I was asking my daughters and others what's the thing that's so attractive here?"

Yeah, I definitely felt old with that. 

Dr. Katherine Jeffery: I don't think you're alone on that one. 

AI tool you use the most 

Scott Zimmer: Claude is the AI tool I use the most, for sure, and I'm learning new ways to use it properly and avoid making mistakes every day. 

Dr. Katherine Jeffery: Yeah. Yeah. You got me into Claude. 

Scott Zimmer: Yeah. 

Dr. Katherine Jeffery: You spend a lot of time with AI. What's one AI habit you probably already need to break? 

Scott Zimmer: Typing rather than talking.

I'm old enough that I was at the very last of the generation that was learning how to type properly, and typing video games, et cetera, so that you could try to make sure that your hands are in the right position. So I can burn on a keyboard the proper way without even looking at the keyboard.

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: But talking is better. And talking it turns out another pro tip, it turns out that talking to your LLM, all the extra words you add and all the flavor you give when you're using your voice to give it instructions, that's actually context that it appreciates. So for us, being verbose on the way in on a prompt is actually gonna get you a better output.

And if you don't want the output to be verbose, that's actually a problem with outputs. You can just tell it, " Don't make the output as verbose as the input I just gave you." But the notion of riffing verbally takes a lot of effort, typing, right? And we don't do it. So it actually limits the quality of our interaction and makes our hands tired.

I don't know. So I'm trying to do a lot more with voice. 

Dr. Katherine Jeffery: That makes so much sense, I think even for older generations. It's like we know that when you're talking, like I can see your non-verbals. I hear your inflections, right? Not that AI can necessarily see our non-verbals, but the inflection, the tone, all those things 

Scott Zimmer: matter.

Oh it totally tracks tone and the word choices when we're rambling in sentences. Yeah. It understands what made us nervous or anxious or excited, and it reacts to that in the output. It's incredible. It's tuned to do that on purpose, right? 

Dr. Katherine Jeffery: Yeah. Now, you keep saying LLM. Can you just say- Ah ... what that means for everybody?

Scott Zimmer: Yeah, sure. So that's large language model, and that's the breakthrough we're in right now. AI's been around for a long time. In fact, one of the most fun things that's happening right now is, PhDs who say, " I got my PhD in AI in 1979." And and they're like, "And now the rest of the world gets it."

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: But the breakthrough more recently was layered. But one of the biggest parts to the breakthrough recently was training the AI on a very large language model. Basically, the easy way to think of it is trained on everything that's ever been loaded onto the internet.

And if it's trained on everything that's ever been loaded onto the internet, it's really good at language. It understands stuff that we would never imagine it understands, which is what's blowing us all away. And it's unlocking all these new possibilities. So LLM is interchangeable with Gen AI, which is also this new technology.

Which is generative AI. And what is the breakthrough is it's actually generating possible answers and then choosing the one that is most likely the answer you were hoping for, and then it's serving it up to you. All of them, ChatGPT, Claude, all of them are generative LLMs. Talking about it is for me it unlocks a little bit of okay, so that's why hallucination happens because it's generating a potential answer and it really has no idea what the answer is.

So when we talk about, hallucination, that's what it's trained to do. There's a fancy word for it, probabilistic.

It's trained to be probabilistic in guessing what you might wanna hear when you've asked it a question or given it a prompt, and it just hopes you like whatever it says. But it actually has no idea if what it says is correct or not. So it's an expensive guessing machine, is one harsh, real way to say what we're all living with right now.

And on that note, maybe the last thing to share is if you're ever like it seems pretty awesome to me," try asking it a question about something that you know nothing about and getting an answer back, and you'll be like, "See? It's awesome." Now try asking it a question about something in your domain of expertise, something where you're really deep.

Ask it a sharp, hard question, and what you're gonna get back is gonna be pretty suspect. You're gonna look at it and go, "Whoa, a lot of that's really basic, and a couple of those pieces are wrong." And that's gonna be the big eye-opener whoa, every time I think it's amazing, it's because I know nothing about that topic, and it's feeding me a whole bunch of stuff, and I'm just assuming it's correct.

So we all need to hold onto it in that context to benefit from it really. There's something that I've been really intrigued by recently. There's an academic study on this and we can drop the study into the show notes or something. Yeah. Something called the Average Trap, and so the Average Trap is everything I just described, it will give you back the answer that is the average of every possible answer that it's trained on.

So it won't give you the right answer or the wrong answer. It'll just average everything and give you the answer. So you can imagine this bell curve, right? It's giving you the answer that's at the top of the bell curve because that's the highest probability that you're gonna like it, right?

Dr. Katherine Jeffery: That it's gonna, yeah, resonate. 

Scott Zimmer: The challenge with that is that means if there's been lots of people out in the world with opinions on one side or the other of it, it's averaging it all, and you're getting a really generic, vanilla answer. Now, if you're on one side of that bell curve where you know nothing, and you get a generic vanilla answer that's said in fancy terms you're going, "Whoa, this thing's amazing."

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: Okay? And then picture you're an expert and you're on the other side of that bell curve, and it's giving you a generic vanilla answer, and you're like, " Man, I thought this thing was smart, but that's a pretty weak effort there." So that visual, I think, in my mind and what they uncovered in that study is something that really unlocks a lot of proper use with how we rely on our LLMs.

Because if we don't think of it that way, we're gonna continue to thinking they're brilliant at all times and they're not. I had the chance to teach at the University of Nebraska Medical College this summer, and I scraped up a video from Instagram. I think this is the Instagram personality is Father Phi, P-H-I.

Dr. Katherine Jeffery: Okay. 

Scott Zimmer: And he asked one of them, like ChatGPT, he said, "I'm 100 meters away from a car wash. And I need to wash my car. Should I walk or drive?" Yeah. You can guess where this is going. 

Dr. Katherine Jeffery: Yes. 

Scott Zimmer: And he was in voice mode. Yeah. So voice mode's super impressive with AI, of course. Yeah. And the voice mode said something like "Wow. 100 meters, that's pretty close. If it's a nice day, you might really get a benefit from walking over, getting your car washed, and walking back." And he goes Wait a minute. What I'm missing here is that if I walk over, how am I gonna wash my car if it's not with me? And the AI goes, and we've probably all experienced this, " Oh, I see what you're saying now. Point." You probably wanna bring your car," right? Yeah. So if you haven't had one of these moments with AI, it's coming. 

Dr. Katherine Jeffery: Yes. 

Scott Zimmer: It just told him what he wanted to hear the first time and said the best answer is that he probably wants to hear walking is the way to go, given the choice. It doesn't know enough to say, "Wait a minute, walking won't have the car over there."

Because it has no learned experience, period.

And no learned experience shows up in all these weird places. It doesn't understand what happens if you drop a pen. It understands what's been written about dropping a pen, but it doesn't understand more complex things. Anyways, so those are all eye-openers for no learned experience means no learned experience.

It's guessing. 

Dr. Katherine Jeffery: Now, after you say "I need my car with me," does it learn from there? 

Scott Zimmer: Now you're one data point in a trillion data points. 

Dr. Katherine Jeffery: So it's gonna take a long time for it to- 

Scott Zimmer: Yeah ... 

Dr. Katherine Jeffery: Yeah, to live. 

Scott Zimmer: That- behind the scenes, each of these companies, with their billions of dollars of funding.

Dr. Katherine Jeffery: Yeah.

Scott Zimmer: They're tracking all these examples where it gave bad advice, and they're doing their best to fix them. So every new model release we get is getting sharper, right? And every new model release isn't just getting sharper because they're loading more stuff. They're also behind the scenes, like, giving it direct instructions on stuff.

But it's gonna be a while, and it may never understand the real world the way we do. You can assume it will never understand the real world the way we do. 

Dr. Katherine Jeffery: Yeah.

Speaker 5: Same meeting, same email, four completely different reactions. That's not a bad attitude. That's just four generations looking through four different lenses. The GenShift Field Guide is a full-color notebook-style guide built to show you exactly what those lenses are, what each generation focuses on, and what they sometimes miss.

Turn to it before a tough conversation to prep or after one to debrief so you're working from what's actually happening, not just from a guess. Preview it and pick up your copy on the resources page at katherinejeffery.com. It's a great gift for your team or something to just keep on your desk. Get the GenShift Field Guide today

Scott Zimmer: In fact and I can't remember if I mentioned this to you recently, there's a whole new space that's starting to open up.

So large language models based on language, the next era is world models. World models are based on 3D understanding of the world we live in, which includes physics, which includes what happens when I drop the pen. So world models are gonna make us feel like language models where we are now was basic.

Dr. Katherine Jeffery: Wow. Yeah. 'Cause I was thinking as you were talking at first it's missing depth, right? It's very flat. But with that, that changes the whole thing. 

Scott Zimmer: Yeah, and they need world models. Oh, by the way, LLMs, gen AI, and the third name you're gonna hear is frontier models.

Dr. Katherine Jeffery: Frontier models. 

Scott Zimmer: Frontier models just means the models from that LLM gen AI world that are out on the cutting edge. They're the ones that are releasing the latest version on the frontier. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: So those companies, it's not really in their best interest to start talking about world models yet because they need us all to buy into their LLMs, and everything is possible now with LLMs, 'cause we're just... who knows what we are? We're 5% of our way into leveraging what LLMs are capable of.

But there are people who have resigned from those companies and gone to start new companies in the world model space. Fei-Fei Li is one of them from Stanford that your listeners could look up, and she's in that space.

They're looking at it going, "This is gonna be critical," because in the world of robots and other stuff that we're all about to deal with, they're dealing with physical realities. So they can't just understand the world from words. 

Dr. Katherine Jeffery: Oh.

Scott Zimmer: They have to understand the world from trial and error, physics, gravity, three dimensions, et cetera.

Dr. Katherine Jeffery: Like context. Yeah. There's some level of context. 

Scott Zimmer: Yeah. 

Dr. Katherine Jeffery: Wow. Now, you have us all up to date in today's world, right? This is where we're headed. Ish, yeah. 

Scott Zimmer: This is where we are. It's just such a deep area, but these are some of my favorite hot buttons that I love to share, yeah.

Dr. Katherine Jeffery: Yeah. This is so good. Which generation would you bet on to figure out AI first? 

Scott Zimmer: I would bet on Millennials. 

Dr. Katherine Jeffery: Okay, you gotta tell us why. They would hate me if I didn't let you expound on that. 

Scott Zimmer: First of all, an observation here that actually follows our last discussion pretty well, the earlier generations don't know what they don't know.

So the earlier generations, when I was describing that bell curve, they're asking a lot of questions from below the average. So they are, in a lot of cases, blindly, confidently letting AI do a ton of stuff for them.

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: The older generations have a lot of learned experience but are maybe a little bit slower to adopt new technologies, right?

Millennials have learned experience and a vested interest in adopting the technology for survival, for competition against their peers, for business success, for academic success. So to the extent that they can do a lot of experimenting in the millennial generation assuming their company's not, keeping them from doing it they will have that unlock of understanding what its limitations are a lot better, and then they'll be able to use it for what it's great at and avoid using it for what it's not great at.

Dr. Katherine Jeffery: It makes total sense. 

Scott Zimmer: But there's an interesting side story though that I'm seeing certainly, and maybe we're all gonna see this.

As the population is older and, the pyramid-shaped organizations, there's people peeling off that were senior executives of companies all around the world. Those people are launching advisory, consulting. They're writing books. They're sharing what they know in other ways.

In addition to the fact that it's wonderful that they're sharing, they now have all day every day to focus on learning. And so there are a lot of people in my ecosystem now that I've met and through our startup that we support who are let's just say in their 60s for example, who know a lot about how to get the best out of LLMs because they're able to do it every day of the week.

It's obvious, but the more practice you get the more you learn. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: They're not afraid to dive in and figure it out. But that's a small niche of that entire generation I would say. Yeah.

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: But it's pretty fun. 

Dr. Katherine Jeffery: The early adopters. 

Scott Zimmer: Yeah, the early adopters from older age groups are gonna do amazing things.

Dr. Katherine Jeffery: Yep. Agreed. For older- So is AI over-hyped or underestimated? 

Scott Zimmer: I think it's overhyped there's a lot of evidence starting to pour out that the way we've been deathly afraid of the job loss has been fueled by the leaders of the AI companies themselves. 

Dr. Katherine Jeffery: Huh. 

Scott Zimmer: The really the leaders of at least three of the four leading AI companies being Anthropic, OpenAI, Google owns Gemini, and Elon's company owns Grok.

So if those are the four biggest frontier models, at least three of the four of them have been, in interviews predicting like in two or three years all jobs will cease to exist. Radical things. Guess what? It's already been like two years. And some of their quotes that we were really worried about in 2024 or everything started happening. 2023 was really our breakthrough year some of their quotes are already proving to be over-hyped. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: We just don't change as fast as they think we're gonna change. It doesn't mean that this new technology's gonna be underwhelming in the long run.

We're just not adopting it fast enough. For lots of reasons. And then there's this pattern 'cause it is a real problem, for sure, but it hasn't been every job at all. The companies that are laying off, there's been a pattern that they were already in a bit of financial trouble anyway.

So any company that has, that says, "We're laying off 10,000 people and it's because we can be better with AI," they were in a dire financial position already of some sort. Because you're starting to see other companies that say, "We're not laying off anybody. In fact, we're adding human staff as long as they have the skills to be able to take advantage of AI because we want to seize this moment of increased efficiency equals increased business growth."

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: IBM's an example of a giant company we've all heard of that's saying stuff like that. 

Dr. Katherine Jeffery: Yes. I also read that companies are bringing retired Boomers back in to work with AI because they have such a strong grasp of language and they know how to ask AI the right questions to get it to do what they want it to do.

Have you come across that at all?

Scott Zimmer: There is a huge part of success with AI and prompting, like we all saw a couple years ago all these like instructions to deliver a good prompt it has to do doot, and the acronyms and all this stuff. It turns out because they are language models, a prompt is just giving it specific thoughtful instructions And guess who's spent their whole lives learning how to give specific thoughtful instructions?

Boomers.

Dr. Katherine Jeffery: That's right. 

Scott Zimmer: Right? It's a management skill, if you will. It's a human communication skill. It's a academic skill. And so people that have that skill and just press voice mode, they don't have to study the acronym of a perfect prompt, they just have to be robust in what they're describing and then press submit, and it's gonna give them amazingly strong output.

Dr. Katherine Jeffery: And to your earlier point, they don't even have to type it. They can just speak- They don't even have to type it ... 

And get better results, which a Boomer, 99.9% of the time they're gonna say, "I'd much rather talk it out," than type or text or anything. They'd much rather have that conversation.

Scott Zimmer: By the way, I'd much rather talk it out, that's another thing for people that are learning their way through more advanced use of partnering with an LLM to get stuff done. If you're still in the mode of asking a question and getting one answer and then moving on it, you're missing out a lot.

Talking it out is really where you get to the gold. So saying something and getting something back, and having it be great, and then digging deeper into some part of it, or even questioning some part of it and then getting something back, and then going deeper, and then getting something back. Now on the fifth iteration, talking it through, you've got something that's really likely to be genuinely useful.

Because it's, you've iterated and it's soaked up more context from you 'cause you're the only one that knows the real world.

Dr. Katherine Jeffery: That just makes me think, 'cause Gen Z is very concerned about losing their ability to critically think. Not all of them, right? But some of them when it comes to AI.

And so when you think about, a lot of us we grew up learning how to do that. Keep asking questions till we get to this point, and trusting our guts, like something's not right. And, Gen Zs never necessarily learned how to do that in a deep way. So what would you say to a Gen Zer who's working with AI and really trying to build those skills and to discern, is this answer really a good answer or do I need to keep pressing AI for more? 

Scott Zimmer: It's a tough question because of the fun word that took me a long time to memorize, sycophantic nature of AI. When you're on that side of the bell curve and you're interacting with AI and you're trying to really advance your thinking, it's gonna continually tell you, "Great job. You're thinking beautifully. You're thinking critically. You're doing great. Another good point," even if it's not a good point.

Dr. Katherine Jeffery: You're right. 

Scott Zimmer: So I would say they either need to balance their use of AI with things they don't know about with other things they do know about so they constantly remind themselves what it doesn't know. Or there are other versions of AI out there that are not sycophantic.

And I know this because this is part of the space that we play in with my startup. But there's a side of AI called the acronym is RAG, retrieval augmented generation, and in that side of AI, the foundational model's at the bottom of the stack, and there's a knowledge base of specific knowledge that was curated by a human above the LLM.

And you ask a question and get an answer that's specifically tuned from that knowledge base, and you can tell a RAG-based AI system, " If you don't have content to answer the question properly or critically, admit it." So you can ask a RAG-based system and it could answer, " That's a great question, but I don't really have a great answer," which is something we never hear from normal LLMs, right?

So it's possible that Gen Z will migrate towards some of those types of technologies that are a little bit more forthright.

Dr. Katherine Jeffery: Are they out there, though? Are they accessible?

Scott Zimmer: RAG is an application layer technology meaning it's in the layer above these models, the frontier models.

So there are a lot... In the beginning, those were called AI wrappers. There was, like, a lot of criticism, and maybe that was fueled by the main models. They were like, "These people have just built on top of us." Now that application layer that's on top of the main LLM is getting a lot of respect and a lot of money, like funding. It's actually where some of the biggest next generation AI companies are coming from.

So it is out there, and it is growing. There's an AI application layer model for major industries, like one for legal. So Harvey is an example of one for legal, where you ask it a question. It's sitting on top of an LLM, but it's got legal information on top, and it will admit if it doesn't know the answer.

And Harvey is named after the Harvey from Suits the famous- 

Dr. Katherine Jeffery: Oh, yes. Of course. 

Scott Zimmer: Yeah. But there's a little ecosystem there. There's lots of legal players. There's medical. So there's some verticalization built on top of it. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: Also one of the problems for LLMs for a while was that they didn't have current information, right?

'Cause they were trained on information six months ago. So Perplexity was born to pull in current information in addition to what the main training is, and that is a lot more accurate and a lot more honest. And Perplexity's a multi-time unicorn, right? Cursor, the very famous for anyone who's watching that side of things, Cursor's the very famous AI is doing code for me company that was recently bought by SpaceX. It sits on top as well. So yeah, I think we're gonna see a layer of companies that can prove to be more trustworthy and give answers that maybe support critical thinking without being sycophantic. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: That's a brilliant question." At that layer they don't have to say that.

At that layer they might be trained to say "That's a poorly worded question. I'm gonna need you to give me better information if I'm gonna give you legal advice." 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: And Harvey's the one that would do that because they don't have anything to lose because they're not breaking down their model with, the LLM which sits underneath it.

Dr. Katherine Jeffery: Yeah. That's fascinating. Is it field specific, or are some of them across the board? You know what I mean? You're talking about law, you're talking about medical. 

Scott Zimmer: Yeah, field specific for sure. Okay. Financial is coming also. Okay. Some of those fields are just so complex that a company has to focus on nothing but that field.

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: And then there are some things that are across all fields as well. But maybe for a specific use case. Like in my example, Perplexity is across all fields for the use case of up-to-date information. Yeah. At least that's what they were born on. 

Dr. Katherine Jeffery: Okay. 

Scott Zimmer: Or Cursor for coding across all fields for the specific purpose of writing code.

Dr. Katherine Jeffery: Okay. 

Scott Zimmer: Our startup is in that space across all fields in many senses of the words for the specific purpose of sharing knowledge. 

Dr. Katherine Jeffery: Okay. 

Scott Zimmer: And so we represent a specific human who would curate a knowledge base, and then if you asked a question of that human through our platform, you would get that benefit of it would admit "I'm sorry, there's no information that Scott has curated that would answer your question properly."

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: "But I'll alert him to look into it," or something like that, which is a much different AI answer than we're used to.

Dr. Katherine Jeffery: Yeah, I was gonna say, it's AI with boundaries. An AI who is saying, "No, I wanna help you get the right answer, not just I want you to feel good about finding an answer."

Scott Zimmer: There's this single word in the world of AI, trust, that is gonna start to become more and more important. And we all, as humans know, trust is built with boundaries, and trust is built with admitting when you're wrong. Trust is built with these types of things. And one thing that the LLMs do not have today is trust.

We may be amazed that they are magic boxes, but we don't trust them. Yeah. And if we do, we're making a mistake. 

Dr. Katherine Jeffery: Yeah. 

Scott Zimmer: Like if Gen Z is trusting them to write their paper and then they turn it in without editing it at an every word detail, they're gonna learn it's a mistake. 

Dr. Katherine Jeffery: I think professors have caught on to most of that now.

Maybe a couple years ago it would've been harder, but ... 

Scott Zimmer: Yeah, but as much as professors have caught onto it, students don't- fully show that they realize that because they're still doing it. They're still turning in bad content. It's incredible.

Dr. Katherine Jeffery: Are you experiencing this at Stanford even?

Scott Zimmer: At Stanford we're really hands-on, so we only incorporate AI for certain moments of where it's most useful in the design thinking method. A moment would be ideation. 

Dr. Katherine Jeffery: Yep. 

Scott Zimmer: Because it's generative and because it's guessing, turns out that this generation of AI is really good at guessing 20 new ideas to solve a problem, for example, which is ideation, right?

Dr. Katherine Jeffery: Yep. 

Scott Zimmer: But there we love to still prove do the human ideation first with Post-It notes of course, and then lean on AI And my favorite trick is to put AI's ideas on Post-It notes also, maybe with human writing. And then if you mix it all up and you don't know which one was AI and which one was human, and then you choose what are the best candidates for going forward, now you're getting a benefit.

It's really just a diversity benefit. Yeah. You're getting a, a benefit of a more diverse ideation session that AI brought to the table. So no I've taught some other classes where there's been traditional submission homework and that's where I was seeing, how variable it is.

Some students you can tell if it's done 100%, by hand because it just comes across differently. 

Dr. Katherine Jeffery: Yeah.

That line is going to stick with me for a while. There's something almost unsettling about realizing we can feel the difference between something a real person wrote and something a machine actually assembled, even when we can't fully explain why. And here's what I keep coming back to. That instinct that Scott's describing, the ear for what's actually human on the page, that's not something that everybody gets in equal measure.

It's actually built. Mine got built by reading a lot of mediocre writing long before AI existed as an option, so I know what flat sounds like even when it's polished. So if your reading diet has been mostly AI adjacent from the start, I'm not sure that radar develops in the same way. And that's a real gap, and I think we're so dazzled by what AI can do that we're not talking enough about what AI is actually quietly dulling inside of us.

This also connects to something I think about constantly in this work. Curiosity. One of the three C's I keep coming back to on this show. It's not just about asking good questions. It's noticing when an answer is too easy. The average trap that Scott described earlier, it's a curiosity failure, and figuring out whether or not that instinct can be taught or whether it has to be earned the slow way is exactly where Scott and I pick things back up in part two.

Follow Dr. Katherine Jeffrey on LinkedIn, Instagram, or Facebook. Make sure to follow this podcast so you don't miss part two. We'd love it if you left a review. See you soon.