Tearsheet Pro Live #1: ChatGPT, fact and fiction: What FIs should know about the future

Dev Patnaik is the CEO of Jump Associates, the leading independent strategy and innovation firm. He’s a board member of Conscious Capitalism. Dev has been a trusted advisor to CEOs at some of the world’s most admired companies, including Starbucks, Target, Nike, Universal and Virgin. Dev is a frequent keynote speaker at major forums, and his writing has appeared in BusinessWeek, Forbes, Fast Company, and many others. He is the author of the book Wired to Care, named one of the best books of the year by both Fast Company and Business Week. Malcolm Gladwell called Wired to Care “just what we need for the lean years ahead.” When not at Jump, Dev’s also an Adjunct Professor at Stanford University, where he teaches social science methods to MBA and design students.

Watch the Episode

Listen to the episode

Subscribe: Apple Podcasts I SoundCloud I Spotify I Google Podcasts

The following excerpts were edited for clarity.

ChatGPT as an opening

Dev: I was amazed by ChatGPT’s 3.5 release, and we all got to see it. And I know, what is particularly impressive is that some of that technology had been sitting on the shelf for a little while, then they decided to pull it forward, dusted it off and, and got it ready for human consumption. But you know, even for our talk today, just to prepare for this yesterday, I asked ChatGPT to tell me why AI is great, the biggest challenges that are going to be there, and then, tell me the biggest reasons you think that financial service CEOs will think that they don't need to worry about AI or ChatGPT. And it’s spot on, just a really nice primer

It didn't tell me anything surprising, but it got me to baseline.

Moses: I think it will have an impact on every industry. I think it's probably the easiest interface to interact with large corpuses of knowledge base. 10 years ago, it was oh, it's all bots that do something blah, blah, blah. And then that technology. I mean, there was promise but it never matured enough to be good as a product, like an actual product you want to use. 

I think that the huge jump that we saw in the last couple of years made that technology available. And now what we were promised 10 years ago is a reality, which is really sci-fi. I think that l just find it’s a better way to interact with a lot of information that is usually very hidden behind – whether these are websites or documentation, or maybe some regulations. It's just a lot better at summarizing a lot of very complex information for you and making that accessible and the interfaces. Just asking and getting the result in human language, that's something that we've dreamed about for years. And now it's a reality.

Impact on financial services?

Dev: I think it's already having an impact, right? If you talk to any financial services firms of any size, they've had folks with PhDs and AI already working in their organization for a few years now. The biggest difference was they were ignoring those people. Execs were just not paying attention to them or saying, well, that's good, that's cute, but we're not going to give you too much money, and we're not going to have it affect our strategy at all. 

So, the real value of ChatGPT is that it is a form of machine learning, a form of AI that human beings, ordinary civilians can actually interact with and get freaked out by, because it sounds like hedge funds were using similar versions, earlier instantiations of this technology for some time for investing. It's not like other people weren't using these models, developing these models to help figure out where a bank should spend their marketing dollars. But it's just that the CMOs were ignoring that information that was coming out.

I don’t want to get too academic, but heck, I moonlight as a professor at Stanford. There's a difference between an invention and an innovation. An invention is something cool and new. And innovation is something cool and new that actually has a socio economic impact. So I was talking about which is a bigger innovation: a hovercraft or disposable diapers?

Hovercrafts walk on water, and disposable diapers are just some plastic and some tape strapped to babies. But disposable diapers are a much greater innovation in terms of how it has changed how we raise our kids, what we do to the environment, and so forth. 

ChatGPT is the start of what has been happening in the space of invention, turning into real innovation, that it's time to pay attention.

Moses: I think it's all about the size of your target audience in terms of what you can approach or who you can approach. You're very accurate at saying before it was mostly researchers, just kind of a niche, if you think about the financial sector. And now you have an interface that basically anyone can interact with a knowledge base the same. But that has an impact. As before, someone had to make a conscious decision to use it, which statistically they didn't. 

But now when everyone can experience the potential, now it becomes organic, those decisions, those use cases, because more people are exposed, more people can use it. And, by the way, it doesn't mean that it's easier to integrate, it's probably more complicated to actually integrate in terms of the product itself. But it becomes more tangible to decision makers and a lot of people that actually need to take part in this process. And that's a huge difference.

The dark side of AI

Dev: Microsoft already released something –do you remember what it was? And then Microsoft basically said, Okay, so the easiest fix is to basically make sure that no conversation is longer than x. Because it will take you a certain amount of time until you get what’s called Sydney or whatever. And that was the fix. 

I think that's a testament of the complexity of actually controlling it. Like, there is no way to just say, oh, don't go there. No, that's way too complex. So stick on topic, just assume that it'll take you a few sentences, you're limited, every few sentences will be a clean slate. And this solves it when you think about conversations. 

But if you really want to get into it, then you will not be able to spend two hours talking to Sydney about how to take over the world, which I remember the gist of half the conversation just before confessing its love. But I think that the main issue with those systems, it's a lot of blackbox and training. You know what gets in, but it's very hard to control the output,

Even if this is just a bot that tells you hey, this is your option here, a sim website of the bank, Like, for a customer, where do I need to get? Where do I go to get a loan? And the bot needs to answer. And then the second one is, do you really need one? Maybe that's not what the bank wants the bot to say. There's no way to limit it at the moment. And this means that it'll be very hard to integrate for the end customer without setting very clear boundaries of what is considered, okay and what isn't.

There's some group of people out there who are very excited, they can see the potential in these large language models. 

And then for folks who are like, oh, this isn't that impressive, I would just say, you're looking at very early days for this. it would be the equivalent of evaluating the promise and potential of social media by looking at Friendster and saying, well, Friendster isn't really that great.

We have the Friendster of AI right now, where we find we finally have one that is ready to show ordinary human beings. It's still a little crunchy. And if I even talked to you back then and I said, Friendster is going to undermine our democracy, come on. That took a few years.

How to think about these advances if you’re an FI 

Moses: I think that the main thing is understanding the kind of framework itself and the borders around what you consider acceptable, whatever that means. And I think that for a lot of FIs, it will mean that they will have to work harder, a lot harder to add compliance into blackbox systems. And at the end, this is what it is: it's, relatively speaking, a blackbox system. You push data, but you don't actually understand how to control it, which means you have to be very specific in what you consider an acceptable output. And then have that as part of that school, the training process, and have and make sure that this is part of the feedback. 

And if I'm being realistic, it's like, that's the tip of the iceberg. We're missing a lot of tools and infrastructure in order to be able to control it and move it from, oh, this is really cool, to a product that you can use and deploy without constantly looking and monitoring.

Compliance

Dev: In the near term, there's going to be a point where compliance is going to be eaten by AI. And there's a lot of different parts of financial services, like any sort of KYC, where you look at it, and you say, you have a very clear idea of what success looks like. And you have a very clear idea of what the rules are, what’s fair or not. It's that kind of decision making, which is a lot of financial service decision making, right? Is this loan a worthy loan to make? 

In any of those cases, those are the first jobs to go. And so I would be incredibly concerned if I was a compliance professional. But even beyond that, but particularly after 2008, we've had every large financial services institution complain about how much compliance and how much regulation happens. But at the same time, when you talk to them behind doors, they'll say, you know what, it's really great, because it's kind of a competitive moat. It's such a pain in the ass, nobody else wants to do it. We have small armies of people to do it, it's great, who else wants to come in and take our business? Google's never gonna bother, they really become a bank. Amazon will never become a bank. 

But when those incredibly complicated decisions become table stakes, if anybody can buy a bank in a box, suddenly your competitive moat goes away. All of the reasons why people should stick with you goes away. Then it goes beyond just a kind of challenge to my career to, well, a challenge to my kids’ college fund.

Addressing tougher questions

Dev: You'll hear all the time that, well, our products are sold, not bought. That's right there with really our strength is in our human relationships and in our human interactions. You better make sure that it actually is true, right? Depending on the sector or the sub sector, there are some segments of insurance where the average career lifespan of an agent is two years, two and a half years, meaning you join this company, you sell insurance to all your friends and family, you run out of your local market, and you quit and you go do something else. Go be a real estate agent. How much really great advice are your people providing today? 

AI will force us to say, okay, if human beings are really our strengths, we better up our game, and most people are only doing that at a fairly low level of play.

What customers really want today

Moses: I think it's mostly about the lowest hanging fruit: KYC, for example, is the first thing you automate, it's very clear what you need to do. And it's a lot of hassle. Half of it is images. And you have to do OCR, and the other half is matching photos and looking through databases. That's the first thing financial firms need to automate. And these are the order forms or deposits, all the things that you think, oh, that should have been there a long time ago, but now it's mature enough. 

I think it's all about confidence in the system. And I think that for a lot of those algorithms, the confidence is high enough for an FI to actually adopt, because the framework is quite clear on what you're getting. And I think that's the lowest hanging fruit and for most financial interests, that's where they are at at the moment. They still have a long way to go until they embrace large language models as a way to interact with data, even internally, because the cost of understanding that you got the wrong answer is way too high.

If you need to verify every answer, that means that you're doubling your job, your work, but if you have to ask, and then you have to verify where people were lazy, we won't do it. And then at the end, someone will make a mistake. 

Dev: Large companies are like large countries. When you have that many people inside an organization, you get a bell curve distribution. And here's what the social science tells us: about 16% of human beings are future focused, when confronted with something new, like ChatGPT, they are, alright, we got to get going. Let's get moving. 

There’s another 14% inside any group of folks who are completely past focused. These are the people who said, come on, Zack, taxi cabs will never go away. Uber is just a blip. They're the folks who said, ou'll always want to walk into a corner bank branch and physically see a teller, right? And because their entire worldview is based on the past and what they've seen before.

70% of human beings are completely present focused. What does that mean? Their reaction is, when you show them something new, they go, you're right, the world is changing, but we need to focus on this quarter. And that's actually the worst of all three combinations: the past focused people, you can disprove and the future focussed people, they're off to the races, they're gone. But 70% of us accept the premise that the world is changing, and we're driving over the cliff anyway. We're talking like the future focused folks and acting like the past focused people. And, that's why large companies run aground because for most of us, it's not a question of technologies. It's everything that Moses was talking about its adoption in an effective use of that technology, hindered by our own mindset.

Let's take something that's a little bit more consumer facing so that everyone can understand. How do you decide on your advertising spend if you're a large bank, right, and I know folks in a few institutions who say, we can now make those decisions better using machines than what any human being could do. But most marketing execs still want to make the personal choice themselves, and that's completely reasonable. This is BMW can drive a car better than I can. But I still want a manual stick shift. I don't know, it makes me feel like I'm driving. That is the state of decision making in most companies today.

You have to address the decision making at very high levels on the one hand. On the other hand, not too high, it's mid to high levels that have the the ability to actually take those decisions to the next step, to budget them in, and have the foresight to understand the value of those decisions. And that's usually where you target your buyers, if you will, in terms of adopting machine learning.

Biases

Dev: It is a huge issue. I mean, then we're only talking about small matters, like whether you get a loan or not. It's when we're usually using AI for things like criminal sentencing that the real horror stories are coming out. Now this person or this person probably shouldn't be up for parole. And then you discover later, that what the AI is doing is deciding if you're black or not, those things are already happening today in more than a few instances. And so we actually have to take a deliberate hand in saying what the outputs of this are.

Moses: Even before you get to more sci fi use cases of AI, even if you think about credit lines and the data that comes into the credit line, if your data is fed with a huge bias, because it was taken from a certain area, or socio economic state, or a lot of things create that bias into the model that later no one will know. And that means that basically you'll have a niche feature that actually affects whether you'll get a loan or on your credit line or on whatever, without creating a lot of visibility into the decision making. 

The problem is when it's wrong. When it's right, everyone is happy. But when it's wrong, and sometimes even when it's right statistically, but you're saying yeah, but we should not do that. Like the fact that you're asking for a loan at a certain branch, that location of a branch, not in a very good neighborhood, statistically, you will not be able to basically pay back your loan statistic, it kinda makes sense. Would that? Should that affect the ability of you taking a loan just because he walked into the wrong branch? No. Statistically, yes, of course, it makes sense. 

So someone needs to create that visibility into the process and understand. That's probably what's happening, we should stop that, even though if you look at the numbers, it’s perfect. And that's a huge risk. And that's usually the problem with adopting those models without understanding how they work.

I want you to imagine your credit score dropped 100 points. And so you're calling and you wait on the phone three hours to actually get to a human being, and you say, Why did my credit score drop 100 points? And they say, I don't know. So well, I could talk to somebody who does know and the real truth is nobody knows.

Computers: it's just a fancy way of computer says no, which is horrible as the consumer and imagine that is affected by something like the model Googled you and your social media accounts. Why would that affect me? And you have no idea that that's what's going on, but based on some statistical model, it has an effect. You're not aware of it. So imagine the end data itself. It's all public and the connection between your real world entity and the social entity, that's not very hard to do.

Dev: The counter argument, though, is that large groups of human beings are not necessarily any better either. The classic example in financial services is looking at homeownership. Several decades ago, in the United States, the gap in homeownership between white folks and black folks was huge. And so they said, why is this so? People are being discriminated against. So they put in all of these laws to make it illegal to do things like redlining to make it illegal to withhold loans. And now you look a few decades later, and the gap is even larger. So in the individual instance, people are, at least in principle, being less discriminatory. But the overall effect seems to be we've gotten worse.

AI + embedded finance makes strong competition

Dev: There are a few tech companies doing exactly that. And they're doing wonderfully. Stripe is doing that – they make an API for you to basically instead of working directly with a bank, you're working with them. That's terrific.

Think of it as three different classes of potential here. Let's say you want to be a better bank. You adopt these things. I don't really want to be a bank, I want all the benefits of being a bank, but I don't want all the hassle. Then you can rent a bank on the back end – this is how Apple  did its credit card with the Green Dot folks.

 And then there's the third aspect, you know, really one of the big implications of AI that we've been talking about, which is like, no, you can buy a bank in a box. Just like a bank with all of its compliance structures with all of its decision making, all of its credit evaluation is actually a box that sits on a rack in your systems or something you can rent from the cloud. 

What you start to get out of this is you get potential for it to be much easier. What's the biggest thing that gets in the way of stage two or stage three is regulatory. It's like, will the OCC allow this? Will the SEC allow this depending on the other segment? But even regulators are learning and moving along.

Moses: I tend to fully agree, and we can see the potential in the crypto world, which basically was a bank on a shelf (obviously, with a huge risk attached). Without regulations on top, or actual visibility, it's even more important than that. So in theory, it works. In practice, you really have to make sure that you're getting what you think that you're getting.

There's something driving that and crypto is a good comparator. Because there's people who are gonna look at this AI and say, Look, they're the same people who are selling me on crypto three years ago, and now telling me they're bankrupt.

AI vs. crypto innovation 

Dev: What the fundamental difference underneath this is for myself, personally, is I was very crypto skeptical. And the biggest reason comes from a place of human need. The class I teach is called Need Finding. And I've yet to see a compelling use case for crypto if I'm not a criminal. 

That is not true when we're talking about ChatGPT. Since December, my brain has been firing on all the things I could be doing, about all the needs that could be satisfied, not just in KYC compliance, credit evaluation, but yes, even in basic Customer Experience matters.

Looking a few years ahead

Moses: It's scary. It's important how quickly this is adopted. And in five years, I don't think that we'll see an actual implementation.

Dev: I would put an inverse on this, which is the advice I have for execs in this space: don't try to predict the future. Just get in the game and start experimenting and start learning. You'd be shocked at the number of financial services companies where to do business with this organization, it requires you to get online, print out a form, fax it back into them – fax, like, you know, like, it's 1940. Or the number of times where you need a wet signature, or the number of times where there's a financial service from where their mobile app doesn't adequately replicate their experience of brick and mortar. 

We're so why are so many organizations finding themselves behind is because they spent a decade arguing about what's going to happen and why you need it, or why you don't need it. Get started and experiment with it at a small scale, so that you can keep up and learn at the same pace as the rest of us as we're doing it.

I think that connects to like the 70% living in the present – it's basically we have more important things to care about, especially these days. And that means that there is never a budget to actually build for the next phase, just oh, we'll wait until I don't know, stabilize or there is a start of school or some sort of that will never happen with technology, you'll just get the next wave and the next wave. And this is how they end up with a lot of tech debt, basically, by doing very little.

Southwest Airlines, their meltdown in December was all a story of tech debt. They just kept pushing out investment, like every financial service firm in the world should look at what happened to Southwest and say there but for the grace of God go I.

Partnerships between FIs and AI firms

Moses: For sure. I see a lot of digital or AI service companies providing that additional level and capabilities for financial services – developing in-house is not realistic. And actually, it's probably not very cost effective as well.

Dev: There tend to be firms who are really great at partnering and ones who are really great at optimizing and doing. It's so easy to see it in the technology space. Microsoft has historically been really good at partnering with other firms, and they follow that through all the way to open AI. As opposed to Amazon, which is very good at doing it themselves. So it's going to depend on what your company is great at. Are you great at partnering? Are you great at copying what somebody else does, just doing it faster, better, cheaper?