Why 70% of banks don’t report the outcomes of Gen AI deployments 

For most of 2023 and 2024, banks in America had a “no comment” philosophy on Gen AI. But over the past year, things have changed.

Banks like Bank of America, Citizens, and JPMC are announcing how Gen AI is being deployed in internal use cases to fuel employee productivity. In fact, BofA is leading the way in making the tool available to all its employees, according to research by Evident.

The senior executive outlook on the technology is positive as well:

Katie Hainsey, Managing Director and Head of AI/ML and Data & Analytics for Digital, Marketing, and Operations at JPMorgan Chase shared that the tech is already being deployed in the firm in key areas: for call center employees, an LLM Suite for all employees, and coding tools for tech teams.

Similarly, Bank of America’s CTO Hari Gopalkrishnan hinted that the tech is likely to become a part of its customer-facing chatbot: “As we look at the emergence of Generative AI, we actually see that classification can actually get a lot better. You can actually talk even more naturally in a natural language. So that is just a natural sort of expansion of where we go with Erica.”

But what’s the ROI and long-term impact, I ask? The answer is… it’s complicated. 

Research shows that nearly 70% of AI use cases don’t have any reported outcomes. Simply put, banks are marketing the number of use cases they have under production and R&D but not what impact those deployments are yielding. 

Research suggests there are at least four reasons for the lack of information around ROI:

1. Progress driven by Gen AI is hard to separate from overall growth, in particular because proof points are overshadowed by enterprise-wide operational metrics. 
2. It’s difficult to map out a cost impact from operational efficiency gains, specifically because most Gen AI deployments are internal-facing i.e. for employees. 
3. Gen AI deployments are costly and results take time to materialize. 
4. Overall ROI reporting may not be factoring in the entirety of the cost outlook, underestimating the role played by tech enablers such as cloud infrastructure and data platforms. 

But is there a potential fifth reason?

It’s unclear what success with Gen AI looks like. In the past when banks have moved into new territories like embedded finance or Gen Z, the target has been somewhat clear. Either the organization is reshaping its role in the market by re-capturing an array of customer relationships and revenue streams (embedded finance) and/or the firm is future-proofing its tech and products for a particular emergent customer segment (Gen Z).

While Gen AI’s internal deployments offer efficiency gains, they don’t fundamentally rework the role a bank plays in the market. On the other hand, consumer facing tools like chatbot integrations may offer a user experience boost but might not necessarily rework overall brand image or vision. 

The Gen AI noise is deafening 

All of this is not to say that Gen AI isn’t important. It is important, and it is cool. Gen AI is changing how people work and think. But that… is what tools tend to do. Tools offer infrastructural and workflow improvements, old-school artificial intelligence has been doing this for decades, and Gen AI deployments have offered further augments. 

There is a cognitive dissonance between what results the Gen AI hype suggests we should see: an era-defining remake of banking. And we should also see results in the form of: $X saved versus $Y costs.

The hype is putting the cart before the horse. The fundamental shifts that the Gen AI hype seems to suggest are on the horizon come from human-first strategies enabled by tech. For example, financial health enabled by PFMs, or banking outside the branch and office hours enabled by banking apps and websites. 

But saying so takes a sledgehammer to “innovation” marketing, so we aren’t saying it. Instead, banks are reporting the number of use cases they have in production, as the whole industry struggles to put words to what these proof of concepts would mean for consumers’ lives.