Agentic AI Has an Explanation Problem: Technology Vendors Are a Big Reason Why

DISPATCHES FROM THE FINTECH SNARK TANK

I asked Claude to check out the websites of a dozen or so agentic AI vendors selling to banks and credit unions in an effort to answer a simple question: what are they telling the market about what their tools do?

Across the board, the vendors led with outcomes: faster decisions, higher throughput, reduced analyst time, more approvals without additional risk.

One vendor’s site boasts about going from “lead to term sheet in minutes, not days” before explaining how its platform works. Another’s homepage stacks three metric callouts above the fold before offering a sentence of product description.

I get the logic. CEOs respond to efficiency, cost, and revenue metrics. Explaining what an AI agent actually does inside a lending workflow is a harder homepage headline than “80% faster decisions.” Vendors optimize for the pitch that closes deals.

The problem: Unlike technologies they’ve purchased before—digital banking platforms, loan origination systems—bankers don’t understand how AI tools get to “80% faster decisions.”

After signing a contract for a generative or agentic AI tool, bank execs have to:

• Address the board’s questions about model governance;

• Explain how credit decisions are made and who’s in the loop to examiners; and

• Make the Chief Compliance Officer comfortable enough to let the tool run.

“80% faster decisions” doesn’t help with any of that.

What banks need—but what not all vendors provide—is a plain-language explanation of the vendor’s AI: what data it uses, where that data comes from, what it does with the data, how it handles exceptions, where humans are in the loop, how it embeds the bank’s credit policies, and more.

That’s not a nice-to-have for banks and credit unions—that’s the purchase decision.

Pharmaceutical sales is a good analogy. Drug companies figured out decades ago that selling to physicians requires more than efficacy data. Physicians need to understand how the drug works before they’ll prescribe it, because they’re the ones who have to explain it to patients and defend it to peers. Pharma reps who lead with “patients recover 40% faster” but can’t explain the underlying biology don’t get far.

Tech vendors selling AI to bankers are in the same position.

What the Pitch Promises vs. What the Examiner Asks

Here’s the gap between what vendors pitch and what examiners ask:

• Pitch: Our AI tools achieve an 80% reduction in total underwriting time. Examiner: What data inputs does the model rely on, and how do you validate that they’re accurate?

• Pitch: We deliver 35% more qualified leads through consistent policy application. Examiner: How does the system document exceptions, and who has authority to override the model’s recommendation?

• Pitch: AI agents complete end-to-end credit workflows with little human effort. Examiner: Where exactly does human review occur, and how do you demonstrate that humans are meaningfully in the loop rather than rubber-stamping outputs?

• Pitch: AI agents can be decision-ready analysis in less than three minutes. Examiner: How do you test for fair lending compliance, and what’s your process for identifying disparate impact in the model’s outputs?

The examiner questions aren’t hypothetical. They’re drawn from the OCC’s model risk management guidance, the CFPB’s fair lending examination procedures, and interagency AI guidance that’s been circulating since 2021.

Examiners have been building their list of AI questions for a few years. Vendors, for the most part, are still building ROI slides.

Vendors Need to Help Banks Develop an “AI Vocabulary”

A few vendors get it. The smart ones build educational infrastructure into their sites by providing guides on how specific workflows operate, product taxonomies organized by what the AI does rather than what the bank saves, and demos that show the decision logic, not just the dashboard. Most of the market hasn’t gotten there yet.

The irony is that the vendors selling agentic AI—the category where the process-related questions are most pressing—are often the most outcome-heavy in their messaging (based on the websites Claude reviewed).

Agentic AI is new to most bank and credit union buyers. These are systems that take actions, make sequential decisions, and operate autonomously inside a compliance-driven workflow.

The bar for “I understand what this thing does before I deploy it” is higher than it’s ever been. And the pitch is more metrics-heavy than ever.

There’s an opportunity in that gap.

My take: Vendors that build a reputation for explaining the technology in language a credit officer can use in a committee meeting will earn trust faster than vendors promising the best ROI.

What’s missing, in many cases, is a “AI vocabulary” to help the bank discuss AI opportunities, risks, and policies (how many times have I said over the past two years, “we need to stop using the term AI!”?). Banks need vendors to give them that vocabulary, not just the headline numbers.

Take the distinction between robotic process automation and agentic AI. It matters, both for what banks can expect the technology to do and for how it’s governed.

But a lot of vendor messaging treats the two as interchangeable or it glosses over the difference entirely with language like “agents that learn, think, and act.” That phrase—and variations of it—is doing a lot of work in the market right now and it’s not doing it honestly. True agentic AI:

• Sets and pursues goals dynamically rather than executing a predefined workflow;

• Decides about what action(s) to take based on context rather than rules;

• Handles novel situations outside its training parameters; and

• Orchestrates multiple tools or sub-agents autonomously.

That’s a high bar. Most of what’s being sold as agentic AI today doesn’t clear it.

What many of these systems do is more like intelligent RPA—document classification, data extraction, rule-based decisioning, straight-through processing triggered by document inputs.

I’m not saying that’s not useful. Automating 60% of consumer loan decisions by routing the easy ones away from human underwriters and enabling them to focus on the more complex cases is solid execution.

But that’s been happening for years. Calling it agentic AI because there’s an LLM handling document classification creates expectations that the technology can’t meet and governance questions that banks aren’t prepared to answer.

The tell is usually in the vendor’s language. Claiming to have “APIs trained on company data that execute predefined workflows” is sophisticated automation. Calling those APIs “agents” is marketing.

Technology vendors have an opportunity—and an obligation—to close the vocabulary gap rather than exploit it. The ones who do will earn something more valuable than a closed deal: a customer who can explain and defend the purchase.

Bottom line: Speed and cost reduction are table stakes. Smart vendors will help bankers answer the questions the board and examiners will ask: How does this work, and how do you know it’s working right?

Come into the Fintech Snark Tank and tell me why I’m wrong.