Guillermo Delgado Aparicio is Global AI Leader at Nisum.
Discover top fintech news and events!
Subscribe to FinTech Weekly's newsletter
Read by executives at JP Morgan, Coinbase, Blackrock, Klarna and more
AI in fintech spans a range of use cases, from fraud detection and algorithmic trading to dynamic credit scoring and personalized product recommendations. Yet, a Financial Conduct Authority report found that of the 75% of firms using AI, only 34% know how it works.
The issue isn't just a lack of awareness. It's a profound misunderstanding of the power and scope of data analytics, the discipline from which AI arises. The mass adoption of generative AI tools has brought the topic to the C-suite. But many of those choosing how to implement AI don’t understand its underlying principles of calculus, statistics, and advanced algorithms.
Take Benford’s Law, a simple statistical principle that flags fraud by spotting patterns in numbers. AI builds on that same kind of math, just scaled to millions of transactions at once. Strip away the hype, and the foundation is still statistics and algorithms.
This is why AI literacy at the C-level matters. Leaders who can’t distinguish where analytics ends run the risk of overtrusting systems they don’t understand or underusing them out of fear. And history shows what happens when decision-makers misread technology: regulators once tried to ban international IP calls, only to watch as the technology outpaced the rules. The same dynamic is playing out with AI. You can’t block or blindly adopt it; you need judgment, context, and the ability to steer it responsibly.
Fintech leaders must close these gaps to use AI responsibly and effectively. That means understanding where analytics ends and AI begins, building the skills to steer these systems, and applying sound judgment to decide when and how to trust their output.
The Limits, Blind Spots, and Illusions of AI
Analytics analyzes past and present data to explain what happened and why. AI grows out of that foundation, using advanced analytics to predict what will happen next and, increasingly, to decide or act on it automatically.
With its exceptional data processing skills, it’s easy to see why fintech leaders would see AI as their magic bullet. But it can’t solve every problem. Humans still have an innate advantage in pattern recognition, especially when data is incomplete or "dirty." AI can struggle to interpret the contextual nuances that humans can quickly grasp.
Yet, it's a mistake to think that imperfect data renders AI useless. Analytical models can work with incomplete data. But knowing when to deploy AI and when to rely on human judgment to fill in the gaps is the real challenge. Without this careful oversight, AI can introduce significant risks.
One such issue is bias. When fintechs train AI on old datasets, they often inherit the baggage that comes with them. For example, a customer’s forename may unintentionally serve as a proxy for gender, or surname inferred cues about ethnicity, tilting credit scores in ways that no regulator would sign off on. These biases, easily hidden in the math, often require human oversight to catch and correct.
When AI models are exposed to situations they weren’t trained on, this can cause model drift. Market volatility, regulatory changes, evolving customer behaviors, and macroeconomic shifts can all impact a model's effectiveness without human monitoring and recalibration.
The difficulty of recalibrating algorithms rises sharply when fintechs use black boxes that don’t allow visibility into the relationship between variables. Under these conditions, they lose the possibility to transfer that knowledge to the decision-makers in management. Additionally, errors and biases remain hidden in opaque models, undermining trust and compliance.
What Fintech Leaders Need to Know
A Deloitte survey found that 80% say their boards have little to no experience with AI. But C-suite executives can’t afford to treat AI as a “tech team problem.” AI accountability sits with leadership, meaning fintech leaders need to upskill.
Cross-analytical fluency
Before rolling out AI, fintech leaders need to be able to switch gears—looking at the numbers, the business case, the operations, and the ethics—and see how those factors overlap and shape AI outcomes. They need to grasp how a model’s statistical accuracy relates to credit risk exposure. And recognize when a variable that looks financially sound (like repayment history) may introduce social or regulatory risk through correlation with a protected class, such as age or ethnicity.
This AI fluency comes from sitting with compliance officers to unpack regulations, talking with product managers about user experience, and reviewing model results with data scientists to catch signs of drift or bias.
In fintech, 100% risk avoidance is impossible, but with cross-analytical fluency, leaders can pinpoint which risks are worth taking and which will erode shareholder value. This skill also sharpens a leader’s ability to spot and act on bias, not just from a compliance standpoint, but from a strategic and ethical one.
For instance, say an AI-driven credit scoring model skews heavily toward one customer group. Fixing that imbalance isn’t just a data science chore; it protects the company’s reputation. For fintechs committed to financial inclusion or facing ESG scrutiny, legal compliance alone isn’t enough. Judgment means knowing what is right, not merely what is allowed.
Explainability Literacy
Explainability is the foundation of trust. Without it, decision-makers, customers, and regulators are left questioning why a model came to a specific conclusion.
That means executives must be able to distinguish between models that are interpretable and those that need post-hoc explanations (like SHAP values or LIME). They need to ask questions when a model’s logic is unclear and recognize when “accuracy” alone can’t justify a black box decision.
Bias doesn’t appear out of thin air; it emerges when models are trained and deployed without sufficient oversight. Explainability gives leaders the visibility to detect those issues early and act before they cause damage.
AI is like the autopilot on a plane. Most of the time, it runs smoothly, but when a storm hits, the pilot has to take the controls. In finance, that same principle applies. Teams need the ability to stop trading, tweak a strategy, or even pull the plug on a product launch when conditions change. Explainability works hand in hand with override readiness, which ensures C-suite leaders understand AI and remain in control, even when it’s operating at scale.
Probabilistic Model Thinking
Executives are used to deterministic decisions, like if a credit score is below 650, decline the application. But AI doesn’t work that way and this is a major mental paradigm shift.
For leaders, probabilistic thinking requires three capabilities:
- Interpreting risk ranges rather than binary yes/no outcomes.
- Weighing the confidence level of a prediction against other business or regulatory considerations.
- Knowing when to override automation and apply human discretion.
For example, a fintech’s probabilistic AI model might flag a customer as high risk, but that doesn’t necessarily mean “deny.” It may mean “investigate further” or “adjust the loan terms.” Without this nuance, automation risks becoming a blunt instrument, eroding customer trust while exposing firms to regulatory blowback.
Why the Judgment Layer Will Define Fintech Winners
The future of fintech won’t be decided by who has the most powerful AI models; rather, who uses them with the sharpest judgement. As AI commoditizes, efficiency gains become table stakes. What separates winners is the ability to step in when algorithms run up against uncertainty, risk, and ethical gray zones.
The judgment layer isn’t an abstract idea. It shows up when executives decide to pause automated trading, delay a product launch, or override a risk score that doesn’t reflect real-world context. These moments aren’t AI failures; they’re proof that human oversight is the final line of value creation.
Strategic alignment is where judgment becomes institutionalized. A strong AI strategy doesn’t just set up technical roadmaps; it ensures the organization revisits initiatives, upgrades teams’ AI capabilities, ensures the company has the required data architecture, and ties in every deployment to a clear business outcome. In this sense, judgment isn’t episodic but built into the operating mode and allows executives to drive a value-based leadership approach.
Fintechs need leaders who know how to balance AI for speed and scale and humans for context, nuance, and long-term vision. AI can spot anomalies in seconds, but only people can decide when to push back on the math, rethink assumptions, or take a bold risk that opens the door to growth. That layer of judgment is what turns AI from a tool into an advantage.
About the author:
Guillermo Delgado is the Global AI Leader for Nisum and COO of Deep Space Biology. With over 25 years of experience in biochemistry, artificial intelligence, space biology, and entrepreneurship, he develops innovative solutions for human well-being on Earth and in space.
As a corporate strategy consultant, he has contributed to NASA's AI vision for space biology and has received innovation awards. He holds a Master of Science in Artificial Intelligence from Georgia Tech, obtained with honors. In addition, as a university professor, he has taught courses on machine learning, big data, and genomic science.
