AI Lending Needs Human Oversight At Critical Decisions

Artificial intelligence is increasingly being used across lending, from credit assessment and fraud detection to document processing and customer servicing. But as lenders give AI greater autonomy, questions around explainability, bias, model drift and human accountability are becoming more important. Tarun Aggarwal, Group Chief Technology Officer, Capri Loans, discusses where AI is making a difference in lending, where rules-based automation still dominates, and why human oversight remains important for high-risk and complex credit decisions. Balancing AI And Accountability What parts of lending are genuinely being transformed by AI today, and which are still mostly automation dressed up as AI? AI is making the biggest difference in credit assessment, risk monitoring and fraud detection through the ability to analyze many data points and recognize trends that could have been missed through conventional approaches. However, there are some applications where automation takes place instead of AI. Document extraction, score workflows, eligibility checks, pricing matrices and collections routing are often valuable but remain largely rules-based automation. What is most important about AI is not its categorization as an application of AI, but the value that it generates. The value generated by AI comes from the ability to make better and quicker decisions for lenders and improve portfolio outcomes. When AI influences a credit decision, what should it be allowed to decide on its own, and where should a human remain accountable? AI must have greater autonomy when faced with scenarios that pose low risk, involve repetition, and are governed by clearly defined rules, such as data validation, document review, fraud detection and initial creditworthiness assessment. At the same time, if a decision is taken in a complicated situation with the presence of many exceptions and high credit risks, there must be human involvement in the decision-making process. Thus, the best way out is not a choice between human and AI, but a path where both can walk together. What happens when the data says a borrower is high-risk, but the business context suggests otherwise? Can an AI system account for that distinction? A conflict between a model's risk signal and business context should trigger structured review, not automatic deference to either technology or intuition. Models may misread temporary setbacks, sector conditions, seasonality or recent improvement. Underwriters should validate the source data and add additional verified evidence to present a more comprehensive picture. Where context justifies a different conclusion, the human override should record the model's concerns, supporting facts, approving authority and monitoring conditions. Any override should be evidence-based and subject to appropriate approval, with the outcome fed back into model validation and ongoing monitoring. How do you explain an AI-driven rejection when even the lender cannot easily trace the reasoning behind the model's output? A lender should never rely on an AI-driven rejection that it cannot explain, trace or defend. Customers should receive clear, specific and actionable reasons such as insufficient cash flow, high existing debt or inconsistent repayment history, rather than technical model terminology. The lender should also provide a meaningful opportunity to correct inaccurate data, submit additional evidence or request human review. Every decision must be reconstructable through documented data, model and policy versions. What are the failure modes you worry about most with AI-led underwriting: bad data, model drift, bias, fraud or something else? The principal failure modes are poor or incomplete data, hidden bias, model drift and fraud. Controls must have verified data history, up-to-date data and monitoring systems to make sure things match. There should be fairness tests for groups and proper checking systems in place for the model. Human review is an essential safeguard, especially for tricky situations and high-impact cases. Systems also need containment measures, including exposure limits, rollback, manual fallback and pause controls, so that problems can be detected early and affected borrowers can be protected in a timely manner. Are lenders finding meaningful uses for GenAI in actual credit decisioning, or is its near-term value largely around servicing, documentation and employee productivity? Yes, GenAI’s near-term value is mainly in servicing, document processing, summarization, compliance support and employee productivity. Additionally, it can also organize unstructured information, identify issues in contracts or financial records, and help analysts build a more complete borrower assessment. However, its outputs can be variable and it is rarely trusted to make final decisions autonomously. Core underwriting should therefore remain anchored in verified data, transparent policy rules and validated predictive models designed for consistent risk estimation. If an AI agent can execute an entire lending workflow, from collecting information to recommending an action, where do you draw the line on autonomy? The autonomy boundary should sit before final approvals or rejections, policy exceptions, adverse actions and material changes to loan specifications. These decisions require accountable human authority, except for narrowly defined, low-risk straight-through cases. Agent permissions should be risk-based and technically enforced through least-privilege access, audit trails and action limits. Automatic escalation, human override and pause controls are essential whenever data conflicts or consequences become material. What would make you stop an AI system from making or influencing lending decisions, even if it appeared to improve efficiency or approval rates? An AI system should be stopped or restricted if it produces unexplained, discriminatory or unreliable outcomes, compromises data or security, or operates beyond approved risk and compliance thresholds. Efficiency or faster approvals should never come at the expense of fairness, compliance, borrower protection or sound credit judgment.