Global Economics Group

Are we winning the war on financial crime? The Financial Crimes Enforcement Network (FinCEN), a unit of the US Treasury Department, recently released its “Year in Review for FY 2023” report. The report highlights the collaboration between US financial institutions and US law enforcement agencies like the FBI, the IRS, and the Department of Homeland Security. Among the notable statistics was that 87.75% of the primary subjects of IRS criminal investigations had suspicious activity requests (SAR) filed against them, and 13.9% of IRS criminal investigations in 2023 originated from a SAR filing. This establishes a clear correlation between financial institutions’ financial crime prevention programs and law enforcement activity. Do these figures clearly indicate the system's effectiveness, or do they leave room for doubt? It is hard to tell. According to the report, approximately 4.6 million SARs were filed in 2023, an average of 12,600 daily. 4.6 million SARs compared to 11,367 FBI case subjects with a SAR filed against them. 0.25% of filed SARs end up becoming part of an FBI case. That seems extremely low, especially considering that the decision to file a SAR comes after the institution has investigated transactions flagged by its transaction monitoring systems. There are many more alerts than filed SARs, and 95-99% of transaction monitoring alerts do not result in a SAR filing. 0.00247% of transaction monitoring alerts contribute to a criminal investigation. Banks have 30 days from when an alert is generated to decide whether to file a SAR, and the investigation time per alert can range from a few hours for simple cases to a few weeks for complicated cases. Complex cases are growing as criminals evade detection by leveraging advanced technology and digital to structure transactions. Detecting and preventing money laundering is critical, and no one would suggest that banks and other financial services providers discontinue their programs. Still, the estimated annual cost of AML compliance programs in the US and Canada is over $61 billion. $61 billion in spending for 11,000 cases, that's about $5 million per case. There must be a better way. Advancements in AI techniques like machine learning have demonstrated that they can detect patterns and make predictions much faster than traditional techniques. Financial regulators have signaled their willingness to embrace innovation and automation to improve the efficiency and effectiveness of financial crime prevention programs. So why isn’t there more adoption of these capabilities across the industry? Institutions are concerned that leveraging these next-generation capabilities will increase false negatives, resulting in fines and other enforcement actions. Society often holds technology to a higher standard than humans, and they should, but continuing to perform an extremely ineffective activity is not sustainable. There must be a better way. #globaleconomicsgroup #AI #AML #csrigg

Can AI Improve Your Client’s Experience with Financial Crime-Related (FCC) Compliance Processes? Most conversations about using AI in financial crime compliance programs focus on efficiency, especially reducing the rate of false positives. Most transaction monitoring programs have false positive rates above 90%, and investigating these alerts is a wasted effort. So, any effort that reduces this delivers actual savings to the bottom line. But what about AI’s potential impact on the client experience? We understand that financial institution clients often express concerns about how compliance-driven processes negatively impact their experience and influence their decisions about where to place their business. Why are these processes creating such a negative experience? One answer is that the individuals and teams involved in executed compliance processes, including CDD, EDD, account opening, payment screening, and refresh, are often disconnected. Clients are asked the same questions multiple times when the information is stored in the bank. It also leads to the customer's expectations being missed when their requests cannot be fulfilled promptly because compliance processing takes too long. Additionally, client-serving personnel are often unaware of the details of the compliance processes, leading them to set the client's expectations inappropriately and erroneously blame compliance for the unfavorable outcome. How can AI help? The first area where AI can help is accuracy. False positives don’t just waste the bank’s time; they can also waste the client’s time. A false positive can result in unnecessary questions being sent to the client, leading to additional time spent and delays in processing their request. It’s essential to recognize that having a more accurate and efficient screening and monitoring capability reduces costs and improves your client’s experience. The second area is risk profiling and KYC processes, including CDD and EDD. AI can help create a more comprehensive and up-to-date profile of a client's actual FCC-related risks and behavior by continuously applying additional contextual and behavioral information. Incorrectly labeling a client as high-risk can result in further information requests and scrutiny of their transaction activity. AI can help strike the right balance between due diligence rigor and smoother customer experience. AI can also improve the identity verification process by automating the interrogation of identity documents, which is being leveraged extensively, and providing additional layers of security through voice and facial recognition. As financial institutions focus on improving their compliance capabilities with financial crimes, they should also consider how these capabilities can enhance their clients' experience and make that an integral part of the design and implementation process. #aml #financialcrime #ai #globaleconomicsgroup #csrigg

Can an AI-based Digital Twin improve the efficiency and effectiveness of your controls? Delivering financial services products requires the coordination of many different processes, capabilities, and organizations. When you consider the detailed attributes of the customer, the product, the delivery channel, and the regulatory jurisdiction for each transaction or interaction, there are almost infinite potential configuration permutations. Managers must add resources and processes to understand and evaluate performance, risk, and compliance. Regulators' continuously increasing demands exacerbate this problem. How do I know if the controls I’ve implemented are working? Are they appropriate for the risks I am looking to manage? Are they effective in mitigating these risks? Are they efficient? Does the realized benefit exceed the cost? A control may seem appropriate at the most granular level, but it may be redundant or superfluous when evaluated against an adjacent control. Based on the nature and magnitude of the targeted risk, a control could be overkill or insufficient. Banks have inventories of controls and processes to evaluate them. But, as the volume of information grows, the ability to analyze this information with traditional means is overwhelmed. Any manually driven process to rationalize a complex control infrastructure will take so long that the business will change during the effort, severely limiting its impact. How can AI and data help? Advanced data management techniques can create a virtualized representation, a “Digital Twin” of the business, including its processes, products, and channels. These techniques, including graph databases, natural language processing, semantic analysis, predictive analytics, and large language models, can help banks better understand the business's performance and the corresponding management controls. Graph databases allow you to better model the interdependencies across an organization and align it to a standardized business ontology. As more data is inserted into the graph, machine learning algorithms can detect patterns humans can’t. Natural language processing can read document text and use semantic analysis to align the document (policy, procedure, org chart) to the correct part of the ontology and extract its meaning and purpose. Predictive models can then generate the key performance and risk indicators that measure the business's health. These models can also simulate the impact on the KPIs and KRIs emanating from proposed business or operating model changes. Lastly, attaching an LLM supports natural language interactions with your operating model configuration, significantly improving the quality and productivity of the measurement and analysis process. The combined impact of these technologies creates a truly fact-based representation of your control infrastructure and the ability to rationalize them. #AI #riskmanagement #compliance #globaleconomicsgroup #csrigg

I've been getting a lot of questions about the new draft of NIST AI 600-1 Generative AI Risk Profile that is now available for public comment: https://lnkd.in/gTvBVu68. If helpful for my ERM/MRM/AI Governance colleagues, here's some basic information about the document and my thoughts on how it might fit into model or AI governance programs. Basic info: * Information for submitting much-needed public feedback is available on pg. iii. * For this document to make sense, you'll need to read the broader AI Risk Management Framework (AI RMF, NIST AI 100-1). That's available here: https://lnkd.in/gs7g6iJD. * This draft document is a cross-sectoral profile of generative AI (GAI) for the AI RMF. It provides general information and guidance across sectors that fit into the AI RMF's MAP, MEASURE, MANAGE, and GOVERN functions. * The AI RMF is a risk-based framework. Riskier products, models, or use cases receive more oversight. The AI RMF is also a socio-technical framework. It seeks to improve the technical and social aspects of AI jointly. * Like the AI RMF, no part of the document is a checklist, and adherence is voluntary. Some personal thoughts on incorporation into governance programs: * The first sections of the document propose 12 GAI risks that can be used to assess the risks of products, models, or use cases and guide risk-tiering. E.g., does a third-party LLM or image generator present intellectual property risks? How severe? What about other risks from the list? * The next parts of the document present GAI actions taken from public comment over Fall 2023. Actions are similar to risk controls but address a wider set of tasks than risk acceptance, mitigation, transfer, etc. The AI RMF Playbook also contains actions. (https://lnkd.in/gmb4e6sS) Draft AI 600-1 actions can be matched to AI RMF functions, categories, and subcategories and augment Playbook actions. E.g., an AI RMF Playbook action for GOVERN 1.1 is: "Align risk management efforts with applicable legal standards." Using this profile, we can augment that with additional actions for GAI: "Establish policies restricting the use of GAI to create child sexual abuse materials (CSAM) or other nonconsensual intimate imagery." Each set of actions also suggests which roles might be best suited to apply the actions. * The last major part of the document focuses on primary GAI considerations: governance, pre-deployment testing, content provenance (e.g., watermarking), and incident disclosure. These are some of the main activities that should be top-of-mind for effective GAI risk management. Many specific actions relate to these more general topics. Finally, it was an honor to contribute to this document with many others at NIST and the GAI Public Working Group. Please read the doc and let NIST know what you think. The more uncorrelated input signals, the better the outcome!

Can sharing data among banks help prevent money laundering? One of the biggest challenges financial institutions face when combating financial crime is the inability to see the bigger picture of their customer's and counterparties’ financial activity. Money laundering involves money movement between entities through financial institutions. When an entity engages in suspicious activity, each institution only has a limited view of the activity. Government entities can see the bigger picture after the SAR is filed and use its powers to pull information from multiple institutions to capture a comprehensive view of the activity. However, this process can take a long time, and much harm can result in the interim period. Financial institutions have long wanted to share information with their peers, but privacy laws forbid it. They can share it with the authorities but not with each other. Enter COSMIC (Collaborative Sharing of Money Laundering/TF Information and Cases) in Singapore. COSMIC allows banks to share customer information with other financial institutions if they are network members as part of their financial crime prevention processes. It was created by the Monetary Authority of Singapore (MAS) and six banks: DBC, OCBC, UOB, Citibank, HSBC, and Standard Chartered. The potential for both good and bad outcomes from this platform is enormous. The Singapore financial ecosystem is smaller than the US, the EU, or China, but it is large enough and adjacent to the larger systems to provide an excellent test bed for the concept. Implications? On the good side, COSMIC could allow bad actors to be detected and caught much faster than before. If multiple banks detect suspicious activity and discover the same activity in their peers, they can move much quicker to shut it down and alert the authorities. Some suspicious behavior may not cross the threshold when viewed individually, but it becomes a much more evident pattern when linked to the same behavior across multiple institutions. On the negative side, some legitimate activities and customers may get caught up in this expanded net and suffer unintended consequences. This will likely result in litigation, which could slow down the whole concept. The participating banks may decide that the risk is greater than the reward. The COSMIC process and enabling legislation contemplated the potential downside. They tried establishing clear criteria to protect customers and institutions, but we don’t know how it will play out. Where could this go? Ultimately, this capability could be extended to create an integrated network of financial transactions and counterparty data that could be continuously surveilled with advanced AI to detect suspicious activity independent of any financial institution. That could be cool or scary, depending on your perspective and trust in the authorities. I'll discuss it in another post. #AML #COSMIC #globaleconomicsgroup #csrigg

Should I use RAG to fine-tune my GenAI Application? Creating and training large language models (LLMs) is an expensive and challenging process that most organizations don’t have the resources and capabilities to undertake. So, how do I get an LLM to provide relevant and accurate information enough to automate a specific business process? The solution is to enhance the LLM's capabilities by incorporating additional authoritative data sources and leveraging specific features within the LLM during response generation. This is where RAG comes in. The decision to add data to an LLM is complicated as it impacts many essential aspects of an AI application, including accuracy, relevance, scope, and safety. We’ve already discussed prompt engineering and prompt transformation as methods to manage interaction and some of the advantages and pitfalls associated with their use. What about RAG? RAG stands for Retrieval Augmented Generation. This technique intercepts the user’s query and performs a directed search for additional data. Then, it takes the search results and inserts them along with the query into the LLM’s prompt. RAG allows you to add a layer of data to the AI application without training or retraining the model, significantly reducing the cost of creating a business process-specific AI application. What about the risks? The most significant risk introduced by RAG is the potential that the search and integration of the application-specific data into the LLM query process produces an unreliable or dangerous result. You are also adding more business logic to the application, which can have quality and security issues. The search process needs to produce relevant results that can be formatted to enrich the query, and the data needs to be relevant. Also, the response from the augmented prompt needs to be interrogated to ensure that proprietary data is not unintentionally leaked. There is also the risk that RAG will significantly increase the cost of interacting with the LLM without improving output quality. The more data you insert into the prompt, the more processing power the LLM may have to deploy to generate a response. How do I mitigate these risks? The key is to factor the risks into the design of the primary components, including data, search, prompt construction, and response processing. Each layer of the RAG delivery architecture needs to be hardened. Data should be rationalized to eliminate unnecessary elements, reducing the risk of leakage. The search process needs to be tailored to the data to produce usable responses, and the results must be analyzed before submission to a prompt. The tremendous potential of generative AI is easy to see when interacting with LLM via a chat window. The challenge is figuring out how to harness that general-purpose technology to perform specific tasks reliably at the right level of cost and risk. #generativeAI #AI #globaleconomicsgroup #csrigg

What are Few-Shot Prompts? We’ve discussed how variability in a large language model (LLM) output is one of the biggest challenges to creating an effective AI-driven application. If you ask the same question to an LLM-based chatbot, you may get different answers depending on the bot's configuration and the parameters used in the prompt. The user can force an LLM to interact with a query in a specific way by manipulating the variables inserted into the prompt. This is called prompt engineering; significant information and training are available on how to engineer prompts. The question is, how do we achieve this same objective when constructing an AI application? The same prompt engineering techniques available to users in the context window are available via the API. The challenge is designing the API calls to maximize the output's accuracy, relevance, and predictability while minimizing the potential degradation of the model’s performance. One approach is called few-shot prompts, which involves passing in sample queries and their answers along with the user's query. These examples help the model learn the task or understand the context, enabling it to generalize and perform adequately on similar inputs. Transformer-based LLMs, like ChatGPT, are designed to allow these small examples to influence the nature and structure of their output materially. This approach enables the application to adapt a generalized model to perform specific tasks even when available training data is limited. What are the risks? While few-shot prompts can be an effective method to improve the accuracy and performance of an LLM-based application, there are some risks, including: 1. Coverage – it may not be suitable for tasks with a broader or more nuanced knowledge base. Variability in training data is critical to teaching a model the nuances associated with some topics, and the limited examples provided by a few-shot prompt may result in corrupt or garbage data when interacting with edge cases. 2. Similarity—The few-shot technique's applicability is limited by the level of similarity between the base model's training data and the target task. Getting a model to produce outputs materially different from its initial training set requires more additional data and context than few-shot prompts can provide. 3. Overhead—Constructing the few-shot examples requires precise knowledge of the target business process and will increase the project's design and testing time. 4. Memorization—Since the supplemental training is based on a small amount of new data, the model may memorize the examples and incorrectly generalize new inputs to them. As with any of these design techniques for interacting with an LLM in your application, careful consideration of the trade-offs between functionality and risk is required. #GenAI #AI #globaleconomicsgroup #csrigg