How do you explain AI limitations to stakeholders?

Explaining the limitations of AI to stakeholders is crucial for managing expectations and ensuring responsible AI deployment. AI systems, as powerful as they are, are not infallible and have their shortcomings. These can include biases in the data, difficulty in understanding complex human nuances, and limitations in their decision-making abilities. Misunderstandings about the capabilities of AI can lead to unrealistic expectations and, ultimately, disappointment. Moreover, the inappropriate application of AI due to a lack of understanding of its limitations can have serious consequences, especially in fields like healthcare. Transparency about these limitations helps to build trust and ensures that AI is used appropriately and effectively.

The success of any real-world project is determined by the clarity of scope comprehended by the key stakeholders. Clear communication enables the effective utilization of resources, such as manpower, capital, and time. Given the experimental nature of the AI space, it can be challenging to clearly outline what is possible and what is not. However, doing so enables the technical team to conduct their experiments without the fear of failure. When the limitations are clearly communicated to the leadership, it helps them develop strategies that encapsulate these limitations effectively. Although this may initially cause temporary discomfort, it increases the probability of success for the entire project.

You will need to explain the limitations of AI to different stakeholder groups — primarily without a tech background. To your leadership, you need to explain what the technology can and cannot do, and what prerequisites need to be in place (data, skills, infrastructure, problem definition, …). To your business stakeholders, you need to explain what AI can do for their business function and how that helps them achieve their goals — and where they still need people to check and review. And to the end-user who’s most affected by the change of using AI, you need to explain *how* it changes their day-to-day — and what the individual benefits to them are, and where they need to do tasks themselves or double-check what the AI has done.

To an executive leadership, the message should contain the opportunity AI brings, business value, limitations of technology and innovation path, and risk and mitigation frameworks. If there are regulatory impacts, it is advisable to provide a brief on the same topic, as executives may require to de-brief the board. The business stakeholders will be interested in how AI aligns with their vision and empowers them to deliver value. The end users who are the potential beneficiaries of AI-enabled solutions should know how their experience changes and impacts daily life with your product.

Explaining AI's limits helps people understand that AI isn't perfect. It needs good data to work well, can sometimes reflect unfair biases, and can't solve all problems. This helps build trust as people know what to expect. It also avoids legal issues and helps people make better decisions about using AI. Simply put, knowing what AI can't do helps us use it better and more fairly.

In my experience advising leaders on AI roadmaps for 10 years, I've seen 4 common challenges that limit AI's potential: 1. Wrong problem: Often, AI is force-fit onto an incorrect usecase. If the fitment isn't right to start with you ned up with big issues. 2. Wrong data: Data of the right type, quality, and volume is often missing. 3. Wrong model: AI model is suboptimally engineered and needs fine-tuning. 4. Wrong rollout: Implementation of AI in the real world and change management is often overlooked leading to big usage issues. Explore which of these you're missing.

Consult, research, pilot, pivot, and innovate is a successful strategy in adopting AI-enabled technologies. Identifying limitation, value, and opportunities these tools brings to an enterprise will be visible after a carefully crafted pilot study. It should lead us to understand the risk, changes in the as-is state of the technology landscape, and risk. Once it is identified, it is easy to craft the messaging because the leadership likes limitations and how home-grown invitations or partnerships can support the adoption of AI platforms.

Understanding limitations in both generative and non-generative AI models is vital. For all models, inspect data for biases, test complex scenarios, evaluate understanding of context, and compare to human decisions. Non-generative (discriminative) models are best tested on their ability to correctly classify or predict. Generative models, which create new data or scenarios, need additional testing to ensure the output's quality and relevancy. By considering the unique nature of these models, you can identify limitations and promote responsible AI use.

To determine and communicate the limitations, you can run a comparison. How does the current process run and how long does it take/ how much does it cost? Compare that to running the process with additional help from AI. How many recommendations does your model make correctly vs incorrectly? How does that compare to the status quo? That’s how I’ve seen teams identify and communicate the limitations.

Limitations are best explained using analogies - it helps to be well versed in a handful that any layperson can reasonably relate to. Having these in your back pocket, available to pull out at will, means you can simplify several discussions by abstracting away from the technicalities and converting those into pure brainstorming sessions. That’s where stakeholders are the most productive and feel confident offering suggestions.

Given the novelty of AI systems and their inherent blind spots, it is crucial to take stakeholders concerns into account and act on key concerns promptly. In my experience, employing a hybrid approach proves highly effective when inviting stakeholders to pose questions or voice their apprehensions. This approach involves utilizing surveys to gather quantitative information, while simultaneously handpicking a group of key stakeholders for in-depth group discussions or one-on-one conversations. By adopting this approach, we can elicit a broader range of inputs and ensure a more comprehensive understanding of the stakeholders' perspectives.

Are your stakeholders not convinced yet? This might as well be a sign of how effectively you communicate, not just their inability! This is why self-reflection and empathy are important. Put yourself in their shoes and see what are they missing. And, ask yourself what else are *you* missing in the communication. This shift in your perspective could be the key to a breakthrough.

The first step is to hear the objections and understand the grounds for challenges. As decision influencers, we have to see the side of the story from the other side of the desk. To find common ground, we must address the objection with facts, evidence, quantifiable benefits, and values. Respecting every stakeholder's view and helping them make informed decisions is essential.

Handling objections and challenges is a matter of providing transparency, education, and demonstrating a commitment to continuous improvement. Address concerns by sharing the steps being taken to mitigate AI shortcomings, such as actively reducing bias in datasets or incorporating new advances to overcome current limitations. Showcasing instances where AI has successfully enhanced processes or decision-making can also help build confidence. It's essential to create a dialogue that allows for open discussions about AI, its potential, and its limitations.

At the end of the day it is humans who will be using the AI model and it’s critical to assess the readiness of people to adopt an AI into their workflows. We must ensure that benefits of the AI system are framed as value in terms of customer outcomes rather than features of the AI system. Whether it’s an AI system or something else, humans seek tools to achieve their specific outcomes. The benefits, trade offs, limitations must be framed in the users’ POV to help build trust and ensure better adoption.

Understanding AI limitations is important. However, understanding AI impacts is even more important. Impacts are heavily linked to the business context and purpose of the AI application. Risks related to an AI engine recommendations are very different from the ones related to autonomous cars. Similarly, the acceptability of AI model accuracy rate is dependent on impacts. We may consider that 90% accuracy rate of a movie recommendation engine is good but a 90% accuracy rate for an autonomous car is unacceptable.

Understanding AI's constraints is crucial, but it's just as essential to consider societal impacts and the need for regular AI system updates. A holistic view of AI, encompassing its strengths, weaknesses, and effects on society, is key to successful implementation and usage.

Bad business processes won't be fixed by AI alone, Businesses need to optimise their systems fix, then look at AI to streamline and scale.