Title: The A.I. Infatuated CTO: Reaching for the High Fruit of AI in Business (part 2)

Welcome back to my "A.I. Infatuated CTO" series. In the first part, we explored the 'low-hanging fruit'—applying generative AI and Large Language Models (LLMs) to unearth hidden value in existing static data. Today, we'll journey upward to the 'high fruit' and into the future of conversational AI.

I predict that as generative AI matures, the emergence of conversational UI will reshape how we interact with applications. Traditional web and mobile interfaces will gradually cede ground to more fluid, conversational interfaces. This shift doesn't mean the immediate extinction of traditional web and mobile interfaces, but a gradual evolution where the complexity of today’s interfaces with their date pickers, and dropdown boxes, and input validation, gives way to our user effortlessly “talking” to our applications. (Whether the user is using actual speech or typing and reading text is an unimportant detail.)  As leaders, how do we prepare for and embrace this inevitable shift?

I believe that we will shift from the expensive and complicated traditional web and mobile user interfaces to more direct and simple interactions. The winners in the next frontier will be those companies whose applications are simply a conversation between their users and their data.

But here's the rub -- out-of-the-box, commercially available LLMs like ChatGPT, Claude, or PaLM, don't have internal knowledge of our systems or data. Thank goodness! So, we can't just ask ChatGPT to talk with our data on our customers' behalf. Our task is to architect systems that create a conversational interface by connecting to and using our existing systems, application logic, and data.

In a nutshell, we can still use the data structures, APIs and application logic we’ve already created, and combine them with a new natural language interface using AI.  Our user interfaces will become simpler, and the data we all fought hard to capture and create becomes even more valuable.

The benefits are clear. Users can interact with systems naturally, and we make the most of our existing investments in our databases, APIs, and application back-end processes. The marriage of real-time data with LLMs will help us take a giant step forward in customer engagement and success.

In this future, our customers will not feel as if they are using an app. They will feel like they are talking to their data.

This path is not without challenges. Data privacy, IP protection, and the security of commercial AI models are critical considerations. The commercially available LLMs are making progress in meeting these concerns, but this is still a new game and as a community we are not yet sure if our data and our customers’ data will be safe.  Which models will we use? Where are our inputs and outputs stored.  Is a commercial model being trained on our data and possibly making it available to others? Are our transactions containing sensitive information secure from leaks? These are solvable problems, but they do require attention and diligence.

And let’s not forget everybody’s favorite LLM problem:  'hallucinations'! This is when language models, left unmanaged, generate inappropriate or inaccurate responses. This is a real risk. We have the responsibility to maintain close oversight, create guardrails, and to ground our models' responses to prevent wild or embarrassing answers to our users’ questions.  

Fortunately, hallucination and grounding issues are solved by creating our applications as a hybrid. By using the LLM for the conversational UI only, and combining with our own APIs and data, we avoid most of these problems. By tightly integrating the conversational capabilities of the LLMs with the structure of our existing systems, we also optimize customer value and increase customer engagement all while minimizing the risk of rogue AI behavior.

Reaching for this 'high fruit'—melding the benefits of the new conversational AI paradigm with real-time data—isn't easy.  It is an engineering challenge, but a worthy one. What we know for certain is that we cannot sit idle if we want to thrive in the AI era. Remember this: while AI is a new and potent tool, the real value will always lie in our data and the applications we build to serve our customers.

In a subsequent installment, we'll discuss practical ways to integrate generative AI into our existing applications with our own real-time data and logic.  Until then, let's keep the conversation going.

Yours truly,

Pete

Your A.I. Infatuated CTO