Propelix lets companies easily build their own generative AI chatbots

00:00 [This transcript was auto-generated.]
Hi, everybody, welcome to DEMO, the show where companies come in and show us the new features of their products or platforms. Today, I'm joined by John Spottiswood, he is the president of Propelix. Welcome.
Hi, Keith, Thank you.
How are you? And you have a great story here because how did you end up building Propelix? Because you're also involved with another company called Jerry. So tell us about Jerry. And then how this how Propelix came out of Jerry?
Great. Let me start by saying what Propelix is so Propelix is essentially GitHub for AI-based virtual agents. So it enables companies to launch, test, deploy, and then monitor virtual agents that can answer a complex questions even solve problems for their customers, but also for employees or partners. The initial use case for Propelix is primarily to automate and improve customer service in regulated industries like insurance and financial services. And so companies are, are using it to essentially reduce their operational costs, as well as to improve the effectiveness of their support teams.
And you started out by building this for the company, Jerry. So tell us a little bit about Jerry. And what that is too.
That's right. So like many of the best products, we built this to solve a need that we had at Jerry. So Jerry is the largest and fastest growing digital broker of insurance and loans in the United States. And we were growing extremely rapidly, we could not hire enough agents to keep up with our customer and prospect growth. And so you know, call, wait holds, etc, we're getting very long. And we have complex problems in insurance and loans. And we also work in 50 states with over 60 different insurers and carriers. So it kind of felt like every conversation was somewhat unique. So we were one of the first companies to become a client, commercial client of open AI, we built a system that enables us to manage very create very complex, specialized virtual agents that can solve really detailed questions for customers without hallucinating and in do so very effective.
Right. And it wasn't just because you found that there wasn't a solution out there. That was viable enough for your business?
That's right, I mean, well open AI was very viable, but there wasn't a framework around it for doing it. And so we had to build it ourselves. We launched it in May of last year, within a few months of going live, it was responding to over 90% of our customer conversations without having to escalate to a human. And our response times went from hours down to an average of less than 13 seconds. So very effective. It also was averaged on average saving us about 4 million in operating costs. So we decided that it was important to make this available to other companies as well. So that's how Propelix was born.
Okay, so show us what you're going to demonstrate here.
Okay, so I'm going to start with sort of the end result I'm going to play for you a conversation that happened a couple of days ago in our call center. So this is going to be a voice conversation. But these same sorts of conversations are happening across chat, SMS and email 1000s of times every day on our platform
Okay, so in the caller is human but the agent virtual.
and you'll be able to tell but hopefully not too much.
Oh, Hi Daniel. My name is Casey and I'm very virtual assistant. How can I help you today? Hi, hey, I went up here for you guys. Realize I write down my information
I'm here to assist you, Daniel. I see that you've requested close with us. Would you like to hear about the quote options available to you?
Great Daniel, the cheapest car insurance often available to you is with Progressive at a monthly premium of 270.67. This does not include comprehensive and collision coverage. Would you like to know more about this option or hear about others? Fine.
Daniel, you can request to confirmed rate by choosing the quote you like and clicking to confirm your rate. Confirmed rates are typically available within 5-10 minutes of requesting. Simply click on the notification that will be sent to your phone in the next few minutes to request a confirmed rate. That's the price for my five cars right?
yep, Daniel. That's correct a quote for 270.67 per month with Progressive covered all five of your cars. The 2009 Honda Odyssey EX, 2003 Toyota Corolla CE, 2007 Toyota Scion TC, 2004 Honda Civic LX and 2009 Honda Fit. This option does not include comprehensive and collision coverage.
You've sped that up a little bit. So it did. It's not that
That's correct. It's just a 1.25. Just for the sake of this.
What also impresses me was was how the system could understand the caller because it didn't sound like the caller had a great connection and he was talking pretty fast as well.
That's right. It's really, really good at listening and responding. And you could also see that it leveraged information about his quotes, as well as about his cars. In order to move the conversation forward. Obviously, he doesn't know how that conversation is going to. So overall a good conversation, and he did end up buying insurance through Jerry. So what I'd like to do now is show you sort of the system that we built that enables that this is, again is architected for a digital insurance broker for cars and for insurance and loans, but it can be used for any type of company. Okay, great. So let's dive in. So it all starts with the all agent screen. This is where you create your agents very simple to add a new agent, you can see we have 22 agents. It starts with a routing agent. So that's the mother of all agent, it basically decides which of the sub agents can answer the question best. And then we have a bunch of detailed sub agents like the app screens, which knows every screen in the Jerry app and what users can do with it, the payment screen the payment agent, escalation, agent, policy agent bind assistance, many more. If these individual agents can either respond to a customer, they can take an action on behalf of the customer, or do both, or they can escalate it. The escalation agent here knows all of our human agent groups, which ones are best to answer which types of questions and it can escalate appropriately. We also have some monitoring agents you can see down here the the analysis agents, so message categorization, and user sentiment, those run after the response agents to categorize and gauge the effectiveness of the of the conversation. So if we dive into one of these, for example, the payment sub agent, this is a summary screen that shows the priority the description, you can choose which large language model you want to use out of any of OpenAI, Anthropic, Google's or Mistral's, right, so you're not tied to OpenAI, you're not tied to a particular model, you can test them all. These also shows you all of the tests that you have for this particular agent, as well as all the prior versions. So we're on version 47. But if I click into one of these, this is where you actually build the agent. So it's broken up into blocks. This is a required block, for example, which means it will always get sent along with the conversation, it tells the agent who Jerry is, as well as who it's supposed to be a payments focus, customer service assistant. And it has some rules. For example, it says, you are only to provide service in English or Spanish, it can speak 36 languages, but we only have agents that can speak to so we limit it, you see a lot of blocks here. Some are required, some are not this payment increased block is a good example of a block that's inserting a lot of data, each of these blue fields is used in this block. And so for example, we use an open source framework here for scripting, to insert the users first name to create a conditional that says if the user has more than one car policy, use this and it can get very detailed. So for example, here, it's saying that if Keith had a missed payment in the last 31 days, then please inform Keith, that this missed payment happened a certain number of days ago. And that's why there was this payment increase. So it enables you to answer very specific questions for for specific users. The other way besides these inserted database variables that you can insert data into the system is through a knowledge base block. So here, for example, is a product guide, knowledge base block, and it has metadata associated with that particular document, which is the state the carrier and the policy type like car, and it only takes the chunks from that specific document that's relevant to this customer that is relevant to the question. So if the question was about, is there a late payment, it would look at the product guide for Progressive for example, in Pennsylvania, right and get you that specifically, right?
So it's not giving me information if so I live in Massachusetts, it's not giving me information for Pennsylvania. Exactly. Right. Is that one way it can reduce errors in accuracy, you know, improve the accuracy?
Yes. And because it's only sending the relevant information that the actual prompt that gets sent to the virtual agent is very concise. If we'd sent information from all the states, it would have to pick through it and find the specific one for Pennsylvania.
Right, and it could make a mistake too.
It could easily. So if we go back up to the top here, after you've created this agent, you're going to want to test it right? So we built a very simple but powerful framework for testing. So we can create any sort of dialog here. Here, I'm going to say how do I make a late payment, I can change any variables. Normally, these variables are sent from your database. But if we wanted to, we could go in and change, for example, the carrier from progressive to some other carrier, but I'm going to leave these variables as is for now. And when I run the test, it's going to take that question or dialog, it's going to take this existing new prompt that we just are working on, and it's going to send it in this case to OpenAI GPT 4, and here it comes back with a response that says, we don't currently support making a late payments to progressive in the app, but you can link directly to the progressive place where you can make that payment and it gives you the link to the wallet where you can do that. So if I like this, I can save it as the expected response. And I can take that particular URL if I wanted to. And I can save that as an expected keyword. And I can also save an expected action. So for example, an action here I might want to take is to refresh the wallet, make sure the most current payment information is available in the wallet. And now I can save all of those as a new test so that in the future, if I'm testing this, I can always get this response for this type of question. And if you then go into the saved tests, as we saw in the summary, there are 96 saved tests for this particular agent. And I can run all 96 of those. And we do every time before we launch a new version of the agent to make sure that it's answering all those questions correctly with the right keywords and the right actions.
And something else that you were showing me too is you can compare the results from one LLM to another as well.
I'm going to show you that yes, that's right. So we do this is the knowledge base where you can create documents and document types. But what you're referring to is experiments. So testing typically happens before you go live experiments run when you're live. So what you do is you as you say, a set of sub agents or agents that you want to test against your production version. So So you've come up with a new version of the routing agent and a new sub agent. And you want to see how that system works versus the current production system before you launch it, you run it in production, and for three or four hours, it will take the answers that the new system comes back with and it will store them in the database, so it doesn't send them to the customer, the production version goes to the customer. And then you can compare the production version response to the new system response. And another way you can use that, as you mentioned, is to test new types of models. So when me straw, for example, came out or with or in this case, when Anthropic came out with Claude Opus, just a few weeks ago, we ran Claude Opus. And so you can see in the experiment, the results, the test version here has all of our production versions running against Claude Opus, whereas the the control version is running against the current version of GPT. Four. And you can see here, for example, that the agent handover rate is like 6.46% for the test and 6.49 for the control. So very comparable, you can look at all the different which sub agents are invoked. And you can also get down to a real detailed level here, where we look at the disagreement, just those situations where Claude and open AI disagreed with each other. And we could look at the details of those and decide, is opening AI better? Or is cloud better? And honestly, Claude Opus is the first one we've come on, where it's actually looking neck and neck with Open AI GPT. For Yeah, so it's exciting. So it's basically you're kind of future proofed with Propelix because you can always test the current version of every new model that comes out and make sure that you find the one that works best for your specific set of use case.
Sure. And you're constantly looking at new new ones, as new companies come out, or or existing companies update their their LLM versions, right. That's exactly right. Was there anything else that we did for the demo? Okay, so So what were the results of of this, you have results for your own company for Jerry, right? Where, once you implemented this, you saw some improvement with your call rates, right?
Massive improvement. Yeah. So again, it's responding to more than at this point, it's about 93 to 94% of all inbound conversations are being responded to by virtual agents and handled all the way through without escalation. So really significant reduction in the workload that goes to our human agents. And now the human agents because they're dealing with a smaller load, they're getting people usually they're getting to people within 15 to 20 seconds. So the people are happy. And and they have the full context of the conversation with the digital agent, so that they don't have to go through the discovery process. So it's a much better experience for the humans, as well as obviously a more efficient experience with customers.
So how can companies take advantage of Propelix?
Yeah, so it's actually really easy to implement. So it's, it's based on Amazon Web Services, it's SaaS software, we can, we can provision accounts immediately. And typically, a customer can have a proof of concept up within a few hours. So we've got very good Training and Documentation resources available. We also assign a customer success manager to every new client, and they can help with the accessing the necessary database fields, as well as building out the knowledge base. And typically our clients can have production systems up and running with both database and knowledge base access within four weeks.
All right, and where can people go for more info.
For more info, go to Propelix.ai or you can email me at John at Propelix.ai.
Wow, usually they don't give up the email address. So that's that's that's confidence right there. All right, John, thanks for being on the show. And thanks for your demo.
Thank you, Keith.