Theory Ventures reposted this
We suffer from a cognitive bias : work performed by a human is likely more trustworthy because we understand the biases & the limitations. AIs are a Schrodinger’s cat stuffed in a black box. We don’t comprehend how the box works (yet), nor can we believe our eyes if the feline is dead or alive when we see it. New AI product on-boarding will need to mitigate this bias. One path may be starting with low-value tasks where the software-maker has tested exhaustively the potential inputs & outputs. Another tactic may be to provide a human-in-the-loop to check the AI’s work. Citations, references, & other forms of fact-checking will be a core part of the product experience. Independent testing might be another path. As with any new colleague, the first impressions & a series of small wins will determine the person’s trust. Severe errors in the future will erode confidence, that must be rebuilt - likely with the help of human support teams who will explain, develop tests for the future, & assure users. I recently asked a financial LLM to analyze NVIDIA’s annual report. A question about the company’s increase in dividend amount vaporized its credibility, raising the question : is it less work to do the analysis myself than to check the AI’s work? That will be the trust fall for AI. Will the software catch us if we trust it?
We suffer from a bigger bias: that cogent language production indicates cogent understanding. For example, if an intern does a simple job correctly, we can use the quality of the work to judge the intern's conceptual understanding and readiness for the next task. Trust grows from inference. But an LLM's ability to do a simple task doesn't necessarily reveal conceptual understanding, just accuracy at that task. So any trust gained may be the result of the user's cognitive error. Ask an LLM questions about tic tac toe. If that gains your trust, boy will you be surprised when you ask it to play a game of tic tac toe!
Until hallucinations are fixed, Gen AI will not be trusted in enterprise use cases.
Jeff Dodge
3mo
Great piece. Your point about humans trusting self-driving cars is particularly poignant as the data is pretty overwhelming that lives would be saved if self-driving cars were broadly adopted in suitable circumstances. Ultimately humans do want to know "who is accountible" and "who will be punished if something goes wrong" even if there are less occurances of something going wrong. When it's a human at fault, society punishes that human and it 'feels right' ie justice is served. When AI is wrong, the only 'person' to hold accountable in most circumstances is a corporation, and generally a trillion dollar corporation and they have a lot to lose finincially so the risk/reward for those corporations is tricky and even more so the legal ramification.
David Schlosberg
3mo
Need better error reporting. And ideally reporting on lame UI/UX so rampant in software in general
Kirill Soloviev
3mo
Trust but verify, this is what we always say at ContentQuo! We’ve built the company to help buyers of AI-powered services develop trust towards their vendors based on easy to understand & openly shared quality metrics. In our first vertical, this really helped alleviate trust issue over time and move to constructive data-based vendor performance conversations and procurement processes. Expecting to see more of it as LLM space grows!
Konstantin Ristl
3mo
love your post! I’m a big advocate for pointing out that not only AI but also humans make mistakes. When trusting someone with a task for the first time, I’m more skeptical than after seeing them perform it multiple times. As for people: Not every task fits every person, just like not every task fits AI.
Brian Sowards
3mo
Human evals (and structuring product outputs for feedback) are a key step in the product design for AI. Right now most platforms seem to have a "take it or leave it" approach to outputs, playing into the "magical AI" value prop many users default to if they haven't actively used AI in their work. The big opportunity is "done with you" where your own cognitive load is lower and the path to confidence in the output is shorter.
Susan Tejuosho
3mo
Social engineering is a significant factor in the multi-modal function of LLMS. My working theory is that the bias is not with the output but that the onus is placed on humans to rectify another task they've been told AI can outsource.
SaaStr Annual 2025 is May 13-15 in SF Bay!! See You There!!
3moIt in essence passes the test by default for jobs no one is willing to do anymore