Delphina

🚀 New Newsletter Alert! I’m thrilled to share the latest edition of my newsletter, Vanishing Gradients, where I dive into some exciting topics in the world of data science, ML, and AI. Here’s what you’ll find in this week’s edition: 1. Kicking off Data Dialogs: I’m launching a new series of conversations for data leaders, starting on August 22. Our first session features Brad Klingenberg, where we’ll explore how traditional ML is evolving with Generative AI and what that means for data science leaders. 🗓️ 2. Cutting AI Assistant Costs: We recently conducted a study where we compared CALM (Conversational AI with Language Models, Rasa) with LangChain/LangGraph. The results were compelling—CALM reduced operational costs by up to 77.8%, delivered 4x faster responses, and maintained high reliability by adhering to crucial business logic (with Alan Nichol, Daksh Varshneya). 📊 3. Podcast with Dan Becker Becker & Hamel H.: I had the chance to chat with two veterans in the AI space about their experiences teaching LLMs to thousands of data scientists. We discuss everything from the challenges of fine-tuning large language models to the future of AI education. 🎙️ 4. The AI Revolution with spaCy Creators: I did a live podcast recording about NLP and AI with Ines Montani and Matthew Honnibal from Explosion. We discussed their journey with spaCy, the role of open-source in AI, and how they’re pushing the boundaries of what’s possible in NLP. 🌐 🔗 in comment 🤗

If you ask most data leaders, “What drives your users to take the highest value actions in your product?”, they’ll gaze back at you with a pained look on their face. They’ll probably respond through gritted teeth, “That’s a hard question.” And they’re right. It’s not that they don't care; the opposite in fact. But it’s an incredibly complex puzzle, and they wish they had better answers. Throughout my career, understanding the drivers of high value actions has been *the* burning analytics question. At Wealthfront, we obsessed over what led customers to transfer their other investment accounts to us. At Uber, it was the factors behind frequent trips and subscription sign-ups. At Gopuff, it was what drove large orders and purchases of high-margin products. The problem is, traditional analytics tools like BI dashboards and spreadsheets can’t untangle the web of factors that lead to high value actions. Answering these questions requires high-dimensional causal factor analysis, decomposing outcomes across dozens, or hundreds, or even thousands of input variables. In other words, they require machine learning. This is what the most advanced analytics teams are doing — using ML to find the needles in the haystack and unveil unexpected relationships between behaviors and outcomes. The good news: upgrading your product analytics with ML is within reach. In our latest article, Jeremy and I break down three core techniques you can use today. The topic is on our mind because we’re coming across it frequently at Delphina. We're eager as always for feedback and reactions, and if you’re tackling a similar problem and want to brainstorm, reach out! #datascience #analytics #machinelearning #artificialintelligence

Love the Delphina shoutout Rob! Agents are going to change everything. And we're just getting started 🔥

I’ve interviewed or advised hundreds of new PhDs trying to break into data science in tech. Even the most accomplished academics tend to struggle in the same area: the onsite. They’re coming from a world where their papers and CV form an encyclopedic track record of their expertise to a new reality where a single 45-minute interview can decide their fate. They’ve never done this before, and they’re typically wildly unprepared. On the other side of the table, hiring managers are in a tough spot. They can’t afford to blindly disqualify talented candidates because of an awkward interview: there won’t be many candidates left! Falling prey to the cardinal sin of recruiting — hiring someone who’s good at interviewing instead of someone who’s actually good at the job — is way too easy with new PhDs. How can you cut through the noise? Over time I’ve built a short list of attributes I look for: 1/ Excitement to learn all parts of the job: have they internalized that working in tech means actually _building technology_? 2/ Evidence of multidisciplinary thinking: do they have a sledge hammer for which they must find a nail, or do they have a real toolbox? 3/ Ability to navigate multiple levels of altitude: PhDs tend to love the microscopic — but can they zoom out to see the whole ecosystem? 4/ A burning need to get things done: do they live and breathe a need for speed? 5/ Bonus: public speaking superpowers: explaining things clearly to an audience is incredibly valuable – and is a spike that’s easy to miss in standard interview loops. I unpack these in my latest post with Jeremy, including what questions to ask (and how pro job seekers respond), below. Let us know what you think, and please share: what do you look for? #hiring #leadership #datascience #machinelearning

When I was leading various data science groups at Uber there was an ever-present and uncomfortably-high likelihood that a Data Science Disaster would derail my day. It might be Thursday around 7pm; perhaps someone on a partner team noticed something looking funky in the evening rush hour of an important metro. All of a sudden, that uBeer I was about to grab would turn into a whirlwind of analysis, strained uChats, and jams to dig in. Sometimes there was a major problem, and sometimes it was nothing. At the time, I thought Uber’s culture was just intense. (That was true!) But in hindsight I’ve also realized: ML itself is fundamentally super intense. ML can be a lever that moves the world — when it works well, it delivers incredible, transformative value. But when it doesn’t work, or even when it’s just mediocre, that lever can snap and slap you in the face. ML that’s merely mediocre can be downright dangerous to the business. Unlike many business functions, the returns to quality in ML are highly nonlinear, and not-great ML is impossible to tolerate. That’s because ML’s scale is massive, it’s hard to monitor, and it’s too easy to inadvertently make mistakes. ML is also immature as a domain – many leaders don’t know problems to look for, and many data scientists don’t have the acumen to foresee where things can go wrong. Problems can burn silently for months, costing companies dearly. In ML, quality is surprisingly binary, and Jeremy and I call the area of mediocre quality the “Danger Zone”: quality that would be uninspiring but tolerable in many functions but is simply unacceptable in ML. In our latest post we unpack why, how to identify it, and how to keep your company in the clear. How have you navigated the danger zone? Let us know what you think! #datascience #machinelearning #artificialintelligence

Really excited for this hybrid virtual + in-person (Seattle) meetup with Yurui (Rui) Tong, Wee Hyong T. and Duncan Gilchrist. We'll be discussing how to build (or in some cases, try to build...) AI products from the ground up, and how Remitly, Delphina, Corgi Labs and Microsoft approach this, plus how the generative AI wave has helped us shorten the time from 0 to 1 (and the time for 0 to 1 recursions when the results don't turn out as you expected...) It should be an awesome discussions, and even better, it's set for a APAC, EMEA and PST friendly time to foster global inclusivity!

🤔 How many stats on gen AI adoption and impact are skewed by confusion over what counts as generative AI vs ML? Yesterday Scale AI released it’s Zeitgeist AI Readiness Report, which opens “We surveyed over 1,800 **ML practitioners** to understand the state of AI development and adoption…” (emphasis added) That got my attention, since I don’t think of ML practitioners as necessarily on the front lines of gen AI projects. Clicking into the report (which is quite insightful, despite my introspection here), even the experts at Scale are blending the two, with questions like “what is the current stage of your AI/ML project?” (image below from the report) At Delphina we generally think of ML as a subset of the broader AI field (and sometimes call ML “predictive AI”), with generative AI yet another subset; though admittedly talking about this stuff in ways that are both simple yet technically correct is tricky. Is this something you’re seeing? Does an LLM embedding inside an ML application like search count as gen AI? Any other glaring examples? #artificialintelligence #machinelearning #datascience