Michael Morett’s Post

Important read for anyone exploring how to rapidly deploy GenAI workflows that actually make it to production via Palantir Technologies.

Excited to share my and Arnav Jagasia's latest blog post in our Engineering #ResponsibleAI series: a field manual for developing effective AI #evals! Drawing from our work with customers, we dive into the tradecraft behind testing AI. From designing a test plan, basic building blocks like wielding ground truth data, and advanced techniques like #LLM-as-a-judge — use this as a guide through the common workflows and challenges in #AI testing and evaluation.

"Success isn't just about accuracy - it's about consistency and robustness." In transportation, where decisions impact safety and efficiency, this resonates deeply. Our testing must go beyond "does it work?" to "does it work reliably?" Great parallel with how we're approaching AI in fleet optimization at Resultant. What testing approaches are you using to ensure AI reliability in your operations? #AI #FleetTech #Transportation #ResponsibleAI

I am pleased to share a compelling new blog post that explores the latest advancements in AI technology. Our latest analysis reveals that DeepSeek-R1-Distill-Qwen-1.5B has surpassed GPT-4o in certain key benchmarks, highlighting its impressive capabilities and potential applications. This milestone marks an exciting development in the field of AI and provides valuable insights for professionals looking to leverage cutting-edge technologies. To read the full article and gain a deeper understanding of these advancements, please visit: https://ift.tt/hcv2gMt.

A novel model-agnostic explainable AI method that considers collinearity to explain the model globally: "Explainable Artificial Intelligence for Dependent Features: Additive Effects of Collinearity". #explainable_AI, #interpretable_AI, #machine_learning https://lnkd.in/e6xGHpyE

https://lnkd.in/e5ThpQsi ARTIFICIAL INTELLIGENCE - Deciding Whether to Automate With AI? 6 Key Practices to Consider: Dan Milczarski believes before deciding if or how to use AI in life sciences, it’s critical to weigh the pros and cons. There is a vital need to customize constantly evolving AI applications and innovations to create tailored, effective technologies that reflect life science organizations’ regulatory and organizational frameworks.

What research are AI companies doing into safe AI development? What research might they do in the future? To answer these questions, Oscar Delaney, Oliver Guest, and Zoe Williams looked at papers published by AI companies and the incentives of these companies. They found that enhancing human feedback, mechanistic interpretability, robustness, and safety evaluations are key focuses of recently published research. They also identified several topics with few or no publications, and where AI companies may have weak incentives to research the topic in the future: model organisms of misalignment, multiagent safety, and safety by design. (This report is an updated version that includes some extra papers omitted from the initial publication on September 12th.) https://lnkd.in/g_6DnqgX

AI at a Crossroads: Navigating the Future of Artificial Intelligence Explore the pivotal moment of AI development. Understand the challenges, opportunities, and decisions shaping the future of artificial intelligence in society and industry.https://premier-consultancy.com/ AI stands at a crossroads, shaping the future with unprecedented potential and challenges. As advancements accelerate, society faces critical decisions about its ethical use, regulation, and integration. The path chosen will define AI's impact on industries, innovation, and daily life.https://https://lnkd.in/g49Fcv9y . . . . #AIInnovation #FutureOfAI #ArtificialIntelligence #AIEthics #TechnologyTrends #AIChallenges #pcil #premier

Ensuring Generative AI systems are robust and reliable is more critical than ever. Our latest blog post explores essential testing and evaluation (T&E) strategies. Learn how to leverage ground-truth data, incorporate advanced techniques like LLM-as-a-Judge, and perform perturbation testing to make your AI systems more effective and reliable: https://lnkd.in/gSAm9yBM

Excited to share our latest blog post on #AI Testing & Evaluation! As Generative AI continues to transform industries, ensuring the reliability and effectiveness of these systems is going to be increasingly important. Our team at #Palantir has been working closely with customers to develop strategies and tools for testing Large Language Models (#LLMs). In this blog post, Colton Rusch and I share insights and practical tips for how to: • Design a testing plan • Evaluate Generative AI systems with and without ground-truth data • Implement more advanced techniques for evaluation, like LLM-as-a-Judge • Test for robustness and consistency in LLM outputs These approaches are designed to help you deploy more reliable AI solutions in real-world contexts. Check out the full blog post for more details, and stay tuned for more on #ResponsibleAI from Palantir’s Privacy and Civil Liberties Engineering team in the year ahead.