Coalition for Health AI (CHAI)

We’re excited to announce the launch of CHAI’s Testing and Evaluation Frameworks for Sepsis Risk Prediction and Note Summarization – Patient Discharge Summary—now available on GitHub. These frameworks provide structured best practice guidance for: 🔹Sepsis Risk Prediction: Supporting developers and implementers of predictive AI solutions to assess responsible AI use in sepsis risk prediction. 🔹 Note Summarization – Patient Discharge Summary: Providing evaluative methods for generative AI solutions used in healthcare organizations to summarize patient discharge notes. We encourage you to explore, implement, and provide feedback as we continue refining AI evaluation best practices. #HealthAI #AI #HealthcareOnLinkedIn https://lnkd.in/eAphNwi8

Coalition for Health AI (CHAI) is officially one year old! Proud to share our Year 1 Impact Report that showcases what our small but mighty team at CHAI alongside our incredible members have achieved together. As we head into Year 2, we’re focused on turning momentum into impact — making CHAI resources more practical, accessible, and inclusive, especially for under-resourced health systems. Last week we also launched our Testing and Evaluation frameworks for Sepsis Risk Prediction and Note Summarization – Patient Discharge Summary—now available on our GitHub page (https://lnkd.in/ehEQgmdQ) and our Responsible AI Content site (https://lnkd.in/ef4C7K7e). Merage Ghane, Ph.D. Brenton Hill, JD, MHA Anthony DiDonato Sumanth Ratna Brian Anderson, MD Tom Kirby 120/80 GROUP Susan Stoner Zook Nicoleta Economou Nirav R. Shah Ed Middleton Thank you also to our awesome interns that helped us create this impact report Karolina Pencina Shelby Boulware-Johnson Daniel Oo Siri Ural Serena W. on content and the remarkable Ann Li who designed the entire report in under a week. John Halamka, M.D., M.S. Michael Pencina Nigam Shah Suchi Saria Jennifer Goldsack Kyu Rhee, MD, MPP Eric Horvitz Morgan Cheatham

Thrilled to share that the Coalition for Health AI (CHAI) and SingHealth singed an agreement and partnership that will launch CHAI's first international chapter in Singapore. Together our organizations are committed to building best practice frameworks and policies that support the rapid innovation of safe and ethical health AI. Partnering with Daniel Ting, and Singapore's Ministry of State in signing this important agreement signifies our commitment to working together, bringing private sector innovators and public sector policy makers together, across the globe! 🚀 👏

Last month, I had the privilege of meeting with Senator Joni Ernst of Iowa to discuss CHAI’s mission to build trust, accelerate innovation, and ensure security for health AI. Coalition for Health AI (CHAI) AI is driven by the experience and expertise of our members and the feedback of our broader health ecosystem and the public. I am grateful to share CHAI’s work with policymakers in Washington and across state governments as we work to bring life-saving innovations to patients, clinicians, and health provider organizations. Thank you Senator Ernst for sharing your time and perspective on advancing the safe and secure use of AI in healthcare. We look forward to future conversations with policymakers as we work to fully realize the promise of AI to transform patient care. #HealthAI #HealthcareOnLinkedIn #AI

“We use the model card to accelerate adoption of our solutions and gain trust with AI governance committees through the sharing of streamlined, rich, technically accurate and relevant information. It helps everyone get down to the real core issues of a discussion instead of working through tons of superfluous steps.” Read our recent Q&A with insights from Demetri Giannikopoulos, Chief Transformation Officer at Aidoc, one of our valued members and active contributor in our working groups. Read the full Q&A here: https://lnkd.in/ecnRnxHg #HealthAI #AI #HealthcareOnLinkedIn

“The model registry, like the model card itself, will give health systems concrete information to answer standard questions about a model besides looking at a sales pitch or marketing materials.” Thank you to Emma Beavins of Fierce Healthcare for speaking with CHAI’s Merage Ghane, Ph.D. about how CHAIs Model Card registry will benefit health systems, developers, and pilot users. Read more here: https://lnkd.in/ey5vr6tg #HealthAI #AI #HealthcareOnLinkedIn

“A vendor has to send the same thing to 20 different health systems, or health systems have to ask for something 20 different ways. Why not come together and figure out a standard way to get this information so we're not all duplicating resources, duplicating efforts and wasting time?" Thank you to Alexis Kayser of Newsweek for speaking with CHAI’s Brenton Hill, JD, MHA, about the coalition’s first Model Card registry and future steps to solve overly complex health AI procurement. Read more here: https://lnkd.in/e5qXss3Q #HealthAI #AI #HealthcareOnLinkedIn

As we head into HIMSS Global Health Conference, we are excited to share another milestone for CHAI Applied Model Cards, our “nutrition label” for health AI. We are establishing our first Model Card registry to support the rapid growth of model cards used to simplify procurement among health systems and solution providers.    Powered by Avanade’s Smart AI Governance Engine (SAIGE), the registry will enable health system AI solution evaluation and decision making teams to easily access Model Cards. For solutions providers, it will give greater visibility and access to a community of potential customers while streamlining the initial stages of procurement. Read the full story here: https://lnkd.in/eungCMaX #HealthAI #AI #HealthcareOnLinkedIn 

Today Michael Pfeffer, Percy Liang and I are delighted to introduce a benchmark to test LLMs on real world clinical tasks. MedHELM comprises a taxonomy -- validated by 29 clinicians from 15 specialties -- and 31 datasets mapped to tasks spanning 22 subcategories to evaluate 6 models. 🔹 GPT-4o leads in 2/5 categories. 🔹 Llama-3.3-70B-instruct in Patient Communication and Education. 🔹 Gemini 1.5 Pro in Clinical Note Generation. The best model achieves a 0.72 mean win rate across healthcare tasks! Leaderboard: https://lnkd.in/gZ7CkNqc Blog: https://lnkd.in/gsVQUc2P MedHELM is collaboration of Technology & Digital Solutions - Stanford Medicine, Microsoft HLS and researchers in the Stanford University School of Medicine and Human-Centered AI Matthew Lungren MD MPH, Eric Horvitz, Alyssa Unell, Hejie Cui, Suhana Bedi, Miguel Ángel Fuentes Hernández, Nikesh Kotecha, Michael Wornow, Juan M. Banda, Shrey J., Hao Qiu, Mert Öz, Yifan Mai, Akshay Swaminathan, Mehr Kashyap, Philip Chung, Roxana Daneshjou, Jonathan H. Chen, Emily Alsentzer, Keith Morse, Vanessa Kennedy, Sanmi Koyejo, Kevin Schulman, Jennifer Lee, MD, PhD, MBA, Timothy Keyes, Nerissa Ambers, Carlene Lugtu MCiM, BSN, RN, Aditya Sharma, Bilal M., Alex Alekseyev, Vicky Zhou, Vikas K., Jarrod Helzer, Jason Fries, Anurang Revri.

The UT System is among 12 health systems to support the Coalition for Health AI (CHAI) Applied Model Cards, AI "nutrition labels" designed to help health systems evaluate AI solutions more efficiently, streamlining validation and testing. As the eighth largest clinical operation in the country by physicians, our health enterprise has a tremendous impact and responsibility. https://bit.ly/41jwqdG