Atalgo’s Post

What research are AI companies doing into safe AI development? What research might they do in the future? To answer these questions, Oscar Delaney, Oliver Guest, and Zoe Williams looked at papers published by AI companies and the incentives of these companies. They found that enhancing human feedback, mechanistic interpretability, robustness, and safety evaluations are key focuses of recently published research. They also identified several topics with few or no publications, and where AI companies may have weak incentives to research the topic in the future: model organisms of misalignment, multiagent safety, and safety by design. (This report is an updated version that includes some extra papers omitted from the initial publication on September 12th.) https://lnkd.in/g_6DnqgX

Trusting that an AI will know when to ask for help is a key to more seamless adoption by users.

Researchers at UC San Diego and Tsinghua University have developed an innovative AI model that can recognize when it doesn’t have an answer and ask for human help, representing a major step forward in AI reliability. Unlike typical models that rely on confidence thresholds, this “self-aware” AI identifies its own limitations, signaling for human assistance to prevent errors and build trust in applications like healthcare and finance, where accuracy is crucial. This approach challenges the idea that simply increasing model size makes AI better. Instead, it suggests that smaller, focused models trained to detect uncertainty can be more effective and safer. This trend emphasizes efficiency and responsible AI, similar to specialized tools like OpenAI’s Whisper and Meta’s LLaMA, which prioritize precision and reliability over sheer scale. Read more at https://bit.ly/4fDUgGj

Researchers at UC San Diego and Tsinghua University have developed an innovative AI model that can recognize when it doesn’t have an answer and ask for human help, representing a major step forward in AI reliability. Unlike typical models that rely on confidence thresholds, this “self-aware” AI identifies its own limitations, signaling for human assistance to prevent errors and build trust in applications like healthcare and finance, where accuracy is crucial. This approach challenges the idea that simply increasing model size makes AI better. Instead, it suggests that smaller, focused models trained to detect uncertainty can be more effective and safer. This trend emphasizes efficiency and responsible AI, similar to specialized tools like OpenAI’s Whisper and Meta’s LLaMA, which prioritize precision and reliability over sheer scale. Read more at https://bit.ly/4fDUgGj

Google DeepMind's researchers have unveiled a new method to accelerate AI training, significantly reducing the computational resources and time needed to do the work. This new approach to the typically energy-intensive process could make AI development both faster and cheaper. Proposed approach—multimodal contrastive learning with joint example selection (JEST)—surpasses state-of-the-art models with up to 13× fewer iterations and 10× less computation. JEST operates by selecting complementary batches of data to maximize the AI model's learnability. Unlike traditional methods that select individual examples, this algorithm considers the composition of the entire set Paper 👉 https://lnkd.in/g2r8trzZ

This research report by RAND identifies the common causes of AI project failure to include (1) misunderstanding the problem that needs to be solved, (2) lack of necessary data, (3) pragmatic use of new tech, (4) insufficient IT infrastructure, and (5) AI not capable enough to solve problem efficiently. The report also video great recommendations to enable these projects to succeed. #DigitalTransformation #AI #RiskManagement

Key Analysis around AI: Risk v. Reward "It is harder to put [GenAI] systems into practice, because they're unstable, and error prone... People are rushing to put these systems in." - Ran Balicer, HIMSS Board of Directors According to McKinsey's latest AI study, 80%+ of budget is still biased towards Machine Learning. Why? They're "simpler," have a decades long track record of stability and success, and requires far less computation and power needs than the new-to-the-neighborhood GenerativeAI models. In the field of healthcare, hallucinations from GenAI models like LLMs may have farther reaching detrimental effects than other industries as patients and healthcare workers are reliant on accurate information. There's also the risk of revealing certain Personal Identifying Information, and violating HIPAA. Its just as important to analyze when you shouldn't be leveraging a GenAI solution than when you want to. #healthcare #artificialintelligence #ai #machinelearning https://lnkd.in/gB4GMg9p

If you’re not experimenting with AI and studying use-cases and lessons learned, you’ll be behind in months - not years. This report is exactly the kind of material worth studying to learn from others’ efforts!

The new AI index report highlights the lack of consensus in academia or industry on the evaluation of LLMs, specifically on AI responsibility features. Another interesting takeaway is that the key stakeholder in LLMs development is the industry due to the prohibitive cost of developing LLMs.

Evaluating Persona Agents and LLMs It's great to see research around evaluating persona agents. Persona agents is probably going to become one of the most common and useful ways to use LLMs but there is very little research on how to properly assess them. This work proposes a benchmark to evaluate persona agent capabilities in LLMs. Finds that Claude 3.5 Sonnet only has a 2.97% relative improvement in PersonaScore compared to GPT 3.5 despite being a much more advanced model. Custom agents that act as specific personas makes sense in domains like education, healthcare, creativity, productivity, entertainment, and more. The interesting thing is that current models are explicitly trained to operate this way but a lot of people are interested in using them like that. https://lnkd.in/ewjPaHfe ↓ Follow my weekly summary of the top AI and LLM papers. Read by 70K+ AI researchers and developers: https://lnkd.in/e6ajg945