THE DECODER

1/ OpenAI has launched MLE-bench, a new benchmark to measure the capabilities of AI agents in the development of machine learning solutions. The test includes 75 Kaggle competitions from different domains such as natural language processing and computer vision. 2/ In initial experiments, the o1-preview model with the AIDE framework achieved the best results. It achieved at least a bronze medal in 16.9% of the competitions. More trials per competition and longer processing times led to better results, while additional GPU power had no significant impact. 3/ OpenAI sees MLE-bench as an important tool for evaluating core competencies in ML engineering, but acknowledges that the benchmark does not cover all aspects of AI research. In order to avoid possible contamination effects, various measures such as a plagiarism detector were implemented. 👇 Read more #AIagents #o1 #OpenAI

1/ AMD has announced details of its upcoming Instinct MI355X AI accelerator, which is expected to be released in the second half of 2025. 2/ The chip is based on the new CDNA4 architecture and will be manufactured by TSMC using the 3-nanometer process. 👇 Read more #AIChips #AMD

1/ Tesla unveiled its Cybercab autonomous taxi, which looks like a smaller version of the Cybertruck. According to CEO Elon Musk, it will be produced from 2026 and will cost less than $30,000. Tesla also unveiled an autonomous minibus called Robovan. 2/ The humanoid robot Optimus could be even more lucrative for Tesla. At the presentation, five Optimus units performed a dance and demonstrated skills such as different accents. Musk predicts that Optimus could generate up to $25 trillion in sales. 3/ Alongside the product launches, it was revealed that Tesla has lost four senior executives in the past week. Former employees report burnout and frustration due to Musk's management style and frequent reorganisations. 👇 Read more #Robotics #Robots #Tesla

Google is rolling out its latest image generation AI, Imagen 3, to all Gemini users worldwide, including free accounts. The company claims that Imagen 3 is its most powerful image model yet, outperforming competitors such as DALL-E 3, Midjourney v6, and Stable Diffusion 3 in internal testing.

1/ Defense contractor Anduril Industries has unveiled two new autonomous drones with AI capabilities: Bolt, for reconnaissance and search and rescue missions, and Bolt-M, an ammunition-carrying variant designed to provide ground troops with precision firepower. 2/ Both drones feature advanced computer vision and machine learning software that enables them to track targets from user-defined standoff positions and maintain tracking even when the target is obscured, with Bolt-M capable of attacking from any angle with high precision. 3/ Anduril emphasizes the ease of use and quick deployment of the drones, requiring minimal training for effective operatio. The company is also involved in developing highly autonomous drones for the US Air Force's Collaborative Combat Aircraft program.

1/ The Japanese start-up Rhymes AI has released Aria, which it claims is the world's first open-source, multimodal Mixture-of-Experts (MoE) model that is designed to match or outperform specialized models of comparable size. 2/ Aria has been pre-trained in four phases with a total of 6.4 trillion text tokens and 400 billion multimodal tokens and shows SOTA performance in benchmarks on multimodal, language and programming tasks, including long inputs such as videos with subtitles or multi-page documents. 3/ Rhymes AI has released Aria's source code under an open source license and is collaborating with AMD to optimize the performance of its models by using AMD hardware, as in the BeaGo search application developed for consumers. 👇 Read more #GenerativeAI #OpenSource

1/ study by researchers from Technion University, Google and Apple shows that large language models often know the right answers internally, even if they provide incorrect outputs. 2/ The researchers focused on the "exact answer tokens" in AI responses. They found that these tokens contain most of the information about whether an answer is correct or incorrect. It turned out that the AI models sometimes "knew" the correct answer internally, but still gave an incorrect answer. 3/ These findings could lead to new approaches to improve the reliability and accuracy of AI systems. The fact that models regularly "know" more internally than they show in their outputs opens up possibilities for improved error detection and correction mechanisms. 👇Read more #GenerativeAI

1/ Researchers from Princeton and Yale University investigated how language models solve tasks in chain-of-thought (CoT) prompts. They identified three influencing factors: Probability of the expected outcome, implicit learning from pre-training, and the number of intermediate steps in reasoning. 2/ A case study on decoding shift ciphers showed that GPT-4 combines probabilities, memorization, and a kind of "noisy reasoning." The model can transfer what it has learned to new cases and uses two strategies: forward or backward shifting of the letters. 3/ The explicit output of the intermediate steps in the chain of thought proved to be crucial for the performance of GPT-4. Surprisingly, the correctness of the content of the example chain in the prompt hardly played a role. The researchers conclude that CoT performance reflects both memorization and a probabilistic form of genuine reasoning. 👇 Read more #GenerativeAI

1/ The Royal Swedish Academy of Sciences has awarded the 2024 Nobel Prize in Chemistry to three researchers who have made important advances in deciphering and designing protein structures. 2/ David Baker from the University of Washington receives one half of the prize for his work in the computer design of proteins. The other half is shared by Demis Hassabis and John Jumper from Google DeepMind for the development of the AlphaFold AI system for predicting protein structures. 3/ AlphaFold 2 uses an AI mechanism known as "Attention" and takes into account evolutionarily related sequences as well as physical and geometric constraints on protein folding. The prizewinners' discoveries enable a better understanding of biological processes and accelerate the development of new drugs. 👇 Read more #AIandscience

1/ A new study shows that ChatGPT GPT-4 is surprisingly good at recognizing faces, determining gender, and estimating the age of people in photos - even though it was not explicitly trained to do so. 2/ For gender recognition, ChatGPT GPT-4 achieved a perfect hit rate of 100 percent on a dataset of 5,400 balanced images, outperforming the specialized DeepFace model. It also demonstrated remarkable age estimation capabilities. 3/ The researchers were able to bypass ChatGPT's safeguards by claiming in the prompt that the image entered had been generated by an AI. This shows that the robustness of large language models needs further investigation.