Fahad Jalal’s Post

Thinking more about LLM planning after having really excellent experiences with o1-preview for coding. It zero-shotted a Neovim plugin for Zig versions, LSP and builds - https://lnkd.in/e4gKs6ST I found an excellent talk from Noam Brown (one of o1's foundational contributors) about how much planning improves model performance on complex tasks: https://lnkd.in/e-3VYqY6 In AI poker, integrating search algorithms allowed the model to achieve in days what would have otherwise required 100,000 times more training data and parameters. This isn't just a minor tweak—it's a transformational leap. Imagine an AI that doesn't just generate immediate responses but strategically plans its output, leading to more accurate, insightful, and human-like interactions. By scaling test-time compute—allowing models to invest more processing power during inference—we enable them to delve deeper, reason better, and provide smarter solutions. How can people working on their own LLMs implement this?

Montecarlo-based tree search will improve reasoning and you can use older models too

Table Augmented Generation - The future which may replace Text2SQL due to its various capabilities. With this comes to LOTUS model. LOTUS (LLMs Over Tables of Unstructured and Structured Data) provides a declarative programming model and an optimized query engine for serving powerful reasoning-based query pipelines over structured and unstructured data! We provide a simple and intuitive Pandas-like API, that implements semantic operators to extend the relational model with a set of modular language-based operators. Programmers can easily compose semantic operators along with traditional data operations to build state-of-the-art AI systems that reason over vast knowledge corpora. credits : https://lnkd.in/gpqSk552 credits : https://lnkd.in/gT2REC-8 #genai #llms

Title: The Unreasonable Effectiveness of Eccentric Automatic Prompts Comment: The efficacy of prompt engineering via chain-of-thought strategies suitable for your problem type is basically understandable (you’re compensating for the lack of intelligence/understanding processes in the model). But, helping them by including in your prompt encouragement with “positive thinking” is weird! 😜 Eg., including in the prompt snippets like: “…you’re an expert mathematician”, and “… so let’s roll up our sleeves and solve the problem”. I am certain it’s not due to “emotional stimulation”, perhaps just directing it to statistically find tokens consistent with text sources associated with these tags, eg., texts linked to expert mathematicians, and detailed explanations. If so, this is evidence that these systems are lazy stochastic parrots! 😜 See… https://lnkd.in/e4gT8-Gn Large Language Models (LLMs) have demonstrated remarkable problem-solving and basic mathematics abilities. However, their efficacy is highly contingent on the formulation of the prompt. This study endeavors to quantify the influence of incorporating "positive thinking" into the system message of the prompt, then compare that to systematic prompt optimization. We assess the performance of 60 combinations of system message snippets, tested with and without Chain of Thought prompting, across three models with parameters ranging from 7 to 70 billion on the GSM8K dataset. Our findings reveal that results do not universally generalize across models. In most instances, the inclusion of "positive thinking" prompts positively affected model performance. Notably, however, Llama2-70B exhibited an exception when not utilizing Chain of Thought, as the optimal system message was found to be none at all. Given the combinatorial complexity, and thus computation time, of experimenting with hand-tuning prompts for large black-box models, we then compared the performance of the best "positive thinking" prompt against the output of systematic prompt optimization. We show that employing an automated prompt optimizer emerges as the most effective method for enhancing performance, even when working with smaller open-source models. Additionally, our findings reveal that the highest-scoring, automatically-optimized prompt exhibits a degree of peculiarity far beyond expectations. cc: Ronald Cicurel Gary Marcus

🤯 ULTRAQUERY: Answering Complex Logical Queries on ANY Knowledge Graph! 🧠 Researchers have developed ULTRAQUERY, a groundbreaking model that can answer complex logical queries on any Knowledge Graph (KG), even those it has never seen before! The Challenge of Complex Logical Queries: Most existing methods struggle to generalize to new KGs with different entities and relations. They require extensive training on each specific graph, limiting their practicality. ULTRAQUERY's Breakthrough: This innovative model utilizes inductive reasoning, meaning it can learn general principles from one KG and apply them to others. It achieves this by: ➤Inductive Relation Projection: Employing a pre-trained model that dynamically builds relation representations, enabling generalization to new relations. ➤Inductive Logical Operations: Implementing logical operators like AND, OR, and NOT as vocabulary-independent functions, allowing them to work with any entities and relations. Results: ULTRAQUERY demonstrates impressive zero-shot performance on various KGs, outperforming existing baselines on both standard and newly curated datasets. It even excels at tasks like: ➤Faithfulness: Accurately recovering answers reachable by simple graph traversal. ➤Answer Cardinality Estimation: Predicting the number of answers for a given query. This research allows efficient and scalable knowledge graph reasoning across diverse domains. _____________ ✔️ Click "Follow" on the Cohorte page for daily AI engineering news.

Let's think step by step and verify, but without humans. 🔥 Improve Mathematical Reasoning in Language Models by Automated Process Supervision from Google DeepMind proposes a new divide-and-conquer style Monte Carlo Tree Search (OmegaPRM) to improve multi-hop reasoning chains with process supervision without the need for human annotation. OmegaPRM identifies the first error in the Chain of Thought (CoT) with binary search and balances the positive and negative examples to then train a Process Reward Model (PRM).👀 Implementation 1️⃣ Define the reasoning task (e.g., math problem-solving) and provide the question and correct answer. This serves as the ground truth for the MCTS process. 2️⃣ Sample multiple Chain-of-Thought (CoT) reasoning steps for each problem. (Rollouts) 3️⃣ Compare the final answer to the ground truth and select the incorrect ones. 4️⃣ Locate the error using a binary search, where reasoning is split in the middle. Then, take the first half and sample multiple times. If one of them is correct, the error might not be in the first half. Then, take the second half and split again; take the first half of the second split and redo it until you find the reasoning step with the error. 5️⃣ Label and calculate Monte Carlo estimations for each step (node). 6️⃣ Train the Process Reward Model (PRM) with pointwise soft label training objective using the Monte Carlo estimation of balanced data. Insights 🔢 Achieves 69.4% on MATH, a 36% relative improvement from the 51% base model performance 💡 Pointwise Soft Label (using Monte Carlo estimation) objectives yield the best results for RPM (> Pairwise loss or hard label) 📈 Generated over 1.5 million process supervision samples 🏆 OmegaPRM data outperformed PRM800K (OpenAI) and Math-Shepherd 🤖 Supervising up to the first incorrect step in MCTS is sufficient to train a PRM 🔍 Using Binary search to identify errors reduces computational costs and time ⚖️ A balanced distribution of positive and negative examples improves RPM accuracy 🤖 Replaced human labeling with binary search + inference 🙋♂️ Human supervision is still necessary, needs the correct ground truth answer Paper: https://lnkd.in/ewA75Bvc

Excited to share that our paper, OOD Detection with NNK (https://lnkd.in/gvmypcE7), has been accepted to #EMNLP! 🎉 Congratulations to my brilliant co-authors. In this work, we extend NNK-Means to the language domain and introduce an information-theoretic approach for concise dataset summarization. 💪 Our method achieves state-of-the-art results in OOD detection while significantly improving memory and computational efficiency. 🚀 #NLP #LLM #MachineLearning

<insert clickbait line> Just built a neural network using Keras, on Lending Club dataset, basically to predict whether or not someone will repay their loan based on historical data, here's the breakdown: 1: Exploratory Data Analysis (EDA): Basically, I stared at some charts until they made sense. 2: Manage Null Values: Efficiently addressed null values to maintain data integrity. 3: Categorical Values: Dropped a few and converted some to dummy variables. Who knew being a "dummy" could be so useful? 4: Train/Test Split: Split the dataset into training and testing sets to ensure model reliability. 5: Scaling Data: Scaled both training and testing data for optimal model performance. 6: Create the Model: Chose a sequential neural network model, starting with layers equal to the number of features, using ReLU as the activation function. Added Dropout layers to prevent overfitting, and used sigmoid activation in the final output layer for binary classification. 7: Train the Model: Fitted the model on training data with multiple epochs, using validation data to monitor and mitigate overfitting. 8: Model Evaluation: Assessed model performance on test data and fine-tuned for better accuracy. You can find the code in my GitHub repository: https://lnkd.in/gQxE3Qts

As I continue exploring the challenges of scaling AI, it's becoming clear that Maximum Likelihood Estimation (MLE), while foundational, struggles when applied to large models. Many of these challenges are closely tied to cross-entropy, which is often used as the loss function in modern machine learning models. Cross-entropy loss measures the difference between the predicted probability distribution and the true distribution. It’s mathematically grounded in information theory and is crucial for models that rely on probabilities, such as classification tasks. In simple terms, it helps a model understand how well it’s performing by calculating how “off” its predictions are. However, when we scale models and datasets, cross-entropy becomes more sensitive to noise and more prone to overfitting. This happens because large models tend to memorize noise in the data, which inflates cross-entropy values and results in models that don’t generalize well. Cross-entropy is also affected by outliers in much the same way as MLE: a few extreme values can disproportionately impact the loss, skewing the model’s performance. For those looking for a deeper dive into the math behind cross-entropy, see the following: https://lnkd.in/gTzGRYew Soo… Why Does This Matter for Scaling? As we push the boundaries of AI and train larger models on increasingly complex datasets, these issues compound. The combination of MLE and cross-entropy means that large models become more brittle, often struggling to adapt to unseen data or fluctuating environments. For example, in natural language models, scaling often results in the overfitting of common phrases while ignoring rarer, but important, linguistic patterns. While techniques like regularization and data augmentation help mitigate these issues, they don’t fully address the fundamental limitations when working with vast datasets and large-scale architectures. So, What Comes Next? There's a lot of conversation around the limits of cross-entropy, particularly as it relates to scaling models. But what if we shifted the paradigm? What if the reliance on cross-entropy itself is part of the problem? I'll leave it at that for now. Stay tuned for a deeper dive into what could be a new direction in AI training methodologies—one that might challenge the very foundation we’ve been relying on… #AI #MachineLearning #CrossEntropy #MLE #ScalableAI #DeepLearning #DataScience #AIResearch

Comparing Embeddings A good data scientist compares data representations against each other. Just like a good parent compares its kids. Depending on what representations you are looking at there is a way to reduce or increase the difference between them. If it’s a point probability, you’d use cross entropy. If it’s distributions, you could use KL Divergence. And if it’s actual input representations generated by a neural network, you can use a contrastive loss. A contrastive loss helps to differentiate embeddings based on whether we want to be similar or not. It allows to create a margin between the similar and dissimilar embeddings. Came across one of the recent posts written on it to explain along with some insights why LeCunn used it and how he used it. Check it out! https://lnkd.in/g_rB2JZN