Snorkel AI’s Post

Data will continue to be the #1 source for models to learn from. Teams that prioritize quantity over quality often face model plateau and increased "out-of-distribution" issues. With Snorkel Flow, you can focus on ensuring data quality first, then train your models. When fine-tuning LLMs, having a small set of well-curated examples can yield amazing results. Want to learn how to achieve these results? Check out the link below!

For best AI responses with the o1 reasoning model, use clear, simple prompts without overexplaining or asking for step-by-step reasoning. Structure inputs with delimiters if needed, and provide only the most relevant context. No need of over-prompting https://lnkd.in/dv_C5368

It appears that OpenAI o1 includes 'reasoning tokens' in its pricing. "Most interestingly is the introduction of “reasoning tokens”—tokens that are not visible in the API response but are still billed and counted as output tokens. These tokens are where the new magic happens." See https://lnkd.in/gFs4v3ex "While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens." #pricing #AI #generativeAI #GenAI #tokenpricing

How to Improve latency across a wide variety of LLM-related use cases? This guide from open ai covers the core set of principles you can apply to improve latency across a wide variety of LLM-related use cases. These techniques come from working with a wide range of customers and developers on production applications, so they should apply regardless of what you're building – from a granular workflow to an end-to-end chatbot. https://lnkd.in/e4z4vRsY #LLM #ai #machine_learning #generative_ai #deep_learning

Looking to get the most out of Generative AI? Crafting effective prompts is the key! You would like to refer to these six strategies from OpenAI for prompt engineering which offer practical tips, including how to ask the model to adopt a persona, how to use "few-shot" prompting, "chain of thought", retrieval augmented generation (RAG) for more tailored and impactful responses. Check it out here: https://lnkd.in/gMjzUydp #AI #PromptEngineering #ArtificialIntelligence #Productivity #RAG #GenerativeAI

🚀 Exciting news! The GPT-4o-mini from OpenAI, the smartest and most cost-effective model in its class, is now available for fine-tuning! Take advantage of our built-in integration with Weights & Biases to seamlessly track your fine-tuning experiments: https://lnkd.in/e4W8YCEx. We can't wait to hear about the amazing projects you'll create with it! Happy experimenting. #foundationalmodels #finetuning #experimenttracking #AI #ML #wandb #OpenAI #GenAI

Noticed an increase in token usage in your results? This might be one of the reasons why. 🤔 The o1 models by #OpenAI introduce an intriguing concept: reasoning tokens. These tokens enable the models to "think" more deeply by breaking down their understanding of a prompt and evaluating multiple approaches to generate a response. Here's how it works: https://lnkd.in/gkXUZTUU 🔹 Reasoning Tokens: Used internally by the model to explore various response strategies. 🔹 Completion Tokens: The final answer output, visible to the user, whereas reasoning tokens are discarded after use. For instance, in a multi-step dialogue between a user and an assistant, input and output tokens are retained each step, but reasoning tokens are not. 🧠 This method allows for a more thoughtful and nuanced interaction with AI, enhancing the depth and quality of the responses generated. Have you explored how reasoning tokens impact your interactions with AI? Share your thoughts and experiences with us! 💬

OpenAI rolls out the meta-prompt, making it easier to craft high-quality prompts by building in best practices from real-world use. A must-use framework for anyone employing AI in their work. Core principles include: Understanding the task's main goal and requirements Making minimal changes when improving existing prompts Emphasizing reasoning steps before drawing conclusions Including high-level examples when appropriate Using clear, specific language Applying markdown for better readability Preserving user-provided content as much as possible Specifying the most suitable output format

I was too busy during this period, and the students around me were trying to do AI-related projects. This article is a guide I read a year ago. It can be said that it is one of the most important articles in my introduction to AI. It is very helpful for you to understand the work and development of AI. I recommend everyone to take a closer look. https://lnkd.in/e4VAkuwn

Here comes a tool to help adjust/generate system prompts for Open AI models: "Creating prompts and schemas from scratch can be time-consuming, so generating them can help you get started quickly. The Generate button uses two main approaches: Prompts: We use meta-prompts that incorporate best practices to generate or improve prompts. Schemas: We use meta-schemas that produce valid JSON and function syntax." https://lnkd.in/eQbA9qzh