Arize AI

Bay Area: Join us tomorrow night (live and in person) for an AI tools deep dive. Come for the snacks, stay to connect with other devs, learn something, and bring valuable insights back to your team. 💪 You'll hear from us and our friends at Airbyte, and have lots of opportunities to meet other folks thinking about the best tools to include in their AI tech stack. Limited capacity for this one so register soon! https://lu.ma/7r4obqsh

Announcing our latest integration: crewAI! 🤖 🤝 CrewAI is an awesome framework that lets you define and coordinate multiple agents to work collaboratively. Imagine you have a crew of employees with different specialties, working together within a hierarchy to accomplish a larger task. Check out this walkthrough on how to build a research team using CrewAI and Arize Phoenix. https://lnkd.in/gwFDesVb

Join Arize AI's Hakan Tekgul and Julia Gomes as they demo aspects of their platform that assist developers in developing safe and effective LLM-based apps. https://lnkd.in/dCn8wkRh #AIObservability #ModernSaaS #GenAI

💫 Tracing 💫 is a powerful observability technique that helps AI engineers better see what goes on inside their LLM applications. In a tangle of prompt-response pairs, it's easy to quickly lose the ability to iterate effectively due to poor visibility. Tracing solves this issue by letting you see into the black. In his latest post, Evan Jolley explains how tracing works and the various use cases where it can be invaluable -- diving into a hands-on example on how to implement tracing. https://lnkd.in/gr3gyUf7

Next week, we’re talking to Kyle O'Brien, Applied Scientist at Microsoft, about his paper: Composable Interventions for Language Models. This paper has implications for how we can keep expensively trained models up-to-date over extended deployments. The discussion, led by Sally-Ann DeLucia, will cover key findings from extensive experiments, revealing how different interventions—such as knowledge editing, model compression, and machine unlearning—interact with each other. The research here offers some important guidance for current practice if you want to keep your models running efficiently, error-free, and responsibly. Join us live: https://lnkd.in/dmEY6C8F

Super excited to announce our latest integration: deepset's Haystack! 🚜 Now you can collect telemetry data on your Haystack pipelines with a single call to our auto-instrumentor. Get call-by-call insights into what is happening in your application, view latency and token usage, and run evaluations to measure performance. Check out the integration here: https://lnkd.in/grM7_h2K

Updating your prompts can feel like guessing. You find a new prompting technique on arXiv or Twitter which works well on a few examples, only to later run into issues. The reality of AI engineering is that prompting is non-deterministic; it’s easy to make a small change and cause performance regressions in your product. A better approach is evaluation-driven development; leveraging Arize, you can curate a dataset of key points that you’re trying to test, run your LLM task against those key points, and use code or LLMs or user-generated annotations to evaluate the output with aggregate scores. This allows you to test as you build and verify experiments before you deploy to customers. I run through a quick demo and accompanying notebook creating a user research AI and how I iterate on prompts below.

📣 Announcing… Annotations in Phoenix! We’ve long supported adding evaluations to spans in Phoenix, but we’ve found this didn't always cover the use cases we see in the community. Maybe you’re wanting to mark a particular run of your application to use for few-shot prompting. Or maybe you’re adding in a 👍 👎 user feedback system that you need to track. That’s why we’ve added annotations to the Phoenix platform. Annotations can be logged via UI or API, and allow you to add custom labels to your spans and traces. Use annotations to: 👍 Collect human feedback on your application responses 🏷️ Tag the best (or worst) runs of your application 📊 Build datasets based on annotations to power few-shot prompting or fine-tuning. To see how to set up annotations, check out our latest blog post: https://lnkd.in/eWTY_xsV

Implementing guardrails for AI systems is a delicate balancing act. While these safety measures are important for responsible AI deployment, finding the right configuration can be tricky. To help manage guards as system complexity grows, many are turning to tools like Guardrails AI or Nemo AI to manage their guards along with tools like Arize’s AI search, which can be helpful in identifying clusters of problematic inputs to allow for targeted guard additions over time. Here are a few of the major types of guards we see teams implementing. INPUT VALIDATION AND SANITIZATION 🚧 Syntax and Format Checks: while basic, these checks verifying that the input adheres to the expected format and structure are important for system integrity 🚧 Content Filtering: removing sensitive or inappropriate content before it reaches the model, critical for things like customer-facing chatbots 🚧Jailbreak Attempt Detection: the guards that prevent massive security breaches and keep your company out of news headlines OUTPUT MONITORING AND FILTERING 🛑 Preventing Damage: system prompt protection, NSFW or harmful language detection, and competitor mentions  🛑 Ensuring Performance: critic guards – which use a separate LLM to critique and improve your pipeline’s output before sending it to the user – and guards to prevent hallucinations are often helpful; here, developers face a choice between using guards to improve your app’s output in real-time or running offline evaluations to optimize your pipeline or prompt template Of course, other non-guard strategies like fencing your app from other systems, red-team prelaunch and monitoring your app post-launch are also critical! More in the blog by Evan Jolley and John Gilhuly: https://lnkd.in/eSqq7EZw

Virtual fireside chat happening tomorrow: Cam Young will be talking about ML monitoring, evaluation, and guardrails with LLMs in mind. 🔥🔥 RSVP to attend live, ask questions, connect with the community, and get the recording. Thanks to Sage Elliott and Union.ai for hosting! Register: https://lnkd.in/gDZCWWNX