MLflow

MLflow 2.17.1 adds OpenAI Swarm autologging+tracing support and expands integrations with Unity Catalog and Databricks Genie Spaces support. This release enhances MLflow's integration capabilities with major platforms while improving core functionality. Updates include: 🔄 OpenAI Swarm: Added tracing and autologging support 🔗 Unity Catalog: UC connections now supported as model dependencies 🚀 Genie Spaces: New support for Databricks model resources 💬 Improved custom chat endpoint support 🛠️ Various bug fixes and documentation improvements Full release notes: https://lnkd.in/gX68ggMu #MLflow #MachineLearning #AI #OpenSource #LLMOps #MLOps

Build advanced RAG systems with MLflow and LlamaIndex Workflows! Go beyond the basics: 🔍 Combine vector search, keyword-based search (BM25), and web search in parallel 🧠 LlamaIndex Workflows for flexible, event-driven orchestration 📊 MLflow for experiment tracking, reproducibility, and evaluation Key benefits: • Break complex tasks into manageable steps • Easily debug and improve your LLM applications • Measure quality, speed, and cost of your system Check out the full blog post: https://lnkd.in/g2jWJw4J

LlamaIndex Workflows let you build powerful AI pipelines. Complex flows need robust tracking and testing. MLflow provides the MLOps foundation you need: experiment tracking, reproducibility, tracing, and evaluation. Our new blog shows how to build a sophisticated RAG system using both tools, walking through: 🔍 Implementing hybrid retrieval (vector, BM25, web search) 📊 Tracking experiments across different retrieval strategies 📈 Evaluating answer quality and latency 🔬 Debugging retrieval issues with MLflow Tracing Complete tutorial with working code by Yuki Watanabe. Read it here: https://lnkd.in/g2jWJw4J #mlops #llmops #machinelearning #ai

🔄 Upgrading to the updated Anthropic Claude-3.5-sonnet model? MLflow AI Gateway makes it easy to test and compare! With a single configuration file and unified API, you can: - Query both Claude versions side-by-side - Compare responses between model versions - Switch with zero code changes - Keep your older version as backup Check out this guide for more details: https://lnkd.in/etuhUMie. It shows: - How to securely manage API keys - Complete configuration examples for multiple providers - Python client code for easy model comparisons - REST API endpoints for non-Python applications - Rate limiting and monitoring capabilities #MLflow #Claude #AI #MLOps #LLMOps

MLflow 2.17 introduces audio file support in the UI, enabling direct playback of logged audio artifacts. This new feature makes it easier for machine learning practitioners and AI engineers to integrate audio outputs into their experiment tracking workflows. As text-to-speech (TTS) models from providers like OpenAI and ElevenLabs become increasingly prevalent in AI systems, the ability to keep track of audio outputs and listen to them directly within MLflow's artifact viewer enables researchers and developers to more effectively track and analyze audio outputs alongside other metrics, models, and parameters. The accompanying screenshots demonstrate how to log audio artifacts generated from the OpenAI speech model using MLflow and show the resulting audio file playback interface within the MLflow UI. #mlflow #ai #machinelearning #mlops #llmops

MLflow's updated ChatModel class enables rapid development of standardized chat interfaces. MLflow's ChatModel class offers a streamlined approach for building and deploying custom chat models: 🚀 Reduces boilerplate compared to PythonModel while maintaining flexibility 🔗 Provides a standards-compliant chat interface with minimal configuration 🔄 Integrates seamlessly with MLflow's model management and deployment features 🛠️ Supports tool calling, tracing, and evaluation out of the box ChatModel is particularly useful for developers creating compound AI systems or agents that perform multiple operations before returning a chat response. It offers a consistent, deployable interface across various LLM providers and custom models. Learn more about implementing ChatModel in our new tutorial: https://lnkd.in/g2UzcQGs #MLflow #MachineLearning #AI #LLMOps #MLOps

MLflow 2.17 enhances ChatModel, improves GenAI evaluation, and adds audio support in the artifacts UI, among other updates! This release expands ChatModel capabilities with tool calling support and a simplified interface, making it easier to implement LLM-compatible models. Additionally: 📊 Tracing UI now provides structured output for retrieved documents 🔊 Audio file playback supported directly in the MLflow UI 📈 GenAI metrics are now callable, the first step toward expanded mlflow.evaluate features 🔗 Enhanced integrations with LlamaIndex Workflows and LangChain 🚀 MLflow AI Gateway is no longer deprecated Full release notes: https://lnkd.in/gk2hcrfD #MLflow #MachineLearning #AI #OpenSource #LLMOps #MLOps

ICYMI - A brief update on MLflow's current and upcoming GenAI features! 🚀 I recently shared insights on the remarkable progress of the MLflow team at the MLOps Community's Data Engineering and AI Conference. We delved into tracing https://lnkd.in/eDMNvm2z, GenAI framework integration support https://lnkd.in/e6aCnhQF, and the advanced application abstraction of ChatModel https://lnkd.in/eDHJdMPy. If you couldn't attend, you can catch up on the talk or access the transcription here: Link to talk and transcription https://lnkd.in/en3BASGB Exciting times ahead for MLflow and GenAI enthusiasts! MLflow #GenAI #MLOps MLOps Community

With just a few lines of code, MLflow's LangChain flavor enables powerful observability and experiment tracking for your existing LangChain projects. If you're using LangChain, integrating with MLflow provides significant advantages: 🔗 Log LangChain models with minimal changes to your existing code 🧪 Leverage MLflow's experiment tracking, dependency management, and model registry 📊 Utilize MLflow Evaluate for automated language model assessment Key benefits: ✅ Rapid comparison of different LangChain configurations and model versions ✅ Consistent environments from development to production ✅ Enhanced observability with MLflow Tracing Learn more and get started: https://lnkd.in/gbSV5tBk #MLOps #LLM #LangChain #MLflow #LLMOps

We're excited to share a new MLflow blog post showcasing how to use Large Language Models (LLMs) as judges for evaluating AI systems using MLflow! In this guide, Pedro Andrade Azevedo and Rahul Pandey demonstrate: - How to create custom metrics for cultural sensitivity and faithfulness - Implementing LLM-based evaluation techniques - Using MLflow Evaluate for standardized experiment setups - Applying these methods to real-world scenarios in the travel industry This blog gives detailed examples of several key MLflow features, including: ✅ Using mlflow.evaluate() for model evaluation ✅ Creating custom metrics with make_genai_metric() ✅ Using built-in metrics like toxicity ✅ Integrating with OpenAI models Whether you're new to MLflow or an experienced user, this blog offers valuable insights into advanced model assessment techniques. Read the full article here: https://lnkd.in/g8VYV6ZT #llm #evals #ai #mlflow