Giskard’s Post

𝗜𝘀 𝘆𝗼𝘂𝗿 𝗖𝗼𝗺𝗽𝗮𝗻𝘆 𝗥𝗘𝗔𝗗𝗬 𝗳𝗼𝗿 𝘁𝗵𝗲 𝗟𝗟𝗠𝗢𝗽𝘀 𝗥𝗘𝗩𝗢𝗟𝗨𝗧𝗜𝗢𝗡? LLMs are revolutionizing various fields, but their development demands specialized handling. LLMOps emerges as a solution, streamlining #LLM creation, deployment, and administration throughout their lifecycle. Mirroring MLOps, LLMOps fosters collaboration across data science, engineering, and IT teams. However, it caters to LLM-specific needs like cost efficiency, immense computational resources, and #promptengineering. While traditional MLOps validation applies, #LLMOps explores A/B testing and LLM-specific evaluation tools for optimal performance assessment. LLMOps empowers businesses to unlock the full potential of LLMs, ushering in a new era of AI-powered applications. lakeFS enables data practitioners to implement data version control for pipelines, reducing data duplication and storage costs while guaranteeing #dataquality. It is an essential piece in your LLMOps infrastructure.

[New on our blog] LLMOps: What It Is, Why It Matters, and How to Implement It by Stephen Oladele TL;DR → LLMOps involves managing the entire lifecycle of Large Language Models (LLMs), including data and prompt management, model fine-tuning and evaluation, pipeline orchestration, and LLM deployment. → While there are many similarities with MLOps, LLMOps is unique because it requires specialized handling of natural-language data, prompt-response management, and complex ethical considerations. → Retrieval Augmented Generation (RAG) enables LLMs to extract and synthesize information like an advanced search engine. However, transforming raw LLMs into production-ready applications presents complex challenges. → LLMOps encompasses best practices and a diverse tooling landscape. Tools range from data platforms to vector databases, embedding providers, fine-tuning platforms, prompt engineering, evaluation tools, orchestration frameworks, observability platforms, and LLM API gateways. — (link to the full article in the comments) #ML #LLM #LLMOps

Got one hour? Then you can learn how to implement rules-based testing to assess your LLM application. 👍 We're really excited about this course. Here's what you'll be able to do after completion: ✍️ Write robust LLM evaluations to cover common problems like hallucinations, data drift, and harmful or offensive output. 🏗 Build a continuous integration (CI) workflow to automatically evaluate every change to your application. ✅ Orchestrate your CI workflow to run specific evaluations at different stages of development. Ready...Set...Learn: https://circle.ci/428thxe Thanks to DeepLearning.AI for partnering with us on this! #LLMOps #CICD #MLOps

This course from DeepLearning.AI and CircleCI is excellent. It directly addresses some of the biggest concerns (dare I say “fears”) about managing non-deterministic probabilistic LLM output in your AI-enabled applications. Highly recommended. Take the course. 👨🏾💻

Developing LLM based services? You might want to have a prompt (or generally model) versioning mechanism. Tracking your experiments and having a dedicated model repository is a well known MLOps practice. Working with LLMs, I've found that you might want to package all of these components as part your experiment or your (production) model: - Model name and version - Prompt template - Code (pre/post processing steps) - Data & Evaluation metrics (measure model's performance with some eval set) You'll always be able to rollback to older models if anything goes wrong whenever you promote your designated model into production. Feel free to DM me if you need any help! #mlops #llm #llmops

[New on our blog] 📖 LLMOps: What It Is, Why It Matters, and How to Implement It Reading time: 11 min — (link to the full article in the comments) #ML #LLM #LLMOps

What does 2024 hold for MLOps and LLMOps? Here’s how Piotr Niedzwiedz sees it: - The use of proxy services between GPT APIs and production for security and guardrails will become more popular. - The number of discussions around regulations and ethics will grow, though smaller players will likely face fewer regulatory challenges. - The tension between the USA and China will push us faster towards AGI and exploring the limits. - The competition among the top players in the tech industry will become more aggressive and faster-paced. - The economic situation in tech will not improve much, and only the strongest companies will survive. - The MLOps/LLMOps tooling landscape will become more refined and easier to navigate. — (link to the full episode in the comments) #MLOps #LLMOps #ML

LMOps vs MLOps - 5 Key Differences

In this notebook, learn how to evaluate various #LLMs and RAG systems with #MLflow, leveraging simple metrics such as perplexity and toxicity, as well as LLM-judged metrics such as relevance, and even custom LLM-judged metrics such as professionalism. ✅ 🔗 Check it out: https://lnkd.in/engnbkvE #opensource #oss #linuxfoundation #mlops #llmops

How are you planning on mastering MLOps & ML Engineering in 2024? On January 24th at 11:30am EST I'll be discussing key strategies on handling your MLOps in 2024, including the latest developments in ML engineering and operational challenges and solutions in MLOps. Join the discussion and get a head start on your 2024 planning here: https://lnkd.in/d9dwCzSf Qwak #mlengineering #mlops