Heka.ai’s Post

Operating research is a major lever of #decarbonisation for sea #freight. However OR needs a powerful digitalization prerequisite - a semantic coherence of data. Read our article to learn more… #transportation #shipping Sia Partners Thibaut Dion Guilhem Catinon Kiki Wang

The road to next-gen AI applications is not going to build itself – though some genAI advocates see that capability around the corner. For now, the work of IT shops resolutely centers on getting useful output from LLMs that are still as much art as science. The balance will shift to the science side as efforts like Dynatrace’s enter the space. At its Dynatrace Perform 2024 event, it will launch extensions to the plaform to target LLM and GenAI observability.

We have released our latest report on 'Cost Reduction Methods for Running LLMs'. The report provides a detailed overview of methods and tools that organizations using APIs of Large Language Models (LLMs) can leverage to balance application usage costs and inference performance. Some of the key findings of the report: 1. Cost and Performance Tradeoffs: - When working towards cost optimization through prompt size reduction methods, the accuracy of results may get affected - Therefore, understanding the trade-off between costs and benchmark parameters such as latency and cold start time is critical.   2. Organizations focus on Token Optimization to reduce LLM costs - As technologies like chain-of-thought (CoT) prompting and in-context learning (ICL) evolve, the prompts provided to LLMs are growing more extensive, sometimes surpassing tens of thousands of tokens. - The cost of an LLM query increases linearly with the size of the prompt. The cost can be reduced by reducing the size of lengthy prompts.   3. Prompt Compression, Model Routing / Cascade, LLM Caching, Optimizing Server Utilization, and Cost Monitoring and Analysis are identified as the key methods for reducing the cost of running LLMs. Read the complete here: https://lnkd.in/gmikNDUP Some of the key companies offering tools in this space are listed below: Deci AI Unify Neutrino AI Teneo.ai Not Diamond Zilliz Portkey Dataiku LangChain Modal Labs RunPod Anyscale Together AI Inferless Replicate FriendliAI Baseten Mystic AI Aporia Arize AI Datadog BricksAI 🔭 Galileo LLMetrics SingleStore Helicone MongoDB #AIMResearch #FrugalGPT #OpenAI #GenAI #LLMs #LLMOps #CostReductionTools #PromptCompression #LLMCaching #ModelRouting #Batching

By looking at sessions as part of the LLM track at ODSC West, we get a pretty good understanding of where the field is going. Here are 8 trends that show what's big in LLMs right now, and what to expect next. #datascience #AI #ArtificialIntelligence https://hubs.ly/Q02Tc3h20

How come OpenAI was able to beat Google in the AI race? What does this have to do with our region? The innovation paradox is a phenomena that is well studied and observed, yet rarely implemented or avoided. We have all it takes to unlock the power of radical innovation and shape the future, together. _____________ AcceMind We redefine how organizations innovate and grow. www.accemind.com Book a Discovery Session Today ➡️ https://accemind.as.me/ 👉Newsletter: https://lnkd.in/d8nC-CUn👈 _______ Business Teaser Video: https://lnkd.in/dWkj4jbE What is AcceMind? https://lnkd.in/dUQs4YKC _______ What Problems Are We Trying To Solve? . 0️⃣ Context Setting: https://lnkd.in/dDjVz9Zf . 1️⃣ Introduction, reducing dependency on crude oil: https://lnkd.in/dzvTvXbz . 2️⃣ GCC Government Efforts: https://lnkd.in/dBUPiqVT . 2.1 Why is relying on Oil & Gas risky: https://lnkd.in/dWnZVeWp . 2.2 The role of organizations in reducing dependence on crude oil: https://lnkd.in/d8KgtMgs . 2.3 Tailing behind in the Innovation Race: https://lnkd.in/d8yUkYnN . 3️⃣ The roll of Patent Applications: https://lnkd.in/dvPnKSwY . 4️⃣ The Significance of Addressing the Innovation Deficit: https://lnkd.in/dgR7C-Nv . 4.1 Are you aware of the innovation deficit in the GCC? https://lnkd.in/dxbMKnfE . 5️⃣ The role of Academic Publications: https://lnkd.in/dBGgP8V9 . 6️⃣ GCC Paradox and The Drivers of Radical Innovation: https://lnkd.in/dewZEYHp . 7️⃣ What is the main Driver of Innovation? https://lnkd.in/dZgP-Sxv . 8️⃣ Localization policies, are they hurting your business? https://lnkd.in/di33YSTz . 9️⃣ This is why TAX in the GCC will not work! https://lnkd.in/dE2mUmVQ . 🔟 What's Next? https://lnkd.in/dQ6Wdm5F

A neat change: Substack added a sidebar "table of contents" for all articles. This is especially handy for The Pragmatic Engineer articles: they are longer, and have a clear structure. It works for all existing articles on the platform, though. To see it, just tap on the left sidebar to open, as it's hidden by default! Somewhat similar to how Google Docs does it. The screenshot is from last week's deepdive on how Anthropic built Artifacts, and how the AI scaleup operates from an engineering point of view: https://lnkd.in/eFMXrBGc

Built on the robust foundation of RapidsDB's distributed, in-memory database, Rapids VectorDB delivers substantial value to our customers. With its lightweight yet high-performance framework, we prioritize system reliability and data security without compromise. 💡   Today, we're thrilled to unveil the final three chapters of our white paper, "Rapids VectorDB: Fueling the AI Revolution"! 📑 In addition to the common use cases associated with vector databases—think text, image, and video retrieval, personalized recommendation systems, and intelligent customer service—Rapids VectorDB is empowering more advanced AI application scenarios. 🤖 This includes its unparalleled support for large language models, marking a significant step forward in the democratization of data within organizations. 🌐

How come OpenAI was able to beat Google in the AI race? What does this have to do with our region? The innovation paradox is a phenomena that is well studied and observed, yet rarely implemented or avoided. We have all it takes to unlock the power of radical innovation and shape the future, together. _____________ AcceMind We redefine how organizations innovate and grow. www.accemind.com Book a Discovery Session Today ➡️ https://accemind.as.me/ 👉Newsletter: https://lnkd.in/d8nC-CUn👈 _______ Business Teaser Video: https://lnkd.in/dWkj4jbE What is AcceMind? https://lnkd.in/dUQs4YKC _______ What Problems Are We Trying To Solve? . 0️⃣ Context Setting: https://lnkd.in/dDjVz9Zf . 1️⃣ Introduction, reducing dependency on crude oil: https://lnkd.in/dzvTvXbz . 2️⃣ GCC Government Efforts: https://lnkd.in/dBUPiqVT . 2.1 Why is relying on Oil & Gas risky: https://lnkd.in/dWnZVeWp . 2.2 The role of organizations in reducing dependence on crude oil: https://lnkd.in/d8KgtMgs . 2.3 Tailing behind in the Innovation Race: https://lnkd.in/d8yUkYnN . 3️⃣ The roll of Patent Applications: https://lnkd.in/dvPnKSwY . 4️⃣ The Significance of Addressing the Innovation Deficit: https://lnkd.in/dgR7C-Nv . 4.1 Are you aware of the innovation deficit in the GCC? https://lnkd.in/dxbMKnfE . 5️⃣ The role of Academic Publications: https://lnkd.in/dBGgP8V9 . 6️⃣ GCC Paradox and The Drivers of Radical Innovation: https://lnkd.in/dewZEYHp . 7️⃣ What is the main Driver of Innovation? https://lnkd.in/dZgP-Sxv . 8️⃣ Localization policies, are they hurting your business? https://lnkd.in/di33YSTz . 9️⃣ This is why TAX in the GCC will not work! https://lnkd.in/dE2mUmVQ . 🔟 What's Next? https://lnkd.in/dQ6Wdm5F

While the #GenAI news cycle can't stop announcing new models, model cost and evaluation continue to be crucial for both developers and businesses. Our latest publication dives deep into #opensource tools that help evaluate models while keeping costs low. These include Prometheus by KAIST AI, an open-source model for LLM-as-judge evaluation; Mozilla.ai's very own lm-buddy, a tool we developed and open-sourced to scale our own fine-tuning and evaluation tasks; and llamafile, a Mozilla Innovation project that brings LLMs into single, portable files. Davide Eynard shows how these components can work together to evaluate LLMs on cheap(er) hardware, and how we assessed the evaluators’ performance to make informed choices about them. https://lnkd.in/dJx8bxUp

Happy to finally share my first post for Mozilla.ai's blog! If I were to choose a sentence from it to capture your attention that would probably be: "[...] That excited me not much for the results I got (to be fair, my laptop’s battery died before we landed), but rather for the feeling of control it gave me: I did not have to rely on a third-party inference service or cloud GPU for doing LLM-as-judge evaluation." 😁