OctoAI (now NVIDIA)

What's your favorite GenAI use-case? How has generative AI made tasks in your life easier? Alyss Noland details her favorite in the clip below. Check out the full episode of AI Unscripted on Youtube: https://bit.ly/3AJuAJe

At OctoAI, we're committed to transparency and control through open-source innovations. We believe that cross-stack, interoperable services are the key to unlocking the full potential of GenAI. Our large multimodal inference engine is designed to be flexible and adaptable, supporting a full range of enterprise tools, data types, and use cases. Supporting the broader organizational needs of enterprise customers, from fine-tuning to evaluation and more. Contact us to request a no-cost proof of concept to demonstrate the value of the OctoAI Inference Engine. Read the full article here: https://bit.ly/47bfIzo

What if you could transcribe conversations in real-time, extract key information, and generate summaries - without physically writing a single word? Pedro T. shows the Electronic Health Record Demo which has some key features: 🎧 Real-time audio transcription ✒️ Named Entity Recognition for crucial medical data ➕ Automatic summary generation 🔒 Secure deployment using Snowflake Container Services This solution goes beyond healthcare. Any industry requiring customer support or process documentation can customize to fit their needs. Watch now - https://bit.ly/3z7AKm4

"Before AI, only programmers were able to get computers to do what they wanted by writing arcane programming language texts. OctoAI was created to accelerate our path to that reality so that more people can use and benefit from AI. And people, in turn, can use AI to create yet more benefits by accelerating the sciences, medicine, art, and more." - Jason Knight Thanks to Unite.AI for this great interview! 🔥 Catch the full interview here: https://lnkd.in/es8Rnuky

As enterprise businesses look to leverage Generative AI models, they face unique challenges integrating sensitive data and ensuring security. Rodney Shetler breaks down some of the key thoughts we've been hearing for the enterprise use case. Find out how you can run your choice of models in your environment, including any cloud platform, VPC, or on-premise, ensuring full control over your data with OctoStack. https://bit.ly/3z0ZrAu Watch the full Builder's Roundtable on Secure GenAI for the Enterprise on Youtube- https://bit.ly/3XoCCQx

Get more out of smaller open source models with fine-tuning! ⚙️ Watch the full video on YT to find out how a fine-tuned Llama 3.1 8B model can outperform a proprietary model like GPT-4o when it comes to cost & quality 👇https://bit.ly/3Z6dsYh

What if we could give LLMs the power to make decisions and take actions and go beyond chat interfaces? 🤔 Function Calling allows the model to decide when it needs additional information or when to use external functionalities, bridging the gap between language understanding and real-world actions. Using Llama 3.1 8B and 70B, we demonstrate how you can use function calling with a customer support use-case - streamlining your workflows and saving you valuable time ⏱️ Read more here 👉 https://bit.ly/4dKC4KC

What does it mean to elicit trust in AI? In yesterday's OctoAI panel on secure genAI for enterprises, I made the point that answering this question amounts to two things. **Maximizing upside** Can we trust AI to attain desired outcomes reliably and generate value for us? **Minimizing downside** Can we trust AI to not attain undesired outcomes that may result in financial or otherwise loss? The definition and metrics for desired and undesired outcomes will depend on usage context of an AI system. Within a certain usage context, - Business goals will dictate minimum required desired outcome. - Regulatory and compliance requirements plus ethical guidelines will inform maximum acceptable undesired outcome (i.e. risk tolerance). Eliciting trust in AI broadly amounts to achieving the balance between these two factors in a particular context. --- Did that make sense? What desired outcomes do you prioritize most when it comes to AI, and what potential risks are you most concerned about? Share your thoughts in the comments below!

Inference should be designed to support the diverse needs of developers, from model tuning and evaluation to routing. OctoAI’s multimodal inference engine delivers value to customers that goes beyond the model endpoint, focusing on aligning closely to the essential needs of enterprise customers: 🎡 Inference as the cornerstone of the GenAI flywheel ⚖️ Balancing of unit economics with system reqs for latency, throughput, and quality. ⛏️ Customization capabilities to adapt a set of models to solve user problems. 👓 Greater transparency and control driven by open source innovations, 💪 Flexible and adaptable support for a full range of enterprise tools, data types, and use cases without increasing engineering overhead. Jared Roesch takes a deep dive into the OctoAI Inference Engine 👇 https://bit.ly/3Xo0TGo

Check out the latest post from our CTO Jared Roesch that digs deep into the OctoAI Inference Engine. There are a lot of model endpoints out there, but what sets OctoAI apart is a relentless focus on building toward enterprise needs: * Inference is the cornerstone of the GenAI flywheel but it must be designed to support the full scope of developer needs -- such as model tuning, evaluation, and routing. * Balancing of unit economics with system requirements for latency, throughput, and quality. * Customization capabilities to a adapt a model or set of models to solve user problems. * Greater transparency and control driven by open source innovations, optimized for the enterprise. * Flexible and adaptable support for a full range of enterprise tools, data types, and use cases without increasing engineering overhead. This philosophy has driven the development of the OctoAI platform since day one and enables us to deliver features that are unique in the market: 🔧  Efficiently run a large number of PEFTs on a single node 🖥 Deploy on diverse hardware, including legacy GPUs  🚀 Leverage MLC-LLM for leading performance  📽 Large context sizes crucial for multimodality  ⚙ Configurable and flexible to support next-gen models I am super proud of this effort and the whole team! Hope you enjoy learning more about it.