Hopsworks

We are proud to announce our biggest and most innovative release yet! 🎉 Hopsworks 4.0 introduces game-changing innovations for building AI systems, whether for batch, real-time or LLM applications, all powered by our AI Lakehouse infrastructure. Release Highlights: 🔹 Kubernetes Deployments - Fully containerized services deployable on any Kubernetes cluster, cloud, or on-premises. 🔹 Hopsworks Query Service - Provides Python clients with up to 45 times higher throughput when reading data. 🔹 Vector Search / Embeddings - Added support for vector & similarity search to easily build GenAI applications. The product will be generally available soon! Stay tuned. https://lnkd.in/d2vR-aZ7 

2/3 Real-time Data in LLM Yesterday we showed how to access Lakehouse Data with the LLM, Today, we are moving to real-time; this time, the diagram shows how models handle immediate data requests. When real-time information is needed, the system bypasses the lakehouse and directly queries current data or even a model! ************************************************* What does it mean: If I ask the LLM for real-time information - what's the current stock price, predict if a transaction is fraudulent, what's the best product to recommend right now - the system needs to access the most up-to-date data or even query a live model. By enabling direct access to real-time data sources and models we are empowering the LLM with new context and even futur predictions. ************************************************* Key points: - 1-5ms latency, - Enables instant decision making and personalization; next best action, next best offer, - Supports both batch and real-time AI workloads, - Ensures data freshness. #RealTimeAI #LLM #DataScience #AILakehouse

1/3 Lakehouse Data in LLM Kicking off a series of short posts and visual on advanced AI data integration for LLMs! This diagram shows how LLMs handle queries requiring historical / analytical data, in short, when the model needs to tap into a lakehouse. The system here needs to uses fast retrieval to access offline data stores. ************************************************* What does it mean: If I ask the LLM any form of analytical question - what is the average number of shops in this area, what are the most sold items, what are the best products for a specific segment - chances are this information already exists in the organization's lakehouse. If your system allows building a RAG that directly accesses lakehouse data, you'll increase accuracy, reduce latency, and decrease reliance on secondary systems; you're leveraging data you already have. ************************************************* Key point: - 10-500ms latency, - Direct access to analytical data, - Supports complex queries (for instance; combining data from multiple time series) - Goes Beyond vector DB limitations (cost, consistency with other systems, support for non similarity related queries, etc) #AIAnalytics #DataLakehouse #MachineLearning #AiLakehosue #LLM

Curious about how AI can accelerate disease prediction for doctors?👩⚕️📋 In a recent webinar we showed how you can create a copilot chatbot that analyzes reports and scans to provide near-instant insights. In the full webinar Fabio Buso explains the underlying model and its role in enhancing healthcare and speeding up diagnostics. https://bit.ly/3Zfa0Lh

⚠️ Data Transformation; When AI Systems Fail ⚠️ One of the major reasons AI systems fail is not due to a lack of performance of the models. But mostly a missunderstanding of how data transformation works. Let's resolve that. 🛒 Take the e-commerce scenario of my diagram: From the Data side; → Current cart value (125) →→ This is only available at request time →→→ Transformation = On-demand ------ → Historical Data (count: 5, total spend: 1000)  →→ Required for multiple models →→→ Transformation = Independent to the model From the Model side; 🔮 For inference: Current Cart + Historical Data →  Model-Dependent. 🤼 For Training: Historical Data for a specific Model →  Model-Dependent. This Taxonomy for Data Transformations in AI systems (link below) is critical, we have; 1️⃣ Model-Independent Transformations • Reusable across models • Think aggregations, windowed counts, joins • Stored in feature stores 2️⃣ Model-Dependent Transformations • Specific to one model • For feature encoding, normalization • Applied in both training and inference 3️⃣ On-Demand Transformations • Require real-time data • Used in online inference • Can be backfilled for training Why this matters: - Prevents training-inference skew - Enables data reuse across multiple models - Optimizes storage and compute #AIEngineering #MachineLearning #DataScience #FeatureStore #AiLakehouse

News from Hopsworks 🗞️ Trouble getting started on your Hopsworks journey? This week, we share a few resources to help you become an ML and AI Lakehouse pro!

In our 5-minute interview series we’ve met with several MLOps and ML advocates telling us about their experiences and contributions to the field. In one episode Paul Iusztin told us about his idea of how to master the fundamentals of MLOps in a fast paced field like AI. Watch all interviews here 👉 https://bit.ly/3X37gPq

This is a visual blueprint to create an LLM, End-to-End System using the FTI Framework, based on one of the many content and talks by Jim Dowling during this year. And this might be a bit daunting, so, what is in the diagram here? 1️⃣ Data Collection: Gathering Raw Data From Different Sources 📂 /Data Sources │ ├──── PDFs 📄 │ │ └──── External storage (e.g., S3, Drive) │ ├──── ClickLog 🖱️ │ │ ├──── Last Clicks │ │ └──── Chat History │ └──── Orders 🛒 │ ├──── Name │ └──── Last Buy ────── 2️⃣ Data Transformation: Turning Data into Intelligence 🔧 /ML Pipelines │ ├──── Feature Pipeline 🛠️ │ │ ├──── Extract Text │ │ ├──── Feature Engineering │ │ └──── Compute Embeddings │ └──── Training Pipeline 🧠 │ ├──── Instruction Dataset │ └──── Fine-Tuned Model ────── 3️⃣ Infrastructure: Supporting the System 🏗️ /Data Infrastructure (Hopsworks) │ ├──── Feature Store 📊 │ ├──── Model Registry 🗂️ │ └──── ANN Index 🔍 ────── 4️⃣ User Interaction: Delivering 👥 /Users & Application │ ├──── Inference Pipeline 💡 │ │ ├──────────── RAG (Retrieval-Augmented Generation) │ │ ├──────────── Enriched Prompt │ │ └──────────── Fine-Tuned Model │ └──── Response Logging 📈 ────── If you are curious on how to build such systems; feel free to go the github repo and build it yourself ; https://lnkd.in/gHvAUtwU

Keep your ML models accurate with feature monitoring in Hopsworks 4.0! Track data changes over time and compare with training data for proactive retraining 🤓 https://bit.ly/3W7BCQm