Qdrant

🧠 𝐖𝐡𝐚𝐭 𝐜𝐚𝐧 𝐲𝐨𝐮 𝐥𝐞𝐚𝐫𝐧 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 Qdrant 𝐚𝐧𝐝 DeepLearning.AI 𝐜𝐨𝐮𝐫𝐬𝐞? In just one hour, our Senior Developer Relations Kacper Łukawski covers: ✅ The purpose of tokens in embedding models. No abstract theory — he breaks it down and shows you exactly how they work. ✅ Why tokenization type is crucial for search quality. Learn how prices, brands, dates, and typos "confuse" AI and how to handle this problem practically in Qdrant. ✅ "𝘠𝘰𝘶 𝘤𝘢𝘯 𝘴𝘬𝘪𝘱 𝘵𝘩𝘪𝘴 𝘭𝘦𝘴𝘴𝘰𝘯 𝘪𝘧 𝘺𝘰𝘶 𝘥𝘰𝘯'𝘵 𝘤𝘢𝘳𝘦 𝘢𝘣𝘰𝘶𝘵 𝘵𝘩𝘦 𝘲𝘶𝘢𝘭𝘪𝘵𝘺 𝘰𝘧 𝘺𝘰𝘶𝘳 𝘙𝘈𝘎 𝘢𝘱𝘱𝘭𝘪𝘤𝘢𝘵𝘪𝘰𝘯" ✅ HNSW (Hierarchical Navigable Small World) will no longer be just an abstract concept. You'll learn to optimize its parameters in Qdrant to improve search speed and accuracy. ✅ How can quantization in Qdrant be used to save memory without sacrificing precision? You'll also see practical examples of optimization techniques to apply in Qdrant: indexing, batch uploads, backups and more 👉 Check it out: https://lnkd.in/dh6PjKXD

🚀 #VectorWeekly 𝐨𝐧 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐢𝐧 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬 (𝐚𝐧𝐝 "𝐡𝐨𝐰 𝐭𝐨" 𝐢𝐧 Qdrant) You have millions of documents and need to get answers to your query within milliseconds. Choose your fighter: 𝐃𝐞𝐧𝐬𝐞 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐞𝐫𝐬 Documents are encoded into dense vector representations (usually, 1 document = 1 vector) with a machine learning model, which captures the semantics (meaning) of text (image, audio, video, etc.). The similarity between a query and documents is measured as the distance between their vector representations. 🔗 Qdrant vectors library called 𝐅𝐚𝐬𝐭𝐄𝐦𝐛𝐞𝐝: https://lnkd.in/d8tu2qre 𝐋𝐞𝐱𝐢𝐜𝐚𝐥 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐞𝐫𝐬 Documents are encoded into sparse vector representations built on the statistics of documents' terms. Similarity is a function based on matches between query's and document's terms. Additionally, keyword-based retrieval could be done by filtering metadata fields saved as-is, e.g., those not encoded into vectors. 🔗 Sparse vectors in Qdrant: https://lnkd.in/dWxR5v-n 🔗 Filtering of metadata in Qdrant: https://lnkd.in/dNdDKJiB 𝐇𝐲𝐛𝐫𝐢𝐝 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐞𝐫𝐬 Documents are encoded both into dense and sparse vector representations. Top-by-similarity results are retrieved for each type of vector and then fused. 🔗 Hybrid retrieval in Qdrant: https://lnkd.in/dq5fZX2g 🔗 To dive deeper: https://lnkd.in/ewJRdumD 𝐒𝐩𝐚𝐫𝐬𝐞 𝐍𝐞𝐮𝐫𝐚𝐥 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐞𝐫𝐬 Documents are encoded into sparse vector representations by ML models, which are adapted to produce sparse vectors. These sparse representations can be directly mapped to words in human vocabulary (for example, English). Similarity is a function based on matches between query's and document's words, where an ML model defines these words. 🔗 BM42 in 𝐅𝐚𝐬𝐭𝐄𝐦𝐛𝐞𝐝: https://lnkd.in/e_Tq2SCF 🔗 Splade++ in 𝐅𝐚𝐬𝐭𝐄𝐦𝐛𝐞𝐝: https://lnkd.in/ddnbb79j 𝐌𝐮𝐥𝐭𝐢𝐯𝐞𝐜𝐭𝐨𝐫 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐼𝑡'𝑠 𝑢𝑠𝑢𝑎𝑙𝑙𝑦 𝑢𝑠𝑒𝑑 𝑓𝑜𝑟 𝑟𝑒𝑟𝑎𝑛𝑘𝑖𝑛𝑔 𝑠𝑖𝑛𝑐𝑒 𝑖𝑡'𝑠 𝑐𝑜𝑚𝑝𝑢𝑡𝑎𝑡𝑖𝑜𝑛𝑎𝑙𝑙𝑦 𝑖𝑛𝑡𝑒𝑛𝑠𝑖𝑣𝑒. Documents are encoded into a matrix of dense vectors (vector per term as in #ColBERT or per image patch as in #ColPali) by an ML model. The most similar piece in the document is found for each unique piece in the query, and their similarity score (vector distance) is calculated. The final similarity score is the sum of scores for each query piece. 🔗 Multivector retrieval in Qdrant: https://lnkd.in/dUP93XEn 🔗 ColBERT in 𝐅𝐚𝐬𝐭𝐄𝐦𝐛𝐞𝐝: https://lnkd.in/dJVbhqgG

Your vectors seem to be gone when you check the search results? 😱 Do not worry! They are safe and sound in the collection, but Qdrant does not return them by default. Next time, pass an additional argument to your query and enjoy your embeddings being transferred over the network 🤓 Why don’t we return your vectors by default? There are multiple reasons, but most of our users don’t process their embeddings further. If we sent them, then a lot of unnecessary network traffic would be generated, which in some cases, could have been reflected in your cloud bills 🤫 📄 https://buff.ly/3Y1rnwD

🔍 𝐖𝐡𝐚𝐭 𝐈𝐬 𝐚 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞? From foundational concepts like 𝘸𝘩𝘢𝘵 𝘪𝘴 𝘢 𝘷𝘦𝘤𝘵𝘰𝘳? to advanced features like payload configurations, hybrid search, sharding, multitenancy, quantization, RBAC, and much more, our latest article breaks down everything you need to know about the inner workings of vector databases. Read the full post by Sabrina A.: https://lnkd.in/gEERKC-r

Qdrant is #hiring a Sr. 𝐒𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭. If you have hands-on experience crafting modern Search, Recommender, RAG, etc, solutions by utilizing vector search and enjoy working with customers to help them translate their requirements into the right software architecture, let us know. Job ad https://lnkd.in/eaFzhAf9 Yes, it is mostly written by an LLM. 😴

The new 𝐃𝐢𝐬𝐭𝐚𝐧𝐜𝐞 𝐌𝐚𝐭𝐫𝐢𝐱 𝐀𝐏𝐈 adds the ability to calculate many-to-many distances between stored vectors. This feature is useful for 𝐜𝐥𝐮𝐬𝐭𝐞𝐫𝐢𝐧𝐠, 𝐝𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧𝐚𝐥𝐢𝐭𝐲 𝐫𝐞𝐝𝐮𝐜𝐭𝐢𝐨𝐧, 𝐯𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧, and other data exploration tasks. API docs: https://lnkd.in/dbe743Fe We also extended the Graph Exploration Tool in the Qdrant Web UI, providing a clearer picture of data point relationships. In this example, 900 sample points are retrieved from a mid-journey dataset, with a limit of 5 connections per vector and a tree visualization. Intro on Youtube https://lnkd.in/dn8vBdvA

We welcome Iurie Muradu to Qdrant as a Platform Engineer! Based in Antibes, France, Iurie brings expertise in Kubernetes and platform engineering to our team. Outside of work, he enjoys CrossFit, boxing, and sailing. Welcome aboard! 🎉

🚀 Qdrant engine v1.12 has been released! Check out the change log: https://lnkd.in/dbReM2ct 🎹 Key updates: ➡ 𝐃𝐢𝐬𝐭𝐚𝐧𝐜𝐞 𝐌𝐚𝐭𝐫𝐢𝐱 𝐀𝐏𝐈: Distances calculation between vector pairs. ➡ 𝐅𝐚𝐜𝐞𝐭𝐬 𝐀𝐏𝐈: Dynamic aggregation of unique values. ➡ 𝐓𝐞𝐱𝐭 𝐚𝐧𝐝 𝐆𝐞𝐨 index on disk: Even more memory reduction options. Stay tuned for more details coming soon! Read the release blog post for further insights: https://lnkd.in/dTBMuXfU #Qdrant #Release #VectorDatabase #VectorSearch

𝐐𝐝𝐫𝐚𝐧𝐭 𝟏.𝟏𝟐.𝟎 𝐢𝐬 𝐨𝐮𝐭! 🚀 In this release, we focused on making large-scale data handling not only more efficient but also more insightful. ➡️ 𝐃𝐢𝐬𝐭𝐚𝐧𝐜𝐞 𝐌𝐚𝐭𝐫𝐢𝐱 𝐀𝐏𝐈: Simplifies clustering by calculating distances between vectors in a single request, useful for tasks like grouping similar data points. You can also visualize these results directly in the Web UI’s Graph Exploration Tool. ➡️ 𝐅𝐚𝐜𝐞𝐭 𝐀𝐏𝐈 𝐟𝐨𝐫 𝐌𝐞𝐭𝐚𝐝𝐚𝐭𝐚 𝐂𝐚𝐫𝐝𝐢𝐧𝐚𝐥𝐢𝐭𝐲: Refine searches and identify patterns in your dataset by aggregating and counting values for a specific payload field. ➡️ 𝐆𝐞𝐨 𝐚𝐧𝐝 𝐓𝐞𝐱𝐭 𝐈𝐧𝐝𝐞𝐱 𝐎𝐧-𝐃𝐢𝐬𝐤 𝐒𝐮𝐩𝐩𝐨𝐫𝐭: Reduce memory usage by moving text and geo indices to disk. 🔗 Release Notes: https://buff.ly/3U34sQx 📄 Learn more about the new features: https://buff.ly/3zQNnSS

How to build a scalable Multilingual, Multimodal Vector Search for 100 million TikTok users with Qdrant #VectorDatabase. From choosing the right embedding model (by Jina AI) to optimizing RAM consumption and custom reranking. Described by Mikhail Korotkov, AI Engineer at HypeAuditor. "𝘛𝘳𝘢𝘥𝘪𝘵𝘪𝘰𝘯𝘢𝘭 𝘬𝘦𝘺𝘸𝘰𝘳𝘥 𝘴𝘦𝘢𝘳𝘤𝘩𝘦𝘴 𝘰𝘧𝘵𝘦𝘯 𝘧𝘢𝘪𝘭 𝘸𝘪𝘵𝘩 𝘷𝘪𝘥𝘦𝘰-𝘩𝘦𝘢𝘷𝘺 𝘱𝘭𝘢𝘵𝘧𝘰𝘳𝘮𝘴, 𝘴𝘰 𝘸𝘦 𝘥𝘦𝘷𝘦𝘭𝘰𝘱𝘦𝘥 𝘢 𝒎𝒖𝒍𝒕𝒊𝒎𝒐𝒅𝒂𝒍, 𝒎𝒖𝒍𝒕𝒊𝒍𝒊𝒏𝒈𝒖𝒂𝒍 𝒔𝒆𝒎𝒂𝒏𝒕𝒊𝒄 𝒔𝒆𝒂𝒓𝒄𝒉 𝒆𝒏𝒈𝒊𝒏𝒆 𝘶𝘴𝘪𝘯𝘨 𝘘𝘥𝘳𝘢𝘯𝘵. 𝘛𝘩𝘪𝘴 𝘢𝘭𝘭𝘰𝘸𝘴 𝘶𝘴 𝘵𝘰 𝘴𝘦𝘮𝘢𝘯𝘵𝘪𝘤𝘢𝘭𝘭𝘺 𝘴𝘦𝘢𝘳𝘤𝘩 𝘢𝘯𝘥 𝘧𝘪𝘭𝘵𝘦𝘳 𝘩𝘶𝘯𝘥𝘳𝘦𝘥𝘴 𝘰𝘧 𝘮𝘪𝘭𝘭𝘪𝘰𝘯𝘴 𝘰𝘧 𝘛𝘪𝘬𝘛𝘰𝘬 𝘤𝘳𝘦𝘢𝘵𝘰𝘳𝘴 𝘣𝘢𝘴𝘦𝘥 𝘰𝘯 𝘣𝘰𝘵𝘩 𝘤𝘰𝘯𝘵𝘦𝘯𝘵 𝘢𝘯𝘥 𝘮𝘦𝘵𝘢𝘥𝘢𝘵𝘢." Details on Medium https://lnkd.in/eJpD525r