Timescale

PostgreSQL and pgvector: Now Faster than Pinecone, 75% cheaper, 100% open-source. Introducing pgvectorscale, an open-source PostgreSQL extension that builds on pgvector, enabling greater performance and scalability. Here’s how pgvectorscale helps pgvector outperform specialized vector database like Pinecone: 1️⃣ StreamingDiskANN:  A new vector search index that overcomes limitations of in-memory indexes like HNSW the index on disk, making it more cost-efficient to run and scale as vector workloads grow. Inspired by the DiskANN paper from Microsoft. 2️⃣ Statistical Binary Quantization (SBQ): Developed by researchers at Timescale, this technique improves on standard binary quantization techniques by improving accuracy when using quantization to reduce the space needed for vector storage 3️⃣ Written in Rust, giving the PostgreSQL community to contribute to vector support. 📈The result? On our benchmark of 50 million Cohere embeddings (768 dimensions each), PostgreSQL with pgvector and pgvectorscale achieves 28x lower p95 latency and 16x higher query throughput compared to Pinecone for approximate nearest neighbor queries at 99 % recall, all at 75 % less cost when self-hosted on AWS EC2. We also tested it against Pinecone’s p2 high performance index, see the blog post at the end of this post for full results (spoiler: It’s just as impressive). Pgvectorscale is open-source under the PostgreSQL license and free for you to use on any PostgreSQL database for your AI projects. To get started, see the pgvectorscale github repo: https://lnkd.in/ghXj2e-U Or try it on Timescale Cloud on any new database service. Eager to learn more about pgvectorscale and how it works? Head over to our blog post with all the details: https://lnkd.in/gcMcxrVb

Artificial intelligence has evolved from science fiction to reality, thanks to innovations in generative AI like ChatGPT and Microsoft’s Copilot. To better understand the future of AI, 🐯 Ajay Kulkarni examines its past and offers insights in our latest newsletter. Learn more about the history of AI— its evolution, innovations, impact, and more.👇

🌍 Geo-Spatial Analysis with PostgreSQL and PostGIS: Mapping the Future 🗺️ Discover how to leverage the power of PostgreSQL and the PostGIS extension to perform advanced geo-spatial analysis and mapping. PostGIS is a powerful extension for PostgreSQL that adds support for geographic objects, allowing you to store, query, and analyze spatial data. This combination enables you to build location-based applications, perform spatial analysis, and create interactive maps. Code: sql -- Creating a table with a geographic column CREATE TABLE locations ( id SERIAL PRIMARY KEY, name TEXT, geom GEOMETRY(POINT, 4326) ); -- Inserting a new location INSERT INTO locations (name, geom) VALUES ('New York', ST_GeomFromText('POINT(-74.0060 40.7128)', 4326)); #PostgreSQL #PostGIS #GIS #Geospatial #Spatial

Public datasets provide valuable insights, but combining them with other data often requires a series of steps to normalize the data. 🔁 💡 Data normalization is the process of organizing data to reduce redundancy and improve data integrity. It typically involves dividing a database into smaller, related tables and defining relationships between them, following rules known as normal forms to help ensure consistent and efficient data storage and retrieval. In a recent project, we combined two public datasets— the San Francisco police incident and NOAA weather database, and encountered two challenges: 1️⃣ Different date formats (yyyy-mm-dd vs. mm/dd/yyyy) made joining data difficult. 2️⃣ Gaps in weather data complicated time-series graphs. Read our latest blog to learn how to solve these issues and tackle other common problems: https://lnkd.in/gqpVef4q

Pessoas aprendendo sobre inteligência artificial, um vídeo muito legal em Português sobre o assunto 👏🏼

𝟳𝟬𝟬+ 𝙨𝙩𝙖𝙧𝙨 𝙛𝙤𝙧 𝙥𝙜𝙫𝙚𝙘𝙩𝙤𝙧𝙨𝙘𝙖𝙡𝙚⭐️ 𝙏𝙝𝙧𝙞𝙡𝙡𝙚𝙙 𝙗𝙮 𝙩𝙝𝙚 𝙛𝙚𝙚𝙙𝙗𝙖𝙘𝙠 𝙬𝙚'𝙫𝙚 𝙧𝙚𝙘𝙚𝙞𝙫𝙚𝙙 𝙖𝙣𝙙 𝙚𝙭𝙘𝙞𝙩𝙚𝙙 𝙩𝙤 𝙠𝙚𝙚𝙥 𝙞𝙢𝙥𝙧𝙤𝙫𝙞𝙣𝙜. 𝙋𝙤𝙨𝙩𝙜𝙧𝙚𝙎𝙌𝙇 🚀 𝗜𝗖𝗠𝗬𝗜: Curious about pgvector and want to learn how pgvectorscale can help you? Here's a handy overview: 🐘 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿𝘀𝗰𝗮𝗹𝗲? Pgvectorscale is an open-source PostgreSQL extension that builds on pgvector, enabling greater performance and scalability. By using pgvector and pgvectorscale, developers can build more scalable AI applications, benefiting from higher-performance embedding search and cost-efficient storage. 📈 𝗛𝗼𝘄 𝗱𝗼𝗲𝘀 𝗶𝘁 𝗽𝗲𝗿𝗳𝗼𝗿𝗺? On our benchmark of 50 million Cohere embeddings (768 dimensions each), PostgreSQL with pgvector and pgvectorscale achieves 𝟮𝟴𝘅 𝗹𝗼𝘄𝗲𝗿 𝗽𝟵𝟱 𝗹𝗮𝘁𝗲𝗻𝗰𝘆 and 𝟭𝟲𝘅 𝗵𝗶𝗴𝗵𝗲𝗿 𝗾𝘂𝗲𝗿𝘆 𝘁𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 compared to Pinecone for approximate nearest neighbor queries at 99 % recall, all at 𝟳𝟱% 𝗹𝗲𝘀𝘀 𝗰𝗼𝘀𝘁 when self-hosted on AWS EC2. We also tested it against Pinecone’s p2 high performance index, see the blog post int he comments for full results (spoiler: It’s just as impressive). 🤔 𝗪𝗵𝘆 𝗱𝗶𝗱 𝘄𝗲 𝗯𝘂𝗶𝗹𝗱 𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿𝘀𝗰𝗮𝗹𝗲? Our team at @timescale built pgvectorscale to make PostgreSQL a better database for AI and to challenge the notion that PostgreSQL and pgvector are not performant for vector workloads. ⚙️𝗛𝗼𝘄 𝗱𝗼𝗲𝘀 𝗶𝘁 𝗮𝗰𝗵𝗶𝗲𝘃𝗲 𝘀𝘂𝗰𝗵 𝗴𝗼𝗼𝗱 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲? Pgvectorscale brings specialized data-structures and algorithms for large-scale vector search and storage to PostgreSQL as an extension, including: (1) StreamingDiskANN – a high-performance, cost-efficient vector search index for pgvector data inspired by research at Microsoft, and (2) Statistical Binary Quantization (SBQ), developed by Timescale’s own researchers to improve upon standard binary quantization techniques. 📚𝗟𝗲𝗮𝗿𝗻 𝗺𝗼𝗿𝗲 I've linked the pgvectorscale GitHub and explainer blogs in the comments -- it's a great first step to getting started, whether you already use pgvector or are just curious about vector databases in general. #pgvector #vectordatabase #opensource #postgresql #devtool

Why use PostgreSQL for AI? 🧐 🤖 Check out this article by Simeon Emanuilov, where he introduces pgvectorscale, our new PostgreSQL extension for enhancing AI application performance. 🔥 Building on the pgvector extension, pgvectorscale leverages the Rust programming language and the PGRX framework for improved efficiency. Key features include the StreamingDiskANN index for efficient disk-based vector searches and Statistical Binary Quantization for better storage efficiency. 📈 Not only that, but Simeon ran his own benchmarks to show that pgvectorscale offers lower latency and higher throughput than Pinecone's vector database, making it a cost-effective and high-performance solution for AI applications. 🤝 Shout out to Simeon for the great read! We're thrilled to see the community actively engaging with us and look forward to collaborating further to enhance PostgreSQL for AI applications. Read more about it here: https://lnkd.in/gz52YVZ8

Já pensou em usar IA diretamente com SQL? Isso é possível com PgAI. Uma extensão criada pelo pessoal da Timescale para o postgres. Vem comigo que não é bait. Link do vídeo nos comentários.

While Amazon's RDS excels in managing relational databases like MySQL, PostgreSQL, Oracle, SQL Server, and Db2 in the cloud, it still has some limitations in high-volume or specialized use cases. Learning about its pros and cons can help you determine when to use RDS or consider alternative solutions. 🤷⬇️ Here are some key topics we'll cover in our latest blog: 👉 Amazon RDS services and limitations. 👉 Various RDS alternatives available on AWS. 👉 The database solution best suited for your use case. Check it out: https://lnkd.in/gWsJBwm5

Karbo client, Timescale, released two new open source extensions to challenge the notion that PostgreSQL is not performant for AI workloads. In his recent byline for The New Stack, 🔥 Avthar Sewrathan breaks down why PostgreSQL is the bedrock of the future of data. “As the PostgreSQL for AI ecosystem continues to develop, I hope that even more developers can trade complex, brittle data architectures (juggling multiple databases) for the rock-solid foundation, versatile extensions, and straightforward simplicity of PostgreSQL in their AI data stack,” says Avthar. Read the full article here to discover how Timescale is lowering the barrier for developers adopting and scaling PostgreSQL for AI applications: https://bit.ly/4bGIusw #AIapplications #opensource #database #developers #PostgreSQL