diff --git a/pgml-dashboard/content/blog/announcing-hnsw-support-in-our-sdk.md b/pgml-dashboard/content/blog/announcing-hnsw-support-in-our-sdk.md new file mode 100644 index 000000000..fb64e1fb4 --- /dev/null +++ b/pgml-dashboard/content/blog/announcing-hnsw-support-in-our-sdk.md @@ -0,0 +1,147 @@ +--- +author: Silas Marvin +description: HNSW indexing is the latest upgrade in vector recall performance. In this post we announce our updated SDK that utilizes HNSW indexing to give world class performance in vector search. +image: https://postgresml.org/dashboard/static/images/blog/announcing_hnsw_support.webp +image_alt: HNSW provides a significant improvement in recall speed compared to IVFFlat +--- + +# Announcing HNSW Support in Our SDK + +
+ Author +
+

Silas Marvin

+

September 21, 2023

+
+
+ +PostgresML makes it easy to use machine learning with your database and to scale workloads horizontally in our cloud. Our SDK makes it even easier. + +data is always the best medicine +

HNSW (hierarchical navigable small worlds) is an indexing method that greatly improves vector recall

+ +## Introducing HNSW + +Underneath the hood our SDK utilizes [pgvector](https://github.com/pgvector/pgvector) to store, index, and recall vectors. Up until this point our SDK used IVFFlat indexing to divide vectors into lists, search a subset of those lists, and return the closest vector matches. + +While the IVFFlat indexing method is fast, it is not as fast as HNSW. Thanks to the latest update of [pgvector](https://github.com/pgvector/pgvector) our SDK now utilizes HNSW indexing, creating multi-layer graphs instead of lists and removing the required training step IVFFlat imposed. + +The results are not disappointing. + +## Comparing HNSW and IVFFlat + +In one of our previous posts: [Tuning vector recall while generating query embeddings in the database](/blog/tuning-vector-recall-while-generating-query-embeddings-in-the-database) we were working on a dataset with over 5 million Amazon Movie Reviews, and after embedding the reviews, performed semantic similarity search to get the closest 5 reviews. + +Let's run that query again: + +!!! generic + +!!! code_block time="89.118 ms" + +```postgresql +WITH request AS ( + SELECT pgml.embed( + 'intfloat/e5-large', + 'query: Best 1980''s scifi movie' + )::vector(1024) AS embedding +) + +SELECT + id, + 1 - ( + review_embedding_e5_large <=> ( + SELECT embedding FROM request + ) + ) AS cosine_similarity +FROM pgml.amazon_us_reviews +ORDER BY review_embedding_e5_large <=> (SELECT embedding FROM request) +LIMIT 5; +``` + +!!! + +!!! results + +| review_body | product_title | star_rating | total_votes | cosine_similarity +| ------------------------------------------------- | ------------------------------------------------------------- | ------------- | ----------- | ------------------ | +| best 80s SciFi movie ever | The Adventures of Buckaroo Banzai Across the Eighth Dimension | 5 | 1 | 0.9495371273162286 | +| the best of 80s sci fi horror! | The Blob | 5 | 2 | 0.9097434758143605 | +| Three of the best sci-fi movies of the seventies | Sci-Fi: Triple Feature (BD) [Blu-ray] | 5 | 0 | 0.9008723412875651 | +| best sci fi movie ever | The Day the Earth Stood Still (Special Edition) [Blu-ray] | 5 | 2 | 0.8943620968858654 | +| Great Science Fiction movie | Bloodsport / Timecop (Action Double Feature) [Blu-ray] | 5 | 0 | 0.894282454374093 | + +!!! + +!!! + +This query utilized IVFFlat indexing and queried through over 5 million rows in 89.118ms. Pretty fast! + +Let's drop our IVFFlat index and create an HNSW index. + +!!! generic + +!!! code_block time="10255099.233 ms (02:50:55.099)" + +```postgresql +DROP INDEX index_amazon_us_reviews_on_review_embedding_e5_large; +CREATE INDEX CONCURRENTLY ON pgml.amazon_us_reviews USING hnsw (review_embedding_e5_large vector_cosine_ops); +``` + +!!! + +!!! results + +|CREATE INDEX| +|------------| + +!!! + +!!! + +Now let's try the query again utilizing the new HNSW index we created. + +!!! generic + +!!! code_block time="17.465 ms" + +```postgresql +WITH request AS ( + SELECT pgml.embed( + 'intfloat/e5-large', + 'query: Best 1980''s scifi movie' + )::vector(1024) AS embedding +) + +SELECT + id, + 1 - ( + review_embedding_e5_large <=> ( + SELECT embedding FROM request + ) + ) AS cosine_similarity +FROM pgml.amazon_us_reviews +ORDER BY review_embedding_e5_large <=> (SELECT embedding FROM request) +LIMIT 5; +``` + +!!! + +!!! results + +| review_body | product_title | star_rating | total_votes | cosine_similarity +| --------------------------------- | ------------------------------------------------------------- | ------------- | ----------- | ------------------ | +| best 80s SciFi movie ever | The Adventures of Buckaroo Banzai Across the Eighth Dimension | 5 | 1 | 0.9495371273162286 | +| the best of 80s sci fi horror! | The Blob | 5 | 2 | 0.9097434758143605 | +| One of the Better 80's Sci-Fi | Krull (Special Edition) | 3 | 5 | 0.9093884940741694 | +| Good 1980s movie | Can't Buy Me Love | 4 | 0 | 0.9090294438721961 | +| great 80's movie | How I Got Into College | 5 | 0 | 0.9016508795301296 | + +!!! + +!!! + +Not only are the results better (the `cosine_similarity` is higher overall), but HNSW is over 5x faster, reducing our search and embedding time to 17.465ms. + +This is a massive upgrade to the recall speed utilized by our SDK and greatly improves overall performance. + +For a deeper dive into HNSW checkout [Jonathan Katz's excellent article on HNSW in pgvector](https://jkatz05.com/post/postgres/pgvector-hnsw-performance/). diff --git a/pgml-dashboard/content/blog/tuning-vector-recall-while-generating-query-embeddings-in-the-database.md b/pgml-dashboard/content/blog/tuning-vector-recall-while-generating-query-embeddings-in-the-database.md index f70054f8f..be46ec4bd 100644 --- a/pgml-dashboard/content/blog/tuning-vector-recall-while-generating-query-embeddings-in-the-database.md +++ b/pgml-dashboard/content/blog/tuning-vector-recall-while-generating-query-embeddings-in-the-database.md @@ -144,7 +144,7 @@ SELECT ) ) AS cosine_similarity FROM pgml.amazon_us_reviews -ORDER BY cosine_similarity +ORDER BY review_embedding_e5_large <=> (SELECT embedding FROM request) LIMIT 5; ``` diff --git a/pgml-dashboard/src/api/docs.rs b/pgml-dashboard/src/api/docs.rs index 6245a0d2f..e256adce4 100644 --- a/pgml-dashboard/src/api/docs.rs +++ b/pgml-dashboard/src/api/docs.rs @@ -80,6 +80,8 @@ async fn blog_handler<'a>(path: PathBuf, cluster: &Cluster) -> Result --> <% } %> diff --git a/pgml-dashboard/src/components/navbar_web_app/template.html b/pgml-dashboard/src/components/navbar_web_app/template.html index c716e6d9e..72f691e58 100644 --- a/pgml-dashboard/src/components/navbar_web_app/template.html +++ b/pgml-dashboard/src/components/navbar_web_app/template.html @@ -54,7 +54,7 @@
<% if !account_management_nav.links.is_empty() { %> @@ -82,7 +82,7 @@ <% if !standalone_dashboard { %> diff --git a/pgml-dashboard/static/images/blog/announcing_hnsw_support.webp b/pgml-dashboard/static/images/blog/announcing_hnsw_support.webp new file mode 100644 index 000000000..248a08733 Binary files /dev/null and b/pgml-dashboard/static/images/blog/announcing_hnsw_support.webp differ diff --git a/pgml-dashboard/templates/layout/nav/top.html b/pgml-dashboard/templates/layout/nav/top.html index affb60dc2..1d0c47d24 100644 --- a/pgml-dashboard/templates/layout/nav/top.html +++ b/pgml-dashboard/templates/layout/nav/top.html @@ -23,7 +23,7 @@ --> <% } %> diff --git a/pgml-dashboard/templates/layout/nav/top_web_app.html b/pgml-dashboard/templates/layout/nav/top_web_app.html index e9d62aab6..2035ad186 100644 --- a/pgml-dashboard/templates/layout/nav/top_web_app.html +++ b/pgml-dashboard/templates/layout/nav/top_web_app.html @@ -53,7 +53,7 @@
<% if !account_management_nav.links.is_empty() { %> @@ -91,7 +91,7 @@ <% if !standalone_dashboard { %>