Welcome

How to cite this book

If you use this book in your work, please cite it as:

APA: Chaudhri, A. (2026). Seven Ways to Do Vector Search in Python. Self-published. https://seven-vector-search.github.io/

BibTeX:

@online{chaudhri2026vectorsearch,
  author  = {Chaudhri, Akmal},
  title   = {Seven Ways to Do Vector Search in Python},
  year    = {2026},
  url     = {https://seven-vector-search.github.io/},
  urldate = {2026-07-31}
}

Cover

License

Copyright

Copyright © 2026 Akmal Chaudhri. All rights reserved.

Publication Information

First published: July 2026

The latest version of this book, together with updates, errata and additional resources, is available at:

seven-vector-search.github.io

Book License

Seven Ways to Do Vector Search in Python is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

You are free to copy and redistribute this book in any medium or format under the following conditions:

• Attribution - You must give appropriate credit, provide a link to the license and indicate if changes were made.
• NonCommercial - You may not use the material for commercial purposes.
• NoDerivatives - If you remix, transform or build upon the material, you may not distribute the modified material.

The full license text is available at:

creativecommons.org/licenses/by-nc-nd/4.0

Code License

Unless otherwise stated, all code samples, notebooks, scripts and source files accompanying this book are licensed under the Apache License 2.0.

You are free to use, modify and redistribute this code, including for commercial purposes, subject to the terms of the Apache License 2.0.

The full license text is available at:

apache.org/licenses/LICENSE-2.0

This distinction means that the book’s written content is protected under the Creative Commons license, while the accompanying code remains freely available for use, modification and integration into your own projects.

Trademarks

Product names, company names and logos mentioned in this book may be trademarks or registered trademarks of their respective owners.

Their inclusion is for identification and educational purposes only and does not imply any affiliation with, sponsorship by or endorsement from the respective trademark holders. All trademarks remain the property of their respective owners.

Disclaimer

The information in this book, including all code samples, scripts and notebooks, is provided “as is” without warranty of any kind, express or implied.

The author makes no representations or warranties regarding the accuracy, completeness, reliability or suitability of the information contained herein for any particular purpose.

Examples are provided solely to illustrate technical concepts, patterns and software architectures.

Readers are responsible for independently validating all code, configurations and recommendations before using them in production environments.

To the fullest extent permitted by law, the author shall not be liable for any direct, indirect, incidental, special, consequential or other damages arising from the use of or inability to use, the information, code or techniques described in this book.

About the Author

Akmal Chaudhri is a technical leader, educator and author with extensive experience in databases, AI and developer relations. He specializes in technical writing, developer education and community building, helping engineers and organizations understand and adopt complex technologies through clear, practical and engaging content. He is a frequent international speaker, a published author and a contributor to industry discussions on data platforms, AI and software development.

Today, Akmal works in developer education at Neo4j, where he focuses on technical content, workshops and community initiatives. While his professional role has evolved, this book represents an independent exploration of various libraries.

Based in the United Kingdom, Akmal continues to work at the intersection of databases, AI and developer tooling, helping developers build modern data-driven applications.

For book updates, code samples and additional resources, visit the Book website.

To connect professionally or follow his latest work, visit LinkedIn.

Introduction

Vector search is one of those ideas that sounds complicated until you see it working. You take a piece of text, convert it to a list of numbers and then find other pieces of text whose numbers are similar. That’s it. The Python ecosystem has produced a remarkable variety of tools for doing this efficiently and choosing between them isn’t always obvious.

This book is a practical guide to seven of those tools. We’ll use the same dataset throughout, ask the same questions of each library and measure the results consistently so you can make informed choices rather than guessing. By the end you’ll know not just how to use each library but why you would reach for one over another given your specific constraints.

What this book is about

Each chapter introduces one library, builds an index over a 500,000-record crime incident dataset, runs similarity searches, evaluates recall and latency against a consistent ground truth and discusses honestly what the library is good at and where it falls short. The chapters are designed to be read in order but work as standalone references once you know the context.

Two details are worth knowing about. Scikit-learn runs on a 50,000-record subset, with a ground truth restricted to that subset, and Day 1 measures FAISS’s recall from a single query, while every other library is averaged over 1,000. The last two chapters, Chroma and LanceDB, are embedded vector databases rather than pure libraries. We include them because they run inside your Python process.

This isn’t a survey of vector search theory. We won’t derive the mathematics of HNSW graphs or explain how product quantization works from first principles. There are excellent papers and textbooks for that. What we’ll offer here is the practitioner’s perspective: what does this library actually do when you run it, what are the gotchas and when should you use something else?

How we measure

Every chapter uses the same 384-dimension embeddings, computed once from the MO_TEXT field with all-MiniLM-L6-v2 and normalized to unit length, so cosine and L2 distance rank neighbors the same way. Every chapter also measures the same two things. Recall@10 is the share of the true ten nearest neighbors that an index returns, averaged over 1,000 sampled queries. The true neighbors come from an exact FAISS search in Day 1, which excludes the query itself. Most indexes return the query as their own first result, so each later chapter asks for 11 results and removes the query’s own ID before scoring. Without that step recall can’t exceed 0.9. Latency is the average time for one query, in milliseconds. These timings come from one machine, so treat them as a guide to relative order rather than precise figures.

Who this book is for

This book is for Python developers who need vector search to work and want to understand their options before committing to one. You might be building a RAG pipeline and wondering whether you need a database or whether a library will do. You might be hitting performance limits with your current approach and wondering if there’s a faster option. You might simply be curious about what the ecosystem looks like right now.

We’ll assume comfort with Python and pandas. We’ll use Jupyter notebooks throughout.

The companion book

This book sits alongside Seven Vector Databases in Seven Days, which covers vector databases: PostgreSQL with pgvector, which we run locally, and six cloud services, MongoDB Atlas, Pinecone, Weaviate, Neo4j, Snowflake and Databricks. The two books address different questions. The databases book asks: what infrastructure should I run? This book asks: what Python library should I call?

If your dataset fits in memory and you want to stay in process, this book is your starting point. If you need a persistent, queryable service with replication and access control, the databases book is where to look next. Many production systems use both: a library for rapid prototyping and offline analysis, a database for the serving layer.

Nordvik

Every chapter in this book uses the same dataset: 500,000 synthetic crime incident reports set in Nordvik, a fictional city on Bouvet Island. Bouvet Island is a Norwegian dependency in the South Atlantic and holds the distinction of being one of the most remote uninhabited islands in the world. Its coordinates are real and spatially coherent, so the incident map in the dataset notebook is geographically meaningful, but nothing we’ll generate implicates any real street or person.

The dataset was generated using a template engine rather than a language model. Each record has a structured set of fields - crime type and subtype, neighborhood, property type, coordinates, timestamp, suspect description, suspect vehicle, victim details, entry method, weapon and property stolen - and a free-text MO_TEXT field that reads like an officer’s incident report and is assembled from them using templates. We chose this approach because it’s fast, reproducible and requires no external API. The trade-off is that the MO_TEXT has more template structure than real police data would: each crime type has ten narrative templates, which affects how the embedding model clusters results. We’ll note this where it matters.

Some fields are also sampled independently of one another. Weapons, for example, are assigned to 40% of incidents whatever the crime type, so a burglary or a drug offence can carry a firearm. That’s unrealistic, but it means a weapon filter cuts across every crime type.

How to use this book

The notebooks are on GitHub alongside the book. To follow along you’ll need Python 3.12, Jupyter and a machine with at least 16GB of RAM. Each chapter installs its own dependencies, pinned to exact versions, via a pip cell at the top.

Run generate_dataset.ipynb first to create the Nordvik dataset, then generate_embeddings.ipynb to compute and save the embeddings. Then run the Day 1 notebook, which computes and saves the ground truth that every later chapter scores against. You only need to run these three once.

If you want to skip straight to a specific library, each chapter notebook is self-contained once the dataset, embeddings and Day 1 ground truth files exist.

The Dataset

Before we can evaluate any vector search library we need something to search. This chapter covers how we built the Nordvik crime dataset and why we made the choices we did.

Why synthetic data

Real crime data exists and some of it’s publicly available. We looked at several sources including British Transport Police (BTP) incident records and Dallas Police Department reports. The BTP data lacked free-text narrative fields, which are essential for meaningful semantic search. The Dallas data was richer but comes with licensing friction that would make it awkward for readers to reproduce the experiments.

Generating synthetic data solves both problems. We control the schema, the volume and the distribution. Readers can reproduce everything without requesting access to anything. And we can tune the data to make the vector search experiments as informative as possible.

The trade-off is realism. Synthetic narratives generated from templates have more structural regularity than real officer-written reports. This affects how the embedding model clusters results - we see tighter clusters around template patterns than we would with genuine linguistic variety. We note this in the chapters where it matters most.

Bouvet Island

The coordinates in the dataset are real. We use Bouvet Island as the geographic anchor for Nordvik, our fictional city. Bouvet Island is a Norwegian dependency in the South Atlantic, roughly 49 square kilometers in area and holds the distinction of being one of the most remote uninhabited islands in the world. If you drop any coordinate from our dataset into a mapping tool it will resolve to a glacier, not a street.

We define 15 fictional neighborhoods across the island’s footprint and assign each a centroid coordinate. Individual incidents are placed within 300 meters of their neighborhood centroid using a small random offset. The result is a spatial distribution that looks like a real city and supports meaningful geographic queries and hotspot analysis.

The MO_TEXT field

The central field for vector search is MO_TEXT - a free-text officer narrative describing how each crime was committed. A typical entry looks like this:

At approx 1632hrs, Multiple offenders described as medium build, tall, wearing face covering gained entry to Residence - Single Family in Hartley Cross. Property taken: handbag, tablet. No FO identified.

We generate these using a template engine with 30 templates per crime type, each drawing from randomized structured fields. The templates vary in structure - some lead with a timestamp, some with the location, some with the suspect description, some with police shorthand. We also vary the phrasing of individual components: Property taken: in one record becomes Stolen: or Items removed include in another.

The result is varied enough for the embedding model to produce meaningful similarity scores but structured enough that readers can see immediately whether a search result makes sense.

Crime types and distribution

The dataset contains six crime types weighted to approximate a realistic urban distribution:

Each crime type has its own set of subtypes, entry methods, property types and property stolen values. Burglary incidents are constrained to residential or commercial properties depending on subtype. Vehicle crime incidents use vehicle-specific entry methods. The structured fields are internally consistent even though the data are synthetic.

Generating the dataset

The generation notebook generate_dataset.ipynb produces two output files:

• nordvik_crimes.csv - the full dataset in CSV format
• nordvik_crimes.parquet - the same data in Parquet format for faster loading

At 500,000 records the generation runs in seconds. There is no LLM involved, no API call and no external dependency beyond pandas. The random seed is fixed at 42 throughout so results are fully reproducible.

Computing embeddings

Once the dataset exists we run generate_embeddings.ipynb to embed every MO_TEXT record using all-MiniLM-L6-v2 from the sentence-transformers library. This model produces 384-dimensional embeddings and is a standard baseline for semantic search benchmarks, which makes it the right choice for a book that compares libraries rather than models.

Embedding 500,000 records at batch size 256 takes roughly 8 to 9 minutes on a MacBook Air M4. The output is saved as:

• nordvik_embeddings.npy - float32 array of shape (500000, 384), 732MB
• nordvik_ids.txt - IncidentID strings in matching row order

We normalize embeddings to unit length before saving. This means L2 distance and cosine similarity are equivalent throughout the book and we can compare results across libraries that use different default metrics without worrying about systematic differences.

Ground truth

Every chapter measures recall against a consistent ground truth computed in Day 1 using FAISS’s exact IndexFlatL2. Rather than computing ground truth for all 500,000 records - which would be expensive - we sample 1,000 queries at random and store their exact top-10 neighbors. This ground truth sample is saved as:

• nordvik_ground_truth.npy - shape (1000, 10), exact neighbor indices
• nordvik_ground_truth_indices.npy - the 1,000 sampled query positions

Every chapter loads these files and uses them to compute recall@10. One caveat applies throughout the book: because the ground truth was computed using L2 distance and most libraries use cosine distance, floating point differences in implementation produce slightly different result orderings at the margin. This causes recall to appear capped around 0.89 to 0.90 even for libraries that should theoretically achieve higher. We’ll explain this in each chapter where it appears.

A note on the sanity check

The embeddings notebook includes a sanity check that compares cosine similarity between two burglary incidents against cosine similarity between a burglary and an assault. The result is counterintuitive: the two burglaries score 0.41 while the burglary and assault score 0.49.

This happens because the two burglary incidents use very different templates and different neighborhoods. The assault happens to share structural phrasing with the first burglary. It’s a reminder that embedding similarity reflects linguistic similarity, not categorical similarity. A search for “incidents like this burglary” will find other incidents that are described in similar language - which may or may not be the same crime type. We’ll return to this observation in Day 1.

Day 1: FAISS

FAISS (Facebook AI Similarity Search) is Meta’s open-source library for efficient similarity search and clustering of dense vectors. It’s the natural starting point for this book because it exposes the full index selection journey more clearly than any other library. Rather than hiding the tradeoff between speed and accuracy behind a single API call, FAISS asks you to choose an index type explicitly. That choice - flat, IVF or HNSW - is the central lesson of this chapter.

When to reach for FAISS

FAISS is the right choice when:

• You need exact or approximate nearest neighbor search in Python with minimal dependencies
• Your dataset fits in RAM and you want fast, well-benchmarked performance
• You want control over the index type and its parameters
• You’re prototyping and want a library that scales from 10k to 100M vectors without changing your API calls

The three index types

FAISS offers many index types but three cover the vast majority of use cases.

IndexFlatL2 stores vectors as-is and computes exact L2 distances against every vector in the index at query time. There’s no training step, no approximation and no parameters to tune. It’s the ground truth against which every other index is measured. The cost is linear scaling - query time grows proportionally with the number of vectors.

IndexIVFFlat partitions the vector space into Voronoi cells using k-means clustering. At query time it searches only the nearest cells rather than the entire index. Two parameters control the tradeoff: nlist (the number of cells) and nprobe (the number of cells searched at query time). Higher nprobe improves recall at the cost of speed. Unlike IndexFlatL2, an IVF index must be trained before adding vectors.

IndexHNSWFlat builds a multi-layer graph of vectors where each node connects to its nearest neighbors. Queries traverse the graph from the top layer down, narrowing the search at each level. This produces excellent recall at low latency with no training step required. Two parameters matter: M, the number of connections per node, which is fixed when the index is built, and efSearch, the size of the candidate list explored at query time. Higher efSearch trades speed for recall.

What we found

We ran all three index types against the full 500,000-record Nordvik dataset using a single burglary incident as the query. The results were clear, though every recall figure below comes from that one query.

The flat index is exact but slow. At 500,000 records it takes around 16ms per query averaged over 50 runs. That’s fast enough for batch processing but too slow for interactive applications at scale.

The IVF index showed the most interesting behavior. At nprobe=1 recall dropped to 0.40 - the only setting in this chapter that genuinely loses results. This is the approximation tradeoff in practice: searching just one Voronoi cell missed 60% of the true nearest neighbors. Recall recovered to 1.0 by nprobe=5, which took 0.16ms. At nprobe=10 recall was still 1.0 at 0.29ms - roughly 54 times faster than the flat index with no accuracy loss on this query. Latency at nprobe=1 (0.24ms) measured slower than at nprobe=5, which is timing noise on a single query rather than a real effect. Beyond nprobe=10, extra probing only added cost: latency climbed to 1.21ms at nprobe=50 with no gain in recall.

The HNSW index was the standout performer. At efSearch=64 it achieved recall 1.0 at 0.06ms - more than 250 times faster than the flat index. No training required, no recall degradation on this query, just a larger memory footprint than IVF. Recall was 1.0 at every efSearch value we tried, from 16 to 256, so on this dataset the setting mainly moves latency. The fastest setting was efSearch=32 at 0.04ms, but the differences at this scale are tiny and partly noise.

The scaling chart tells the most important story. Flat index latency grows roughly linearly, from 0.23ms at 10,000 records to 14.43ms at 500,000 (this separate scaling run differs slightly from the 15.75ms measured above because of timing variance). IVF and HNSW grow far more slowly: IVF goes from 0.03ms to 0.46ms and HNSW from 0.03ms to 0.08ms over the same range. We didn’t measure beyond 500,000 records, but extending the flat index’s linear trend to 10 million would put it near 290ms per query, while the approximate indexes would be expected to stay far lower.

A note on the query results

The ten results for our sample burglary query include the query incident itself at rank one with a distance of 0, so there are really nine neighbors. Nine of the ten are labeled Burglary and the last is labeled Other. All ten share the same narrative pattern: offender descriptions, entry to a residence and property taken. All three indexes returned the same ten incidents in the same order. This is the embedding model doing exactly what it was designed to do: finding linguistic similarity. In a dataset generated from templates, linguistic similarity and template similarity are almost the same thing.

This is worth keeping in mind throughout the book. Embedding-based search finds incidents described in similar language. In production with officer-written narratives - which have far more linguistic variety - the clusters would be semantically richer. Our synthetic data are honest about this limitation.

What FAISS does not do

FAISS is a library, not a service. It has no built-in persistence - you must serialize indexes yourself using faiss.write_index and faiss.read_index. There’s no server and no authentication. It also has no native metadata filtering of the kind a database offers. You can restrict a search to a set of IDs, but if you need to filter by crime type or neighborhood before running vector search, you need to build that logic yourself or move to Chroma (Day 6) or LanceDB (Day 7).

The IVF index requires training data representative of your full dataset. If your data distribution shifts over time the index will degrade and need retraining. HNSW doesn’t require training but typically consumes significantly more memory than IVF at the same dataset size.

When to look elsewhere

FAISS is the wrong choice if:

• You need metadata filtering alongside vector search
• You need a persistent queryable store - look at Chroma (Day 6)
• Your dataset doesn’t fit in RAM - look at LanceDB (Day 7)
• You want automatic parameter tuning - other libraries handle this more gracefully

Day 2: Voyager

Voyager is Spotify’s open-source approximate nearest neighbor search library, built and used in production to power music and podcast recommendations across hundreds of millions of users. It uses the HNSW algorithm - the same one we saw in Day 1’s IndexHNSWFlat - but wraps it in a cleaner API with better defaults and built-in persistence.

Spotify describes Voyager as their answer to the limitations they found with Annoy, their original recommendation library. The comparison with Annoy would have been a natural chapter pairing, but Annoy proved to be broken on Apple Silicon - it consistently returned only one neighbor regardless of K, which we confirmed with a minimal test on 1,000 random vectors. We dropped it from the book. Voyager, by contrast, installed and ran without issue.

When to reach for Voyager

Voyager is the right choice when:

• You want HNSW search with a simpler API than FAISS and sensible defaults out of the box
• You need built-in index persistence without writing serialization code yourself
• You’re building a Python or Java application and want index compatibility between both languages
• You want a library that is battle-tested in production at massive scale

What we found

We built two Voyager indexes on the 500,000-record Nordvik dataset: one with Spotify’s default of M=12 and one with M=32 to match the Day 1 FAISS configuration. Both were scored against the same 1,000 sampled queries and ground truth we used in Day 1, with recall and latency averaged over those queries.

M=12 reached a recall of 0.946 at ef=11 and climbed slowly to 0.980 at ef=200. Most of that gain arrived by ef=20 (0.964), so raising ef helps but with diminishing returns. M=32 started higher, at 0.990 at ef=11, and reached 0.997 at ef=200. The denser graph is worth about two to four percentage points of recall, at the price of a longer build: 51 seconds against 34 for M=12.

In Day 1, FAISS HNSW with M=32 achieved recall 1.0, so Voyager at M=32 lands very close. That Day 1 figure came from a single query though, so it isn’t strictly comparable with these 1,000-query averages. Latency stayed between roughly 0.06 and 0.46ms across every setting. Differences between neighboring rows, such as M=12 at ef=11 being slower than at ef=20, are mostly timing noise.

One detail is worth knowing if you run your own evaluation. Voyager returns the query itself as its top result, while the Day 1 ground truth excludes it. If you score Voyager’s top ten directly against that ground truth, each result set holds the query plus only nine true neighbors, so recall can never exceed 0.9. Our first attempt did exactly that and both indexes stalled near 0.89 whatever M and ef we used. The fix is to ask for 11 results and drop the query before scoring. Because we ask for 11 results, the smallest usable ef is 11.

Persistence

One of Voyager’s practical advantages over FAISS is built-in index persistence. Where FAISS requires faiss.write_index and careful file management, Voyager handles this with a single index.save() call. The index file for 500,000 vectors at M=12 came out at 788MB. Loading back from disk with Index.load() is equally straightforward and returned identical results for our sample query.

Dynamic insertion

Voyager supports adding new vectors to an existing index without rebuilding it. We demonstrated this by adding 1,000 new vectors to the built index in about 0.1 seconds. They were searchable straight away with no rebuild: when we queried with each new vector, about 98% were found in their own top ten. That’s a meaningful advantage in production systems where new data arrive continuously.

What Voyager does not do

Voyager is an in-memory library. The entire index must fit in RAM. At 788MB for 500,000 vectors at M=12, extrapolating linearly would make this a constraint at tens of millions of records on typical server hardware (we didn’t measure beyond 500,000).

Like FAISS, Voyager has no built-in metadata filtering. If you need to constrain search results by crime type or neighborhood, you must filter the results after retrieval or move to Chroma (Day 6) or LanceDB (Day 7).

When to look elsewhere

Voyager is the wrong choice if:

• You need exact search - Voyager is approximate only
• You need metadata filtering - look at Chroma (Day 6) or LanceDB (Day 7)
• Your dataset doesn’t fit in RAM - look at LanceDB (Day 7)
• You want a purely static index with the smallest possible memory footprint and don’t need dynamic insertion

Day 3: Scikit-learn NearestNeighbors

Scikit-learn’s NearestNeighbors is the vector search library you already have. If you have scikit-learn installed - and most Python data science and machine learning projects do - you have a capable nearest neighbor search implementation ready to use without any additional dependencies.

Unlike Voyager and FAISS’s HNSW index, which use graph-based search, scikit-learn offers three distinct algorithms: brute force, ball tree and kd-tree. Each has different characteristics and the library selects the most appropriate one automatically based on your data if you ask it to. This makes scikit-learn a useful starting point for understanding what’s actually happening inside a nearest neighbor search before reaching for a specialized library.

When to reach for scikit-learn

Scikit-learn NearestNeighbors is the right choice when:

• You’re already using scikit-learn and want to avoid adding dependencies
• Your dataset is small enough that brute force is fast enough - typically under 50,000 vectors at 384 dimensions
• You want exact results rather than approximate and FAISS feels like overkill
• You’re teaching or learning nearest neighbor search and want to experiment with different algorithms side by side

A practical note before we start

On Apple Silicon you may see divide by zero and overflow warnings from the underlying matrix multiplication. In our runs they came from numpy and appeared even with the embeddings cast to float64. The notebook suppresses them and checks that the similarities are finite and that each query matches itself. Note that float64 doubles the data each search scans, so scikit-learn’s latencies aren’t strictly like-for-like with the float32 libraries in Days 1 and 2.

What we found

We evaluated all three algorithms on a 50,000-record subset of the Nordvik dataset. We didn’t run the full 500,000 records: extrapolating the 100,000-record result linearly suggests about 150ms per query. Recall is scored against an exact ground truth computed on the same 50,000 records, using the 92 sampled queries from Day 1 that fall inside them.

The results confirmed what the theory predicts. Ball tree and kd-tree offered no meaningful advantage over brute force at 384 dimensions. All three algorithms took roughly 22 to 26ms per query - essentially identical. This is the curse of dimensionality in action: tree-based algorithms that are fast in low dimensions (under 20) degrade toward O(n) query time as dimensionality increases. At 384 dimensions they are just a more complicated way to do the same linear scan.

All three algorithms scored a recall of 1.000, as exact search should. That’s the meaningful recall result here: scikit-learn never misses a neighbor, it just takes a long time to find them. Two details matter if you score a subset yourself. The Day 1 ground truth covers all 500,000 records, and only about 10% of its neighbors fall inside our subset, so scoring against it caps recall near that share. And each index returns the query itself as its top result, so we ask for 11 neighbors and drop the query before scoring.

The scaling numbers are the most telling result. Brute force latency is flat at around 13.5ms from 10,000 to 25,000 records, then roughly doubles to 26ms at 50,000 and reaches 31ms at 100,000. We haven’t isolated why the start is flat. A fixed per-query cost is one possible explanation. Meanwhile FAISS HNSW at 500,000 records takes 0.06ms, which is roughly 400 times faster than scikit-learn at 50,000 records and a tenth of the data.

What scikit-learn does not do

Scikit-learn’s NearestNeighbors has no dedicated index persistence (you can pickle a fitted estimator, but there’s no save and load designed for neighbor indexes), no approximate search and no GPU support. Its brute force implementation is well-optimized with multi-core support via n_jobs=-1, but O(n) scaling makes it impractical above roughly 50,000 vectors at 384 dimensions for latency-sensitive applications.

There’s also no built-in metadata filtering, no document storage and no path from scikit-learn to a production serving layer. It’s a research and development tool, not a deployment target.

The conclusion

Scikit-learn NearestNeighbors is the right answer to a specific question: “I need nearest neighbor search, I already have scikit-learn and my dataset is small.” For anything larger or more demanding, the libraries in the remaining chapters are the better choice. The value of this chapter is not in the performance numbers - which are unimpressive - but in seeing clearly where scikit-learn’s limits are and why they exist.

When to look elsewhere

Scikit-learn is the wrong choice if:

• Your dataset has more than 50,000 vectors at high dimensionality - latency will be too high for interactive use
• You need approximate search to trade recall for speed - scikit-learn only does exact search
• You need index persistence - scikit-learn has no dedicated save and load for neighbor indexes
• You need the lowest possible latency - every other library in this book is faster at scale

Day 4: PyNNDescent

PyNNDescent is a pure Python implementation of the Nearest Neighbor Descent algorithm, developed by Leland McInnes at the Tutte Institute for Mathematics and Computing. If you’ve used UMAP - one of the most widely used dimensionality reduction libraries in data science - you’ve already used PyNNDescent. It’s the nearest neighbor engine that powers UMAP under the hood.

Unlike Voyager and FAISS’s HNSW index, which build HNSW graphs, PyNNDescent builds a k-neighbor graph by iterative refinement. It starts from a random graph and repeatedly improves it by checking whether neighbors of neighbors are better neighbors. This descent process converges to a high-quality approximate graph without requiring the careful parameter tuning that HNSW demands.

When to reach for PyNNDescent

PyNNDescent is the right choice when:

• You want high recall approximate nearest neighbor search with minimal parameter tuning
• You’re already using UMAP and want consistency between your dimensionality reduction and search pipelines
• You need a pure Python library with no C++ compilation requirements
• You want scikit-learn compatibility via PyNNDescentTransformer as a drop-in replacement

What we found

We built PyNNDescent indexes on the full 500,000-record Nordvik dataset and scored them against the same 1,000 sampled queries and Day 1 ground truth we used for Voyager.

PyNNDescent landed in the same territory as the HNSW libraries. With n_neighbors=10 recall was 0.958. At n_neighbors=20 it reached 0.991, and at the default of 30 it was 0.997, rising to 0.999 at n_neighbors=50. That’s close to Voyager at M=32 (0.997 at ef=200) and above Voyager at M=12 (0.980). Latency was between 0.03 and 0.06ms for most settings, in the same range as FAISS HNSW (0.06ms from a single query in Day 1). The figures vary between runs, though. The same n_neighbors=30 index measured 0.033, 0.052 and 0.120ms in three passes over the same queries, so we wouldn’t read anything into small differences.

Build time grew with n_neighbors, from 5.8 seconds at n_neighbors=10 to 29 seconds at n_neighbors=50, with 14.6 seconds at the default of 30. That’s less than half of Voyager’s 34 seconds at M=12, though the two algorithms and their parameters aren’t equivalent.

The first query is slow. Before it can answer, PyNNDescent prepares its search structures (pruning the graph and building a search tree) and compiles its search function with Numba. That took about 8 seconds in our run. We run a warmup query before timing, and if cold-start latency matters in your deployment you should factor it in.

Recall also responds to a query-time setting, epsilon, which trades accuracy for search cost. The default of 0.1 gave 0.997 on the n_neighbors=30 index, while 0.0 gave 0.985 and 0.3 gave 0.9995.

One detail is worth knowing if you run your own evaluation. As with Voyager, the index returns the query itself as its top result, while the Day 1 ground truth excludes it. We ask for 11 results and drop the query before scoring. Without that step recall can’t exceed 0.9, and our first attempt stalled at 0.900 at n_neighbors=50.

The FAISS recall comes from a single query. The Voyager and PyNNDescent figures average 1,000 queries.

Query result quality

The query results matched what FAISS and Voyager returned. PyNNDescent’s top ten for our sample burglary query are the same ten incidents, in the same order and at the same distances, as Voyager’s in Day 2. Rank 1 is the query incident itself, so there are really nine neighbors: eight burglaries and one Other at rank 10, at distance 0.1615. This isn’t an error. It reflects a genuine linguistic similarity between certain crime descriptions that crosses the categorical boundary. A forced entry incident described in similar language to a burglary will score high cosine similarity regardless of how it was categorized.

The scikit-learn integration

PyNNDescent provides a PyNNDescentTransformer that’s a drop-in replacement for scikit-learn’s KNeighborsTransformer. This means we can swap PyNNDescent into scikit-learn pipelines that use nearest neighbors without changing our pipeline code (the PyNNDescent documentation shows it feeding Isomap, for example). We demonstrated the transformer on a 5,000-record subset, producing a sparse 5,000 by 5,000 neighbor graph in 0.6 seconds.

What PyNNDescent does not do

PyNNDescent can add data to an existing index through its update method, but it isn’t cheap. Adding 1,000 vectors to the 500,000-record index took 9.4 seconds, against 0.1 seconds for Voyager in Day 2, and all 1,000 new vectors were found in their own top ten afterward. That’s quicker than rebuilding (about 15 seconds to build plus about 8 for the first-query preparation at n_neighbors=30), but it’s not the cheap insert Voyager offers.

The index can be pickled to disk with Python’s standard pickle module, which is simpler than FAISS’s write_index. The file is large, though: about 1.8GB for our 500,000-record index, against 788MB for Voyager. Loading it back took 1.5 seconds, and the first query on the loaded index was immediate in the same session. We haven’t tested loading it in a fresh process.

Build time at 500,000 vectors ranged from 6 to 29 seconds depending on n_neighbors. We haven’t compared it with FAISS here. For datasets that grow continuously, the cost of each update needs to be factored into operational planning.

When to look elsewhere

PyNNDescent is the wrong choice if:

• You can’t tolerate a cold start - the first query on a freshly built index took about 8 seconds
• Your dataset changes frequently - update works but took about 9 seconds per batch at this scale, and Voyager inserts are far cheaper
• You need metadata filtering - look at Chroma (Day 6) or LanceDB (Day 7)
• You need a language-agnostic library - PyNNDescent is Python only

Day 5: USearch

USearch is a fast, compact vector search library developed by Unum Cloud. Like FAISS, it implements the HNSW algorithm - but it’s designed around a different set of priorities: a minimal binary footprint, SIMD-optimized distance computations, support for user-defined metrics and compatibility across a wide range of platforms including iOS, Android and WebAssembly.

Where FAISS is a research-oriented library with a large C++ codebase, USearch is engineered for deployment. The core C++ implementation is a single header file. Its README reports exact brute-force search up to 20 times faster than FAISS’s IndexFlatL2 in a Google Colab test, thanks to SIMD-optimized similarity kernels. We’ll put that claim to the test.

When to reach for USearch

USearch is the right choice when:

• You need the smallest possible binary footprint - important for mobile, edge or embedded deployments
• You want both approximate and exact search from one library
• You need custom distance metrics - USearch supports user-defined Python functions via Numba
• You want a library that works identically across Python, JavaScript, Rust, Java and C without a separate server

What we found

We built a USearch HNSW index with connectivity=32 and expansion_add=200 on the full 500,000-record dataset and scored it against the same 1,000 sampled queries and Day 1 ground truth we used for the other libraries. Build time was about 80 seconds, slower than Voyager (34 to 51 seconds) and PyNNDescent (6 to 29 seconds) in our runs. The saved index came out at 864MB (the library reported 1.27GB in memory), against 788MB for Voyager at M=12. These settings aren’t equivalent, so treat the sizes as a rough guide.

HNSW recall was 0.9945 at expansion_search=16 and 0.9996 at 64, with little to gain beyond that (0.9997 at 256). Latency roughly doubled with each doubling of expansion_search, from 0.16ms to 0.91ms, and repeated almost exactly between two runs. That’s higher recall than Voyager or PyNNDescent reached, at higher latency than PyNNDescent (0.03 to 0.06ms) and FAISS HNSW (0.06ms from a single query), and in the same range as Voyager (0.06 to 0.46ms).

The exact search result

The most significant finding in this chapter is what happened when we enabled exact=True. USearch’s README reports exact brute-force search up to 20 times faster than FAISS’s IndexFlatL2 in a Google Colab test. On our MacBook Air M4 a single exact query took 88ms, about 5.6 times the 15.75ms we measured for FAISS flat in Day 1.

The gap closes substantially when queries are batched. Sending all 1,000 queries in one call, USearch exact search took 14.5ms per query, close to FAISS flat’s single-query figure. That suggests much of the single-query gap is a threading effect, since a batch can spread queries across cores. We didn’t run FAISS in batch mode, so we can’t say which is faster there. The 20x claim didn’t reproduce in either mode, and we haven’t established why. The comparison also isn’t like-for-like, because USearch computed cosine distance while FAISS computed L2. Exact search scored a recall of 0.9999 against the L2 ground truth, so the metric difference itself is negligible.

This is a useful lesson about benchmarks in general: headline numbers are measured on specific hardware and workloads. Always benchmark on your target platform and with your own query pattern before committing to a library.

Custom metrics

One of USearch’s most distinctive features is support for user-defined distance metrics via Numba. You write a Python function decorated with @numba.cfunc, compile it to a CompiledMetric object and pass it to the Index constructor. USearch uses it for both index construction and search.

We demonstrated a weighted cosine metric that upweights the first 64 dimensions of each vector - a toy example of the kind of domain-specific tuning this enables. Those dimensions have no special meaning in our embeddings. To check the metric was really applied, we compared it with a standard cosine index built on the same 50,000 vectors with the same settings. On average 9.7 of the 10 results were the same (including the query itself), and the distances differed: for our sample query, rank 2 was 0.1629 under the custom metric and 0.1638 under standard cosine. Some of the difference in results could come from HNSW’s approximation, but the changed distances confirm the metric was used. The custom index took 6.9 seconds to build against 5.3 for the standard one.

Custom metric APIs have changed between releases, so if you adapt older examples, check where CompiledMetric and MetricSignature are imported from.

Persistence

USearch persistence starts with index.save(path), which wrote our 500,000-vector index to an 864MB file in 0.6 seconds. There are two ways to load it. You can create an empty Index with matching parameters and call index.load(path), or call the class method Index.restore(path), which reads the parameters from the file. Restoring took 0.3 seconds. Passing view=True memory-maps the file instead of loading it, which took 0.03 seconds, and both returned the same results for our sample query.

What USearch does not do

USearch has no document storage and, in its Python API, no filtered search. The README lists predicate-based filtering for C++, C, Rust and Swift but not Python. Like FAISS and Voyager, it’s a pure vector index. From Python, if you need to constrain results by crime type or neighborhood you must filter after retrieval or move to Chroma (Day 6) or LanceDB (Day 7).

Unlike PyNNDescent, USearch copes well with changing data. Adding 1,000 vectors to the 500,000-vector index took 0.27 seconds (about 0.1 for Voyager in Day 2), all 1,000 were found in their own top ten, and removing them took effectively no time and left none in the results. Exact search didn’t beat FAISS flat in our tests: 5.6 times slower for single queries and about level in batches.

When to look elsewhere

USearch is the wrong choice if:

• You expect the headline exact-search speedup on single queries - we didn’t see it on an M4 Mac, and batching only brought it level with FAISS flat
• You need metadata filtering - look at Chroma (Day 6) or LanceDB (Day 7)
• Your dataset doesn’t fit in RAM - look at LanceDB (Day 7)
• You want automatic parameter tuning - USearch requires the same manual tuning as FAISS HNSW

Day 6: Chroma

Chroma is an open-source AI-native vector database designed for building LLM applications. Unlike every library we’ve used so far, Chroma isn’t just a search index - it’s a persistent, queryable store with built-in metadata filtering, document storage and collection management. It’s the bridge between a vector search library and a full vector database.

Chroma runs embedded in your Python process with no server required, which is what makes it relevant in a book about Python vector search libraries. In embedded mode it feels like a library - you import it, create a collection and query it. But underneath it manages persistence, indexing and metadata automatically in a way that none of the previous libraries do.

When to reach for Chroma

Chroma is the right choice when:

• You’re building a RAG pipeline and need a persistent store that survives process restarts
• You need metadata filtering alongside vector search - filter by crime type, neighborhood or weapon alongside vector search
• You want to store documents and embeddings together rather than managing them separately
• You want a simple local development experience, with client-server and hosted options if you outgrow it (we didn’t test those)

What we found

We loaded all 500,000 Nordvik records into Chroma in batches of 5,000, storing the MO_TEXT as the document and including crime type, crime subtype, neighborhood, datetime, weapon and property stolen as metadata fields. The collection persisted automatically to disk with no explicit save call. Loading took 321.5 and 414.8 seconds in our two runs, slower than any index build in the earlier chapters, although it also includes storing the documents and metadata, so it isn’t a like-for-like comparison.

Vector search recall was 0.9967 at 0.532ms, averaged over the same 1,000 sampled queries and Day 1 ground truth we used for the other libraries. As before, we asked for 11 results and removed the query’s own ID before scoring. The query itself came back first for 997 of the 1,000 queries, and we didn’t investigate the other three. That recall is in the same range as USearch at ef=16 and above Voyager, but Chroma is roughly an order of magnitude slower than FAISS HNSW and PyNNDescent for pure vector search, and slower than Voyager and USearch at low ef settings. We didn’t measure where the extra time goes.

Metadata filtering

This is where Chroma earns its place. The previous five libraries gave us no metadata filtering in our examples, so we’d have had to filter the results ourselves. We ran three filtered queries against the same burglary incident, NVK-000004, and checked that every result satisfied its filter, for these three examples and for 50 further queries per filter:

Burglary only. Restricting results to CrimeType = "Burglary" returned ten burglaries, all from Hartley Cross. In the unfiltered search rank 10 was an incident of type Other (NVK-331119, distance 0.1615). With the filter that slot went to another burglary (NVK-025362, distance 0.1638). In both searches rank 1 is the query incident itself.

Armed incidents only. Filtering to Weapon != "none" returned seven burglaries, two robberies and one Other incident, all carrying weapons. The filter crossed crime types, finding similar incidents that shared the weapon characteristic.

Millfield only. Restricting to a single neighborhood produced distances from 0.3428 to 0.3702, compared with 0.1416 for the nearest unfiltered neighbor (excluding the query itself). This is expected and informative: when you constrain the candidate set geographically, the closest semantic matches within that area are further from the query than the global nearest neighbors. The filter doesn’t degrade the search - it answers a different question.

Hybrid search. We also combined a metadata filter with a keyword condition on the document text: burglaries whose MO_TEXT contains 1632hrs, a word taken from the query incident itself. Ten results came back, all satisfying both conditions. Distances beyond the query itself ranged from 0.3376 to 0.4429, larger than in the plain Burglary filter because the keyword narrows the candidates further.

What filtering costs. Filtering wasn’t cheap. Averaged over 200 queries, an unfiltered search took 0.575ms. Filtering to Burglary took 328.1ms, armed incidents 426.4ms and Millfield 126.4ms, roughly 220 to 740 times slower. Every filtered result was correct, so the cost shows up in latency, not accuracy. The three filters match very different shares of the records: about 7% for Millfield, 35% for Burglary and about 40% for armed incidents. Latency rose in the same order, which is consistent with cost tracking the number of matching records, although we didn’t test that directly. If you plan to filter at this scale, measure it on your own data.

Is the filter applied during the search? To check, we picked the rarest combination of crime type and neighborhood with at least 20 records (Other incidents in Northgate, 941 of the 500,000 records) and asked for ten results. Chroma returned all ten, all matching, even though none of those records appeared in the unfiltered top 100 for that query. So the filter is applied during the search rather than to the top results afterward.

A simple RAG pipeline

We built a minimal retrieval function that fetches the five most similar incidents to a query, with an optional crime type filter. It excludes the query incident itself so it doesn’t appear in its own context. This is a common pattern in LLM applications: embed the query, retrieve relevant context, pass it to the language model. We stopped short of the model call. Chroma’s document storage means the retrieved context is immediately available without a second lookup against the original dataset.

With the Burglary filter the function took 384.9ms on average over 100 queries. That’s close to the cost of the filtered search itself, so the filter rather than the formatting is the likely cause, but we didn’t time the two separately. Whether that’s acceptable depends on your application.

Persistence

Chroma’s PersistentClient writes to a local directory automatically. We verified this by creating a second client pointing to the same path, in the same process rather than after a full restart, and confirming the collection loaded with all 500,000 records. Fifty queries returned identical results from both collections. For a production application this means you build the collection once and can reopen it later without any additional code, though we didn’t test a full restart. The directory took 3,639 MiB on disk, about five times the 732 MiB of raw embeddings, because it also holds the documents, metadata and index. We didn’t break the total down.

A note on the API

"ids" isn’t a valid value for the include parameter of collection.query(). IDs are always returned and don’t need to be requested explicitly. Use include=[] to return only IDs or include=["documents", "metadatas", "distances"] for the full result set.

What Chroma does not do

Chroma in embedded mode is a single-process store. It’s not designed for concurrent writes from multiple processes. If you need that, Chroma offers a client-server mode with chromadb.HttpClient. We didn’t test it, but only the client initialization should need to change, not your query logic.

In our tests Chroma’s HNSW search reached high recall but was slower than the other libraries’ approximate indexes, and filtered search was far slower (126 to 426ms). For pure vector search without filtering, FAISS and PyNNDescent are significantly faster. If you need metadata filtering and document storage, the extra cost may be worth paying. If you don’t, an earlier library in this book will serve you faster.

When to look elsewhere

Chroma is the wrong choice if:

• You need the lowest possible query latency - FAISS and PyNNDescent are faster for pure vector search
• Your dataset is larger than available RAM - we didn’t test this case, but our collection took 3,639 MiB on disk, so plan for memory use before scaling up; look at LanceDB (Day 7)
• You need fast filtered search at this scale - our filtered queries took 126 to 426ms
• You need concurrent writes from multiple processes in embedded mode - use client-server mode instead
• You need SQL-style joins between your vector store and relational data - LanceDB (Day 7) has better structured data support

Day 7: LanceDB

LanceDB is an open-source vector database built on Lance, a columnar storage format designed for AI workloads. It’s the final chapter of this book for a deliberate reason: LanceDB sits furthest along the spectrum from pure library to full database and it targets a problem we haven’t tested for in the earlier chapters - what happens when your dataset doesn’t fit in RAM?

LanceDB stores data on disk in the Lance columnar format and queries it without loading everything into RAM. This makes it the natural choice for datasets that are too large for available memory, while still supporting vector search and metadata filtering with SQL-style conditions.

When to reach for LanceDB

LanceDB is the right choice when:

• Your dataset doesn’t fit in RAM - LanceDB queries disk-resident data without loading the whole table into memory
• You want SQL-style metadata filters alongside vector search - where clauses are plain SQL strings
• You want versioned data - Lance tables keep every version and support time travel, restore and tags
• You need tight integration with the Arrow and pandas ecosystem
• You want a zero-server embedded database that can also use cloud storage (S3, GCS) - we didn’t test the cloud side

What we found

We loaded all 500,000 Nordvik records into LanceDB in batches of 10,000, then built an IVF-PQ index with 256 partitions and 96 sub-vectors. Loading took 7.1 and 7.6 seconds in our two runs and the index build took 42.8 and 43.0 seconds, a small fraction of the 321 to 415 seconds Chroma needed to load the same records. The table took 3,407 and 4,271 MiB on disk in the two runs, about 4.7 to 5.8 times the 732 MiB of raw embeddings, and we didn’t investigate why the runs differed. The index added only 49 MiB in both.

The IVF-PQ index type introduces two sources of approximation that distinguish it from the HNSW indexes used elsewhere in this book: the IVF partitioning, which assigns vectors to cells and searches only the nearest cells at query time, and product quantization, which compresses vectors to a fraction of their original size.

With default search settings recall came out at 0.8827 at 3.476ms, averaged over the same 1,000 sampled queries and Day 1 ground truth we used for the other libraries. As before, we asked for 11 results and removed the query’s own ID before scoring. That’s the lowest recall and the highest latency of any approximate index in the book.

Much of the recall gap comes from the default settings. Without refinement, recall stopped improving at about 0.8845 however many partitions we searched (nprobes of 10, 20 and 50 all gave 0.8845), so quantization error rather than partition coverage was the limit. Asking LanceDB to re-score candidates on the full vectors with refine_factor=5 raised recall to 0.9895 at nprobes=5 (4.713ms), 0.9970 at nprobes=10 (5.248ms) and 1.0000 at nprobes=50 (9.216ms). A refine_factor of 20 gave the same recall as 5, only slower. We measured these settings on the first 200 queries.

One detail worth noting: the query incident found itself at rank 1 with a distance of 0.037 rather than 0.000. Re-scoring on the full vectors (refine_factor=1) brought it to 0.000, so the difference comes from product quantization. It’s a visible reminder that IVF-PQ is an approximation at the storage level, not just the search level.

Does it stay out of RAM?

This is LanceDB’s headline claim, so we tested it as far as our laptop allows. We ran 200 queries in a separate Python process that opened the table but never loaded the embeddings or the dataframe. Its peak memory was 137 MiB right after opening and 228 MiB after the queries, against a table of 4,320 MiB on disk. The first query took 10.5ms and the rest averaged 3.559ms, so expect a slower first query after a cold start. We didn’t separate the operating system’s file cache from other start-up costs, and a table larger than the machine’s memory would be the real test.

Metadata filtering

LanceDB uses SQL-style where clauses rather than Chroma’s dictionary filter syntax. This will feel natural to anyone who has worked with a database:

table.search(query_vec).where("CrimeType = 'Burglary'").limit(10)
table.search(query_vec).where("Weapon != 'none'").limit(10)
table.search(query_vec).where("Neighborhood = 'Millfield'").limit(10)

The filtering results showed the same pattern as Chroma. We checked that every result satisfied its filter, for these three queries and for 50 further queries per filter. The Millfield filter produced larger distances (0.7039 to 0.7673) than the nearest unfiltered neighbor (0.2721, excluding the query itself), a ratio of about 2.6 against about 2.4 in Chroma. LanceDB’s distances are approximate and aren’t comparable with Chroma’s.

Filtering cost less here than in Chroma. Averaged over 200 queries, an unfiltered search took 3.369ms against 13.909ms for Burglary, 16.603ms for armed incidents and 10.692ms for Millfield, roughly three to five times slower. Chroma’s filtered queries took 126 to 426ms. The three filters match very different shares of the records: about 7% for Millfield, 35% for Burglary and about 40% for armed incidents. Latency rose in the same order, which is consistent with cost tracking the number of matching records, although we didn’t test that directly.

LanceDB applies the filter before the vector search by default. To check, we picked the rarest combination of crime type and neighborhood with at least 20 records (Other incidents in Northgate, 941 of the 500,000 records). With the default pre-filtering we got all ten results. With prefilter=False we got none, and none of those records appeared in the unfiltered top 100. If you turn pre-filtering off, a selective filter can return fewer results than you asked for.

Versioning

Every write to a Lance table creates a new version, and old versions stay readable. After loading and indexing, our table had 255 versions, more than the 50 writes plus one index build we expected, and we didn’t investigate why. Checking out version 1 showed the first 10,000 rows, and returning to the latest version restored all 500,000. We also tagged a version by name. We didn’t try branches.

Analytics

LanceDB stores data in the Lance columnar format, which is designed for efficient analytical queries. We ran our analytics demonstration on the source parquet with pandas instead, computing crime type distributions and armed incident counts by neighborhood, so we didn’t test SQL or DuckDB access to the Lance table.

The crime type distribution matched our generation weights: Burglary at 35%, Vehicle Crime at 25%, Robbery at 20%, Assault at 10%, Drug Offences at 7% and Other at 3%. The generator picks each neighborhood uniformly from 15, so the armed incident counts are evenly spread (the top ten ranged from 13,247 to 13,553, and we printed only those ten). Real crime data wouldn’t look like this.

API notes

A few things behaved unexpectedly and are worth flagging:

• A .select() that omits _distance triggers a deprecation warning, so include _distance in the select list
• The _distance column is inaccessible via itertuples() because pandas renames underscore-prefixed columns - use iterrows() with row['_distance'] instead
• LanceDB’s documentation says the search metric must match the metric used to build the index, so set distance_type explicitly

Cloud storage

LanceDB supports S3, GCS and Azure Blob Storage as backends. We didn’t test them in the notebook, so check the documentation for how to switch the connection URI and supply credentials. If it works as described it’s a path from local development to production, but we can’t vouch for it.

What LanceDB does not do

LanceDB’s IVF-PQ index trades recall for index size. With 96 sub-vectors on 384-dimensional vectors, each vector is compressed from 1,536 bytes to 96 bytes in the index, a 16x reduction, and the index added only 49 MiB to our table. The full-precision vectors stay on disk, so the table itself doesn’t get smaller. That’s what lets LanceDB search large datasets without holding the vectors in memory. The compression is lossy, and at default settings the recall cost is real. In our tests re-scoring candidates on the full vectors recovered most of it, at the price of extra latency.

For datasets that fit comfortably in memory and where latency matters, the in-memory HNSW libraries from earlier chapters are faster and, at their default settings, more accurate. LanceDB’s advantages are most visible at scale, which we couldn’t test on our machine.

When to look elsewhere

LanceDB is the wrong choice if:

• You need the highest recall at low latency - IVF-PQ compression loses information, and recovering it by re-scoring costs time (5.2ms for 0.997 recall in our tests); use FAISS or PyNNDescent
• Your dataset fits comfortably in RAM and latency is critical - in-memory HNSW libraries are faster
• You want the simplest possible setup - FAISS or PyNNDescent are simpler to get started with
• You need the exact search guarantee - LanceDB’s IVF-PQ is always approximate

Conclusions

We’ve run the same experiment seven times. Same dataset, same embedding model and same ground truth, scored on the same 1,000 sampled queries, with scikit-learn on a 50,000-record subset as the one exception. The results are in and the picture is clear enough to draw some practical conclusions.

The numbers in one place

Notes on the table:

• Recall is averaged over 1,000 sampled queries after removing each query’s own ID. FAISS’s recall and latency come from a single query.
• Scikit-learn ran exact search on a 50,000-record subset, so its latency isn’t comparable with the others, which ran on all 500,000 records.
• The tuned LanceDB row was measured on the first 200 queries.
• “Disk-based” means we checked that the table can be queried without loading it into memory. We did that only for LanceDB. “Not tested” means we didn’t check, not that the library can’t.

A note on the recall numbers: Day 1’s ground truth excludes the query itself, but every index returns the query as its own first result. For every library we ask for 11 results and remove the query’s own ID before scoring, so the figures above reflect the true ten neighbors. Voyager at M=12 (0.980) and LanceDB at its default settings (0.883) give up some recall for speed or index size. USearch, Chroma and PyNNDescent (at n_neighbors=50) reached 0.99 or better.

Scikit-learn’s exact search scored 1.000 on its 50,000-record subset, as expected for a brute-force method, at about 25ms per query.

What we actually learned

There’s no single best library. The right choice depends on what you’re optimizing for. The table above has four dimensions and the libraries trade them off differently. No library wins on all four.

HNSW is the dominant algorithm for in-memory search. FAISS, Voyager and USearch use it and PyNNDescent uses a related graph-based method. The differences between them show up in API design, default parameters, persistence and platform optimization, and in speed, although FAISS’s figure comes from a single query so that comparison is rough.

PyNNDescent is the surprise of the book. A pure Python library posting 0.033 to 0.062ms query latency, depending on n_neighbors, is in the same range as FAISS’s C++ HNSW index (0.06ms from a single query). Numba JIT compilation is a likely reason, although we didn’t test that. Recall ranged from 0.958 at n_neighbors=10 to 0.999 at 50. The first query pays about 8 seconds to prepare and compile, and building took 6 to 29 seconds. If you need fast in-memory search without FAISS’s complexity, PyNNDescent deserves serious consideration.

The exact search claim for USearch didn’t reproduce on our machine. USearch’s README reports exact search up to 20 times faster than FAISS’s IndexFlatL2 in a Google Colab test. On a MacBook Air M4 a single exact query took 87.6ms against 15.75ms for FAISS flat, and 14.5ms per query when batched. We haven’t established why, and we didn’t test Intel hardware. This is one of the most useful negative results in the book. Benchmark on your target hardware before committing to a library based on headline numbers.

Metadata filtering has a real cost. Chroma and LanceDB are the only libraries in this book where we used built-in metadata filtering. Both are slower than the pure search libraries even without a filter (0.532ms and 3.476ms), and filtering adds more. In Chroma, filtered queries took 126 to 426ms against 0.575ms unfiltered. In LanceDB they took 10.7 to 16.6ms against 3.4ms. Chroma’s filtered queries may be too slow for interactive use at this scale, so measure on your data. If you don’t need filtering, the pure search libraries are significantly faster.

Disk-based storage changes the tradeoff. LanceDB is the only library in this book where we tested querying without loading the data into memory. A separate process queried a 4,320 MiB table with a peak memory of 228 MiB. Its default recall (0.883) and latency (3.476ms) are the worst of the group, but re-scoring candidates on the full vectors raised recall to 0.997 at 5.2ms. We couldn’t test a dataset larger than our machine’s memory, which would be the real test. If your dataset is large enough to need this, LanceDB is worth serious consideration.

Annoy is broken on Apple Silicon. We attempted to include it and found that it consistently returned only one neighbor regardless of K on M-series chips, as described in Day 2. We dropped it from the book. If you’re on Linux or x86 hardware it may work correctly, but we can’t verify this.

A decision framework

Here’s a simplified way to choose:

Start with PyNNDescent if your dataset fits in memory, you don’t need metadata filtering and you want the simplest path to fast approximate search. It has sensible defaults, needs no C++ compilation to install (the first query compiles with Numba and takes about 8 seconds) and reaches good recall once you raise n_neighbors.

Reach for FAISS if you need exact results, are working at very large scale within memory or want fine-grained control over index type and parameters. It’s the most flexible library in this book.

Use Voyager if you want HNSW with a cleaner API than FAISS and built-in persistence, and you value that it was developed and used at Spotify. It supports dynamic insertion too, although USearch and PyNNDescent also support updates in our tests.

Use USearch if you’re deploying to edge or mobile environments where binary size matters, or if you want approximate and exact search, insertion, removal and custom metrics in one library. We couldn’t reproduce its headline exact-search speedup on Apple Silicon.

Use Chroma if you’re building a RAG pipeline, need metadata filtering and want persistence without managing it yourself. Budget for the costs we measured: loading 500,000 records took 321 to 415 seconds and filtered queries took 126 to 426ms.

Use LanceDB if your dataset doesn’t fit in RAM, or if you want cloud storage without managing it (we didn’t test the cloud side). Expect lower recall at default settings, and plan to tune nprobes and refine_factor.

Avoid scikit-learn NearestNeighbors for anything beyond small datasets or quick experiments. At 384 dimensions the tree-based algorithms offer no advantage over brute force, and brute force on 50,000 records was roughly 400 times slower than FAISS HNSW on 500,000.

What this book did not cover

We focused on single-node in-process search. We didn’t cover distributed vector search, streaming index updates at scale or the interaction between vector search and traditional relational queries in a production data platform. For those topics the companion book - Seven Vector Databases in Seven Days - covers the infrastructure layer: PostgreSQL with pgvector, MongoDB Atlas, Pinecone, Weaviate, Neo4j, Snowflake and Databricks.

We also didn’t cover embedding model selection. We used all-MiniLM-L6-v2 throughout because it’s a standard baseline that makes the library comparisons clean. In practice, the choice of embedding model often matters more than the choice of search library. A better model with a slower library will often outperform a worse model with the fastest library.

A final observation

The most useful result in this book is not a latency number or a recall figure. It’s the observation from Day 1 that all ten results for our sample burglary query (the query itself and nine neighbors) shared the same narrative pattern, and nine of the ten were burglaries. Embedding similarity largely reflects the wording of the text. The embedding notebook shows it in one comparison: two burglaries described very differently had a cosine similarity of 0.41, lower than the 0.49 between one of them and an assault that happened to share its suspect description. The model found incidents described in similar language and, in a template-generated dataset, that means incidents built from the same template.

With real officer-written narratives we’d expect richer variation, although we haven’t tested that. The fundamental point holds: vector search finds what your embedding model considers similar to your query, which is largely a matter of wording. Understanding what your embedding model considers similar - and whether that matches what you consider similar - is the most important question to answer before deploying any of the libraries in this book.

Free Books

The SingleStore Cookbook: Recipes for Multi-Model, Machine Learning and AI Data Engineering

Read Online ↗

A hands-on cookbook covering SingleStore’s multi-model capabilities, from time series and geospatial data through vector search, machine learning pipelines and AI-powered applications. The recipes draw on first-hand experience building applications with the platform and are organized into four parts:

1. Multi-Model
2. Streaming and Big Data Pipelines
3. Machine Learning
4. AI and Agentic Frameworks

Seven Vector Databases in Seven Days

Read Online ↗

A practical guide that takes one vector database per day and pairs each with a use case chosen to showcase that database’s strengths. Databases covered:

1. PostgreSQL and pgvector - Semantic job search
2. MongoDB Atlas - Recipe finder
3. Pinecone - E-commerce search
4. Weaviate - Research paper discovery
5. Neo4j - Fraud detection
6. Snowflake - Customer support analytics
7. Databricks - RAG over internal documents

Each chapter is self-contained, comes with a Jupyter notebook and gives an assessment of when you’d look elsewhere.

Generative AI: A Manager’s Guide

Read Online ↗

A practical guide for managers, directors and executives who need to make decisions about AI in their organizations, not the engineers building it, but the people responsible for making it work well. The book uses a single central metaphor, the Digital Intern, to frame what AI is genuinely good at, where it falls short and what managing it actually requires. It covers governance, risk, board-level accountability, business case building and the organizational change of moving from pilot to embedded capability.

Seven Ways to Do Vector Search in Python

Read Online ↗

A practitioner’s guide that benchmarks seven Python libraries against the same dataset, measuring recall and latency consistently so you can compare like-for-like. Libraries covered:

1. FAISS
2. Voyager
3. Scikit-learn NearestNeighbors
4. PyNNDescent
5. USearch
6. Chroma
7. LanceDB

Each chapter covers one library, explains what it’s genuinely good at and when you’d reach for something else.

Real-Time Vehicle Tracking with Neo4j, Databricks Lakebase and OpenStreetMap

Read Online ↗

A fleet operations demo that puts ten simulated vehicles onto real road networks loaded from OpenStreetMap. The architecture:

• Neo4j Aura holds the road network graph
• Databricks Lakebase stores live vehicle positions
• Databricks Lakehouse handles historical analytics

Two Streamlit dashboards display live positions and trend data. The primary demo uses the London Borough of Merton, with additional configurations for San Francisco and Singapore.

Real-Time Supply Chain Routing with Neo4j, Snowflake Postgres and Confluent Kafka

Read Online ↗

A supply chain routing demo that combines graph-based route optimization with real-time event streaming. The architecture:

• Neo4j Aura holds the supplier and logistics network graph
• Snowflake Postgres stores product and inventory data
• Confluent Kafka streams supply chain events in real time

Route queries traverse the graph to find optimal paths between suppliers, warehouses and distribution centers under live capacity and disruption constraints.

Neo4j Beyond the Graph

Read Online ↗

A hands-on guide to Neo4j’s multi-model capabilities, going beyond graph traversal to explore seven native features of the platform. Each chapter pairs one capability with a use case chosen to showcase it:

1. Native Graph - London Underground routing
2. Full-Text Search - Research paper discovery
3. Geospatial - Air quality along the Pyrenees corridor
4. Temporal - S&P 500 stock price analysis
5. Document-Style Properties - Library inventory management
6. Vector Search - Fashion image similarity
7. Graph + Vector - Fraud ring detection

Every chapter comes with a Jupyter notebook and a Streamlit application. All examples run on Neo4j Aura’s free tier.

Seven Ways to Use Jev

Read Online ↗

A practical guide to TypeSafe AI’s structured decision model, exploring seven domains where a single Choice call – structured state in, typed decision out, calibrated confidence alongside – replaces brittle rules-based logic. Each chapter pairs one domain with a hands-on Jupyter notebook and examines what the confidence score reveals about the decision:

1. Transit Routing - Northern Line branch triage using a NetworkX graph
2. Customer Support Triage - Ticket routing by team and priority
3. Financial Fraud Detection - Flag or pass with a three-tier confidence response
4. Content Moderation - Approve, flag or remove user-generated content
5. Supply Chain Disruption - Alternative supplier selection under live constraints
6. Hiring Screening - CV screening against a structured job description
7. Medical Symptom Triage - Care pathway assignment with a conservative bias

A recurring theme across all seven chapters: the confidence score is more informative than the decision itself for uncertain cases and the threshold between automatic routing and human review is always a business decision, not a technical one.

Lessons Learned Putting Jev Inside Neo4j

Read Online ↗

A hands-on account of building a Neo4j function and batch procedure in Java that ask a model to make a decision about a node and store the answer back in the graph. The book follows one running example, 500 synthetic transactions and each chapter adds the code that the previous chapter’s problem called for. The lessons:

1. Why in the Database - Where a decision belongs and what the finished function returns
2. A First Working Function - Converting Neo4j’s types on the way in and out
3. Failures as Data - Returning errors as values so one bad node can’t stop a query
4. Testing Without a Network - Seams for the network and the clock, with 68 fast tests
5. Keeping the Key Safe - Sending a credential only to the host the parser reports
6. Many at Once - A bounded, ordered pool and a measured concurrency sweep
7. Running It Locally - Swapping in a small local model and seeing how the judgment changes

Every chapter with code has its own folder in the repository and each Cypher script also comes as a notebook, so you can use whichever you prefer. The results are reported as measured, including the ones we can’t explain!