02 / ENGINEERING CASE STUDY · Infrastructure
Distributed Vector Database
A C++20 search engine built from storage and distance kernels through an HNSW index and a coordinator/worker architecture.
01 / CONTEXT
The problem
Similarity search is easy to call through an API, but building the engine exposes the tradeoffs underneath it: search quality versus latency, memory layout versus portability, and local indexing versus distributed coordination. I built this project to understand those pieces by implementing them.
02 / SYSTEM DESIGN
Architecture
One query, shard-local searches, one ranked result
- Writes: hash(vector_id) % shard_count
- Each worker: aligned storage + local HNSW
- Transport: Protobuf contracts over gRPC
03 / TRADEOFFS
Engineering decisions
HNSW: spend search effort where it matters
The index descends through a multilayer proximity graph, then expands a bounded candidate set at the base layer. Search breadth controls how much work the engine does. The tradeoff needs to be evaluated with recall alongside latency; a fast answer is only useful if it retrieves good neighbors.
HNSW implementationSIMD with a portable fallback
Vector buffers use 64-byte alignment for AVX-512 loads. L2 and cosine kernels dispatch to the AVX-512 implementation on supported x86 hardware, and use scalar code elsewhere. This keeps the engine usable on machines such as the development Mac without implying that every machine gets SIMD acceleration.
Runtime distance dispatchHash writes; search every shard
Vector IDs determine the destination shard for single and batch inserts. Queries run against all workers concurrently. This keeps write routing simple, but a change in shard count would require moving data; cluster membership and rebalancing are not implemented.
Coordinator and worker interfacesExplicit RPCs and a simple top-k merge
Protobuf defines search and write contracts across the coordinator and workers. Each shard returns up to k candidates; the coordinator sorts the combined list and truncates it to k. This is straightforward to inspect, though merge cost grows with the number of shards and candidates. The result remains approximate because the local HNSW searches are approximate.
gRPC coordinator implementation04 / VALIDATION
Evidence & measurement
The repository contains implementation and correctness tests. It does not yet contain a reproducible recall/latency/throughput benchmark harness, so the performance claims below are not presented as measured results on the project cards.
768-dimensional vectors
The engine tests use the default 768D configuration, including a 128-vector synthetic nearest-neighbor fixture. This is a correctness fixture, not a performance dataset.
View supporting reference64-byte alignment
The storage test checks buffer alignment and padded stride; distance tests exercise L2 and cosine behavior.
View supporting referenceDistributed behavior
Two local shards exercise candidate merging, sorted results, and batch routing. These tests do not establish networked gRPC throughput.
View supporting referenceRecovery and build scope
A snapshot round-trip test saves vectors and restores a searchable index. The README reports scalar-backend testing on Mac and notes that gRPC targets were not compiled in that environment.
View supporting referencePerformance claims: available context
These figures appear in my résumé. Published benchmark conditions are still missing; they should not be read as reproducible results.
| Reported figure | Evidence status | What’s needed |
|---|---|---|
| <2 ms ANN latency | Résumé-reported; no published run | Dataset size, CPU/RAM, SIMD backend, index parameters, k, recall, p50/p95, and warm-up policy are not documented. |
| 10K+ queries / second | Résumé-reported; no published run | Client concurrency, shard count, network setup, test duration, and latency/recall at that load are not documented. |
05 / NEXT ITERATION
What I’d improve next
Implementation references are pinned to repository revision 17791ba. Performance claims originate in the September 2026 résumé; no new benchmarks are claimed here.