Why object-storage vectors use OPANN + Sq16, while RAM uses HNSW
HNSW is an excellent in-memory vector index. It is also a poor object-storage access pattern. Each graph hop depends on the previous hop, so a cold walk turns pointer chasing into serialized range requests.
Infino therefore uses two serving paths over the same vector data: HNSW when the working set is resident in RAM, and OPANN—Infino’s object-storage ANN path—with Sq16 when the vectors are served from Parquet on object storage.
The same corpus has two hardware states
A 1024-dimension fp32 vector occupies 4 KiB before index metadata. Ten million vectors are roughly 40 GiB of raw values. If that working set and its graph fit in the available RAM, spending memory to minimize query CPU and latency is reasonable.
The same decision fails for a large or mostly cold corpus. Keeping every vector and graph edge resident makes storage cost scale with the full dataset, even when only a small fraction receives queries. Moving the graph itself to object storage avoids the RAM bill but replaces memory loads with dependent network reads.
hot working set
query ─▶ HNSW graph walk ─▶ top-k
random access in RAM
cold / object-storage working set
query ─▶ OPANN routing ─▶ bounded cluster ranges ─▶ top-k
contiguous object reads
durable source for both
parquet superfiles in object storage HNSW: random access is a feature in RAM
HNSW builds a navigable graph over the vectors. A query starts at an upper layer, follows promising neighbors toward the query, and expands a candidate set near the bottom. The graph avoids scoring most vectors.
In RAM, those dependent hops are cheap. The next node address may be unpredictable, but it resolves as a memory access rather than a storage request. A graph over the resident Sq16 representation also reduces the bytes touched per score relative to fp32.
The cost moves to build time and memory. The graph stores edges in addition to vector data, and loading or rebuilding it is work a cold worker must pay before it receives the low-latency path. HNSW is the right answer only when the working set is actually resident.
Why graph hops do not belong on object storage
Object storage rewards planned reads. One request can fetch a useful contiguous range at high throughput, but every additional dependent request pays request latency again.
An HNSW walk cannot issue all of its reads in advance: the node selected at step N determines which neighbors exist at step N+1. Even with caching, a cold graph walk creates the exact sequence object storage handles poorly—small, unpredictable, serialized reads.
Downloading the whole graph before searching avoids that request pattern, but turns the first query into a bulk load and requires enough local capacity for every cold namespace. That is pinning under another name.
OPANN: route first, then read contiguous ranges
The object-storage path groups vectors into regions that can be addressed independently. A small routing structure chooses the regions most likely to contain the nearest results. The query then fetches those cluster ranges, scores their compact vectors, and reranks a shortlist.
The important property is not the acronym; it is that the expensive reads are known before the data arrives. Cluster ranges can be requested concurrently, coalesced, cached, and reused. A query touches a bounded subset of the vector region rather than traversing a graph one network-dependent node at a time.
Because the vector index sits inside the same immutable Parquet superfile as the row data, the returned ids resolve against the same snapshot as scalar filters and BM25 results.
Why Sq16 sits between fp32 input and search
Infino accepts fp32 vectors. Internally, Sq16 represents each dimension with 16 bits. A 1024-dimension row drops from 4 KiB to about 2 KiB before surrounding metadata: roughly 40 GiB to 20 GiB across ten million vectors.
That reduction matters in both states. A resident HNSW walk touches fewer bytes per node. An OPANN query transfers and scores more candidates per object-store range. Full-precision input remains the API contract; quantization is an engine choice rather than a format the application has to manage.
Sq16 is still a quantized representation. Workloads requiring a different codec or direct control over every search parameter may prefer a dedicated vector database that exposes those choices. Infino chooses the representation and query parameters from the table’s data and serving state.
The switch is a serving decision, not a data migration
The durable index remains in the Parquet superfiles. A hot working set can load a resident HNSW path over those vectors. A cold worker, a restart, or a table beyond the RAM budget continues through OPANN. Changing serving state does not create a second source of truth or require the application to reinsert vectors.
The object-storage path is the fallback when the resident graph is absent or evicted. Vector search becomes colder rather than unavailable because object storage still holds the data and the range-searchable index.
The measured object-storage path
On the 9.4-million-vector Cohere run (768 dimensions, top-10), the post-drain OPANN + Sq16 path reached 0.995 recall@10 with 9.37 ms p50 and 10.25 ms p99 warm latency. A cold open took a 661 ms median; the first cold query took 4.60 seconds while fetching 2.55 GiB of index ranges.
Those are OPANN measurements, not HNSW measurements. They show the trade directly: planned range reads are fast once resident, but expensive on first touch. The resident HNSW path trades additional RAM and graph load/build cost for a query path that does not perform that cluster scan.
Use the index that matches the medium
There is no reason to force one vector structure onto two different storage media. HNSW turns resident memory into low-latency graph traversal. OPANN turns object storage into a small number of planned range reads. Sq16 lowers the bytes both paths have to move and score.
The shared invariant is the Parquet source beneath them: one durable vector corpus, with the serving path selected by where the working set lives. The implementation is in the Infino repository.