The three costs of a cold query on S3
Object storage holds Infino’s durable copy; compute and local cache can disappear. The first query on a new worker therefore has to discover each file’s layout, open enough connections to read many files concurrently, and populate cache without downloading the whole dataset.
On a 10-million-document table in us-east-1, opening one cold superfile took about 164 ms; a fresh table with a couple hundred files took about 556 ms. Those numbers reflect three categories of cold-path work.
These delays overlap: file opens fan out, cache state differs by file, and connection setup runs concurrently. Together they are the three places a first query can be forced to wait.
1. Discover the file in one GET
Parquet puts its footer at the end of the object. On local disk that is one seek. A textbook S3 reader sends a HEAD request to learn the object size, then a range GET for the tail: two round trips before it knows where the columns and indexes live.
S3 suffix ranges remove the HEAD. A request for bytes=-N returns the tail and reports the full object size in Content-Range. Infino starts with a 64 KiB suffix, enough for the common footer. A targeted second GET runs only when the footer begins before that window.
In-region, the suffix request removed about 25 to 50 ms from a cold open. When the manifest already records exact ranges, the footer and index metadata can be fetched in one parallel batch. Small superfiles can carry their entire open window in that read.
Azure Blob does not accept the same suffix-range request, so a standalone Azure open still uses HEAD plus a bounded read. A table manifest already records the object size, which removes that extra request from the normal query path.
2. Keep connections ready for fan-out
A query may open hundreds of superfiles concurrently. Saving one metadata request does little if every range read first establishes TCP and TLS. Connection setup adds another serialized round trip and becomes visible in the tail.
The pool keeps up to 1024 idle connections per host so fan-out can reuse sockets. S3 connections expire from the pool after 10 seconds, below S3’s roughly 20-second server-side idle close; a 5-second connect timeout prevents one stalled handshake from holding the query. Azure uses a 90-second pool timeout to stay below its longer server-side window.
3. Drop RAM without dropping the file
After a superfile is cached locally, Infino memory-maps it. That consumes two different budgets: resident pages in RAM and the file on local disk. An idle-threshold sweep (and a budget sweep when a memory cap is configured) uses MADV_DONTNEED to drop resident pages while retaining the disk file, so the next access faults from local storage rather than S3.
Pinning every file in RAM would remove that fault, but it would also recreate the always-on storage cost this architecture is meant to avoid. The disk copy is the middle state: slower than resident pages, faster than another object-store read, and disposable.
The cache files are immutable and mapped read-only. POSIX also keeps an existing mapping valid after its file is unlinked, until the last reference disappears. A running query can therefore finish while the cache evicts the file from its directory.
Where the design stops working
The cold-open cost is paid when a worker encounters files it has not seen before. Repeated queries reuse the open metadata, connections, disk files, and resident pages. The architecture works when that cache has time to become useful.
On the same 10-million-document table, a cold analytic that touched every segment ran as high as eight seconds. A workload made entirely of first-touch scans is the wrong fit. For tables that should become warm, prefetch runs at a bounded fan-out of 8, filling the local cache without sending every range request at once.
Object storage becomes practical as the durable database only when the query path bounds its requests, reuses its sockets, and treats local disk as a disposable cache.