Search Benchmark, the Game
Full-text search at Wikipedia scale: Infino with Tantivy, Lucene and iresearch in the open benchmark started by the Tantivy project, run nightly on our public fork.
How it is measured
The benchmark asks every engine the same questions over the same corpus, one query at a time. The harness, the corpus scripts and the engine adapters are all in the fork, which tracks the upstream benchmark.
- corpus
- English Wikipedia, stemming off
- queries
- from the AOL query log, kept if they have at least two terms and match at least one document as a phrase
- query types
- intersection, union and phrase, plus single terms of different frequencies
- collection
- count only, top-k by BM25, and top-k with a count
- timing
- single thread, best of 10 runs, after a warm-up run so data is in the page cache
- harness
- each engine in its own process, timed over stdin and stdout
- hardware
- GCP c3-highcpu-8 (8 vCPU, 16 GiB) since Oct 6, 2026. Earlier runs used an AWS c7i.2xlarge, so timings are not comparable across that date
Results
The full run of Oct 27, 2026: 962 queries over 5M documents. Each figure is an engine's average latency relative to Lucene 10.5 (1.00), averaged over the three top-k shapes and over the two count shapes. Lower is faster.
| engine | version | top-k search | count |
|---|---|---|---|
| Infino | 0.11.9 | 0.72× | 1.12× |
| Infino | main | 0.50× | 0.44× |
| Lucene | 10.5.0 | 1.00× | 1.00× |
| Tantivy | 0.26 | 1.17× | 0.83× |
| iresearch | 26.10.3 | 0.48× | 0.38× |
Released 0.11.9 trails Lucene on counts because of one shape, the top 100 with a count, which main brings back ahead. The per-query timings are on two live dashboards, refreshed by the nightly run.
- The full query set, every query the benchmark runs, which the table above summarises.
- The turbopuffer set, turbopuffer's 31 queries. Its turbopuffer timings come from turbopuffer's own published run on other hardware, so read them as a reference, not a head-to-head.
- The results file, committed to the fork by each nightly run with the engine versions it measured, and its history, one commit per night.
Reproduce it
You need Cargo for the Rust engines and the driver, and Java for Lucene. The ENGINES setting picks which engines to build and run. A full run takes more than half an hour.
git clone https://github.com/infino-ai/search-benchmark-game && cd search-benchmark-game make corpus # download and unpack Wikipedia make index # build each engine's index make compile # build the query runners make bench # run the queries, writes results.json make serve # dashboard at http://localhost:8080/
Common questions
Is Infino faster than Lucene in this benchmark?
In the nightly run of Oct 27, 2026, released Infino 0.11.9 is 28% faster than Lucene 10.5 on top-k search and 12% slower on counts. Infino at main is 50% faster on top-k and 56% faster on counts. iresearch, also in the run, is faster than both. The dashboards show every engine on every query, so the wins and the losses are both there to read.
Can I compare these timings with the vector or ClickBench numbers?
No. This benchmark runs each engine as its own process over a fixed Wikipedia corpus, single-threaded, on its own hardware. The in-process vector comparison and ClickBench use different machines and methods, so their numbers do not line up with these.
Which versions are measured?
The nightly run benches the released Infino crate and the engine at main side by side, so the dashboard shows both what you can install today and what is coming. Tantivy and Lucene run at the versions pinned in the harness.
Full-text, vectors and SQL in one engine. Try it on your own data.
Free tier · No credit card to start
or run it yourself: pip install infino · Apache-2.0 · quickstart →