Infino
talk to an engineer sign up

What is hybrid search?

· engineering

Hybrid search runs two retrievers over one query. A keyword retriever scores documents with BM25 against an inverted index. A vector retriever finds nearest neighbors to an embedding of the query. Each returns its own ranked list, and a fusion step merges the two into one ranking.

The reason to run both is that they miss different things.

Two retrievers, opposite failure modes

BM25 is literal. It finds the error code, the part number, the surname, the exact phrase someone typed, and it misses the same idea written another way.

Vector search is about meaning. It finds the paraphrase, the synonym, the other language, and it will also hand back things that are merely on topic.

A query like retry a failed request needs both: the token retry, and the idea of backoff, try-again, transient failure.

one query, one snapshot keyword · BM25 inverted index catches: codes · names misses: other wording vector · ANN embeddings catches: paraphrase misses: the literal string both run concurrently reciprocal rank fusion merges by rank position out: one ranked list · columns + score

A question in clauses

Take “Which services retried a failed request, by team?”. Each clause asks for something different.

  • Which services is a count: COUNT(DISTINCT service).
  • By team is a GROUP BY: one count per team.
  • Retried is a word to match. BM25 finds the events that say it.
  • A failed request has no fixed wording. Vector search finds “try again”, “transient failure”, “exponential backoff”.
FOUR CLAUSES which services COUNT by team GROUP BY retried BM25 a failed request vector

Fusing two incompatible scores

BM25 similarity rises for a better match, while vector distance falls. The two numbers are not on the same scale and adding them means inventing a weight. Reciprocal rank fusion sidesteps that by using position in each list instead of the scores themselves.

rrf.txt 
contribution(rank) = 1 / (60 + rank)

# first in a list contributes 1/61
# a document first on both sides:  1/61 + 1/61  ≈ 0.0328
# first on one side only:          1/61         ≈ 0.0164
# tenth on both sides:             1/70 + 1/70  ≈ 0.0286

Each list is best-first. A hit at rank r (first place is 1) contributes 1 / (60 + r). The two contributions add, and the sum is the emitted score. A document both retrievers liked climbs above a document that only one of them ranked first.

60 is the constant from the 2009 paper that introduced the method, where fusing runs this way beat every individual system and Condorcet Fuse. It keeps fusion deterministic across queries.

Fusion needs a stable identity to merge on. A row's _id is what makes the same document one document, even when the two indexes store it differently.

k controls recall

Each lane retrieves k hits and fusion keeps the top k by fused score. A result LIMIT is a separate number: how many of those rows the query returns.

  • A result page uses a modest k (50-200) and a small LIMIT.
  • An aggregate over matching events uses a large k (thousands) so GROUP BY and COUNT see the full candidate cohort.
  • The disagreement set (rows only one retriever found) also needs a deep k. Shallow k hides the misses.
  • Raising k costs work on both lanes and increases retrieval depth without changing fusion weights.

Seeing where the two disagree

Fusion hides the split by design. To measure it, run each retriever on its own and join the two relations on _id. The rows on only one side are the ones a single retriever never returns.

disagreement.sql 
-- where do keyword and meaning disagree, and how much does each add

WITH sides AS (
  SELECT CASE
           WHEN v._id IS NULL THEN 'keyword only'   -- literal, no paraphrase
           WHEN k._id IS NULL THEN 'meaning only'   -- the same thing, said differently
           ELSE 'both agreed'
         END       AS found_by,
         k.score   AS bm25,
         v.score   AS cosine
  FROM   bm25_search('tickets', 'body', 'disk full on ingest', 100) k
  FULL OUTER JOIN
         vector_search('tickets', 'embedding', :q, 100) v
      ON v._id = k._id                            -- one snapshot, both sides
)
SELECT   found_by,
         count(*)              AS docs,
         round(avg(bm25), 3)   AS avg_bm25,               -- nulls skip themselves
         round(avg(cosine), 3) AS avg_cosine
FROM     sides
GROUP BY found_by
ORDER BY docs DESC;

Run it per query class. When the “meaning only” bucket grows, the embeddings and the vocabulary are drifting apart. Note that k.score is BM25 (higher is better) while v.score is a distance (lower is better), which is the reason fusion compares rank positions rather than adding those columns.

What sits around retrieval

Hybrid search is the retrieval middle: one keyword query, one vector query, fusion, a ranked list. The quality stages on either side of it belong to the application.

  • Query rewrite sits in front. Several rewritten queries can run as separate searches or a UNION.
  • A cross-encoder reranker sits after, on the dozens of rows that came back. Rescoring candidates inside an ANN index is a different thing that shares the name.
  • Chunking, contextual retrieval, and a labeled eval set are ingest and evaluation work.

In Infino, both indexes live inside the same Parquet files, both lanes read one pinned snapshot, and the fused result is a SQL relation: hybrid search on Infino · the query surface.