Integrates with
your data stack.
Point Infino at Parquet you already have, or append JSON directly. It builds the BM25 index and writes hydrated Parquet back to your bucket. Embeddings can come from the table’s ingest model, from vectors you send, or from a later hydration pass.
infino hydrate s3://your-bucket/logs # the files stay where they are
How you adopt it
Hydration reads the files where they already sit and writes them back to the same prefix. If you have no Parquet yet, append JSON and Infino writes it for you.
- 01 Existing Parquet Already on S3, GCS, or Azure Blob
- 02 Hydration BM25 index; embeddings generated or supplied
- 03 Hydrated Parquet Same file, now searchable
cat query.sql # search and sql, one pass
What one statement expresses over that file
Retrieval, the filters, and the ranking are one statement.
SELECT _id, title, score FROM hybrid_search( -- BM25 + vector, fused by RRF 'logs', 'body', 'disk full', -- the text side 'embedding', :q, 50 -- the vector side, top 50 ) WHERE level = 'error' -- pushed-down filter AND ts > now() - interval '24 hours' ORDER BY score DESC LIMIT 10; -- → ranked hits · one pass over Parquet
RAG: joins, group-bys, and windows over the same result set →
infino diff my-stack # what each tool does today
What it replaces, and what it runs beside
replaces
| tool | what it does today | with infino |
|---|---|---|
| elasticsearch · opensearch | BM25 and vector search over engine-private segment files on cluster disks, sized and paid for at peak. | The same ranking over Parquet in your own bucket. |
| vector database | A second copy of the embeddings in a second store, with keyword and vector hits merged in your client code. | One index. BM25 and vector run in one pass and are fused by RRF inside the query. |
runs beside
Infino does not replace a warehouse. Teams move query workloads onto Infino to reduce cost and improve performance, but Infino does not have the ecosystem of a typical data warehouse.
| tool | what it does today | with infino |
|---|---|---|
| snowflake · databricks · bigquery | SQL and aggregates over the lake. Ranked retrieval means a table scan, or a separate managed search service with its own copy. | Move retrieval and selected SQL query workloads onto Infino. The warehouse keeps BI and governance; both read the same Parquet files. |
| duckdb · spark · trino | They already read your Parquet, and there is no BM25 or ANN index inside the file for them to use. | Hydration puts that index in the file. They keep reading the columns; Infino serves the ranked read. |
cat catalog.md
Iceberg, Delta Lake, and Hudi
Infino hydrates the data files your table already points at, then registers the swap with the catalog. The catalog stays the ACID authority; a query reads one pinned snapshot.
infino spec
- search
- full-text · vector · hybrid
- index
- BM25 (PFOR-delta, FST) · HNSW · OPANN + Sq16
- engine
- Rust
- language
- SQL (Apache DataFusion) · REST · Query DSL
- storage
- object storage, S3 · GCS · Azure Blob · on-prem
- format
- Apache Parquet
- deploy
- on-prem · self-hosted cloud · hosted cloud
- core license
- Apache-2.0
Go deeper: how it works · elasticsearch replacement · agent observability · RAG · docs ↗