--- title: "What’s the SQLite equivalent for full-text and vector search?" description: "Embedded search engines that run inside your application with no server: how SQLite with FTS5 and sqlite-vec, DuckDB, LanceDB, Chroma and Infino compare on full-text, vector, hybrid and SQL search, and when each one fits." canonical: https://infino.ai/blog/embedded-full-text-and-vector-search/ published: 2026-09-28 --- # What’s the SQLite equivalent for full-text and vector search? Sep 28, 2026 · engineering An embedded search engine is a library that runs inside your application’s process and keeps its index in local files, the way SQLite does for relational data. There is no server to deploy, no network hop on a query, and it works offline. For full-text search alone, SQLite’s own FTS5 extension already is that library. The question gets harder when the same application also needs vector search over embeddings, both kinds fused in one query, and SQL over the results. This page compares the in-process options on those four jobs, and says when each one fits. It compares capabilities only, not speed. ## Why search moves inside the application - **Desktop and local-first apps.** A notes app, an email client or a document tool has to search the user’s data on their machine, often offline. Shipping a search server with a desktop app is not realistic. - **Agents and RAG.** An agent issues many retrievals per task. A library in the same process answers each one without a network round trip, and the whole thing runs in a notebook or a test without infrastructure. - **Edge and single-tenant deployments.** One process per customer or per device, each with its own small index, is simpler as a file than as a cluster. The trade-off is the one SQLite has always had. A library shares the application’s memory and CPU, and writes to one index go through one writer at a time. Reads scale with the processes that open the files. When many services write to the same large corpus and query it at a high sustained rate, a dedicated search server or vector database is the better shape. ## How the embedded options compare | option | runs as | keyword ranking | vector search | keyword + vector in one query | SQL over results | storage | | -------------------------- | ----------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------- | | Infino | library for Python, Node.js and Rust | BM25 | ANN index (IVF + 1-bit RaBitQ, full-precision rerank) | built in: BM25 plus vector fused with reciprocal rank fusion, filters applied before fusion | read-only SQL (DataFusion) with each search as a table function, writes through the API | Parquet files on local disk, S3 or Azure Blob | | SQLite + FTS5 + sqlite-vec | SQLite with a loadable extension | BM25 (FTS5) | brute-force KNN in the stable release, ANN indexes in alpha | you write it: two queries fused in your own SQL | full SQLite SQL | one SQLite file | | DuckDB + fts + vss | DuckDB with two extensions | BM25 (fts), index rebuilt by hand after changes | HNSW (vss, experimental, persistence behind a flag) | you write it in SQL | full DuckDB SQL | one DuckDB file | | LanceDB | library for Python, TypeScript and Rust | BM25, new rows indexed after optimize() | ANN indexes (IVF\_PQ, IVF\_HNSW\_SQ) | built in, fused with RRF by default | filter expressions in open source, full SQL in Enterprise | Lance files on local disk or object storage | | Chroma | embedded client for Python and JavaScript | no local ranking: text filters with $contains and $regex | HNSW | Chroma Cloud only (Search API), single-node planned | no, metadata filters | a local directory | All five are open source. Checked against each project’s documentation and release notes in September 2026. These projects move quickly, so check the current docs for the feature you depend on. ## When is each one the right choice? - **Infino** when the application needs keyword, vector and SQL search over the same rows, fused in one query, with the data kept as Parquet on local disk or in object storage. Three limits today: it runs on macOS and Linux, not Windows. You bring your own embeddings. And it is built for batched writes, not a row at a time (see below). - **SQLite with FTS5** when the app needs keyword search only. It ships with most SQLite builds, so it is often already in the app, on every platform SQLite runs on. - **SQLite with sqlite-vec** when the app already stores its data in SQLite and the vector corpus is small enough for exact search. The project says it is pre-v1, and its approximate indexes are still in alpha releases. - **DuckDB with fts and vss** when the workload is analytics first and search second, on data already in DuckDB. Both extensions carry caveats: the vector index is experimental, and the full-text index has to be rebuilt after the table changes. - **LanceDB** when the app works with vectors and multimodal data and wants built-in hybrid search, and filters are enough rather than full SQL. - **Chroma** for the quickest start on a vector-only prototype, with Chroma Cloud when it needs ranked keyword and hybrid search. For keyword search inside a Rust program, [Tantivy](https://github.com/quickwit-oss/tantivy) is a full-text library in the style of Lucene. It ranks with BM25. ## What this looks like in Infino Infino is an open source retrieval library. A table lives in a folder of superfiles, next to a small manifest that says which of them are live. A superfile is a search index that is a valid Parquet file: the data and the BM25 and vector indexes in one file. Point `connect` at a local path for a desktop app, or at an `s3://` URI when the same table should live in a bucket. The code does not change. ```py import infino import pyarrow as pa db = infino.connect("./notes") # a folder on local disk, no server notes = db.create_table( "notes", pa.schema([ pa.field("folder", pa.large_utf8(), nullable=False), pa.field("body", pa.large_utf8(), nullable=False), pa.field("embedding", pa.list_(pa.float32(), 384), nullable=False), ]), infino.IndexSpec().fts("body").vector("embedding", 384, "cosine"), ) notes.append(rows) # a batch of your text and embeddings: one commit # keyword and meaning in one query, fused with reciprocal rank fusion hits = notes.hybrid_search("body", "INV-2291", "embedding", query_vec, 10, projection=["body"]) # the same search as a SQL table function, grouped by folder db.query_sql(f""" SELECT folder, COUNT(*) AS hits FROM hybrid_search('notes', 'body', 'overdue payment', 'embedding', '{vec_literal}', 10) GROUP BY folder ORDER BY hits DESC """) # when the app is idle: build the table-wide vector index, merge small files notes.optimize() ``` An identifier like `INV-2291` is the kind of token embeddings blur. BM25 indexes it as `inv` and `2291`, and the rare number carries the match. The vector half catches notes that say the same thing in other words. A `WHERE` on `hybrid_search` is applied to both halves before they are fused, so a filter ranks within the matching rows rather than trimming a list afterwards. One thing to design for in a local app: every `append` is a commit that writes a new file, so write in batches rather than a row at a time. Nothing runs in the background inside your process. `optimize()` is where maintenance happens, so call it when the app is idle. The [quickstart](https://infino.ai/docs/quickstart) runs the full version in Python, Node.js or Rust, and [Integrating SQL analytics with vector and full-text search](https://infino.ai/docs/guides/sql-analytics-with-search) covers the filtering, grouping and joins over search results. For choosing a store for agent memory in particular, see [Choosing a database for AI agents](https://infino.ai/blog/choosing-a-database-for-ai-agents/). ## Common questions ### Is there an embedded vector database that runs without a server? Yes. sqlite-vec, DuckDB’s vss extension, LanceDB, Chroma’s embedded client and Infino all run inside the application process and keep their data in local files. They differ in whether the vector search uses an approximate index or an exact scan, and in whether keyword search and SQL come with it. ### Can SQLite do vector search? With the sqlite-vec extension, yes. Its stable release does exact (brute-force) nearest-neighbor search, which is fine for small corpora. Approximate indexes are in its alpha releases, and the project describes itself as pre-v1. Combining it with FTS5 keyword search is SQL you write yourself. ### How do I add offline search with embeddings to a desktop app? Use a library that stores its index in local files and runs in the app’s process, then generate embeddings with a local model so nothing leaves the machine. For keyword search alone, SQLite FTS5 is enough. For keyword plus meaning, pick an engine with built-in hybrid search, such as LanceDB or Infino, and check that it ships for the operating systems you target. Batch writes where the engine commits per write, and run its maintenance step when the app is idle. ### What is hybrid search in an embedded engine? Running keyword (BM25) and vector search over the same records and merging the two ranked lists, usually with reciprocal rank fusion. Keyword search finds exact names and identifiers, vector search finds paraphrases, and the fused list keeps what either one alone would miss. [What is hybrid search?](https://infino.ai/blog/what-is-hybrid-search/). ### Can I query my search index with SQL? In SQLite and DuckDB, yes, since search lives inside a SQL database. LanceDB’s open source edition takes SQL filter expressions on queries. In Infino each search is a SQL table function, so search results can be filtered, grouped and joined in one statement. Its SQL is read-only, and writes go through the API. [SQL reference](https://infino.ai/docs/sql-reference).