Where your data lives is chosen by the connect URI:
db = infino.connect("memory://") # in-process, ephemeral
db = infino.connect("./data") # local disk (durable)
db = infino.connect("s3://bucket/prefix") # object storage (durable)
const db = connect("memory://"); // in-process, ephemeral
const db = connect("./data"); // local disk (durable)
const db = connect("s3://bucket/prefix"); // object storage (durable)
use infino::connect;
let db = connect("memory://")?; // in-process, ephemeral
let db = connect("./data")?; // local disk (durable)
let db = connect("s3://bucket/prefix")?; // object storage (durable)
memory:// is in-process and ephemeral. update and delete require durable
storage (a path or s3:// URI).
Object-storage configuration
Credentials go in storage_options (storageOptions in Node.js), keyed by the
standard object_store config strings — aws_*
for S3, azure_* for Azure Blob (the same names the AWS and Azure SDKs use).
Infino reads no credentials from the environment; omit them to use ambient cloud
identity (an IAM instance role or Azure managed identity).
# S3
db = infino.connect(
"s3://bucket/prefix",
storage_options={
"aws_access_key_id": "...",
"aws_secret_access_key": "...",
"aws_region": "us-east-1",
},
cache_dir="/var/cache/infino", # local disk cache for remote-backed tables
cache_budget_bytes=2_000_000_000,
validate=True, # probe the store now, not on first query
)
# Azure Blob
db = infino.connect(
"az://container/prefix",
storage_options={
"azure_storage_account_name": "...",
"azure_storage_account_key": "...",
},
)
// S3
const db = connect("s3://bucket/prefix", {
storageOptions: {
aws_access_key_id: "...",
aws_secret_access_key: "...",
aws_region: "us-east-1",
},
cacheDir: "/var/cache/infino", // local disk cache for remote-backed tables
cacheBudgetBytes: 2_000_000_000,
validate: true, // probe the store now, not on first query
});
// Azure Blob
const db = connect("az://container/prefix", {
storageOptions: {
azure_storage_account_name: "...",
azure_storage_account_key: "...",
},
});
use infino::{connect_with, ConnectOptions};
// S3
let db = connect_with(
"s3://bucket/prefix",
ConnectOptions::new()
.with_storage_option("aws_access_key_id", "...")
.with_storage_option("aws_secret_access_key", "...")
.with_storage_option("aws_region", "us-east-1")
.with_cache_dir("/var/cache/infino") // local disk cache for remote-backed tables
.with_cache_budget_bytes(2_000_000_000)
.with_validate(true), // probe the store now, not on first query
)?;
// Azure Blob
let db = connect_with(
"az://container/prefix",
ConnectOptions::new()
.with_storage_option("azure_storage_account_name", "...")
.with_storage_option("azure_storage_account_key", "..."),
)?;
| Backend | Keys |
|---|
| S3 | aws_access_key_id, aws_secret_access_key, aws_region, aws_session_token, aws_endpoint |
| Azure | azure_storage_account_name, azure_storage_account_key, azure_storage_sas_key, azure_storage_client_id, azure_storage_client_secret, azure_storage_tenant_id |
object_store accepts for the backend; an unknown key
is rejected at connect. For an S3-compatible endpoint (MinIO / R2 / Ceph), set
aws_endpoint (add aws_allow_http: "true" for plain HTTP) alongside the
credentials.
Pass validate (with_validate(true) in Rust) to probe the backend during
connect, so wrong credentials or an unreachable bucket fail there instead of
on the first query. The local cache keeps hot superfiles on disk so warm queries
don’t re-fetch from object storage.
Connect options
| Option (Python / Node.js / Rust) | Description |
|---|
storage_options / storageOptions / with_storage_option(k, v) | cloud credentials & tuning, keyed by object_store’s aws_* / azure_* config strings |
validate / validate / with_validate(bool) | probe the object store at connect to fail fast on bad credentials (default off) |
cache_dir / cacheDir / with_cache_dir | local disk-cache directory for remote-backed tables |
cache_budget_bytes / cacheBudgetBytes / with_cache_budget_bytes | disk-cache budget, in bytes |
cold_fetch_mode / coldFetchMode / with_cold_fetch_mode | how cold-cache misses are serviced |
Limitations
memory:// is ephemeral and can’t be mutated. update/delete need a path or s3:// URI.- Object-storage reads are cached locally. Cold queries fetch from the store; size the cache (
cache_budget_bytes) for your working set.
See also
