Accuracy benchmark

OfficeQA Pro, Databricks' benchmark for grounded reasoning over documents.

corpus
1,435 U.S. Treasury documents · 90 questions
scoring
exact match

Results

  1. 1 Supermodel + Claude Opus 5.5 accuracy: 73.3%
  2. 2 Mixedbread Toast + GPT-5.6 Sol accuracy: 70.0%
  3. 3 Databricks Genie + Claude Fable 5 accuracy: 60.0%
  4. 4 Snowflake Cortex Agent accuracy: 57.8%
  5. 5 Claude Opus 5.5 alone accuracy: 55.6%
  6. 6 GPT Astra accuracy: 49.6%
  7. 7 Claude Code + Claude Fable 5 accuracy: 45.6%
  8. 8 Codex + GPT-5.6 Sol accuracy: 33.3%
  9. 9 Claude Code + Claude Sonnet 5 accuracy: 15.6%

Cost and time per question

  1. 1 Supermodel + Claude Opus 5.5 cost per question: $0.63 accuracy: 73.3%
  2. 2 Mixedbread Toast + GPT-5.6 Sol cost per question: $1.15 accuracy: 70.0%
  3. 3 Databricks Genie + Claude Fable 5 cost per question: $4.00 accuracy: 60.0%
  4. 4 Codex + GPT-5.6 Sol cost per question: $4.70 accuracy: 33.3%
  5. 5 Claude Code + Claude Sonnet 5 cost per question: $5.01 accuracy: 15.6%
  6. 6 Claude Code + Claude Fable 5 cost per question: $37.36 accuracy: 45.6%

Questions

What is Supermodel?

Small models that run next to your data and pass the frontier model only what matters.

How much does Supermodel add to the same model?

Claude Opus 5.5 scores 55.6% on its own and 73.3% with Supermodel, 17.7 points higher, in 74 seconds a question against 156.

Why is Snowflake Cortex Agent missing from the cost board?

Its accuracy, 57.8%, is the only figure we have for it, so it is ranked on accuracy alone.

#1 in accuracy. Try it on your own data.

or run it yourself: pip install infino · Apache-2.0 · quickstart →