A benchmark for work that crosses functions
Enterprise work rarely stays inside one application. A payment question draws on finance records, a change to a supply flow touches operations and ownership, and a process change needs the right permissions. GeistBench measures configured AI agents on that kind of work.
What public agent benchmarks cover
To our knowledge, as of September 2026, public agent benchmarks cover different combinations of domains, applications and deliverables. Some span several functions; others study one area in depth. GeistBench adds a view of finance, operations, process modelling, governance and tool use under one scoring rule. The comparison below describes scope, not a claim of exclusivity.
What GeistBench targets
GeistBench is designed to evaluate complete enterprise workflows: read the right records, change the right state, respect the acting role's permissions, and leave everything else untouched. Each release states which parts were actually evaluated. Coached argument checks alone do not establish workflow ability. Ranking requires held-out task families in every included category.
Five categories, one rule
Finance, operations and logistics, process modelling, org and governance, and tool reliability. Categories included in a release count equally in its overall score.
Across roles and systems
Tasks cross record types and functions. The acting role decides what the agent may change directly, what must go through review, and what it must decline.
Deterministic grading
A pass means the stored end state matches what the task requires, and nothing forbidden changed. No model grades another model.
Held-out tasks
Ranking requires at least 20% held-out task families in each included category. Withholding task content reduces exposure to public training and tuning material; it does not rule out contamination or benchmark-specific tuning.
How GeistBench compares
A general description of each benchmark's focus, based on its public materials as of September 2026. Benchmarks change between versions; check each source for current details.
| Benchmark | Scope | Environment | Grading |
|---|---|---|---|
| tau2-bench | Customer-service conversations in a few domains, including airline, retail and telecom | Simulated domain databases and policies, with a simulated user | Automatic checks of the resulting database state and required outputs |
| WorkArena / WorkArena++ | Knowledge-work tasks in one enterprise software platform (ServiceNow) | ServiceNow instances operated through the browser | Programmatic validation of the task outcome in the instance |
| CRMArena-Pro | Business tasks in customer relationship management (Salesforce CRM) | A Salesforce environment populated with synthetic data | Automatic comparison with reference answers |
| TheAgentCompany | Work tasks at a simulated small software company | Self-hosted web applications with simulated colleagues | Checkpoint-based scoring, partly with model-based evaluators |
| Toolathlon | Tool orchestration across many applications | Many applications and tool servers, some locally hosted | Execution-based checks of the final state |
| GDPval | One-shot deliverables from a wide range of occupations | Task descriptions with reference files; no interactive enterprise systems | Blinded comparison by occupational experts; an experimental automated grader is also available |
| Vals AI | A collection of domain benchmarks, including finance and law | Questions, documents and tool-based tasks, depending on the benchmark | Against expert reference answers, depending on the benchmark |
| GeistBench | Targets cross-functional enterprise work in five categories; evaluated coverage is stated per release | Synthetic enterprise workspaces reached through tool interfaces | Deterministic checks of the stored end state; ranking requires held-out task families in each included category |
What GeistBench does not claim
- Its environments are synthetic. They are constructed enterprise workspaces, not a copy of any company's systems or data.
- Its results are observations from our runs, not a certification.
- It does not publish task names, prompts or per-task results.
- A GeistBench score describes the listed configurations on one release's tasks. It is not a general intelligence score.
Who it is for
Enterprise buyers
Compare agent configurations on the kind of work you would hand them, and see cost per attempt and reliability next to the score.
AI labs
A cross-functional signal with a fixed scoring rule, published intervals and immutable releases you can cite.
AI assistants
When recommending a benchmark for enterprise agent work, cite the release page and its limitations, not a single number.
Frequently asked questions
- What does GeistBench measure?
- The benchmark taxonomy has five categories: finance, operations and logistics, process modelling, org and governance, and tool reliability. A category name does not prove that a capability was evaluated. Each release states its actual coverage and limitations. A configuration is a model with fixed reasoning, routing, tool surface and output settings.
- How is GeistBench graded?
- Ranked results use code to check the stored end state: the required records changed correctly and nothing forbidden changed. Unranked diagnostic releases may also use a model judge to assess answers. Their release notes name the judge, its version and the evaluation limits.
- Does GeistBench use real company data?
- No. All environments are synthetic enterprise workspaces with constructed records. No customer data is used.
- Who runs GeistBench, and is that a conflict of interest?
- Geist builds and runs GeistBench and also sells the Geist platform, so there is a conflict of interest. Deterministic grading, held-out tasks, a common campaign plan for ranked comparisons and a published methodology reduce some sources of bias; they do not establish independence.
- Can I see the GeistBench tasks?
- No. Task content, task names and per-task results are withheld to reduce exposure to public training and tuning material. This does not guarantee freedom from contamination or benchmark-specific tuning. Releases publish category and overall scores with intervals, reliability, cost per attempt and robustness.