GeistBench releases

Releases and changelog

Each release is immutable: its own page, its own JSON file and a summary hash. Releases are never pooled, and scores from different releases are comparable only where the notes say so.

Changelog

Release changelog

Release history and citation
Version	Published	Methodology	Claim	What changed	Links
0.1.0Latest	8 October 2026	category-cluster-mean/1	Diagnostic	Initial diagnostic release: 48 coached tool-contract trials across eight finance and logistics tools, with three repetitions per model. DeepSeek V4.1 Flash passed 24/24; Gemini 3.5 Flash Lite passed 21/24. All three Gemini failures were exact-argument mismatches in the projection-rebuild probe; the rebuild itself succeeded each time. Real API-funded calls used the in-app execution surface, launched through the runtime CLI. Frozen product source: de42f6b4c004181007bfd4b97d87867a2e9659cd. These probes measure supplied-argument adherence, not end-to-end workflow ability or general model quality. No provider ranking or latency comparison. Later write-safety changes and workflow trials are excluded. Recorded cost: USD0.208726, a catalog estimate rather than a reconciled invoice.	Open JSON

Citation

Cite GeistBench

Cite a specific release where possible: its page, version and summary hash identify exactly what you are referring to.

Cite this release

BibTeX

@misc{geistbench-0-1-0,
  title = {GeistBench release 0.1.0: observed results},
  author = {{Geist}},
  year = {2026},
  url = {https://geist.company/geistbench/releases/geistbench-b12-contract-20261008-diagnostic},
  note = {Summary hash bcd88ab15069cf9adc01846328722f7af64e6bff8741f4b6b39b93e64aad08a9}
}

Plain text

Geist (2026). GeistBench release 0.1.0: observed results. https://geist.company/geistbench/releases/geistbench-b12-contract-20261008-diagnostic. Summary hash bcd88ab15069cf9adc01846328722f7af64e6bff8741f4b6b39b93e64aad08a9.

Summary hash: bcd88ab15069cf9a
Permanent link: https://geist.company/geistbench/releases/geistbench-b12-contract-20261008-diagnostic