👉 GnollBench is the Gnoll Overseer's own AI benchmark.
📊 What GnollBench Is For
In a permadeath roguelike, a single confident hallucination can end a promising run. GnollBench tests AI models under the exact instructions and lookup tools of the live Gnoll Overseer chat, using a two-provider AI grading panel and a source-grounded fact-checker to penalize dangerous mistakes.
GnollBench ranks models on intelligence, speed, and cost to select the best ones to offer players. It also finds missing wiki facts and tool issues so that they can be fixed.
🐺 GnollBench at a Glance
| Item | Content |
|---|---|
| Questions | 18 in the standard suite |
| Graders | Two AI graders from two different AI providers |
| Results | Intelligence, speed, and cost |
| Who runs it | Administrators |
📖 Guides
| Guide | For whom | What you get |
|---|---|---|
| Introduction to GnollBench | Newcomers | What GnollBench is and why it exists, in plain language |
| Advanced Guide to GnollBench | Interested readers | How scoring, the grading panel, and repeated runs work |
| Technological Overview of GnollBench | Readers who know AI evaluation | Formulas, statistics, and comparability rules |
| Comparison of GnollBench and Popular AI Benchmarks | Anyone comparing benchmarks | How GnollBench compares with popular public AI benchmarks, and how they complement each other |
📖 See Also
- Gnoll Overseer — All Gnoll Overseer guides in one place