Benchmark Commands¶

Run coding harness benchmarks across multiple agent targets.

benchmark run¶

Run benchmark tasks across one or more targets.

superqode benchmark run <tasks.json> [OPTIONS]

Arguments¶

Options¶

Examples¶

superqode benchmark run tasks.json --target superqode --target opencode --target pi --target deepagents

Tasks File Format¶

The tasks file is a JSON array of task definitions:

[
  {
    "id": "task-001",
    "prompt": "Implement a fibonacci function in Python"
  },
  {
    "id": "task-002",
    "prompt": "Write a markdown parser in TypeScript"
  }
]

Each task contains a unique id and a prompt sent to each target. Results are compared across targets by task ID, producing a side-by-side report of completion rates, time, and output quality.

Supported Targets¶