UmbonoOpen evaluation studioOpen studio

Model decisions, made inspectable

Know which model wins.

Run comparable prompts, inspect operational tradeoffs, and score every output with a rubric your team can explain.

Open studioView on GitHub
Evaluation resultCommunity guidance review
Weighted / 5
  1. 01
    Cedar Reasoner1880 ms p95
    4.48
  2. 02
    Baobab Balanced1540 ms p95
    3.97
  3. 03
    Marula Fast790 ms p95
    3.88
  4. 04
    Karoo Compact1030 ms p95
    3.28
Deterministic demo fixture12 scored outputs

No account required. The demo runs entirely in browser memory.

Bring one API key. Live mode supports OpenAI-compatible endpoints.

Audit the method. Scoring and aggregation are tested pure functions.

From prompt to decision, without the spreadsheet.

Umbono keeps model output, operational metadata, human judgment, and the resulting rank in one reproducible flow.

Configure

Choose an evaluation set, prompt fixture, and up to four models.

Compare

Run requests in parallel and inspect each response beside latency, tokens, and cost.

Score

Apply a weighted rubric that keeps human judgment visible and reviewable.

Decide

Rank quality and operational tradeoffs using calculations defined in the repository.

A real evaluation workflow, with an honest demo boundary.

The included fixtures prove the product without credentials. Live mode swaps in your configured models while keeping keys on the server.

Try the evaluator
Human rubric100%

Explicit weights across clarity, usefulness, creativity, and values alignment.

Parallel runs1-4

Models per comparison, bounded for predictable local use.

Provider supportOpenAI-compatible

Works with hosted gateways and local compatible servers.

Clone to first run

A five-minute setup that starts in demo mode.

Clone, install, and open the studio. Add provider variables only when you are ready to spend live tokens.

Read the README
$ git clone https://github.com/Schramm2/umbono-dashboard.git
$ cd umbono-dashboard
$ npm ci
$ npm run setup
$ npm run dev

Run the same question through more than one lens.

Start with deterministic fixtures, then connect your provider when the workflow makes sense.

Open studioFork the project