From a generator run to a ranked arena

Submit a run#

A submission is one generator run inside one category. submit reads the run manifest where it lies, uploads every candidate with its own attributes, and keys the submission by the generator, the absolute run directory and the sha256 of the manifest bytes. Submitting the same run twice is refused rather than duplicated.

submit also reads wisent-marketing-asset-generator runs, and a plain JSON file of items with --generator text-file, where each candidate is a line of text.

preferences category create --arena wisent --id logo --name Logo --kind image

preferences submit --manifest ~/wisent-logo-generator/runs/preferences-seed/latest/manifest.json --arena wisent --category logo --generator wisent-logo-generator

Rank and accept the winner#

Until the leader's 95% interval lies above every other candidate's, rank exits 69 — not ready yet; the same command succeeds once more votes arrive — and names what is still open: submission <id> has <n> qualified comparisons; the leader's 95% interval still overlaps candidate <id>'s, so no winner is declared yet. A candidate without a ranking row is named the same way. --provisional prints the current order anyway. Once the leader is separated the answer carries the winning candidate id, which is what the generator's own accept command takes.

preferences rank --submission $SUBMISSION --json

winner=$(preferences rank --submission "$SUBMISSION" --json | jq -r .winner.candidate_id)
wisent-logo-generator accept --run ~/wisent-logo-generator/runs/preferences-seed/latest --candidate "$winner"

Hill-climb a user interface#

People cannot say what makes a screen look good, but they pick the better of two. A climb keeps one incumbent HTML page in an html category; each round sets a copy of it, named incumbent, against challenger variants as one submission under the generator climb:<id>, and once the vote separates a leader the round is decided and the leader is the next incumbent, or the incumbent held.

advance reads the round's ranking as rank does. It changes nothing until the leader's 95% interval clears every other's and exits 69 meanwhile, naming what is still open; then it records the winner, writes its HTML to --out, and takes the round's items out of future pairs. Refused: a category that is not html, a challenger named incumbent or two with one name, a directory without .html files, and a second round while one is open. On the site, an html category's management panel has Start climb, Open round, Propose challengers and Advance climb, and /climbs/<id> draws the incumbent and lists every round.

While a round is open, climb show also prints where it stands: each variant's score, 95% interval, comparisons and wins, and whether a leader has separated; its --json carries that as standing, null with no round open. GET /api/climbs/<id>/standing answers the same for anyone who may read the arena, 409 with no round open, and /climbs/<id> draws it under the incumbent, so a voter or a manager sees how far the round is from Advance climb deciding it.

preferences category create --arena wisent --id landing --name "Landing page" --kind html
preferences climb start --arena wisent --category landing --name "Landing hero" --seed hero.html

preferences climb propose --climb $CLIMB --model MODEL --out variants/ "bolder headline" "more whitespace"
preferences climb round --climb $CLIMB --variants variants/

preferences climb advance --climb $CLIMB --out hero.html
preferences climb show --climb $CLIMB

Read what happened after a failed graphical request#

In an html category's management panel, Read climbs reads the current category collection. Start climb is unavailable until that first collection read succeeds. Open round, Open round with these and Advance climb do not interpret a failed or lost response as proof that nothing committed: they read GET /api/climbs/<id>, show the actual observed incumbent and open round, and update the controls from that response. The notice names the failed operation and its cause. An opening failure remains on the climb's controls even if the proposals form disappears because a round is now open.

If the diagnostic read also fails, current state is explicitly unknown, both causes are shown and the previous incumbent is labeled last observed rather than current. A failed Start climb creation attempt reads the category's current collection and shows its actual entries; it does not identify a creation receipt by matching a climb name. A local seed or challenger file-read failure names that operation and says no mutation request was attempted, rather than suggesting an unknown server commit. A proposal failure is labeled as a failed proposal request, not as evidence that an existing preview or a committed round vanished. These reads do not retry the mutation. Successful creation and round responses update the displayed climb directly without depending on another collection read.

Reading the collection again supersedes its older read. A round response cancels a collection read started before that response, so it cannot later restore the older incumbent or round controls. Leaving the panel releases its pending collection read; cancelling a superseded read is not a mutation retry.

Confirmed creation clears both the seed state and the browser's file input, so the same HTML file can seed another climb. Name and seed inputs are held while creation is pending. Challenger file selections belong to the no-open-round form, not the climb's whole lifetime: observing an open round discards the draft even after a lost response. After that round is decided, the new form has no invisible selection left over from the previous round. A local file-read failure keeps its draft available and does not trigger a diagnostic network read for a mutation that was never attempted.

Run the real climb lifecycle#

This journey creates an isolated arena through the real CLI, checks wrong-category and missing, reserved or duplicate challenger-identity refusals without changing the incumbent or opening a round, records actual anonymous votes, observes the winning incumbent and its HTML/history, opens the next round, and removes the arena. It confirms removal through an HTTP not-found response rather than treating any failed read as absence. Commands, exit statuses, HTTP receipts, local source revision/patch/hashes and fixture artifacts remain in .next/tests/. The report identifies the deployment URL; it does not establish its remote source revision or qualify the rendered UI. Use a candidate deployment built from the recorded source for release qualification.

PREFERENCES_URL=… PREFERENCES_TOKEN_ROLE=… node --test tests/climb/journey.test.mjs

Keep taking the next step#

With an open round that has no separated leader, step changes nothing and exits 69, naming the qualified comparisons. Once a leader is separated it decides the round, writes the new incumbent to --out and opens a new round with one proposed variant per direction. With no open round it opens one. People still decide every round. JSON reports decided and opened.

If proposing or opening the next round fails, step reads the current climb and reports its observed incumbent and open round, plus any preceding round this step already decided and the actual failed operation and cause. A failed state read is explicit: current state is unknown, both causes are reported, and the last observed incumbent is labeled as such. The API's status and exit code are preserved. An accepted round is not described as rolled back, and a lost response is not proof that no round opened. --out without a file path is refused before any API or model call.

preferences climb step --climb "$CLIMB" --model MODEL --out hero.html "bolder headline" "more whitespace"