Export the pairs#

A contrastive pair is one positive text and one negative text. The chosen side of each qualified choice is the positive, the rejected side is the negative, and the trait name is the category. ster is the Wisent toolkit that turns such pairs into a steering vector: a direction added to a model's residual stream during generation.

Exporting your own pairs needs an account: an anonymous voter is refused with sign in to export your pairs. An operator can export any voter with --voter and the voter id that preferences profile --json prints. A category that does not hold text is refused with its kind named.

Your own pairs, with an account token

preferences export --arena wisent --category tagline --format ster-pairs > pairs.json

Train and use the vector#

Ster assumes no generation number, so the strength, token budget, temperature and seed are yours to state; ster refuses a command that leaves one out and names it.

Train one steering vector from the pairs

ster train --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --pairs pairs.json --layers 12..20 --method caa --output taste.json

Generate with the vector applied

ster generate --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --vector taste.json --strength <S> --max-new-tokens <N> --temperature <T> --seed <SEED> --prompt "Write one line of copy for a product that ranks generated work by what people prefer."