When you publish a result through L80, you do not send us a page. You send us data — a JSON object — and a template on our side turns it into HTML. That is what makes a published page safe to serve: there is no user-supplied markup anywhere in the pipeline, so there is nothing to sanitize and nothing to get wrong.
It also has a consequence people notice immediately. A template is a shape. If your result has that shape, the page is better than something you would have hand-written. If it doesn't, the page is worse. There is no middle path where you nudge the layout, because you never had the layout to begin with.
So the shape matters enormously, and we picked one on purpose: BetterBench.
What BetterBench actually produces
Most inference benchmarks print one average tokens-per-second at batch size 1. BetterBench refuses to, and that refusal is the whole reason it was worth building a template around.
It reports distributions
Per category, per layer: TTFT p50 and p99, decode median with its IQR, inter-token latency as a 1%-low / median / 99%-high band. The 1% low is what a stutter feels like. An average hides exactly the thing you were measuring.
It admits when it doesn't know
Every percentile resting on too few samples is marked, and the shortfall is written into results.json rather than recomputed at render time. A p99 over 20 runs is labelled for what it is.
There is a third wrinkle. On a stack doing speculative decoding — MTP, EAGLE, Medusa — several tokens arrive in one network write. There is no time between them, so a per-token latency figure would be invented. BetterBench reports the gap it measured instead: update p50, update p99, tok/update. Different columns entirely, for the same question.
The same run, published two ways
Take one BetterBench run — Qwen3-8B on vLLM, eight categories, twenty measured passes. Here is what happens depending on which template you name.
bench.report.v1fits- Categories render as a table, so p50 and p99 sit in the same row and can be read against each other.
- Columns are chosen from what the rows actually contain — a speculative stack gets
update p50, a plain one gets the ITL band, neither gets a grid of dashes. undersampled: trueputs a dagger on the row and a footnote under the table.- Caveats render above the numbers, not below them.
- Model, engine, hardware, harness version and corpus version get a dedicated spec block, so the run is reproducible from the page.
generic.report.v1flattens- There is no table. Twelve flat metric tiles, maximum — and one category alone needs six numbers.
- Eight categories × six figures is 48 values competing for 12 slots. Everything else goes into prose or gets dropped.
- p50 and p99 become unrelated tiles. Nothing tells the reader they describe the same measurement.
- No sample-size concept at all. An under-sampled p99 reads as plain fact.
- No spec block. The model and engine end up in the summary paragraph, if the author remembers.
See both for yourself, rendered by the real pipeline: bench.report.v1 · generic.report.v1
The failure here isn't ugly formatting. It is that a reader of the flattened page cannot tell a solid p99 from one computed over five runs — and BetterBench went to real trouble to make that distinction visible. The generic template throws away the tool's most careful work.
Now the other direction
The specialised template is not a free upgrade. Point it at something that isn't a per-category inference benchmark and it is worse than useless — it rejects the payload outright.
Say you ran a retrieval eval: recall@5, MRR, a confusion matrix, and four paragraphs about which queries failed. bench.report.v1 requires a system.model and organises everything around latency percentiles and concurrency levels. Your eval has none of those. It has accuracy, which the template has no column for, and prose, which it treats as an afterthought.
So that result goes through generic.report.v1, and gets a page that is honest and readable and slightly plain. Title, summary, a row of metric tiles, prose sections. Nothing lost, nothing special gained.
The known limitation, stated plainly
Artifact publishing renders BetterBench output well and everything else adequately. There are two templates. One is shaped around a specific tool's specific output. The other is a generic fallback that will never show a distribution properly, because it has no table and no concept of a sample size. If your result is not an inference benchmark, you are using the fallback.
We would rather say that out loud than let people discover it by publishing a page that quietly misrepresents their numbers. It is a real limitation and it is the honest description of where the system is today.
Why the limitation is temporary
The template registry exists precisely so this does not stay true. Adding a template means writing a JSON Schema, writing a render function, and adding one line to the registry. The CLI is untouched. The auth path is untouched. The publish route is untouched. The skill gains a paragraph.
Three files. That is the entire cost of teaching the system a new kind of result — and because rendering is a pure function of the stored payload, improving a template later re-renders every page ever published under it, with nobody having to republish anything.
If the thing you measure doesn't fit either template, that is useful information and we would like to hear it. The next template gets picked by whatever people actually keep trying to publish.