A $500 RL Fine-Tune That Beat the Frontier

Fermisense published a long case study on training a small open model to do one job better than any frontier model can. The job is e-commerce catalog review: given a listing with images, title, description, claimed brand, and region, put it in the right taxonomy category, extract the attributes that category requires, and decide whether it violates policy. The result is a GRPO fine-tune of a 9B open-source model (Qwen3.5-9B) that reached 87.3% of the achievable score against 76.9% for the best frontier configuration tested, at $0.50 per 1,000 listings against $34 for the strongest frontier model.

Fermisense sells this as a service, so the article ends in a booking link and every number is theirs. What makes it worth reading anyway is that the numbers are unusually complete: the environment, the reward function, the hardware bill, the training duration, the step at which the frontier was passed, and both the strict harness score and the looser training-monitor score are all published.

The environment is the deliverable

The training setup is a "digital twin" of the workflow rather than a dataset of examples. Amazon Berkeley Objects — real product images and listing data — was turned into 177,767 review episodes, one listing each, with controlled policy cases, mismatched images, conflicting brand claims, and deliberately legitimate claims planted as hard negatives, so every episode has a known correct answer.

Inside the twin the model gets three tools: search_taxonomy over roughly 13,000 categories, lookup_brand for whether a brand is registered and protected, and get_attribute_schema for the required attributes of a chosen category. It then commits a category, the attributes, and a policy verdict. A scorer grades each episode on

reward = 0.3·category + 0.3·attributes + 0.4·policy − tool_overage

with the penalties encoding business priorities directly: a missed violation costs 7× a false alarm. That asymmetry is the part that cannot be prompted reliably, and it is doing a lot of the work in the final gap.

Benchmarking the frontier first

Before training, five frontier models — GPT-5.5, GPT-5.6-sol, Gemini 3.1 Pro, Claude Opus 4.8, and Claude Fable 5 — were run on 200 stratified validation episodes with identical tools, images, scorer, and turn budget, in two configurations: a plain prompt, and one with 2,800 characters of optimized instructions covering extraction conventions, lookup procedure, and worked examples.

Two things came out of that. The optimized configurations converged to within a tenth of a point of each other, which is the shape of a ceiling rather than a ranking — and the strongest zero-shot extractor, Gemini, actually got worse with the optimized instructions. And the instructions were not free: they inflated input-token bills by 28–55% depending on the model, on every call, forever. Fermisense calls this the prompt tax, and it is the sharpest framing in the piece: prompted task knowledge is rented per call, trained task knowledge is bought once and sits in the weights.

Their diagnosis of the gap is not that frontier models are less intelligent. A frontier model starts every episode from zero — it has never seen this store's taxonomy, its inventory conventions, which attribute values count as supported, or how the platform wants corner cases resolved, and has to reconstruct all of it from the prompt on every single call. Instructions can compress some of that. The corner cases that decide the score are exactly the ones no instruction list can enumerate.

The training run

Two rented RTX PRO 6000 GPUs, one generating rollouts and one applying gradient updates, with the open-source prime-rl framework. 1,000 optimizer steps, about three and a half days, roughly $500 in GPU time.

Most of that was not needed. The model crossed the frontier band after roughly 250 steps — about a day — and the remaining 750 steps were spent squeezing. The W&B monitor climbs from about 0.50 to 0.671 at step 1,000 under sampled decoding; the strict benchmark harness scores the final adapter at 0.626, which is the 87.3% figure, about ten points above the best frontier configuration and 36% above its own untrained base at 64.2%.

The cost side

That is 40× cheaper than the cheapest frontier option, 68× cheaper than the strongest, and ~340× cheaper than the most expensive — while scoring higher than all of them. At Shopify-scale volume, roughly 40 million decisions a day, the gap between $34 and $0.50 per thousand is about $500M a year versus $7M.

Fine-tuning added roughly 23 points over the base 9B at the same ~$0.50 price, which is the actual claim: not that small models are cheap (everyone knows that) but that on this workload the usual quality-versus-cost trade-off disappears entirely.

The pattern elsewhere

The article's supporting evidence is a set of other companies' self-reported results, each linked to its own write-up. Bridgewater trained an open model on labels from its own investors and reports ~30% fewer mistakes than the best frontier model. Harvey ran RL on an open-weight model for legal due diligence and memo drafting and reports a legal agent that outperforms GPT-5.5 and Claude Opus 4.8 on its own rubrics. Intercom post-trained a vertical support model, Fin Apex, on billions of customer-service interactions and reports higher resolution at lower cost. Shopify classifies products with fine-tuned open models at ~40 million inferences a day, a volume it says commercial APIs cannot economically serve.

An appendix collects eight more with the same shape — AT&T summarizing 900k support calls a day at 17% better personal-data detection than GPT-4o, LinkedIn's candidate matching 75× cheaper than GPT-4, OpenPipe scoring 93% on support QA where o3 scored 50% at 64× lower cost, Checkr's background-check classifier beating GPT-4 on the hardest cases at 5× cheaper and 30× faster. Cognition's entry in that table is swe-1-7, which this wiki covers separately — and reading the two together is instructive, because the SWE-1.7 page catalogues exactly the caveats this article's appendix does not mention: benchmarks run by the vendor, on the vendor's own harness, sometimes on the vendor's own benchmark. Every row here is a company's own reported figure against the model it was replacing. The direction is consistent enough to believe; the magnitudes are not independently checked. The same caution structured-output-benchmark raises applies — one number can flatter a model depending on which axis it measures.

When it is the wrong tool

Two questions settle most cases: how often the task runs, and whether the outcome is verifiable. Fine-tuning pays in the corner where work is both frequent and checkable — routing tickets, extracting fields from documents, checking submissions against policy, classifying products, approving or flagging transactions. Verifiable but rare work is better served by a prompt-optimized frontier model. Non-verifiable work keeps a human in the loop. And when the problem is changing facts rather than judgment, retrieval is the fix at any volume — RAG and RL solve different things, and the twin keeps prices, inventory, and policy text in tools rather than in weights for exactly that reason.

The stated test is blunt and good: if a decision can be scored, a model can practice it; if it can only be debated, it cannot.

A footnote on how owning the endpoint feels

Matthew Saltz's short post from the same week is the ergonomic version of the same shift, and it is worth reading as an anecdote rather than as evidence. He works at Modal, which had just launched managed kimi-k3 endpoints, and rather than upgrade a personal Claude plan for a side project he pointed opencode at his own endpoint — about five minutes of work. What he reports is not a benchmark but a feeling: "I own the endpoint, and my data just goes from my laptop to there and back. It feels like it's mine." He compares it to opening vim after a big fancy editor.

That is one developer, one evening, on a hosted endpoint he did not have to operate, and it says nothing about quality or cost at volume. But it lands on the same argument the Fermisense piece makes at the end of its checklist — that sensitive data never leaving your boundary is a safety property, not just a preference. local-ai-is-not-opus and titit-local-ai make the same case from the self-hosting side, and both are more careful than the anecdote about what open models still do not do well.