#evaluation

6 pages search “evaluation” ↳ all tags

Wiki 6

Evaluating Quantized Models for Deployment ByteShape on why perplexity, KLD, and BPW don't rank quantized models for deployment
JSON-Pass vs Value-Accuracy Gap For LLM structured output, JSON parse rate clears 95%+ on every frontier model, but exact leaf-value accuracy sits 15-30 points lower — schema-compliance benchmarks have been hiding a third of the actual error rate
Reasoning Prefills on Open Models, v1.1 A reasoning-prefill test where Qwen3.8 follows GPT-5.5 Pro's trace far more than other open models, read as a sign of GPT distillation
Reward Hacking in the Wild 3,607 reported agent misbehaviours, LLM-classified into fourteen categories, with the caveats
Structured Output Benchmark (SOB) Interfaze's open benchmark for LLM structured output across text/image/audio with seven metrics; the load-bearing finding is JSON-Pass beats Value-Accuracy by 15-30 points on every frontier model
Which LLMs Write Alike Typebulb notebook that maps which LLMs write alike by cross-entropy over character trigrams