#scaling-laws

2 pages search “scaling-laws” ↳ all tags

Wiki 2

  • Factual Capacity Scaling Stored factual knowledge scales log-linearly with parameters and does not compress — the Shannon-entropy floor that survives the Densing Law
  • Incompressible Knowledge Probes (IKP) Bojie Li's IKP benchmark estimates closed-model parameter counts from factual capacity, R²=0.917 calibration; falsifies the Densing Law for stored knowledge; puts GPT-5.5 at ~9.7T effective params