#llm

159 pages search “llm” ↳ all tags

Wiki 138

0xsid 0xsid's personal blog at 0xsid.com — short opinion posts on agentic coding workflows and developer experience
2 OLMo 2 Furious AI2's OLMo 2 report, fully open 7B/13B/32B models on up to 6.6T tokens with a stability fix list, Dolmino mid-training and the Tülu 3 RLVR recipe
80 tok/s + 128K Context on 12GB VRAM — Qwen3.6 + MTP janvitos's RTX 4070 Super config — Qwen3.6-35B-A3B MTP UD-Q4_K_XL on 12GB via `-fitt 1536`; thread also documents turboquant+MTP on a GTX 1070
A $500 RL Fine-Tune That Beat the Frontier A $500 GRPO fine-tune of a 9B open model beat every frontier config on catalog review, 68× cheaper
A recent experience with ChatGPT 5.5 Pro (Gowers) Tim Gowers reports ChatGPT 5.5 Pro producing an "original and clever" PhD-level result in additive combinatorics in under two hours, with no mathematical input from him
A Safe Path to Open Weights Thinking Machines Lab's release framework for open weights - test the model, stage access, filter dangerous knowledge - applied to its Inkling models
Abliteration Removing an LLM's refusal behavior by identifying and suppressing the refusal direction in activation space
Agentic Coding is a Trap Lars Faye on the supervision paradox — agent oversight requires the very skills that agent use erodes, plus vendor lock-in and token cost volatility
Agentic Coding is Burning Me Out 0xsid on decision fatigue as the next invisible friction — gacha-loop psychology, broken pacing, and the verifier-of-the-verifier regress
Agentic Search Search an agent decides when and how to run, across many context sources each with its own native search tool
Agentic Search for Context Engineering Leonie Monigatti's case that context engineering is ~80% agentic search, with three demos on what each search tool can and can't do
An Empirical Study on Strong-Weak Model Collaboration for Repo-level Code Generation CMU compares strong-weak model splits on SWE-bench Lite; the strong model writing first wins, and no strategy tested is a review stage
antirez (Salvatore Sanfilippo) Creator of Redis; writes prolifically about programming with LLMs; building DwarfStar, a local inference engine
Are Open Models Catching Up? SemiAnalysis reruns era-specific benchmarks on open and closed LLMs and finds catch-up time roughly halving each era, from 19.7 to 4.8 months
arkaung.github.io arkaung's GitHub Pages site with interactive math tutorials; Interactive TurboQuant is the first ingested piece
Armin Ronacher (lucumr.pocoo.org) Armin Ronacher's blog at lucumr.pocoo.org — Flask creator, Sentry co-founder, currently building Pi; long-running posts on Python, infra, and (recently) LLM tooling
Average Is All You Need Rawquery's thesis that LLMs make "average" output cheap, plus a walkthrough of agent-driven email-campaign attribution in five minutes
Awesome DeepSeek Agent (deepseek-ai curated list) DeepSeek's official curated list of 16 AI agent and coding-assistant tools that ship step-by-step integration guides for DeepSeek-V4-Pro and DeepSeek-V4-Flash
Being Linux Torvalds antirez: Linus's rare move was quitting code to hold the design, the role LLM work now asks of you
Benchmarking Opus 5 on SlopCodeBench Dex Horthy runs three Claude models through SlopCodeBench; the best, Opus 5, clears 24% strict pass
Better AI Code Comment Detector kqr rebuilds a calibrated human-vs-LLM code-comment classifier on public data; 77% balanced accuracy, POS-tag n-grams as the strongest features
BF16 vs FP16 BF16 keeps FP32's 8-bit exponent and shrinks the mantissa; FP16 shrinks the exponent and keeps the mantissa — for neural training, range beats precision
Build Your Own OpenClaw 18-step Python tutorial for building a Claude Code-style coding agent from scratch
Claude Code Anthropic's terminal coding agent — the tool that maintains this vault, and the thing most of the toolbox extends
Clean Code in the Age of Coding Agents Code structure matters more with agents — context windows are finite, messy code wastes tokens
Code Mode Token Savings One script against 26 tool calls on agent-swarm's own production data — a self-measured 99.2% cut
Coding-Agent Tests of Local LLMs on 16 GB VRAM Vyacheslav's RTX 5070 Ti tests of Gemma 4 / Qwen 3.6 / Qwen Coder via OpenCode + llama.cpp — Gemma wins 12/12, Thinking mode hurts rule-following
Consent in Crisis: The Rapid Decline of the AI Data Commons Longpre et al.'s audit of 14,000 domains behind C4, RefinedWeb and Dolma - AI crawl restrictions surged in 2023-2024, cutting off the best web data
Context Engineering Assembling the right prior state into an LLM's context before asking it to do anything
Control the Ideas, Not the Code antirez argues line-by-line review of LLM code is mostly wasted effort — own the design, spend the hours on QA and direction
Controllable Thinking Style Training-injected markers that toggle the style of a reasoning model's chain-of-thought without changing the answer pipeline
Cross-model code review Who should check an LLM's code, and why the model that wrote it reviews poorly, why the reviewer must be able to solve the task, and when a weaker writer pays
DeepSeek Chinese AI lab whose V3/R1/V4 model line drove the open-weights reasoning-model cost-collapse story of 2024-2026
DeepSeek V4 Flash 0731 scores 50 on the Artificial Analysis Intelligence Index, 10 points above previous DeepSeek V4 Flash Artificial Analysis puts DeepSeek V4 Flash 0731 at 50, one point under GPT-5.6 Luna for ~60% less per task and on the cost Pareto frontier
DeepSeek-V4 Roleplay Thinking-Mode Instructions A Chinese-language guide documenting DeepSeek-V4's training-injected markers that toggle the thinking trace between immersive monologue and pure analysis
Detecting and countering misuse of AI: September 2026 Anthropic's threat report on Claude misuse from Dec 2025 to Aug 2026, led by seven Chinese labs said to distill Claude and pass it off as their own models
Disposable code When code is cheap enough to write, fail at, and throw away, maintenance dread stops deterring marginal projects and the ROI threshold shifts
Evaluating Quantized Models for Deployment ByteShape on why perplexity, KLD, and BPW don't rank quantized models for deployment
Factual Capacity Scaling Stored factual knowledge scales log-linearly with parameters and does not compress — the Shannon-entropy floor that survives the Densing Law
Frontiers in synthetic data Lambert's 2024 notes on synthetic data in post-training, from SFT on GPT-4 outputs to Gemini Flash being distilled from Pro
GLM-5.2 is the step change for open agents Nathan Lambert on GLM-5.2 as the first open-weight model that works as a general coding agent, released while Claude Fable 5 was restricted
GLM-5.3: How Chinese labs keep stride with the frontier Lambert on GLM-5.3 and why Chinese labs match US models without relying mainly on distillation, plus Z.ai's staged release for cyber capabilities
GLM5.2 on AMD MI355X Wafer's vendor benchmark claiming 2626 tok/s/node for GLM5.2 on MI355X, and how they got there
Gowers's Weblog Tim Gowers's WordPress blog on combinatorics, mathematical exposition, and (recently) LLM-assisted mathematical research
How distillation is used today and what performance uplift it gives to open models Lambert's July 2026 note that distilled data seeds SFT for Chinese labs but matters less as RL grows, written against a Stratechery claim
How far behind are open models? Håvard Tveit Ihle measures open-model lag on 17 benchmarks, finding 8-10 months on private ones, 4-6 on public, and a gap growing since R1
How much does distillation really matter for Chinese LLMs? Lambert reads Anthropic's February 2026 disclosure against DeepSeek, Moonshot and MiniMax and argues distillation helps but is not decisive
I Still Prefer MCP Over Skills Mohl on why MCP is for service access and Skills are for knowledge — the hybrid pattern
I Want Local Models to Work Armin Ronacher on why local-LLM UX is worse than it should be — fragmentation, missing tool-streaming, no critical mass — and his pi-ds4 / ds4.c bet
Improving Code Generation via Small Language Model-as-a-judge Fine-tuned sub-5B judges pick correct Java code about as well as GPT-4.1-mini; small generator plus judge rivals its 8-33B sibling in 4 of 5 families
In Defense of Not Understanding Your Codebase Goedecke's case that partial understanding is the honest baseline in large systems, and one engineering value traded against others
Incompressible Knowledge Probes (IKP) Bojie Li's IKP benchmark estimates closed-model parameter counts from factual capacity, R²=0.917 calibration; falsifies the Densing Law for stored knowledge; puts GPT-5.5 at ~9.7T effective params
Interactive TurboQuant — Rotate, Then Quantize arkaung's visual explainer of the DRIVE → EDEN → TurboQuant lineage; one rotation turns outliers into a known distribution, one fixed codebook serves every input
Interconnects (interconnects.ai) Nathan Lambert's newsletter on how AI models are trained, reasoning models and post-training, and the open-model race between the US and China
Interfaze AI startup building a deterministic-by-design model aimed at structured output and parsing tasks; published the open Structured Output Benchmark in April 2026
itayinbarr.substack.com Itay Inbarr's Substack on agent scaffolds, small-local-model benchmarks, and the scaffold-model-fit framing of coding-agent evaluation
JSON-Pass vs Value-Accuracy Gap For LLM structured output, JSON parse rate clears 95%+ on every frontier model, but exact leaf-value accuracy sits 15-30 points lower — schema-compliance benchmarks have been hiding a third of the actual error rate
Kimi Delta Attention (KDA) DeltaNet linear attention derived from softmax attention up to KDA's per-key-channel forget gate
Kimi K3 Raschka on Moonshot's 2.8T open-weight K3 — LatentMoE, KDA, attention residuals, NoPE everywhere
Kimi K3: The open-weights escalation Nathan Lambert on Kimi K3 as the first true frontier open-weight model, what it says about Chinese labs, and why banning open weights backfires
KV Cache Sizing KV cache memory per token = 2 × KV-heads × head-dim × layers × bytes/element; halving via q4_0 lets you double context for free
Lars Faye Lars Faye's personal site at larsfaye.com — articles on AI in software engineering, vendor dependency, and developer-skill economics
Let's talk about LLMs (Bennett) Bennett applies Brooks' No Silver Bullet to LLM coding — math caps upside, DORA/CircleCI show instability
little-coder — Scaffold-Model Fit on Aider Polyglot Itay Inbarr shows Qwen3.5-9B Q4_K_M goes from 19.11% (default Aider) to 45.56% (little-coder) on Aider Polyglot by redesigning the scaffold around small-model behavior
LLM API Routing Layer A thin gateway that fronts multiple LLM providers behind one endpoint, with fallback chains, model selection rules, and unified billing
LLM as Average Democratizer The argument that LLMs' main economic impact is making "average" output cheap, lifting the floor without raising the ceiling
LLM Distillation Training one LLM on another's outputs, from SFT on generated text to on-policy KD, and the 2025-2026 fight over Chinese labs distilling US models
LLM Quantization Formats Q4_K_M vs UD-Q4_K_XL vs MXFP4 — three approaches to 4-bit weight compression, with the perplexity gap that newer ≠ better
LLM Wiki as agent-memory substrate Karpathy's wiki pattern is being adopted as the storage layer for agent memory products — markdown you can read and grep, instead of an opaque vector store
LLM Wiki Pattern LLM incrementally builds and maintains a persistent wiki from raw sources
LLM-assisted mathematical research Documented cases where LLMs contribute non-trivial mathematical ideas, and what that does to the research-training pipeline
LLMs: Intelligence vs. Cost Guido Imperiale replots Artificial Analysis's intelligence-vs-cost chart on a linear axis with OpenRouter and local-electricity prices
Local AI is not Opus Alex Ellis on why "local Qwen is near-Opus" is wrong, but a tuned local model still earns its keep in a small software business
Local LLMs Deep Dive — Quants, MoE Offload, REAP, ik_llama, Speculative Decoding Vyacheslav's long-form Habr primer covering BF16, K/I-quants, dynamic quantization, `-fit`/`-cmoe`/`-ncmoe`, REAP expert pruning, ik_llama, MTP/EAGLE3, Linux vs Windows
Mesh LLM: Distributed Inference over iroh Pools GPUs across machines as one OpenAI-compatible API; splits big models across nodes over iroh's p2p QUIC transport
Mixture of Experts (MoE) Architecture where a router activates a small subset of "expert" subnetworks per token — splits total params from compute params and changes the local-inference economics
MoE CPU Offload (`--n-cpu-moe`) llama.cpp's MoE-aware offload — keep attention in VRAM, push inactive expert weights to system RAM via PCIe; +55-60% over naïve layer-based offload
Neurosymbolic AI Combining LLMs with symbolic solvers for language understanding plus exhaustive correctness
Neutrino-1 8B Fermion Research's Apache-2.0 8B: a 3.88 GB ternary-coded container for H100, MacBook, and CPU
Notes from inside China's AI labs Nathan Lambert's May 2026 trip report from Chinese AI labs - student-heavy teams, less ego, Claude everywhere, in-house data, and too few Nvidia chips
Obsidian + Claude Code Complete Guide Huashu's LLM Wiki implementation with three-layer architecture, SCHEMA.md, and seven workflows
Old and new apps via modern coding agents (Tao) Terence Tao ports his 1999 math applets and vibe-codes new visualizations with coding agents, and draws a clear line at where the bugs are tolerable
Olmo The Allen Institute for AI's family of fully open language models, OLMo (2024) to OLMo 2 to Olmo 3, released with data, code, checkpoints and logs
Olmo 3 AI2's Olmo 3 report (Dec 2025), fully open 7B/32B Base, Think, Instruct and RL-Zero models with every stage's data, code and checkpoints released
OLMo: Accelerating the Science of Language Models AI2's first OLMo release (Feb 2024), 1B and 7B models on 2T+ Dolma tokens with weights, data, code, logs and 500+ checkpoints under Apache 2.0
On the Self-Verification Limitations of LLMs on Reasoning and Planning Tasks Stechly, Valmeekam and Kambhampati find GPT-4 self-critique loops lose accuracy while a sound external verifier gains it, even with no critique text
On the Societal Impact of Open Foundation Models Kapoor, Bommasani et al.'s 2024 paper - five properties of open-weight models and a marginal-risk framework showing most misuse studies were incomplete
Open and closed models are on different exponentials Nathan Lambert on coding agents proving users pay a premium for top closed models, while open models take the larger, slower diffusion market
Open models in perpetual catch-up Nathan Lambert on why the roughly six-month gap between open and closed models holds steady, plus trends in adoption, specialization and China
Open Source AI is the Path Forward Mark Zuckerberg's July 2024 letter releasing Llama 3.1 405B, arguing open models are better for developers, for Meta, and for safety against China
Open-closed model gap How far the best open-weight LLMs trail the best closed ones - how it is measured, estimates from 4 to 10 months, and whether it closes
PHK's last Bikeshed: the end of FOSS as we know it Poul-Henning Kamp's farewell ACM Queue column — LLM code review is a fad, but age verification, attestation, and EU accountability end BDFL-style FOSS
Prompt Caching in Agents How KV-cache reuse sets the cost, latency and tool design of a coding agent, and what Pi shows
Pulpie: Pareto-Optimal Models for Cleaning the Web Encoder models that label HTML blocks as content or boilerplate at a twentieth of Dripper's cost
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling EleutherAI's 2023 Pythia suite, 16 models from 70M to 12B trained on the Pile in one fixed order, with 154 checkpoints each for training-dynamics research
Qwen4: The Architecture of the Future (Qwen3.8-Flash-Next) Habr piece from reseller gptunnel on Qwen3.8-Flash-Next, the open-weight preview of Qwen4's architecture; Qwen4 itself is not out
Random-Rotation Quantization Rotate vectors by a random orthogonal transform, then quantize each coordinate against a fixed codebook — outliers dissolve, no per-block metadata, one codebook fits all inputs
REAP — Router-weighted Expert Activation Pruning Cut the least-active experts from an MoE model based on a target-domain calibration dataset; 20% size reduction with most agentic-coding capability retained
Reasoning Prefills on Open Models, v1.1 A reasoning-prefill test where Qwen3.8 follows GPT-5.5 Pro's trace far more than other open models, read as a sign of GPT distillation
Reviewer Capability Governs Rejection Targeting, Not Repair Skill A 100-problem pilot where a cross-family reviewer adds 12 points and self-review, despite the best recall, falsely rejects 35% of correct answers
Scaffold-Model Fit Coding-agent benchmark scores measure model × scaffold, not model alone — small local models inherit responsibilities the scaffold assumed they'd self-manage
Sean Goedecke's Blog Sean Goedecke's blog at seangoedecke.com — staff engineer essays on LLMs, careers, and the inside view of working at GitHub and Zendesk
Simon Willison's Weblog Simon Willison's daily blog at simonwillison.net — short link-and-commentary posts on LLMs, datasette, web tooling, and open-source governance
Skill atrophy and the supervision paradox The circularity where agent oversight requires the critical-thinking and code-reading skills that heavy agent use measurably erodes
Slow Software: The Case for High-latency Systems Development Irene Zhang argues critical infrastructure needs deliberate development friction now that AI decoupled importance from build time
Software Engineering Is About Managing Complexity (hack8s) hack8s.com on why cheap AI-generated code makes engineering judgment and real ownership of a codebase scarcer instead of less needed
Speculative Decoding Predict multiple tokens per forward pass, verify in batch — four llama.cpp variants (draft model, MTP, EAGLE3, ngram-mod) trade off draft quality, memory, and applicability
Stealing Reasoning Traces from Proprietary LLM APIs Encrypted chain-of-thought blocks are replayable across sessions, users and models, so a cheap sibling model will decode a frontier model's hidden reasoning
Steering Is Interesting Again (Goedecke) Goedecke's case that steering vectors are worth a second look now that local frontier models exist, with antirez's DwarfStar 4 as evidence
Steering Vectors Manipulating LLM activations mid-inference to bias output toward a concept, without changing the weights
Structured Output Benchmark (SOB) Interfaze's open benchmark for LLM structured output across text/image/audio with seven metrics; the load-bearing finding is JSON-Pass beats Value-Accuracy by 15-30 points on every frontier model
SWE-1.7 — Cognition's Coding Model Cognition's SWE-1.7, RL-trained from Kimi K2.7, reportedly near GPT-5.5 on coding benchmarks — with the strongest numbers on its own FrontierCode
Synthetic Data & Distillation | RLHF and Post-Training Book by Nathan Lambert Chapter of Lambert's RLHF book on synthetic data, from SFT distillation and on-policy KD to AI feedback, Constitutional AI and rubrics
Telnyx Inference Inference provider serving open-weight models on its own GPUs via an OpenAI-compatible API
Terence Tao Fields Medal mathematician who writes first-hand about using LLMs and coding agents for math exposition, tooling, and research
The ATOM Project: American Truly Open Models Nathan Lambert's 2025 memo arguing the US lost open-model leadership to China and needs several 10,000-GPU labs building fully open models
The ATOM Report: Measuring the Open Language Model Ecosystem Lambert and Brand's 2026 adoption study of ~1.5K open models - China passed the US in downloads in mid-2025, Qwen dominates, plus a size-normalized metric
The cheap reverse-engineering of home devices Willison on why cheap agent-written code flips the ROI of reverse-engineering home devices — the unlock is disposability, not possibility
The distillation panic Lambert argues "distillation attacks" wrongly brands a standard training technique and warns US policy could end up banning Chinese open weights
The Gradient of Generative AI Release: Methods and Considerations Irene Solaiman's 2023 framework placing AI releases on a six-level gradient from fully closed to fully open, with the tradeoffs and controls at each
The Load-Bearing Vocabulary of Claude Louis Abraham clusters GitHub PR descriptions by vocabulary; one cluster rose from 0.7% to 39% by mid-2026, led by "load-bearing" and "quietly"
The Myth of unsafe Open Source AI Florian Brand's survey of third-party incident reports - real AI misuse in 2025-2026 ran mostly through closed models, except image abuse
The Peril of Laziness Lost Cantrill on why LLMs lack laziness — human time constraints drive simplicity, unchecked generation bloats
The Z.ai Playbook ChinaTalk interviews Z.ai's Zixuan Li on GLM - why Zhipu open-sources, the coding plan, role-play and translation, release within hours
Thinking-Mode Rule Erosion Reasoning modes degrade rule-following — when models "think," they evaluate whether contextual rules are load-bearing and skip the ones they decide are arbitrary
Thoughts on Coding Agents (rakyll) Agents amplify direction rather than supply it, which removes the permission tax large orgs charge
Unsloth Quantization team behind the UD-Q*-XL GGUF family and Unsloth Dynamic 2.0 scheme; most-cited dynamic-quant publisher for Qwen/Gemma/DeepSeek
Variation in Verification: Understanding Verification Dynamics in LLMs Salesforce study finding errors from stronger generators are harder for any verifier to catch, so a weak generator plus GPT-4o nearly matches a strong one
Vibe Coding and Agentic Engineering Are Getting Closer Than I'd Like Willison admits the line between vibe coding and agentic engineering has collapsed in his own practice — he no longer reviews every line, even for production
We Have Proof Automation Now Langley proves a zstd FSE table builder correct in Lean; LLMs wrote the proof in 20 minutes
What comes next with open models Nathan Lambert on why open models should stop chasing the closed frontier and build small, specialized models that closed agents call as tools
Which LLMs Write Alike Typebulb notebook that maps which LLMs write alike by cross-entropy over character trigrams
Why I build open language models Nathan Lambert's 2024 case for fully open LLMs beyond Meta's self-interest, and what building them at Ai2 looks like day to day
Why Software Factories Fail Dex Horthy on why lights-off factories fail — RL rewards passing tests, never good design
With AI, You Barely Need a Frontend Framework Framework implicitness is the enemy of AI-generated code — a minimal explicit pattern that replaces it
You Should Read "Programming as Theory Building" CodeUtopia on Naur's 1985 essay; programming is building a shared mental theory of the program, with code, docs, and tests as artifacts of that theory
Your File System Is Already a Graph Database Markdown vault with wikilinks is already a graph database; PARA extensions that scale to 52k files
Z.ai (Zhipu AI) Chinese lab behind the open-weight GLM models, from a 2021 Tsinghua paper to GLM-5.3, and how its release practice changed along the way

Toolbox 20

atomicapp Self-hosted knowledge base with built-in embeddings, auto-tagging, wiki synthesis and MCP server
botctl Process manager for scheduled autonomous Claude agents, each defined by a BOT.md file
cc-mirror Clones Claude Code into isolated per-provider variants, each its own command, config and theme
claude-code-router Request-layer router that puts Claude Code on other model providers, with per-task routing slots
CrofAI Cheap hosted inference for open-weights models, unusual for publishing each model's quantization
cursor-bridge Rust binary that runs Claude Code on a Cursor subscription by proxying Anthropic calls to Cursor
deltafin Runs Kimi K3's 2.8T weights on one Apple Silicon Mac, streaming MXFP4 experts over HTTP to disk
Gemma Gem Chrome extension running Gemma 4 on-device over WebGPU to read, click and script web pages
GoModel Go-based AI gateway with OpenAI-compatible unified API across 10+ LLM providers
ik_llama.cpp ikawrakow's llama.cpp fork with IQK quants, Q6 KV cache and faster long-context prompt processing
llama.cpp C/C++ inference engine for GGUF open-weights models, the base most local-LLM tools wrap
local-llm (jamesob) Build guide and configs for a 4x RTX PRO 6000 local inference rig, including PCIe switch tuning
needle 26M-parameter function-calling model distilled from Gemini 3.1 for phones, watches and glasses
OpenCode Provider-agnostic terminal coding agent that points at any OpenAI-compatible or local endpoint
Petals BitTorrent-style distributed LLM inference across volunteer GPUs — working idea, dormant code since 2024
qmd Local-first search over markdown notes combining BM25, vector search and LLM re-ranking
rawquery Hosted data platform whose CLI is meant to be driven by coding agents instead of humans
routing.run Model-routing layer with one OpenAI-compatible endpoint and flat per-request pricing
vamp Single-file TypeScript view layer with explicit DOM bindings, written to be legible to LLMs
vera Language meant to be written by LLMs — no variable names, mandatory contracts, effect rows

talks 1

Nathan Lambert on China's AI Ecosystem and the Open Model Gap Lambert's 2025 recap of open models - Qwen overtaking Llama, a crowded Chinese field, and why the US needs funded, fully open models