Which Tools Do Claude Code, Codex and Cursor Choose?

When a coding agent is asked to "send each invoice by email, find the best solution and implement it", something in the session picks Resend, Postmark or SendGrid. Armature set out to measure what that something is. The post, published 2026-09-03 by "the Armature team", reports 16,893 runs of claude-code, Codex and Cursor on synthetic repositories, of which 5,292 sessions on 51 codebases and 18 sectors were kept as valid for this first write-up which-tools-coding-agents-install.

Who is measuring matters here. The post itself only hints at it, saying it will keep digging for "what's of interest to vendors and developers" and pointing to a public leaderboard. Armature's homepage, checked on 2026-09-14, is explicit: its product is "Be the tool Claude Code chooses", a service in which a growth engineer measures how agents choose tools in a vendor's category and works with the vendor until they choose it, with pick rate on this same repository panel as the success metric. So this is research published by a company whose revenue depends on vendors believing agent choices can be moved. That does not make the numbers wrong, but the study is also the product's sales pitch.

The setup

The panel started from statistics over thousands of public GitHub repositories: languages, frameworks, third-party services, deployment targets, team size, codebase age. Armature says it reweighted those because startups publish more open source than enterprises, then had coding agents build repositories matching the target distribution. The result is 75 repositories in 10 languages with fake company names, fake git histories, fake API keys and real lockfiles. Variants had whole service integrations removed so the agent would have a gap to fill.

Tasks came from four personas — a vibe-coder who describes symptoms, a junior who names the category, a senior who states constraints, and an enterprise engineer who brings compliance and procurement — for 1,163 prompt variations. In 20–25% of them the prompt mentioned cost or usage volume. Each run was in an ephemeral sandbox, rotated across E2B, Blaxel and Daytona.

Two design choices carry a lot of weight. A "simulated human" played by Gemini 3.7 Flash first asked the agent to analyse and recommend, then always accepted the top pick or told the agent to choose. Armature found that asking for implementation straight away pushed agents toward building everything in-house, since they could not ask permission to adopt a vendor, and that adding the human reduced the dominance of market leaders and cloud-native options; in object storage, Cloudflare R2 started winning sessions where S3 had always won before. A second Gemini 3.7 Flash instance judged each session: whether the repository had already pre-chosen a provider, whether a real choice was made (OpenTelemetry alone did not count as picking observability), and which players were mentioned and which one won, reading both the conversation and the diff.

What they report

The three agents gather evidence differently. Cursor bases its decision on the web in about two thirds of sessions. Codex searches in 94% of sessions, and in nine queries out of ten uses operators such as site: to narrow to a vendor's domain. Claude Code mostly relies on its priors and searches in about 30% of cases, but reads three times as many pages as Codex when it does, and in newer categories like sandboxes, where its priors are thinner, it searched about 80% of the time. The three agree on the same tool in only 42% of cells. For voice agents, Claude Code picked Twilio, Codex the OpenAI Realtime API, and Cursor Vapi. Claude Code also builds in-house almost twice as often: 19% against 10%.

The repository matters as much as the agent. The same email task on four repositories gave four winners: Resend on TypeScript (55 of 89 runs), SendGrid on Python (22/24), Postmark on Golang (20/24), Azure Communication Services on Java (22/23). Vercel won on TypeScript, every time when Next.js was present, and was never recommended on Python, where Render dominated.

Being mentioned is not the same as being chosen. PayPal came up 139 times in payments sessions and was never picked; Stripe won 124 of those. Adyen was mentioned 175 times and picked three times, LangChain 194 times and picked four, Netlify 152 times and picked six. Supabase was the most-mentioned database at 242 and still lost to Neon.

Small details on vendor pages flipped outcomes. Mailgun regularly lost to Postmark once the agent read "1-day retention" on its free plan. Supabase kept losing because its pricing bundles auth, storage and realtime when the agent wanted only a database. Of the 5.3k sessions, 388 mentioned platform-management overhead and 195 mentioned cost, and the post says that in a significant share of those, the objection came from how a vendor presented information rather than from a disqualifying fact. It gives no number for that share.

Some categories are barely contested. Stripe won nine times in ten, losing only in EU-regulated cases to Paddle and Mollie. Neon took 66% of database picks, followed by cloud-native offerings. S3 held 45% of file storage, Azure and GCP 20% each. Email was close, with Resend at 35.6% and Postmark at 27.4%.

Reading it carefully

The finding that holds up best is the disagreement: three agents given the same task in the same repository choose differently, and a repository's language shifts the choice more than the task does. That fits what agentic-search-context-engineering says about search shaping what an agent knows, and it is a practical argument for pinning a vendor in a repository's instructions rather than letting whichever agent is running decide.

Several things limit how far the percentages travel. The repositories were written by coding agents, so they may already carry agent habits in their structure and dependencies. The simulated human always accepts the first recommendation, which measures the agent's top pick rather than what a person would end up shipping. Both the orchestrator and the judge are the same Gemini model, and nothing in the post reports how often the judge was checked against a human reading. Agent and model versions are not given, which matters because priors change with every release. Two thirds of the runs are held back for a later wave, with no statement of what excluded them beyond the validity criteria. And no confidence intervals are given, a weakness llm-output-variance explains: cells with 23 or 24 runs are small enough for run-to-run noise to move a winner.

The vendor-page findings are the ones Armature sells, and they are also the most interesting for anyone publishing documentation. They point the same way as llms-txt and the agent-readiness category in specification-website-checklist: an agent reads a pricing page literally, and a caveat written for humans can disqualify a product. agent-reading-test measures the reading side of that problem directly.