Your AI Agent Doesn't Understand Code, It Guesses Confidently

A July 2026 Habr article by artemidoor introducing CodeSlicer, a tool the author built. It is a product announcement, and the benchmark numbers in it are the vendor's own, measured on the vendor's own fixtures. The framing is worth keeping anyway, because the failure mode it names is precise and the honesty conventions it proposes are testable. (The headline says "SLICER"; the body calls the tool CodeSlicer throughout, which is also the repo name.)

The opening quote is from Mo Bitar, creator of Standard Notes, after two years of AI-assisted development: agents write units of change that look good in isolation, are consistent with themselves and with your prompt, and leave the integrity of the whole system unattended. The article's list of what that looks like in practice is concrete. One agent picks a provider without noticing the second implementation. Another changes an HTTP wrapper but never sees the backend route. A third fixes a service without knowing a background consumer reads from it. A fourth updates the frontend client and leaves the stale barrel export in place. Every diff passes its nearest checks; the seam between modules is where it breaks.

Probable is not proved

The article's central example is four lines of Python:

class OrderService:
    def __init__(self, repository):
        self.repository = repository

    def create_order(self, order):
        return self.repository.save(order)

The obvious reading is self.repository → OrderRepository → OrderRepository.save. The call does not establish that. A project can hold OrderRepository.save, PaymentRepository.save, AuditRepository.save, and MockRepository.save, and the object can arrive through constructor DI, a provider, a factory, a registry, or runtime configuration. What an extractor actually observes is only receiver: self.repository, method: save. Connecting that to a concrete method requires walking the constructor parameter, the provider binding, the concrete type, the field assignment, and the method lookup. The article's line for this: confidence without evidence is a nicely formatted guess.

So the tool's design rule is that facts and hypotheses stay separate at every stage. An extractor pulls out what is literally written. A resolver tries to bind that fact to a symbol. Support packs add framework-specific rules. A quality guard checks provenance and contradictions. An impact query walks the graph and marks the weak segments. The AI consumes the graph as context but cannot write a confirmed edge into it.

The output is a GraphDocument where every edge carries its own audit trail:

{
  "from": "OrderService.create_order",
  "to": "OrderRepository.save",
  "kind": "CALLS",
  "resolution_status": "resolved",
  "evidence_class": "static_inferred",
  "validation_status": "not_validated",
  "confidence": 0.86,
  "resolver_id": "typed_receiver_resolver",
  "evidence": [
    "constructor parameter repository: OrderRepository",
    "self.repository = repository",
    "self.repository.save(order)"
  ]
}

Three axes stay independent. resolution_status is resolved, ambiguous, or unresolved. evidence_class is static_proven, static_inferred, or support_pack. validation_status is not_validated or runtime_observed. Confidence answers how strong the evidence chain is; validation answers whether anyone watched it happen. A multi-edge path takes the status of its weakest segment, which stops a chain of four strong edges and one guess from presenting itself as confirmed. When no target can be chosen, the permitted answers are ambiguous, unresolved, unsupported_semantics, and quarantine. Drawing the extra arrow would look better and be worse.

The pipeline and the benchmark

The nine stages run inventory (languages, manifests, local modules, dependency classification), scan planning (excluding node_modules, build output, vendored and generated files), extraction via Python AST and parser-backed extractors, normalization into stable canonical IDs with provenance preserved, semantic binding of imports and parameters and fields and receivers, precision resolution to an exact target or an explicit non-answer, support packs for FastAPI, React, SQLAlchemy and Celery, quality gates checking status and confidence caps and dangling edges, and finally a bounded upward and downward walk for the impact query.

Support packs are versioned rules with attribution, not regexes. A FastAPI edge records its rule_id, rule_version, trust_level, the pattern it matched, and the evidence — local route decorator, parent router prefix, app.include_router prefix. An agent may propose a pack; only validation promotes it to confirmed.

The validation approach is the part most worth stealing. Each fixture declares expected_edges and forbidden_edges, so a run is scored on the false connections it avoided as well as the true ones it found. For the fullstack order-flow fixture, OrderRepository.save must link to OrderService.create_order and POST /api/v1/shop/orders, while the users route and saveOrderDraft must stay excluded despite the name similarity. Mutation testing then deliberately damages the evidence: remove the constructor binding, swap the provider, add a second candidate, delete the import, change the receiver type, remove the HTTP wrapper, change the route prefix. The correct response is that the edge disappears, the target changes, the status drops to ambiguous, a quarantine is created, or confidence falls. An edge is not allowed to survive as confirmed because the method name still matches.

Two further rules keep the honesty from leaking. Determinism: the same project and configuration must produce the same canonical IDs, logical edges, and semantic fingerprint, with timestamps and temp paths excluded from meaning. And runtime observation is not treated as truth. A test that really executed a call proves the call happened in that scenario, environment, and configuration, nothing more, so the strongest available status is STATIC_RESOLVED + RUNTIME_OBSERVED. The absence of a call in one test is not evidence of absence either: the branch may not have run, the object may have been mocked, the instrumentation may not follow a subprocess or an async boundary.

The reported results, on the vendor's labeled scenarios: Python semantic resolution across 21 test scenarios and 29 mutation scenarios, with 20 required semantic edges labeled and 20 true positives, 0 false positives, 0 false negatives. TypeScript and the frontend-backend bridge across 12 test scenarios, 15 mutation scenarios, 4 cross-language scenarios, endpoint precision 1.0, forbidden violations 0. The article states the limit itself: high precision on a small fixture means the specific pattern is demonstrably supported, not that any production codebase is understood. It also says outright that a finished analysis is not supposed to be all green — confirmed, inferred, ambiguous, unresolved, and unsupported_semantics are all legitimate outcomes, and unknown regions are part of the result rather than an interface defect.

Where it sits

The comparison table separates tools that share the word "graph". Sourcegraph is hosted code navigation across an organization; Joern is a Code Property Graph for security research; Semgrep and CodeQL are rule engines for SAST and data-flow findings; Graphify is a local code knowledge graph for assistants; CodeSee is visual codebase maps for onboarding. CodeSlicer's claimed niche is narrower: what a change touches, why the analyzer believes that, and where it does not know. The proposed products on top are PR risk reports that split must-change from should-review, test selection by real chain rather than filename similarity, refactor and migration blast radius, incident response walking from a symptom back through the handler and service, and splitting a large task across several agents so each gets its own slice plus an explicit list of boundaries it may not treat as confirmed.

The article is candid about the business model: a free local engine (CLI, MCP server, parser, GraphDocument, basic public support packs, visual interface) with a paid verified registry of validated framework packs, private registries, CI impact checks, and self-hosted enterprise deployments. The stated constraint is that a paid registry must not require uploading source; rules flow down, code stays local. Whether that holds is a thing to check later, not now.

The problem this attacks is the one agent-principal-agent-problem describes from the reviewer's side. Crawshaw's point is that the reviewer can no longer infer effort from a diff; a machine-checked blast radius is an attempt to give the reviewer a different signal, one that does not depend on guessing how hard the contributor worked. It also lands next to how-do-we-stop-vibe-coding, where Scryer makes essentially the same bet from the model-first direction: show the change as a diff over a structure both the human and the agent can read, instead of asking either to read every line. reviewing-ai-code supplies the reason the reviewer needs help at all, and clean-code-coding-agents the reason a compact, pre-resolved slice beats handing an agent the repository.

Repo: artemnoor/CodeSlicer. Single author, early, and the only published numbers are the author's own — worth revisiting rather than adopting.