Most comparisons of AI coding tools rank products against each other on a single axis and produce a winner. That framing does not survive a week of real use, because the tools are not competing for the same slot in a workflow. A completion engine that finishes the line you are typing and a terminal agent that runs your test suite are not alternatives; plenty of developers run both, on the same file, in the same hour.
So this assessment is organised by category and by criteria, and the criteria are stated before any judgement. What it deliberately does not contain: scores, benchmark numbers, prices or adoption claims. Those either change faster than an article can track or cannot be verified from outside the vendor.
What was assessed, and what was not
Assessed: how each category obtains repository context; what it can change and under whose approval; how it behaves when it is wrong; how it fits into version control and CI; whether it can be extended to reach your own systems; and what data leaves the machine.
Not assessed: raw model quality. The model behind any of these tools changes on a cadence no article can match, and most products let you switch models anyway. Judging a tool by the model it shipped with last quarter is judging the wrong layer. Model-level differences that persist across versions are covered separately in Claude vs GPT from a developer's perspective.
Also not assessed: productivity. Self-reported speedups are unreliable, and honest measurement requires a controlled study nobody writing a tools round-up has run.
Four categories, not one market
Inline completion
The original form: a model predicts the next few tokens or lines from the surrounding code, and you accept with a keystroke. Context is a window around the cursor plus, typically, a few related open files.
The strength is latency and zero interaction cost — you never leave the keyboard. The weakness is architectural blindness. Completion has no view of your service boundaries or your team's conventions beyond what is textually nearby, so it produces locally plausible code that occasionally reimplements something that already exists two directories away.
Chat with repository context
An assistant panel that can search and read the repository. This is where indexing strategy starts to matter: some tools embed the codebase and retrieve semantically, some rely on the model reading files agentically, and many do both.
Strong for explanation and orientation — "where is authentication enforced for this route?" — and for changes spanning a handful of files. Weaker as an author, because the human still has to shuttle code between panel and editor, which is where subtle transcription errors enter.
Agentic editors and terminal agents
The category that has changed the most. The tool runs a loop over your repository: read files, edit them, run commands, read the output, iterate. Some live in an editor, some in the terminal, some in CI. Several are open source, which matters if you need to see exactly what runs.
This is the only category that closes the feedback loop. A tool that can run your tests can tell the difference between code that looks right and code that passes, and it will keep going until it does. It is also the category with the largest blast radius: it edits files and executes commands under your credentials.
Review and CI-time tools
Bots that comment on pull requests, or checks that run in the pipeline. They see the diff, sometimes the surrounding code, and post findings.
The value is that they operate at the point where a human is already reviewing. The persistent problem is precision. A review bot that raises three low-value comments per pull request trains the team to skim its output, at which point it costs attention and returns nothing. Tuning toward fewer, higher-confidence findings matters more than coverage.
The criteria that separate them
| Criterion | Completion | Repo chat | Agentic | Review bot |
|---|---|---|---|---|
| Context source | Cursor window | Index and/or file reads | Reads and runs the project | The diff |
| Can verify its own output | No | No | Yes, via tests | Partially |
| Blast radius | One insertion | Clipboard | Files and shell | Comments only |
| Approval model | Per keystroke | Per paste | Per diff or per command | Human merges |
| Extensible to your systems | Rarely | Sometimes | Yes, commonly via MCP | Rarely |
| Cost profile | Per keystroke, small | Per question | Per run, variable | Per pull request |
| Best at | Boilerplate, repetition | Understanding | Multi-file change | Catching regressions |
The "can verify its own output" row is the meaningful divide. Everything above it produces suggestions a human must validate. Everything at or below it can check itself against something external. That single property changes how much you can delegate more than any model upgrade does.
Context is the whole game
Almost every quality complaint about an AI coding tool is, on inspection, a context complaint. The model wrote a component that ignores your design system because it never saw the design system. It duplicated a utility because it never searched for one. It used a deprecated internal API because the deprecation lives in a comment three files away.
Three mechanisms determine how well a tool solves this, and knowing which one your tool uses tells you where it will fail:
Proximity. Include what is textually near — open files, imports, the current function. Cheap, low latency, and blind to anything not already on screen.
Retrieval. Index the repository and fetch semantically relevant chunks. Scales to large codebases, and inherits every failure mode of retrieval — including that a chunk split down the middle scores poorly, a problem covered in RAG vs AI agents.
Agentic reading. Let the model search, list and open files itself. The most accurate, because it can follow a reference to its definition, and the most expensive, because every read consumes context and every step costs a model call.
The tools that feel noticeably better in a large repository are almost always the ones combining retrieval with agentic reading, rather than the ones with a newer model.
Extensibility is the underrated criterion
The most valuable thing an agentic tool does is not writing code — it is answering questions that require systems outside the repository. Why did this test become flaky? What changed in the config for this service? Which customers are on the code path this change touches?
That requires reaching your infrastructure, and the emerging common answer is the Model Context Protocol: write one server per system, and every compatible client can use it. When comparing tools, whether they speak a protocol you can implement once matters more than any feature on the marketing page, because it decides whether your integration work is portable or captive.
The failure modes to plan for
Confident wrong edits. Agentic tools that cannot run your tests will report success on code that does not compile. Wire up the test command first; it converts the tool from an author into an author with a proofreader.
Context rot on long sessions. Every read and every command output accumulates. Long sessions degrade — the tool starts forgetting decisions from earlier in the same task. Short, scoped sessions with a clear goal beat one marathon.
Dependency invention. Models suggest packages that do not exist, or that exist and are not what the name implies. Anything that adds a dependency deserves a human check — this is an active supply-chain attack surface, not just a quality issue.
Silent scope creep. An agent asked to fix a bug reformats four unrelated files. Review the diff, always, and prefer tools that stage changes rather than writing directly to the working tree.
Data egress. Know what leaves the machine: file contents, repository names, shell output, environment variables. For most teams this is a procurement question with a real answer, and it should be settled before adoption rather than after.
Run your own evaluation
Vendor demos are built on greenfield repositories. Your repository is not greenfield. The only assessment that transfers is one run on your own code, and it takes an afternoon.
Pick five tasks from your actual backlog, spanning difficulty:
1. Add a nullable column and migrate — a task with an obvious correct answer.
2. Fix a bug with a failing test — verifiable, no ambiguity.
3. Extend a feature across three files — tests conventions and consistency.
4. Explain an unfamiliar subsystem — tests context retrieval, not generation.
5. A task the codebase makes genuinely hard — tests whether it says "I can't".Then score with a rubric applied identically to each tool, by a person who did not choose the tools:
export const rubric = {
correct: 'Does it work? Tests pass, behaviour matches the request.',
idiomatic: 'Does it match conventions already in this repository?',
scoped: 'Did it change only what was needed?',
honest: 'When it could not do the task, did it say so?',
effort: 'How many human interventions to reach a mergeable state?',
};
export function summarise(runs) {
return runs.map((run) => ({
tool: run.tool,
task: run.task,
interventions: run.interventions,
mergeable: run.mergeable,
notes: run.notes,
}));
}Task five carries more information than the other four combined. A tool that produces confident output for an impossible task will produce confident output for a subtly impossible one, and you will merge it.
What not to buy on
Ignore any comparison built on public coding benchmarks. They measure self-contained puzzle-solving, and your job is mostly navigating a large codebase with implicit conventions and load-bearing history. Ignore leaderboards for the same reason. Ignore demos on a repository you have never seen.
Buy on context mechanism, on approval model, on whether the tool can verify its own work, and on whether integrations you write are portable. Those properties persist across model upgrades. Everything else is next quarter's changelog — and the broader shift they are driving is covered in how AI agents are changing software development.