Your IDE indexes your codebase. Why doesn't your AI?

Open IntelliJ. Cmd+click any function. Instant. Jump to definition, find all usages, see the call hierarchy. It knows your entire codebase — every symbol, every dependency, every type relationship.

Now open Claude Code or Codex on the same codebase. Ask it to change a function.

What does it do? It greps. It searches file by file. It reads things it doesn’t need. It misses things it should’ve found. It burns through context tokens exploring code that a simple index would’ve surfaced in milliseconds.

We went from having AI that can reason about complex systems to… making it read files like a junior dev on their first day.

There’s a better way. And it’s not even new — it’s the same idea your IDE has used for 20 years, adapted for LLMs.

How your IDE actually works

When IntelliJ opens a project, it doesn’t read every file on every action. It builds an index — a pre-computed map of your entire codebase.

This index knows:

Every class, function, and variable name
Where each symbol is defined
Where each symbol is referenced
The dependency graph — who calls what, who imports whom
Type relationships — inheritance, interfaces, implementations

When you Cmd+click, IntelliJ doesn’t search. It looks up. One hash table lookup. Instant.

The technology behind this is called an Abstract Syntax Tree (AST). Your IDE parses every file into a tree structure that represents the code’s meaning, not just its text. It’s the difference between knowing that processOrder is a 12-character string somewhere in your codebase, and knowing it’s a method on the OrderService class that takes a Cart parameter and is called from 7 places across 4 files.

LLMs don’t have this. They see your code as text. Flat, unstructured text.

How Claude Code and Codex navigate code today

When you ask Claude Code to “update the payment flow,” here’s roughly what happens:

It reads your CLAUDE.md (if you have one) for project context
It greps through files looking for keywords — “payment,” “checkout,” “order”
It reads matching files, often pulling in more context than it needs
It builds a mental model from what it found
It makes the change

Steps 2 and 3 are where the waste happens. A codebase with 500 files might have “payment” mentioned in 40 of them. Claude reads all 40. Maybe 6 are relevant. That’s 34 files worth of wasted context tokens — context that could’ve been used for better reasoning about the actual change.

On a large codebase, this gets worse fast. I’ve watched Claude Code burn through half its context window just finding the right files before it even starts thinking about the solution.

Your IDE would’ve found the 6 relevant files in under a second.

The tools that fix this

1. CLAUDE.md — The simplest win

You probably already know about this, but most people do it wrong.

CLAUDE.md is a file at your project root that Claude Code reads at the start of every conversation. Think of it as giving the AI a cheat sheet before it starts exploring.

What most people put:

This is a Next.js project with TypeScript.
We use Tailwind for styling.

What actually helps:

# Architecture
- API routes: src/app/api/[resource]/route.ts (REST pattern)
- Database: Prisma with PostgreSQL, schema at prisma/schema.prisma
- Auth: NextAuth with Google + credentials, config at src/lib/auth.ts
- Payments: Razorpay integration in src/lib/razorpay.ts — webhooks at api/webhooks/razorpay

# Key patterns
- All DB queries go through src/lib/db/ — never call Prisma directly from routes
- Error handling: AppError class in src/lib/errors.ts, caught by middleware
- Feature flags: src/lib/flags.ts, checked via hasFeature(userId, 'flag-name')

# Commands
- npm run dev — starts on port 3000
- npm test — runs Vitest, needs TEST_DB_URL in .env.test
- npm run db:migrate — runs Prisma migrations

The second version tells Claude where things are before it starts searching. Instead of grepping for “payment” across 500 files, it knows to look at src/lib/razorpay.ts first.

But CLAUDE.md has limits. It’s a static file. It can’t answer “which functions call processOrder?” or “what changed in the auth module this week?“

2. Aider’s Repo Map — The AST approach

Aider solved this problem with something called a repo map. Here’s how it works:

It uses tree-sitter (the same parser that powers syntax highlighting in VS Code, Neovim, and most modern editors) to parse every file in your codebase into an AST. From that, it extracts:

Every function signature
Every class definition
Every import statement

Then it builds a dependency graph — which files reference which other files. It ranks these using an algorithm similar to Google’s PageRank: files that are referenced by many other files are more “important.”

When you ask Aider to make a change, it doesn’t send your entire codebase to the LLM. It sends a condensed map — the top-ranked symbols and their relationships. Usually about 1,000 tokens. The LLM sees the full API surface of your project without reading a single implementation.

This is the difference between giving someone a building’s blueprint vs. making them walk through every room.

You can use this outside of Aider with RepoMapper, which extracts just the repo-map functionality.

3. Cursor’s Embedding Index

Cursor takes a different approach: semantic search via embeddings.

When you enable codebase indexing in Cursor:

It parses your code into chunks using tree-sitter (splitting at function/class boundaries, not arbitrary line counts)
Each chunk gets converted into a numerical embedding — a vector that captures its meaning
These embeddings are stored in a vector database
When you ask a question, your query gets embedded too, and Cursor finds the most semantically similar code chunks

The result: you can ask “where do we handle authentication errors?” and it finds the relevant code even if it never uses the word “authentication” — maybe it’s called verifyCredentials or checkSession. Grep would miss these. Semantic search catches them.

Cursor reports this improves response accuracy by 12.5% on average.

4. MCP-based Code Intelligence — The new frontier

This is where things get interesting for Claude Code users.

MCP (Model Context Protocol) lets Claude Code connect to external tools. A few projects are building code intelligence servers that Claude can query:

codebase-memory-mcp: Indexes your codebase into a persistent knowledge graph. Functions, classes, call relationships — all stored and queryable. In benchmarks across 64 repos, five structural queries consumed ~3,400 tokens vs ~412,000 tokens via file-by-file exploration. That’s a 99.2% token reduction.

Think about what that means. Instead of Claude burning half its context window finding files, it asks the MCP server “what calls processOrder?” and gets a precise answer in 50 tokens.

CodeRLM: A Rust server using tree-sitter that exposes a JSON API. Claude can query it for symbols, callers, tests, even grep — but through a structured interface that returns exactly what’s needed, not everything that matches.

This is the equivalent of giving Claude its own IntelliJ index.

What actually works today — a practical guide

Let me cut through the tools and give you what I’d actually set up today on a real project.

For small projects (under 10K lines)

Just use CLAUDE.md well. Write 50-100 lines covering architecture, key files, patterns, and commands. Claude Code can grep a small codebase fast enough that indexing doesn’t save much time.

Also: use subagents. When Claude Code spawns a subagent to explore code, it does so in a separate context window. The subagent reports back a summary. Your main conversation stays clean. This is built into Claude Code — you don’t need any extra tools.

For medium projects (10K-100K lines)

CLAUDE.md + feature-level documentation. Create markdown files per feature area:

docs/
  PAYMENT_FLOW.md    — Razorpay integration, webhook handling, refund logic
  AUTH_SYSTEM.md     — NextAuth setup, providers, session management
  SEARCH_FEATURE.md  — Elasticsearch queries, indexing pipeline, relevance tuning

Claude can read the relevant doc when working on that feature instead of exploring the whole codebase.

Also consider aider’s repo map if you use aider, or Cursor’s codebase indexing if you use Cursor. Both give you semantic navigation out of the box.

For large projects (100K+ lines)

MCP-based code intelligence is worth the setup. Install codebase-memory-mcp or CodeRLM, index your project once, and let Claude query the knowledge graph instead of grepping. The token savings are massive.

Also consider the research-plan-implement workflow that Anthropic’s own engineering team recommends:

Research phase: Claude reads relevant code and creates a research document mapping out files, functions, and data flows
Plan phase: Using the research doc as context, Claude creates an implementation plan
Implement phase: Using the plan as context, Claude writes the code

Each phase uses only the context it needs. The research doc becomes a reusable artifact — next time someone touches the same feature, the research is already done.

The mental model shift

Here’s what I want you to take away:

Context is not free. Every file Claude reads costs tokens. Tokens are a finite resource — there’s a context window, and when it fills up, quality degrades. The research says performance is best when context utilization stays between 40-60%.

Pre-computed knowledge beats runtime exploration. Your IDE proved this 20 years ago. An index that takes 10 seconds to build saves hours of file-by-file searching. The same principle applies to LLMs.

The best AI engineers aren’t prompt engineers. They’re context engineers. The difference between a productive Claude Code session and a frustrating one is almost never about the prompt. It’s about what context the LLM has when it starts working.

Give it an index, not a haystack.

Try this today

If you don’t have a CLAUDE.md: Create one. Spend 15 minutes. Include your project structure, key file paths, build commands, and coding conventions. This alone will make Claude Code noticeably better.
If you use Cursor: Enable codebase indexing in settings if you haven’t already. It’s off by default on some installations.
If you’re curious about MCP: Try codebase-memory-mcp. The setup is straightforward and the token savings are real.
For any tool: Try the research-plan-implement workflow on your next complex task. Have the AI create a research doc first, then plan, then implement. Three separate conversations. You’ll immediately feel the difference.

The AI tools are good enough. The bottleneck isn’t intelligence — it’s context. Fix the context, and the intelligence follows.

Building AI tools weekly at @gabrubuilds. Next up: a tool that actually solves this indexing problem for Claude Code users.

Frequently asked questions

What is codebase indexing for AI coding tools?

Codebase indexing is the process of pre-computing a structured map of your code — functions, classes, dependencies, and call relationships — so that an AI coding assistant can look up relevant files instantly instead of grepping through every file. It’s the same concept your IDE has used for 20 years (Abstract Syntax Trees), adapted for LLMs.

How do I make Claude Code faster on large codebases?

Three approaches, depending on project size: (1) Write a detailed CLAUDE.md with architecture, key file paths, and patterns for projects under 10K lines. (2) Add per-feature markdown docs for 10K-100K line projects. (3) Set up an MCP-based code intelligence server like codebase-memory-mcp for 100K+ line projects — benchmarks show 99.2% token reduction compared to file-by-file exploration.

What is the difference between Aider’s repo map and Cursor’s codebase indexing?

Aider uses tree-sitter to parse code into ASTs and builds a dependency graph ranked by PageRank — it sends a condensed ~1,000 token map of your project’s API surface. Cursor uses embeddings: it chunks code at function/class boundaries, converts each chunk into a vector, and performs semantic search. Aider’s approach is structural (follows code references), while Cursor’s is semantic (finds conceptually related code even when naming differs).

MCP (Model Context Protocol) allows AI assistants like Claude Code to connect to external tool servers. Code intelligence MCP servers like codebase-memory-mcp index your codebase into a queryable knowledge graph. Instead of Claude reading 40 files to find relevant code, it asks the MCP server “what calls processOrder?” and gets a precise answer in ~50 tokens.