Build your knowledge base

Index the code graph

Turn your mirrored repos into a queryable code graph: incremental indexing, and the full node and edge model across 14 languages, Terraform, SQL, and web topology.

Indexing turns your mirrored repos into a queryable knowledge graph. contextlake index --workspace ~/work walks every git repo under a folder and builds the graph. Runs are incremental by default; --force rebuilds from scratch.

Incremental and time-travel#

index --workspace is incremental, it re-indexes only repos whose git HEAD moved since their last index, so a scheduled (cron) run stays cheap; pass --force to rebuild everything, or --watch [--interval N] to keep re-indexing in a loop (the same --watch / --interval flags also drive connect and embed). Every indexed snapshot is kept, so query "<text>" --repo R --as-of <commit> does time-travel, it searches repo R as it was at a previously-indexed commit.

Parallelism and noise-pruning#

Repositories are parsed across worker processes (CPU-bound work) while the SQLite store is written serially from the parent. The spawn start method is used on every platform, so behaviour is identical on Linux, macOS, and Windows, with an automatic serial fallback if a worker pool can't start. It defaults to cpu_count - 1 workers (capped at 8); set [kb] index_workers to tune it (1 forces serial).

The parser also skips machine-generated and derived files (*.designer.cs, *.min.js, AssemblyInfo.cs, and @generated / <auto-generated> headers) plus code files larger than [kb] max_file_bytes (5 MB). That's derived noise, not real source, and every skip is reported (no silent gaps). Set [kb] skip_generated = false or raise max_file_bytes to index them anyway. Discovery also skips vendored nested repos: an upstream clone carried inside the mirror with its own .git under a module-federation path segment, which is not your source and would flood the global graph with upstream-demo nodes. Each such skip is logged. (node_modules trees are already pruned before discovery descends into them.)

To exclude your own paths, drop a .contextlakeignore at a repo's root: one glob per line (# comments and blank lines ignored), matched against each file's path relative to the repo and its name, so *.lock ignores by name anywhere and vendor/ ignores a directory and everything under it. It's a small, dependency-free subset of gitignore syntax (no negation, **, or anchoring), enough to drop vendored trees and lockfiles from the graph.

Health and maintenance#

contextlake doctor checks the environment (FTS5, git / glab on PATH, the store, the embedder, and the ANN index) and exits non-zero if anything is wrong, so it doubles as a CI health gate. contextlake lint audits the graph itself, reporting stale repos (HEAD moved since they were indexed) and dangling edges (an edge whose endpoint node is missing). Both exit non-zero on problems, so they're CI-friendly.

What the graph captures#

Indexing builds a typed graph of your source. tree-sitter extracts files, classes, functions/methods, interfaces, imports, an intra-repo call graph, and an inheritance graph (inherits edges for extends / implements / base classes), so "what extends BaseController?" is one hop, and changing a base class shows its subclasses in blast_radius. Every node kind and edge relation below is colored and styled the same way here, in contextlake graph, and in the dashboard:

Languages#

tree-sitter covers 14 languages across 13 grammars (TypeScript and TSX share one grammar), and the parser registry is pluggable:

Frameworks are indexed through their base language: React / Next.js / Node.js are JS/TS(X), Angular is TS (its templates are HTML), and .NET is C#.

Infrastructure: Terraform / HCL#

.tf files build an infrastructure dependency graph: resource / data / variable / output / module / local definitions with depends_on edges resolving var. / module. / data. / resource references across files in a repo. resource nodes are semantically searchable.

Resolution is repo-wide, so a block address defined identically in separate root-module directories (for example environments/prod and environments/staging) surfaces as an AMBIGUOUS edge; directory-scoped resolution is a future refinement.

Databases: SQL DDL#

.sql files build a referential graph: table / view / procedure definitions with references edges from foreign-key REFERENCES clauses, resolved across files in a repo. table and view nodes are semantically searchable.

It uses a regex DDL extractor (the fleet's T-SQL/PL-SQL defeats a tree-sitter AST), so it targets the high-value defs and FK references and is a deliberate undercount.

Web topology: endpoints and routes#

Two web-topology layers sit on top of the definitions.

HTTP endpoints a repo exposes or calls (Express/Fastify/Nest, FastAPI/Flask, ASP.NET minimal-API, and Next.js App Router route.ts handlers) become shared endpoint nodes that join across repos into flow edges, from the caller repo to the exposer repo.

Frontend routes become repo-scoped, embeddable route nodes from three frameworks:

The object-literal forms use a tree-sitter AST walk anchored on the route-table container (a Routes-typed declarator, or the array argument to forRoot / provideRouter / create*Router), so nested children compose into full paths and bare {path:...} config objects are never mis-read as routes.

Skipped rather than guessed. Not yet extracted: Luigi navigation configs, Angular lazy loadChildren sub-trees, React loader / lazy, realtime / WebSocket channels, templates, and stylesheets.

Manifests and cross-repo dependencies#

Indexing also reads manifests (pyproject.toml, package.json, *.csproj, pom.xml) to build a cross-repo dependency graph through shared package nodes. Agents traverse all of this over MCP, from finding a definition to cross-repo blast_radius ("what could break if I change this"); see the full tool list under Serve.

The same change-impact walk is a one-liner from the shell: contextlake impact <symbol> [--hops N] lists what calls / depends on a node, no editor needed. When a symbol name (e.g. Node, Order) is defined in more than one repo, impact lists the candidates and you narrow it with --repo <repo> rather than getting a silent best-guess.

See also#

• Connect and enrich
• Semantic search
• Serve it to your editor