contextlake
Build your knowledge base

Connecting and enriching

Link repos to their issues, docs, and designs, manage sources, and pull grounded external facts into the knowledge layer with query-driven enrichment.

Beyond code, contextlake links each repo to its external context (issues, docs, and designs) and can pull grounded facts from those sources into the knowledge layer. connect links repos to items; enrich queries connected sources with codebase-derived terms and stores what comes back.

flowchart LR
  C(["tracker, designs, merge
requests, chat"]) --> CON["kb connect"]
  I(["files, web, api, graphql,
another MCP server"]) --> ING["kb ingest"]
  Q(["the searchable ones: Atlassian,
or an MCP search tool"]) --> ENR["kb enrich"]
  G[("the code graph")] -.->|"repo name and top symbols
become the search terms"| ENR
  CON --> P[("isolated partitions")]
  ING --> P
  ENR --> P
  P -.->|"linked to the repos and
symbols they name"| G
a rectangle is something that runs a cylinder is something that persists a rounded box is a start or an end point

Each stage writes its own partition, so re-indexing a repo's code never disturbs its external links.

Connectors#

connect enriches the graph with external context. Four connectors ship, sharing one seam:

Adding another connector is a small, self-contained module, and its output lands in an isolated graph partition, so re-indexing a repo's code never disturbs its external links. Configure connectors by copying examples/kb.toml.example to ~/.contextlake/kb.toml.

Managing sources: the source command family#

Editing kb.toml by hand works, but for everyday use contextlake kb source commands let you add, test, and manage connectors without touching the config file. They rewrite kb.toml while preserving your comments, and work alongside hand-editing if you mix approaches.

The commands:

An example workflow:

contextlake kb source add                # interactive: what type? which workspace?
contextlake kb source list               # show what you've configured + status
contextlake kb source test my-atlassian  # does it work? what's in scope?
contextlake kb connect                   # now link repos to their items

init can also prompt you to connect a source during first-run setup, and doctor reports per-source reachability as part of its environment check, so hand-editing is optional; the CLI guides you through the whole flow.

Every fact carries its receipt. Each is provenance-stamped (source file + verified date) and confidence-tagged as one of three tiers, EXTRACTED (read straight from source/AST), INFERRED (a resolved call or link), or AMBIGUOUS (an unconfirmed candidate), and sanitized before it reaches an agent. The dashboard and the graph legend use these same tiers.

Query-driven enrichment#

contextlake kb enrich performs query-driven enrichment: it derives search terms from each repo's code graph (the repo's name and its top searchable symbols by graph degree, which is the symbol kinds the embedder covers, not files or packages) and queries your connected sources (Atlassian Rovo search, or any mcp source with a tool and arg_template configured) with those terms, then stores the returned documents in a searchable, embedded @enrich:<repo> partition, idempotent and re-runnable across the whole fleet or a single repo:

contextlake kb enrich --workspace ~/work     # all indexed repos
contextlake kb enrich acme/forecast-api        # one repo

Which symbols become terms. Ranking is by graph degree, and by nothing else. There is no per-kind quota, so a repo whose highest-degree definitions are all classes and functions gets classes and functions, and no field or endpoint name at all, even when it holds many of them. That is on purpose: a repo gets its name plus 9 symbol names by default, and reserving one for each of the 19 embeddable kinds would leave the ranking two or three slots to decide.

Prerequisites. Two things must be true:

  1. The code graph is indexed, via contextlake kb index.
  2. At least one term-searchable source is configured. That means either an mcp source with tool and arg_template keys, or an atlassian source.

Sources without those capabilities, such as a plain files or web source, are skipped gracefully.

What you get. Each repo's enrichment documents live in their own partition, so they can be re-fetched without clobbering earlier results.

With the semantic tier enabled they are embedded too, so they appear in semantic search as document nodes tagged with their source in attrs.source.

A result that names one of the repo's symbols is linked straight to it with a documented_by edge, so the enrichment lands on the graph rather than beside it.

How to read the run. Each repo prints the terms tried, the documents returned and the edges attached to code. Both numbers are reported because they answer different questions: documents returned says whether your sources had anything, edges to code says whether a question about the code can reach it.

The closing line puts every targeted repo in one of five buckets, and the five add up to the number of repos the run planned to touch:

Bucket What it means
enriched Documents came back and at least one names a symbol in the repo.
nothing returned The repo was searched and the sources had nothing.
returned but unattached Documents came back and none names a symbol in the repo.
failed The store or shard write failed for that repo. The run continues.
skipped No graph shard, so no terms were built. Run kb index first.

returned but unattached is a normal outcome, not a failure, and the run still ends with a ✓. Symbol matching is whole-word and ignores names under three characters, so a ticket that discusses the repo in prose without naming any of its code correctly attaches to nothing. If you expected attachments, check the repo is indexed and that the document text spells the symbol names the way the code does.

After contextlake kb wiki runs, these documents become an attributed "External context" section in the curated wiki, grounded to the code graph's terms.

Configuring document sources, including the built-in files source, plugin packages and MCP endpoints, is in Document sources and RAG.

When a source stops answering#

connect and enrich are the only stages that leave the machine, and a fleet run asks each source once per repo. If a source goes down mid-run, contextlake stops asking it rather than paying its timeout on every remaining repo: after three consecutive failures the source is skipped for 60 seconds, then one call is let through to see whether it came back. A run against an unreachable MCP server finishes in seconds instead of timeout x repos.

You will see this in the output, it is never silent, because "the source was down" and "the source had nothing" would otherwise look identical:

resilience: circuit OPEN for mcp:npx:https://mcp.example.test after 3 consecutive
  failure(s) (TimeoutError) -- further calls are skipped for 60s
resilience: skipping mcp:npx:https://mcp.example.test for 60s -- circuit open after 3
  consecutive failure(s) (TimeoutError); results from this source will be incomplete

The name in that line identifies the endpoint by transport and host only, never the rest of the URL, since a hosted MCP endpoint can carry a token in its path or query.

Failures the server rejected rather than failed on, an unknown tool, a bad token, are reported as themselves and never trip the skip: no amount of waiting fixes a wrong request. Raise timeout on the source if the server is merely slow.

When one repository is unreadable#

A repository that fails outright costs that repository, not the run. connect names it, skips it, and carries on with the rest:

  api-gateway: 'utf-8' codec can't decode byte 0x96 in position 99486: invalid start byte
⚠ Connect complete: 143 external link(s) stored
⚠ 1 of 20 repo(s) failed and were skipped; the rest were enriched. Re-run to retry
  them, or narrow with `contextlake kb connect <repo-id>`.

The exit code is non-zero when any repository was skipped, the same verdict kb index gives a workspace where one repo failed to parse: the graph an agent will cite from is not the one you asked for, so the run should not read as clean.

See also#

Next steps

Searching semanticallyGenerating the wiki