contextlake
Build your knowledge base

Connecting and enriching

Link repos to their issues, docs, and designs, manage sources, and pull grounded external facts into the knowledge layer with query-driven enrichment.

Beyond code, contextlake links each repo to its external context (issues, docs, and designs) and can pull grounded facts from those sources into the knowledge layer. connect links repos to items; enrich queries connected sources with codebase-derived terms and stores what comes back.

flowchart LR
  C(["tracker, designs, merge
requests, chat"]) --> CON["kb connect"]
  I(["files, web, api, graphql,
another MCP server"]) --> ING["kb ingest"]
  Q(["the searchable ones: Atlassian,
or an MCP search tool"]) --> ENR["kb enrich"]
  G[("the code graph")] -.->|"repo name and top symbols
become the search terms"| ENR
  CON --> P[("isolated partitions")]
  ING --> P
  ENR --> P
  P -.->|"linked to the repos and
symbols they name"| G
a rectangle is something that runs a cylinder is something that persists a rounded box is a start or an end point

Each stage writes its own partition, so re-indexing a repo's code never disturbs its external links.

Connectors#

connect enriches the graph with external context. Four connectors ship, sharing one seam:

Adding another connector is a small, self-contained module, and its output lands in an isolated graph partition, so re-indexing a repo's code never disturbs its external links. Configure connectors by copying examples/kb.toml.example to ~/.contextlake/kb.toml.

Managing sources: the source command family#

Editing kb.toml by hand works, but for everyday use contextlake kb source commands let you add, test, and manage connectors without touching the config file. They rewrite kb.toml while preserving your comments, and work alongside hand-editing if you mix approaches.

The commands:

An example workflow:

contextlake kb source add                # interactive: what type? which workspace?
contextlake kb source list               # show what you've configured + status
contextlake kb source test my-atlassian  # does it work? what's in scope?
contextlake kb connect                   # now link repos to their items

init can also prompt you to connect a source during first-run setup, and doctor reports per-source reachability as part of its environment check, so hand-editing is optional; the CLI guides you through the whole flow.

Every fact carries its receipt. Each is provenance-stamped (source file + verified date) and confidence-tagged as one of three tiers, EXTRACTED (read straight from source/AST), INFERRED (a resolved call or link), or AMBIGUOUS (an unconfirmed candidate), and sanitized before it reaches an agent. The dashboard and the graph legend use these same tiers.

Query-driven enrichment#

contextlake kb enrich performs query-driven enrichment: it derives search terms from each repo's code graph (the repo's name and its top symbols by graph degree) and queries your connected sources (Atlassian Rovo search, or any mcp source with a tool and arg_template configured) with those terms, then stores the returned documents in a searchable, embedded @enrich:<repo> partition, idempotent and re-runnable across the whole fleet or a single repo:

contextlake kb enrich --workspace ~/work     # all indexed repos
contextlake kb enrich acme/catalog-api         # one repo

Prerequisites: the code graph must be indexed first (contextlake kb index), and at least one term-searchable source must be configured: either an mcp source with tool and arg_template keys, or an atlassian source. Sources without these capabilities (e.g. a plain files or web source) are skipped gracefully. Each repo's enrichment documents are stored in their own partition so they can be re-fetched without clobbering prior results, and are embedded (when the semantic tier is enabled) so they surface in semantic search results as document nodes tagged with their source (attrs.source). A result that names one of the repo's symbols is also linked straight to it (documented_by), so the enrichment lands on the graph rather than beside it. After contextlake kb wiki runs, enrichment docs are incorporated into the curated wiki as an attributed "External context" section, grounded to the code graph's terms.

Configuring document sources, including the built-in files source, plugin packages and MCP endpoints, is in Document sources and RAG.

When a source stops answering#

connect and enrich are the only stages that leave the machine, and a fleet run asks each source once per repo. If a source goes down mid-run, contextlake stops asking it rather than paying its timeout on every remaining repo: after three consecutive failures the source is skipped for 60 seconds, then one call is let through to see whether it came back. A run against an unreachable MCP server finishes in seconds instead of timeout x repos.

You will see this in the output, it is never silent, because "the source was down" and "the source had nothing" would otherwise look identical:

resilience: circuit OPEN for mcp:npx:https://mcp.example.test after 3 consecutive
  failure(s) (TimeoutError) -- further calls are skipped for 60s
resilience: skipping mcp:npx:https://mcp.example.test for 60s -- circuit open after 3
  consecutive failure(s) (TimeoutError); results from this source will be incomplete

The name in that line identifies the endpoint by transport and host only, never the rest of the URL, since a hosted MCP endpoint can carry a token in its path or query.

Failures the server rejected rather than failed on, an unknown tool, a bad token, are reported as themselves and never trip the skip: no amount of waiting fixes a wrong request. Raise timeout on the source if the server is merely slow.

When one repository is unreadable#

A repository that fails outright costs that repository, not the run. connect names it, skips it, and carries on with the rest:

  api-gateway: 'utf-8' codec can't decode byte 0x96 in position 99486: invalid start byte
⚠ Connect complete: 143 external link(s) stored
⚠ 1 of 20 repo(s) failed and were skipped; the rest were enriched. Re-run to retry
  them, or narrow with `contextlake kb connect <repo-id>`.

The exit code is non-zero when any repository was skipped, the same verdict kb index gives a workspace where one repo failed to parse: the graph an agent will cite from is not the one you asked for, so the run should not read as clean.

See also#

Next steps

Searching semanticallyGenerating the wiki