contextlake
Reference

Model providers

The pluggable embeddings and wiki backends: auto, built-in CPU, Ollama, OpenAI, Anthropic, and agent-CLI, with data-sharing posture and setup.

Both the embeddings and wiki tiers are pluggable and take a provider, defaulting to "auto". Pick by what may leave your machine and what hardware you have.

flowchart LR
  E["kb embed"] --> P{"which provider?"}
  W["kb wiki"] --> P
  P -->|"builtin"| BI["a small model,
in-process on CPU"]
  P -->|"ollama"| OL["a local Ollama daemon"]
  P -->|"openai, plus anthropic
for the wiki"| API["a hosted API, key
read from an env var"]
  P -->|"cli, for the wiki"| C["an agent CLI you
already pay for"]
  BI --> IN(["nothing leaves your machine"])
  OL --> IN
  API --> OUT(["your prompts leave your machine"])
  C --> OUT
a rectangle is something that runs a rounded box is a start or an end point a diamond is a decision

auto, the default, is the one value that picks for you: it resolves to a local Ollama that already has the model pulled, and otherwise to the built-in CPU model.

How provider=auto resolves: if a local Ollama is reachable AND already has the target model pulled, use it; else if the built-in extra is installed, use the built-in CPU model; else skip the tier.

Data-sharing posture per backend. Pick by what may leave your machine:

Backend Data leaves the machine? Auth
builtin / auto→builtin No: fully local CPU model none
ollama No: local daemon none
cli Yes: to whatever provider that CLI uses reuses the CLI's own login
anthropic / openai Yes: to the API endpoint env-var key (never stored)

Configuring the wiki LLM#

Two lines is enough (passing --llm on the CLI implies enabled = true), or set both in ~/.contextlake/kb.toml:

[llm]
enabled  = true
provider = "ollama"        # auto | builtin | ollama | openai | anthropic | cli
model    = "qwen2.5:3b"    # provider-specific model id (table below)
# base_url    = "http://127.0.0.1:11434"   # optional: defaults per provider (table below)
# api_key_env = "OPENAI_API_KEY"           # openai: env var holding the key (never the key)
# timeout    = 300          # seconds per model call; raise it for a slow CPU (ollama/openai)
council_size = 3           # review lenses that run (1-3); fewer = fewer calls per page
accept_score = 0.7         # mean council score a page must clear to be written
# review_provider = "anthropic"      # optional: have the COUNCIL review with a different
# review_model    = "claude-haiku-4-5"  # (usually stronger) backend than the one generating

Unset, review_provider means the council reviews with the same client that generated the page, which is the historical behavior. Setting it lets a cheap local generator be gated by a real model; it costs pages × council_size extra calls against that provider, so it is opt-in and never inferred from an API key that happens to be in your environment. See Generate the wiki for the full rationale.

provider example model notes
builtin OpenVINO/Qwen2.5-Coder-0.5B-Instruct-int4-ov a HF repo id holding a pre-converted OpenVINO IR model
ollama qwen2.5:3b, llama3.1, llama3.2:3b must be ollama pulled first
openai gpt-4o-mini, or your server's model id base_url defaults to https://api.openai.com/v1; point it at a local openai-compatible server to override
anthropic claude-haiku-4-5, claude-opus-4-8 base_url defaults to https://api.anthropic.com; key from ANTHROPIC_API_KEY
cli n/a, the CLI picks its own model command = the agent CLI to invoke (default claude); reuses that CLI's login

base_url is resolved per provider, so you only set it to reach a proxy or a local openai-compatible server, never just to make anthropic/openai work.

CLI flags override the toml and now work on bootstrap too: contextlake bootstrap --llm ollama --llm-model qwen2.5:3b.

Installing the built-in LLM#

contextlake doctor --fix llm-local     # or: pip install "contextlake[llm-local]"

An ordinary wheel install. No compiler, no custom index, no --only-binary pin. Install and upgrade covers each channel, including why the standalone binary and the full Docker image need nothing at all.

What it runs, and what it costs you#

The builtin wiki model is a small int4 OpenVINO IR model driven by openvino-genai. The default is OpenVINO/Qwen2.5-Coder-0.5B-Instruct-int4-ov: Apache-2.0, published pre-converted by the OpenVINO project, 349 MB, downloaded on first wiki run into ~/.contextlake/models.

The runtime pulls openvino and openvino-tokenizers and nothing else -- in particular neither torch nor transformers, which is why a local generative tier does not cost you a multi-gigabyte install. "Pre-converted" is the load-bearing part: converting a stock checkpoint to OpenVINO IR yourself needs optimum-intel, which does pull torch.

Why not llama.cpp#

Earlier releases ran a GGUF through llama-cpp-python, and it needed a per-accelerator wheel index because upstream publishes no wheels to PyPI at all -- so a plain pip install compiled C++ from source and wanted cmake and a compiler. It also had no wheel for CPython 3.14 on any x86_64 platform, which pinned the whole project's container base image to an older Python.

openvino-genai ships ordinary manylinux wheels for CPython 3.10 through 3.14, so the index, the --only-binary pin and the compiler requirement are all simply gone. If you want GPU inference, use Ollama (below) rather than a different index.

Using Ollama for the wiki#

Ollama is a standalone local model server. It sidesteps the native Python build, and a 3B-8B model writes much better wiki pages than the 0.5B built-in.

A) Ollama inside WSL / Linux (simplest, localhost just works):

curl -fsSL https://ollama.com/install.sh | sh   # installs + starts the daemon
ollama pull qwen2.5:3b                            # ~1.9GB, one-time
contextlake bootstrap --llm ollama --llm-model qwen2.5:3b   # whole layer in one command
# or per repo:  contextlake kb wiki <repo> --llm ollama --llm-model qwen2.5:3b

contextlake defaults base_url to http://127.0.0.1:11434, so usually nothing else to set.

B) Ollama on Windows, contextlake in WSL (the cross-boundary case). WSL2 is a separate network namespace, so a Windows Ollama bound to 127.0.0.1 is not reachable from WSL (localhost:11434 → connection refused). Two fixes:

Pull the model on whichever side runs Ollama: ollama pull qwen2.5:3b.

How much does the model matter?#

The wiki's quality is bounded by the model behind it. The graph facts fed in are identical; the difference is how well the model turns them into prose (and the verification council rejects weak pages regardless, so a smaller model mostly means more rejections and blander accepted pages).

The constraint that decides this for most people is hardware, not model quality. Generating one wiki page is not one model call: it is the page, plus one review from each council lens. A rejected page is reported and skipped, not rewritten, so a rejection costs you the page rather than another round of calls. So the question is not whether a 1.5B-3B model writes better prose than the built-in 0.5B (it does) but whether your host can finish that many calls: if a single call takes minutes, which is the ordinary case for a 1.5B-3B model on a CPU with no GPU, the run reaches the default 300-second timeout below before a page is done. The built-in 0.5B stays under it, and pays for that with thin, generic prose the council often rejects. The same 1.5B-3B models answer in seconds once Ollama has a GPU (including Ollama on a Windows host with a discrete GPU, reached from WSL). So:

If your local model is slow, raise the per-call timeout instead of letting every page fail silently: timeout in [llm] (seconds, default 300):

[llm]
provider = "ollama"
model    = "qwen2.5:3b"
timeout  = 1200        # give a slow CPU room; default is 300s (5 min)

If the endpoint stops answering rather than being slow, raising timeout makes things worse: a wiki run calls the model once per page. After three consecutive endpoint failures (a timeout, a refused connection, a 5xx) the provider is skipped for 60 seconds and then probed once, so a dead Ollama daemon or an unreachable API costs the run a few timeouts instead of one per page, and says so in the log. A rejected request is exempt and always reported as itself, so model "..." not found, try pulling it first keeps telling you to run ollama pull however many pages are left.

Notes: behind a TLS-inspecting corporate proxy the first built-in download needs your OS CA bundle (export REQUESTS_CA_BUNDLE / SSL_CERT_FILE; see docs/releasing.md). Don't switch the embedder model/dimension against an existing vector store without re-embedding from scratch, a guard refuses the mismatch. The prebuilt Docker image (ghcr.io/sayak-sarkar/contextlake) bundles these models so nothing downloads at runtime. See examples/kb.toml.example.

See also#

Next steps

Generate the wikiInstall and upgrade