contextlake Docs
Build your knowledge base

Model providers

The pluggable embeddings and wiki backends: auto, built-in CPU, Ollama, OpenAI, Anthropic, and agent-CLI, with data-sharing posture and setup.

Both the embeddings and wiki tiers are pluggable and take a provider, defaulting to "auto". Pick by what may leave your machine and what hardware you have.

How provider=auto resolves: if a local Ollama is reachable, use it; else if the built-in extra is installed, use the built-in CPU model; else skip the tier.

Data-sharing posture per backend. Pick by what may leave your machine:

Backend Data leaves the machine? Auth
builtin / auto→builtin No: fully local CPU model none
ollama No: local daemon none
cli Yes: to whatever provider that CLI uses reuses the CLI's own login
anthropic / openai Yes: to the API endpoint env-var key (never stored)

Configuring the wiki LLM#

Two lines is enough (passing --llm on the CLI implies enabled = true), or set both in ~/.contextlake/kb.toml:

[llm]
enabled  = true
provider = "ollama"        # auto | builtin | ollama | openai | anthropic | cli
model    = "qwen2.5:3b"    # provider-specific model id (table below)
# base_url    = "http://127.0.0.1:11434"   # ollama, or a local openai-compatible server
# api_key_env = "OPENAI_API_KEY"           # openai: env var holding the key (never the key)
# timeout    = 300          # seconds per model call; raise it for a slow CPU (ollama/openai)
council_size = 3           # review lenses that run (1-3); fewer = fewer calls per page
accept_score = 0.7         # mean council score a page must clear to be written
provider example model notes
builtin Qwen/Qwen2.5-0.5B-Instruct-GGUF a HF GGUF repo id; model_file picks the quant
ollama qwen2.5:3b, llama3.1, llama3.2:3b must be ollama pulled first
openai gpt-4o-mini, or your server's model id base_url = the API's /v1

CLI flags override the toml and now work on bootstrap too: contextlake bootstrap --llm ollama --llm-model qwen2.5:3b.

Why the built-in LLM needs a prebuilt wheel (or a compiler)#

The builtin wiki model runs a GGUF model through llama-cpp-python, Python bindings around llama.cpp, a C++ inference engine. Native (C/C++) packages ship as prebuilt binary wheels, one per (OS, CPU, Python version). Two consequences explain why an extra step is sometimes needed and why contextlake can't do it for you:

  1. A dependency can't carry an index URL. contextlake[llm-local] can only name llama-cpp-python; Python packaging (PEP 508) deliberately forbids pinning an --extra-index-url in a package's metadata (for reproducibility and supply-chain safety). So contextlake cannot make pip look anywhere but your configured indexes (PyPI by default), only your pip command can add one.
  2. PyPI lags brand-new Pythons. Wheels are uploaded per interpreter version; a just-released Python (e.g. 3.14) often has no wheel on PyPI yet, so pip falls back to the source tarball and tries to compile, which needs cmake + a C/C++ compiler you may not have installed. That is the build failure you saw.

The maintainer also publishes a prebuilt CPU wheel index carrying wheels PyPI doesn't have yet, so pointing pip at it skips compilation entirely, no compiler needed:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

Belt-and-suspenders: --only-binary :all:. On a compiler-less machine, add this flag so pip refuses to fall back to a source build for any package, you get a clean "no matching distribution" message instead of a wall of cmake/compiler errors. :all: is the all-packages token (its opposite is --no-binary). Combined with the CPU-wheel index, this installs the built-in LLM on a brand-new Python with no toolchain:

pip install --only-binary :all: llama-cpp-python \
  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

The trade-off is deliberate: if a wheel genuinely doesn't exist for your platform, the command stops with an actionable error rather than attempting a build that can't succeed.

On a mainstream Python (3.10-3.13) none of this applies: pip install "contextlake[llm-local]" finds a PyPI wheel and Just Works. It is specifically the bleeding-edge-interpreter case that needs the extra index. The cleanest way to avoid the native build altogether is to use Ollama (below), a standalone binary with no Python compile step.

Using Ollama for the wiki#

Ollama is a standalone local model server. It sidesteps the native Python build, and a 3B-8B model writes much better wiki pages than the 0.5B built-in.

A) Ollama inside WSL / Linux (simplest, localhost just works):

curl -fsSL https://ollama.com/install.sh | sh   # installs + starts the daemon
ollama pull qwen2.5:3b                            # ~1.9GB, one-time
contextlake bootstrap --llm ollama --llm-model qwen2.5:3b   # whole layer in one command
# or per repo:  contextlake wiki <repo> --llm ollama --llm-model qwen2.5:3b

contextlake defaults base_url to http://127.0.0.1:11434, so usually nothing else to set.

B) Ollama on Windows, contextlake in WSL (the cross-boundary case). WSL2 is a separate network namespace, so a Windows Ollama bound to 127.0.0.1 is not reachable from WSL (localhost:11434 → connection refused). Two fixes:

Pull the model on whichever side runs Ollama: ollama pull qwen2.5:3b.

How much does the model matter?#

The wiki's quality is bounded by the model behind it. The graph facts fed in are identical; the difference is how well the model turns them into prose (and the verification council rejects weak pages regardless, so a smaller model mostly means more rejections and blander accepted pages).

A measured A/B on one repo (contextlake, 1810 graph nodes, identical facts + 3-lens council) on a CPU-only host (no GPU, e.g. WSL2 without GPU passthrough):

model where speed result
built-in Qwen2.5-0.5B (GGUF) in-process CPU fast enough page written (~119 words), accurate but thin and generic
Ollama qwen2.5:1.5b CPU (no GPU) ~1.7 tok/s timed out, a full page + reviews needs ~10 min/repo
Ollama qwen2.5:3b CPU (no GPU) ~0.85 tok/s timed out, ~20 min/repo

The lesson is about hardware, not model quality: a 1.5B-3B model writes better prose than the 0.5B, but on a CPU-only box it is too slow to finish a page in reasonable time (and, at fleet scale, wholly impractical). Those models shine when Ollama has a GPU, e.g. Ollama on a Windows host with a discrete GPU generates in seconds, not minutes. So:

If your local model is slow, raise the per-call timeout instead of letting every page fail silently: timeout in [llm] (seconds, default 300):

[llm]
provider = "ollama"
model    = "qwen2.5:3b"
timeout  = 1200        # give a slow CPU room; default is 300s (5 min)

Notes: behind a TLS-inspecting corporate proxy the first built-in download needs your OS CA bundle (export REQUESTS_CA_BUNDLE / SSL_CERT_FILE; see docs/releasing.md). Don't switch the embedder model/dimension against an existing vector store without re-embedding from scratch, a guard refuses the mismatch. The prebuilt Docker image (ghcr.io/sayak-sarkar/contextlake) bundles these models so nothing downloads at runtime. See examples/kb.toml.example.

See also#

Next steps

Bootstrap and keep freshDashboard