contextlake
Operate it

Scheduling runs

Measure a run, work out an interval, and install a systemd timer or crontab entry that keeps it current on its own, set/reset/unset, ad-hoc jobs, and the interval formula.

contextlake schedule measures how long a run takes and how often your repositories change, works out an interval from those two numbers, and installs a background job that keeps the mirror and the knowledge layer current on its own. It runs on the core tier, so it works even without the [kb] extra. This page covers reading its recommendation, installing it, the set/reset/unset model, ad-hoc jobs, the interval formula, its config keys, platform differences, and containers.

Prerequisites#

schedule recommend: what interval your measurements suggest#

$ contextlake schedule recommend
⚠ Recommended interval: 6h
  Because: no measured runs yet, so this is the built-in default of 6h, not a measurement. The first completed run replaces it.
  Nothing has been measured yet. Run `contextlake mirror sync` or `contextlake bootstrap` once, or install the schedule and let the first run replace this default.

  Install it:  contextlake schedule install

That is the cold-start default: nothing has run yet, so there is nothing to measure. Once a few runs have completed, the same command reads real numbers:

$ contextlake schedule recommend
✓ Recommended interval: 70m
  Because: duty cycle: the median incremental run takes 7m, and at 10% of wall-clock time that needs 70m between runs
  From 3 measured run(s) over 0.1 day(s)
    duty-cycle floor: 70m
    activity floor:   20m

  Install it:  contextlake schedule install

The glyph names the answer: ⚠ for a default nobody has measured yet, ✓ for one built from real runs. recommend changes nothing. Add --json for the same answer as a machine-readable payload (interval, basis, reason, floor_duty_seconds, floor_activity_seconds, history, and so on).

schedule install: measure, decide, and install#

contextlake schedule install

This is idempotent: run it again after a config change or after more runs have accumulated, and it recomputes and rewrites the job in place. It writes to three places:

What Where
The job record (name, command, interval setting, failure count, last run) schedule-jobs.json, beside the project cache
The measured run history, appended to by every run schedule-history.jsonl, beside the project cache
The platform unit that fires a systemd user timer (~/.config/systemd/user/contextlake-<name>.timer + .service), or a marked block in your crontab

"Beside the project cache" is the same directory cache_dir in Configuration resolves to: ~/.cache/contextlake/<workspace>-<id> by default.

install picks systemd if a systemd user session answers, cron otherwise, and prints the platform's own unit file (or crontab line) when it cannot write one, so you can install it by hand. Force a specific platform with --platform systemd or --platform cron.

The default job runs contextlake bootstrap on most cycles, and switches to contextlake bootstrap --force (a full rebuild: every repository re-parsed, every node re-embedded) once schedule_full_every has passed since the last successful full run.

Set, reset, and unset#

Three layers decide what interval a job runs on, and only one of them is read back as the truth:

Layer Role Changed by
The INI (schedule_interval, or --interval on install) Supplies the starting value when the job is created. Ignored after that. Editing .contextlake.ini before the first install
The job record (schedule-jobs.json) Authoritative. Holds auto or a pinned duration, plus the failure count that drives backoff. install, interval, reset, uninstall
The platform unit (the systemd timer or the crontab line) Rendered from the job record's resolved interval. Never read back as truth; status reads it only to report drift or cron's rounding. Rewritten every install / reset / interval

Three commands change the job record:

Discarding history is recoverable#

--purge and reset --history both ask first, printing the count of runs and the span of days they cover, because a useful median takes days to earn back:

$ contextlake schedule --history --yes reset
About to discard 3 measured run(s) spanning 0.1 day(s) (2026-08-24T01:00:00Z to 2026-08-24T03:00:00Z).
  The recommender starts cold and re-learns from the next run.
✓ Discarded 3 measured run(s).
  Saved to /home/you/.cache/contextlake/work-ab12/schedule-history.jsonl.discarded in case you want it back.

The discard renames the history file to a .discarded sidecar instead of deleting it. To get it back, move the sidecar over the live file yourself:

mv schedule-history.jsonl.discarded schedule-history.jsonl

Only one sidecar is kept. A second discard replaces it, so this is undo for the discard you just did, not an archive of every one you have ever run.

Ad-hoc jobs#

schedule interval creates or replaces a named job that runs any contextlake command on its own interval, separate from the default bootstrap cycle. Give it a duration (or auto), the word run, a -- separator, then the command:

contextlake schedule interval 6h run -- kb wiki --force
contextlake schedule --job wiki-refresh interval 2h run -- kb wiki --namespace acme/widgets

The -- separator is not optional the moment your command has its own flags. schedule parses everything before -- as its own flags and action; without it, a flag meant for the trailing command is read as a flag of schedule itself and the whole thing fails:

$ contextlake schedule interval 6h run kb wiki --force
✗ '--force' isn't a flag on 'schedule'

It's used by: bootstrap, kb embed, kb index, init, kb steer, kb wiki.

Run 'contextlake schedule --help' to see schedule's own flags.

A flag that no other contextlake command uses fails the same way, with Unknown flag: '<flag>' (on 'schedule') instead. Either way, the fix is the same: add --.

The command is validated before the job is written: it must parse, and if it needs the [kb] extra and that extra is not installed here, interval warns rather than installing a job that fails on every run. A job named schedule itself is refused outright, because that would recurse.

Every job appends to the same history file, and each record carries the job that wrote it. Reads are scoped to one job: a job's full-rebuild schedule and its measured duration come from its own runs only. Without that, a rebuild by one job satisfied schedule_full_every for all of them, and a two-minute kb index and a forty-minute bootstrap shared one median. Records written before the job name was recorded carry no job and count as the default job's.

The interval formula#

floor_duty     = median(incremental run duration) / duty_cycle
floor_activity = k / change_rate
interval       = clamp(max(floor_duty, floor_activity), schedule_min, schedule_max)

floor_duty is a cost bound: never occupy more than duty_cycle of wall-clock time. floor_activity is a freshness bound: there is no point running more often than the fleet produces roughly k changed repositories. Whichever bound needs the longer gap wins.

floor_activity needs a measurement that only the index stage produces: how many repositories changed on a run. That stage is part of the knowledge layer, so on an install without the kb extra nothing ever records it and the activity bound never engages. The interval then rests on floor_duty alone, which is correct behaviour rather than a fault. contextlake schedule recommend says so on the activity floor line, and its --json output carries an activity field reading not-measured, no-change or measured.

Three worked examples, all at the default duty_cycle = 0.10 and k = 1.0:

Median incremental run Change rate floor_duty floor_activity Interval Why
7 min ~3 repos/hour 70m 20m 70m Duty wins: a busy fleet, so the cost bound is what limits how often you can afford to run
7 min ~0.1 repos/hour 70m 10h 10h Activity wins: a quiet fleet, so there is nothing to gain from running every 70 minutes
40 min ~3 repos/hour 6.7h 20m 6.7h Duty wins again: a bigger, slower fleet costs more per run, which raises the cost bound past the freshness one

Kubernetes and OpenShift#

contextlake schedule --platform k8s install            # applies with kubectl, or oc
contextlake schedule --platform k8s install --json     # see the manifest without applying

One adapter covers both, because OpenShift is Kubernetes with a stricter default security context and the manifest is written to satisfy the stricter one. Three properties of it are deliberate:

The schedule is a cron expression, so it rounds the way cron does and says when it rounded. Nothing is patched in the cluster on its own: changing an interval means re-running schedule install. A process that rewrote a CronJob in the background would need cluster-write rights for the life of the schedule, which is more authority than the benefit is worth.

AWS and Azure#

contextlake schedule --platform aws install       # EventBridge Scheduler firing an ECS task
contextlake schedule --platform azure install     # a Container Apps Job on a cron trigger

On EKS and AKS, use --platform k8s instead. Both are Kubernetes, so the CronJob adapter serves them and brings concurrencyPolicy: Forbid with it.

Neither cloud service has an equivalent of Forbid. Both will start a second execution while the first is still running, so single-writer protection there rests on the store's advisory lock: the second run takes it, fails, and skips with a logged reason. That is the same protection a bare schedule run has, and weaker than what a cluster gives.

Both need persistent storage for the same reason a container does anywhere: EFS on AWS, Azure Files on Azure. An ephemeral store re-indexes the whole fleet every run instead of doing an incremental pass, which schedule run refuses without --allow-ephemeral.

The two round differently, and that is not an inconsistency. EventBridge takes rate(N minutes), so AWS rounds to whole minutes. A Container Apps Job schedule trigger is a cron expression, so Azure rounds the way cron does and cannot express 70 minutes at all.

Neither adapter ships an SDK. Both shell out to an already-authenticated aws or az.

Configuration#

Ten schedule_* keys, read from the same .contextlake.ini [contextlake] section as work_dir and the rest of the mirror settings. Full descriptions and defaults are in Configuration; the worked block below shows all ten together:

[contextlake]
schedule_interval = auto
schedule_min = 1h
schedule_max = 24h
schedule_duty_cycle = 0.10
schedule_full_every = 7d
schedule_adjust_threshold = 0.5
schedule_gate_retry = 10m
schedule_on_battery = skip
schedule_require_idle = false
schedule_max_load =

A typo or an out-of-range value logs a warning and falls back to the built-in default, rather than stopping a run at 3am when nobody is there to fix it.

Platform differences#

Seven adapters exist today: systemd and cron on Linux, launchd on macOS, Task Scheduler on Windows, a CronJob for Kubernetes and OpenShift, EventBridge Scheduler with ECS on AWS, and a Container Apps Job on Azure.

Cluster and cloud adapters are never auto-detected. kubectl, aws or az being on your PATH does not mean a schedule belongs in whatever cluster your kubeconfig points at or whatever account your credentials reach, so reach them with --platform k8s|aws|azure. They are still registered, so schedule list reports an orphan there like anywhere else.

Two of the seven are verified by running them: systemd and cron. The development machine is Linux with no cluster and no cloud account, so the other five (launchd, Task Scheduler, Kubernetes, AWS, Azure) are verified by asserting what they render and the exact launchctl, schtasks, kubectl, aws and az arguments they would run. That is a real check and a weaker one than execution, and the difference is worth knowing before you rely on one.

The rendered documents are parsed rather than string-matched: the Kubernetes manifest through a YAML loader, the AWS schedule through a JSON one. A document that contains the right words but is malformed fails here instead of in your cluster or account.

systemd (user timer) launchd (LaunchAgent) Task Scheduler cron
Interval Exact, any duration Exact, any whole number of seconds (StartInterval) Rounded to whole minutes (/SC MINUTE /MO n), reported the same way cron's rounding is Rounded. Cron's minute field only divides the hour, so */70 does not mean "every 70 minutes". The adapter picks the nearest interval it can express, rounding down above one minute and up below it, and tells you the difference
Missed a run while asleep or off Replays it (Persistent=true) Replays it on wake Lost. schtasks cannot set StartWhenAvailable Lost. cron has no equivalent
Runs while logged out Only if linger is on: loginctl enable-linger $USER. schedule status reports when it is off No. A LaunchAgent runs in the user's GUI session Yes, a task runs under its registered account Yes, cron runs independent of any login session
Skips on battery ConditionACPower=true in the unit itself The schedule_on_battery gate at run time. launchd has no equivalent condition The schedule_on_battery gate at run time The schedule_on_battery gate at run time
schedule_require_idle Same limitation as cron: see below Same limitation: a LaunchAgent sets no XDG_SESSION_ID either Same limitation Inert. Neither cron nor a systemd timer sets XDG_SESSION_ID, which idleness detection needs, so the gate cannot tell and passes every time. status and the run itself both warn when this is on

Containers#

A container's writable layer is thrown away on every restart. Running the scheduler there without persistent storage would mean every cycle re-indexes the whole fleet from scratch, so schedule run refuses:

$ contextlake schedule run
Refusing to run: this container's state does not survive a restart, so every run would re-index the whole fleet from scratch.
  Mount a volume at the cache directory (a PVC on Kubernetes, EFS on AWS, Azure Files on Azure), or pass --allow-ephemeral if that is what you want.

The check is whether the cache directory sits on its own mount point, not what filesystem type it reports: a container's own writable layer is part of the root mount, while any volume you attach (a PVC, an emptyDir, a bind mount, EFS, Azure Files) shows up as its own mount, whatever its underlying filesystem. Fix it by pointing cache_dir at a mounted volume, or pass --allow-ephemeral if a from-scratch run every cycle is what you want. There is no daemon to install inside a container; run the cycle in the foreground instead:

contextlake schedule run --foreground

--foreground loops here, sleeping between cycles, and does not re-read the config file mid-loop; restart the container to pick up an edited schedule_interval.

Troubleshooting: it is installed but nothing runs#

Start with contextlake schedule status. It reads the job record, the platform unit, and the measured history, and reports every way they disagree:

  1. Linger is off (systemd only). A user timer does not fire while you are logged out unless linger is on. status reports this; fix it with loginctl enable-linger $USER.
  2. The interpreter has moved. status resolves the path the installed unit runs and reports it missing if the venv was moved or deleted. Re-run contextlake schedule install from the current install to fix it.
  3. The unit was removed outside contextlake. status reports "recorded but NOT installed" when the job record exists but the systemd timer or crontab block does not. Re-run contextlake schedule install.
  4. The job record was removed and the unit was not. schedule list reports these as orphaned units, by name and platform. Such a unit keeps firing on schedule, and uninstall cannot reach it, because it resolves a job name through the record that is gone. Recreate the record with contextlake schedule --job NAME install and then uninstall it, or delete the unit on the platform. list also names any platform it could not enumerate, so an empty result is never mistaken for one that was checked.
  5. schedule_require_idle is on and you expected it to gate a run. It cannot: see Platform differences.
  6. Consecutive failures are backing off the interval. status reports the failure count. Fix the underlying command failure (run it by hand first) and the next success resets it to zero.
  7. Nothing has fired yet at all. Check the platform directly: systemctl --user list-timers for systemd, crontab -l for cron. A gap between what status reports and what the platform shows is the bug to chase.

See also#

Next steps

Bootstrap and keep it freshConfiguration