SwarmMarshal
Function-scoped proof
A route is admitted only if a current, non-expired benchmark receipt exists for the exact function it will perform. A model that aces chat can still be rejected for the message pipeline — proof doesn't transfer between jobs.
Technology deep dive
Model Scout: routing on evidence, not vibes.
Most apps let you pick "the model" from a dropdown and hope. SwarmMarshal runs a background steward that continuously studies every model it could use — cloud APIs, local Ollama and LM Studio installs, and subscription CLIs — and only admits a route when it holds current, function-specific proof that the model can do that exact job.
No receipt, no route.
Every automatic route decision — an advisor's pick, an internal override, a fallback — is treated as untrusted input. Before it can reach a provider it must be re-admitted through a fresh Model Scout decision carrying an audit correlation id.
Durable audit chain
Every selection writes a routing-audit receipt: the route before and after, the hard requirements, the prompt hash, and a correlation id. If the receipt cannot be committed, the model call does not happen. There is no unexplainable route.
Self-recovering
Missing proof never blocks a user-facing request — Scout queues bounded background "proof" work instead, deduplicated into recovery episodes that notify you when a lane heals. Cold benchmarks run behind the scenes, not in your critical path.
User pins stay sovereign
LocalOnly enforced before any cloud advisor
Hard requirements checked at one capability boundary
Four evidence streams feed one ranking.
Scout ranks candidates on a three-axis score — capability, speed, cost — with configurable weights and Economy / Balanced / Frontier tiers.
| Source | What it contributes |
|---|---|
| Local benchmark catalog | Every calibration run this install has ever performed, keyed by function, prompt hash, and hardware. |
| External model feeds | Model catalogs and pricing feeds provide priors on capability, context window, and per-token cost — normalized across each provider's divergent pricing format. |
| Published function benchmarks | SHA-versioned benchmark bundles published on this website, built from privacy-safe aggregate results — never raw prompts or personal data. |
| Runtime telemetry | Observed latency, error rates, and health from this machine's actual calls — the ranking learns what your hardware really delivers. |
Same model, different machine? Different decision.
Capability verdicts are cached under a composite key: prompt schema hash, provider and model, local model digest, runtime version, quantization, and a hardware fingerprint. Change any of them and stale proof invalidates itself — but a daily rebuild doesn't.
JSON-contract probe
Local candidates get an active probe — a controlled, tiny structured-output request — before any real work. Failures are classified precisely: empty response, thinking-without-content, parse failure, wrong model, timeout. Transient failures are distinguished from definitive ones so a model isn't permanently negative-cached over a hiccup.
Tool-use calibration
Native tool-calling is proven per model, per schema, per runtime — not assumed from a marketing page. Only definitive outcomes (passed, failed, excluded) become durable; the classic "this local model returns zero tool calls" failure is caught before it wastes a turn.
Sometimes the model is fine and the serving layer is sick.
A local model that re-pays a 22-second prompt prefill on every call isn't a bad model — it's a cache-eviction problem. Scout's serving-health lane diagnoses the serving layer separately from model quality.
-
Capture serving telemetry Prefill time, decode rate, and model-load events are recorded on every local call, alongside runtime introspection of what's actually loaded and how.
-
Run deterministic detectors Named anomaly signatures — repeated full prefill, alternation eviction, warm-up lag, load thrash — fire on the telemetry, not on guesswork.
-
Fire synthetic probes Same-prefix-repeat and alternation probes confirm a suspected pathology before anything is changed.
-
Remediate and verify A two-tier remediation catalog applies fixes — an in-app prefix-cache primer stays silent; environment tuning that changes your machine asks first — then re-probes to confirm the fix took.
One GPU, many callers, zero churn.
The local runtime manager serializes inference to one in-flight request, with a drain gate that lets user-waiting calls jump ahead of background work.
Priority drain gate
Background enrichment yields to your question. The queue drains interactive work first, so a batch job never makes the assistant feel slow.
Keep-alive lanes
Vision, embedding, and interactive workloads each get their own keep-alive policy, so the runtime stops paying model load/unload churn every time work alternates.
Observable
Hot-model-per-lane and switch counts are tracked, so "why was that slow?" has an answer in data instead of a shrug.
Advisors recommend. The governor decides. The guard verifies.
Pluggable, precedence-ordered advisors shape routes before the proof gate — each one defers cleanly under privacy or urgency constraints.
Budget pressure
As spend approaches the cap, work steps down to local or economy tiers before the cap is hit — a soft landing instead of a hard cutoff.
Quota step-off
When a Claude Code or Codex subscription lane is saturated, the advisor steps off to paid API headroom (or local) rather than queueing behind an exhausted plan.
Urgency & timing
User-waiting calls can justify a faster or better route than overnight background work — the same function routes differently at different moments.
Your coding-agent subscription is a first-class model route.
SwarmMarshal detects, installs, and authenticates subscription-backed CLIs (Claude Code, Codex, Gemini CLI) from inside the app — cross-platform executable discovery across npm, volta, nvm, scoop, and desktop-app install locations, CPU-architecture mismatch detection, auth-state probing, and OAuth launch with API-credential scrubbing so the CLI session can't silently bill your API key instead of your subscription.
Already-paid capacity used first
Newest-semver resolution
Auth probed, not assumed
Keep reading
Where the proof gate bites hardest.
The message pipeline is Model Scout's strictest customer — every promotion runs through calibration gates and production replay.