v0.3.5 · evidence-bound

Your LLM has no immune system.

Screen untrusted text with structural checks and optional sacrificial-model probes before your production model acts. Open source. Self-hosted. Coverage-aware.

Star on GitHub

Apache 2.0 | uvx --from little-canary==0.3.5 little-canary --version

How it works

Three steps. The local Ollama path needs no cloud account or API key; provider-backed modes use their configured endpoint.

01

Install

One command. No containers, no cloud keys.

Pinned PyPI release: little-canary 0.3.5
02

Detect

Behavioral probing, not pattern matching. The canary gets attacked so your LLM doesn't.

Input → Structural filter → Canary probe → Your LLM
03

Results

Block, flag, or pass. Your rules, your thresholds.

verdict.safe verdict.blocked_by verdict.advisory
app.py
1from little_canary import SecurityPipeline
2
3pipeline = SecurityPipeline(canary_model="qwen2.5:1.5b", mode="full")
4verdict = pipeline.check(user_input)
5
6if not verdict.safe:
7    return "Sorry, I couldn't process that request."
8
9# Prepend advisory to your existing system prompt
10system = verdict.advisory.to_system_prefix() + "\n" + your_system_prompt
11response = your_llm(system=system, messages=[{"role": "user", "content": user_input}])

Evidence, with boundaries

Current checks, model-dependent holds, and historical material are kept in separate lanes so a passing build never masquerades as a performance certificate.

Current package

v0.3.5

Verified on PyPI and in the canonical source metadata

Luna pilot

400 assigned

Fixed TensorTrust sample with all-assigned failure scoring

Structural corpus

180 cases

Executed twice with an identical decision vector

Archived source

SWH snapshot

Stable Software Heritage identifier linked below

PASSControlled experiment · 2026-08-15

Advisory results were directionally favorable, not statistically decisive.

Little Canary's canonical advisory was supplied to Luna as a developer-role instruction while the original TensorTrust user prompt stayed unchanged. The result is scoped to Luna through the Codex final-answer scaffold using ChatGPT OAuth—not ChatGPT generally and not production safety.

Sample
200 cases
400 assigned evaluations
Advised calls
103
297 exact raw reuses
Retries
0
One call per flagged assignment
Error policy
All assigned
Missing and errors count as failures

HRR

Hijacking robustness

+3 pp

71% → 74%

Exact 95% intervals: 61.1–79.6% raw; 64.3–82.3% advised. Paired p=0.508.

ERR

Extraction robustness

+2 pp

82% → 84%

Exact 95% intervals: 73.1–89.0% raw; 75.3–90.6% advised. Paired p=0.625.

DV

Defense validity

-0.5 pp

53.5% → 53%

Exact 95% intervals: 46.3–60.6% raw; 45.8–60.1% advised. Paired p=1.000.

Zero retries · tools disabled · low reasoning and verbosity · parser and technical failures score as failuresOpen evidence artifact

Reproduced · 2026-08-13

Deterministic software checks

These checks exercised the v0.3.3 evidence boundary without calling a model. They support installability, offline behavior, and a repeatable structural decision vector—nothing broader.

source 8fc21d4fa3adf23f75ba4d32ca5a6e7af371c44a

Offline software test suite

342 / 342 tests passed

PASS

Python 3.14.6 against the v0.3.3 code boundary at revision 8df689c3; network behavior is mocked. Later canonical-main changes are metadata-only relative to that runtime boundary.

Committed structural corpus

180 cases executed twice with an identical decision vector

PASS

Structural layer only: 25/160 expected attacks blocked and 0/20 expected-safe cases blocked. This is not a full-pipeline rate.

Python static lint

Ruff completed with no findings

PASS

Covers little_canary and tests at the v0.3.3 runtime boundary, revision 8df689c3.

Provider-dependent

Scoped model evidence

The current Luna pilot has a frozen sample, model and transport identity, scoring rules, complete call ledger, and hash-bound receipt. Its result remains local to that evaluated configuration.

Admitted model replay

REPLAY UNAVAILABLE

HOLD

Version 0.3.3 intentionally packages no provenance-bound replay fixture.

TensorTrust Luna advisory pilot

HRR 71% → 74%; ERR 82% → 84%; DV 53.5% → 53.0%

PASS

Controlled 200-case / 400-assignment pilot on Luna through the Codex final-answer scaffold using ChatGPT OAuth. The attack deltas were directionally favorable but not statistically decisive (paired p=0.508 and p=0.625).

Production safety and generalization

No production or model-general safety rate is licensed

NOT_ESTIMABLE

TensorTrust is a fixed stress-test sample. This run does not estimate deployment prevalence, unseen-model behavior, production false-positive rate, or universal prevention.

Historical cross-model tables

Retained as legacy material; not reproduced for 0.3.3

HOLD

Repository history includes earlier TensorTrust and multi-model artifacts, but the current benchmark guide does not admit them as a performance certificate.

Legacy record · not a current certificate

Historical TensorTrust and cross-model figures

Values retained from the prior site for audit context. They were not reproduced against the 0.3.3 evidence boundary and are not current performance claims. The prior site recorded 396/400 caught, including 241 structural blocks and 4 bypasses. Those figures remain visible here for traceability, not endorsement.

HOLD

Opus 4.6 API

HOLD
Recorded alone
93.2%
Recorded + Canary
99%

Qwen3 4B

HOLD
Recorded alone
91.8%
Recorded + Canary
96.8%

Llama 3.2 3B

HOLD
Recorded alone
86.8%
Recorded + Canary
94.8%

Llama 3.1 8B

HOLD
Recorded alone
86.2%
Recorded + Canary
94.5%

Dolphin3 8B

HOLD
Recorded alone
85.5%
Recorded + Canary
94.2%

Mistral 7B

HOLD
Recorded alone
83.6%
Recorded + Canary
93.5%

Gemma3 12B

HOLD
Recorded alone
80.5%
Recorded + Canary
92.2%
Current evaluation boundarySoftware Heritage snapshotEvaluation evidence class: controlled experiment

Security tools shouldn't be black boxes.

Every layer of Little Canary is open source under Apache 2.0. You can read the detection logic, audit the behavioral analysis, and inspect the tests, corpora, historical artifacts, and evaluation boundaries.

We believe the security layer protecting your LLM should be as transparent as the models it guards. No obfuscated classifiers. No API-only access. No trust-us-it-works.

Apache 2.0

Deployment modes

Run it your way. No vendor lock-in.

Self-Hosted

Run entirely on your infrastructure with Ollama. No data leaves your network. Pull a 1.5B model and go.

Design Partner Program

Pro

Open source now. If your team wants to co-build deployment patterns, become a design partner or request integration support.

Become a Design Partner

Little Canary is fully open source. If your team wants guided rollout, hardening, or policy/use-case tuning, request integration support.

Tell us your use case. We'll reply with design partner or integration options. No spam.