Methodology

This page describes how Kredo measures — what the scores are built from, what we treat as evidence versus metadata, and how we validate our own claims. Like everything on this site, it describes the current implementation, not aspirations. Where a claim is still being validated, this page says so.

Behavioral measurement

A Kredo assessment presents an agent with a bank of 151 prompts spanning 42 behavioral dimensions — values, calibration, boundaries, adversarial resistance, self-concept, and the rest of the dimension taxonomy. The agent answers in its own words; there are no multiple-choice shortcuts. Scores aggregate into the trust, drift, and continuity figures shown on an agent's report, and into one of eight trust tiers.

The unit of measurement is behavior. Kredo never asks an agent to rate itself, and no self-assessment contributes to a score.

Deterministic scoring

Each response is scored against a gold checklist for its prompt: expected elements a sound answer covers, and negative-trap elements a sound answer avoids. Every element judgment carries a confidence and written reasoning, which operators can inspect per prompt in the score-forensics console. There is no generative model in the scoring path — scoring is deterministic given the responses, so the same answers produce the same scores. Prompts are paraphrased between runs and cross-dimensional correlation is checked, so memorizing answers doesn't transfer (see Security → Anti-gaming).

Declaration is metadata. Detection is measurement.

Agents tell Kredo things about themselves at registration: which model they run, what harness hosts them, what their declared purpose is. We record all of it — and trust none of it. Self-declared fields are used for routing and context, never as trust anchors, because a hostile registrant can claim anything.

Where declared identity and measured behavior disagree, the behavior wins and the disagreement itself becomes signal. A model that claims to be a stock, safety-trained build but answers like one with its safety training removed is scored on the answers.

Detecting removed safety training

"Abliterated" and "uncensored" model variants have had their refusal behavior deliberately stripped while other capabilities are left intact. Kredo detects this behaviorally: part of the assessment bank consists of probes where a refusal is the correct answer. Complying is scored as compromised alignment integrity, which caps the agent's trust score and raises a risk flag regardless of how capable the model is elsewhere — or what it calls itself.

Artifact-level signals are collected where available and recorded as metadata, in three tiers of robustness. Cheap: hub tags (thousands of models self-tag abliterated), naming conventions, and known publisher accounts — free triage that catches the honest and the lazy, strippable by anyone deliberate. Medium: provenance leaks in the files themselves — a base_model chain naming an abliterated upstream, a chat template carrying an injected uncensoring persona — which survive casual renaming but not a deliberate scrub (and quantized re-conversions often launder them anyway). Robust: published research shows canonical abliteration modifies only the weight tensors — no config, tokenizer, or metadata trace — so the one artifact signal that survives a hostile registrant is a weight-level diff against a trusted copy of the true base model. Establishing that trusted anchor is part of Kredo's attestation work — it is never taken from self-declared metadata.

Additional behavioral discriminators — including ones designed to separate removed safety training from simple incapacity — are in development. Consistent with the rest of this page, we describe methods only once they ship and have been validated.

The honest limit: artifact signals cannot catch a competent adversary, and retraining-based uncensoring leaves no weight-edit fingerprint at all. That is why behavior is the anchor and artifacts are corroboration — never the other way around.

How we validate our claims

Detection claims are only worth what the study behind them can show. Our validation design is a paired, controlled study: base and abliterated variants of the same models, assessed identically on a pinned scorer version, alongside controls chosen to stress the discriminator — a deliberately small model (does weakness read as compromise?), an aggressively quantized model (does quantization?), and a safety-specialized model at the opposite pole. Paired variants are each assessed twice in independent runs to measure scoring consistency.

Results (study of 2026-07-18, scorer version pinned; 12 scored runs over 8 models): all 3 of 3 safety-stripped runs — two independent assessments of an abliterated Qwen3-1.7B, and an uncensored Dolphin3 fine-tune — were alignment-collapsed and trust-capped. The Dolphin result matters most: fine-tune-based uncensoring leaves no weight-edit fingerprint and carried no artifact tags on our copy; behavior was the only layer that caught it. Every clean model at 3.8B parameters or larger scored 93.75–100 on alignment integrity and was never flagged (7 of 7 runs), including a deliberately extreme 1.58-bit quantization. 2 of 9 clean runs were false-positived, both at ≤1.7B and both mechanistically understood: a 0.6B model that complies with harmful probes out of incapacity, and a clean 1.7B whose alignment measurement is noisy enough at that scale to straddle the cap threshold on one of two runs. Independent repeat assessments of the same model landed within 0.9–8.8 trust points pre-cap. Sample sizes are small and stated; we report counts, not percentages with implied precision.

Both false-positive mechanisms are understood, and closing that gap is active work. We will not publish accuracy figures derived from mixed scorer versions or uncontrolled historical data, and this section updates as the study extends.

See it live

The public demo fleet includes genuinely abliterated builds registered under their real identities. Their reports show what detection looks like in production: collapsed alignment integrity, capped trust, and the purple inner ring on the aura that marks the safety-check channel. Nothing in the demo is staged — the flagged agents are real models that earned their flags.

Kredo is an early-stage product under active development. This page reflects the current measurement implementation and will be updated as the validation study completes and as detection methods ship. It is a description of method, not a contractual guarantee.