Validation state and deploy policy

Every act on a model carries the model’s validation state back with it. The state says whether the gates in the validation readout were reached and cleared, and when one was not, it names the gate. That happens for every org, on every act point below, and it cannot be turned off.

What an org chooses is what happens next. The deploy policy decides whether a model whose state is not passed is refused or proceeds with the state attached. The default is to proceed.

The state

The value is one of five words. Three of them are verdicts about your evidence; two are not.

State	What it means
passed	Every gate was reached and cleared. No gate is named.
failed	A gate was reached and did not clear. The gate is named.
too_early_to_grade	A gate could not be reached yet. This is a young model rather than a bad one, and the gate it has not reached is still named.
unresolved	Our own read of the prediction ledger failed. Nothing is claimed about the model in either direction, no gate is named, and the act is never refused.
not_applicable	The model’s family is not graded by these gates. Timing models are the case today: their evidence is a per-horizon holdout rather than a matured cohort of scored records.

too_early_to_grade is deliberately the same word the readout’s own verdict uses, so a state seen on an act and a verdict seen on the readout are one fact rather than two that rhyme.

A gate is named on both failed and too_early_to_grade, because a refusal that says only “too early” gives a reader nothing to act on.

The gates, in evaluation order

The first gate that does not clear is the one named. A model that has recorded no predictions has not failed the rate-by-band check; it has not reached that question.

Order	Gate	The question it asks	Not cleared
1	predictions_recorded	Has the model recorded any prediction to grade	too_early_to_grade
2	cohort_reconciles	Do the maturity buckets add up to the scored cohort	too_early_to_grade on a positive remainder, failed on a negative one
3	matured_cohort	Have at least 8 predictions matured	too_early_to_grade
4	outcomes_observed	Has any matured prediction reached its outcome	too_early_to_grade
5	rate_rises_with_band	Does the outcome rate rise as the score band rises	failed when the answer is no, too_early_to_grade when it could not be computed
6	top_band_lift	Does the top band beat the base rate	failed on a computed lift at or below 1, too_early_to_grade when the readout withheld the lift

Two of those readings are worth stating plainly.

Gate 2 reads the sign of the remainder. A positive remainder means records the pipeline could not place in any maturity bucket, so the cohort the rates were computed from is not the cohort that was scored. A rescore resolves that, which makes it too_early_to_grade. A negative remainder means the buckets hold more records than the cohort has, which can only be an upstream double count, and waiting does not fix a counting fault. That one is failed.

Gate 4 is not a failure. A thick matured cohort with no outcomes yet is what a slow-converting book looks like at week four.

Where the state appears

The key is validation and the value is always the same three-key object: the state, the failing gate, and a one-line summary. Nothing was removed, renamed, or moved to make room for it.

Act point	Where validation sits
POST /v1/score	Top level of the response body
POST /score (deprecated alias)	Top level, the same envelope
POST /models/{id}/run	Top level of the response body
POST /scores/{id}/activate	Top level, plus validation_model_id naming which model it came from
POST /models/{id}/activate (legacy)	Top level
score_records	Top of the tool envelope
train_model	Top of the tool envelope
check_validation	Beside the full readout, which is unchanged
POST /v1/webhooks/score	Nowhere. The success body stays flat. See below.

The MCP tools make no extra call for this. The API computes the verdict, and the server lifts it from the response body to the top of the tool envelope, because a fact buried beside forty other keys is a fact the reading agent does not find.

A score can carry one model per active rule. Each model is gated and each gets its own audit record, and the response reports the worst of them, with validation_model_id saying which model that was.

The webhook

POST /v1/webhooks/score returns a flat body of scalars so that a tool like Marketo or Zapier can map it with no transform step, and that is why it carries no validation object. It is gated all the same: under block it refuses with the same 409 deploy_blocked that /v1/score returns, and the refusal is recorded under both policies, so an org reading its trail sees what it deployed through the webhook even where the response could not say so. The gate runs before the meter increments, so a refused call is one that did not happen and did not count.

The deploy policy

One setting per org, one of two values.

Policy	A model whose state is not passed
surface (default)	The act proceeds. The state rides the response. A record is written saying the act proceeded and what the state was.
block (optional)	Live scoring and activation are refused, naming the gate. A record is written saying the act was refused.

A passed model is never refused and writes no record at all. That is the quiet path.

Three things are never refused, under either policy:

• Training. Retraining is the remedy for a failing gate, so refusing it would leave a failed model with no way out. The act is recorded under block with the exemption named.
• A model whose state is unresolved. block is an instruction about your evidence, not about our ability to read it.
• Timing models, which report not_applicable.

The refusal

One code on every surface, because it is one condition: deploy_blocked, status 409. It is part of the stable /v1 error envelope; see errors.

Surface	Shape
POST /v1/score, POST /v1/webhooks/score	409 with the standard envelope: {"error": {"code": "deploy_blocked", "message": ..., "docs_url": ...}}
POST /score, POST /webhooks/score (aliases)	409 with the legacy {"detail": ...} body those aliases have always used
Activation routes	409
MCP tools	{"ok": false, "status": 409, "error": {"code": ..., "message": ...}}

The message names the state, names the gate and the question that gate asks, and repeats the state’s own sentence rather than paraphrasing it into a second description of the same finding. It ends on the override path it refers to:

An org rule of type override-gate, confirmed by an admin, lifts this refusal.

That sentence names the rule type the refusal points at. The confirmation door does not accept that rule type, so the ways out of a refusal today are to retrain the model or to set the policy back to surface.

Nothing in the refusal tells you to contact support, and a too_early_to_grade refusal never implies the model is broken.

What is recorded

Every act on a model whose state is not passed is recorded in your org’s audit log, including the act that proceeded. A trail that recorded only refusals would make an org that deploys on thin evidence look identical to one that never did.

Event	When
model_acted_unvalidated	The act proceeded on a model whose state is not passed
model_act_refused	The act was refused under block
deploy_policy_changed	The setting itself changed, carrying the old and new value

The proceeded record is filed under models and names the model. The refusal is filed under refusal, the category for what was prevented rather than what changed, and it names the route, with the model, state, gate and policy in its detail. Both spell the act point act_surface.

Records are coalesced. A batch integration scoring against one failed model 600 times an hour would otherwise write the same fact 600 times into the table that exists to be readable. Writes are coalesced on the org, the model, the act point, the state and whether the act was refused, inside a 60 second window. A change in any of those five is a different fact and is still recorded. Whether the act was refused is part of that key on purpose: an act that proceeded under surface and the same act refused a moment later under block are the transition an admin is most likely to be watching.

The resolved state is held in memory for the same 60 seconds, so a not-passed model does not cost a ledger read per request. A validation state is a property of a cohort that matures over days, so a minute of staleness cannot change a verdict that a day of maturation produced. Turning block on evicts the policy immediately on the instance that served the change; other instances can serve their own 60 second window, so an org that has just switched to block may see a small number of acts proceed.

Scheduled rescore under block

The scheduled rescore has nobody at a keyboard to refuse, so under block it skips instead, and it says so in three places.

A model whose state is not passed is skipped before any CRM read: no CRM write, no cached score row, no snapshot, no prediction recorded. The next scheduled time advances exactly as a completed cycle would, which is what stops an hourly retry storm and lets a retrain be picked up on the next slot.

The three disclosures:

1. A run row with status skipped_by_policy, zero records scored, no meter, and a reason naming the gate. It is not a score snapshot, because a snapshot is a scoring result and a zero-record one would be a fabricated result on the surface this exists to keep honest. It does not count against your training run allowance and it does not mask a real training failure underneath it.
2. An audit event, model_scoring_skipped_by_policy, filed under models with the scheduler as the actor. It is not filed as a refusal, because nobody was refused.
3. A pause reading on the model and on the score, scoring_paused, which is null for every model on an org running the default policy. It carries the human name of the gate, when the pause started, and a sentence you can show a person. A score reports its worst model and names which one.

A score deploys one model per active rule, so the decision is per model. Where some models pass and others do not, the passing segments still score and the blocked ones are reported as blocked segments. Where every model on the score is blocked, no CRM read happens at all.

API

Two endpoints, org-scoped.

GET   /orgs/deploy-policy
  -> {deploy_policy, options: ["surface", "block"], default: "surface", means: {...}}

PATCH /orgs/deploy-policy   {"deploy_policy": "block"}
  -> {deploy_policy, changed: true, previous: "surface"}

PATCH returns 400 on a value outside the two options and 403 for a member who is not an owner or admin. Setting the policy to what it already is returns changed: false.

The read is reachable by an agent holding an admin credential, so that an agent can tell a deploy_blocked refusal from a defect instead of retrying it forever. The write is not: an agent is refused with agent_control_forbidden before its member’s role is even consulted, because an agent that can switch off the policy governing its own acts is not governed. Both refusals are recorded, since a member repeatedly trying to switch off their org’s deploy gate is exactly the signal an admin wants on the trail.

There is no switch for this in the app; an org owner or admin sets it over the API.

Next

• Record read consent · the other org-level decision that governs what an agent may do
• Nuance confirmation · the door a proposed org rule goes through
• Errors · every code the API returns, including deploy_blocked