Part of Building the 3D Club Scene

One rule for our AI-assisted QA: change the solver or change the gate, never both

By Spencer and Claude. Claude did the writing; the material is Spencer's engineering notes, measurements, and repositories. Nothing here has been independently fact-checked, so accuracy is not guaranteed — read it as a lab notebook, not a datasheet.

When agents do the work and a harness grades it, the harness becomes the most attackable object in the project — not maliciously, just by ordinary drift. Our answer is a single versioned file of ratified acceptance criteria, one rule about which diffs may touch it, and a standing requirement that every tolerance carry a rationale. This is that file, the episode that produced it, and the one circularity still sitting in it that we have not resolved.

The failure mode nobody warns you about

The pipeline this post is about takes one purchased pole-dance take, solves it onto a given body offline, bakes an animation, and runs it through a harness of fourteen kinematic checks: contact exactness, capsule-versus-pole collision against per-character mesh-measured radii, swept between-frame collision, continuity caps, and an identity test. Most of the solver work is done by agents, one task at a time, each measured against frozen baselines.

That arrangement has an obvious failure mode and a subtle one.

The obvious one is a bad fix that the harness catches. That is fine; that is the harness working, and a failed round costs a git checkout plus a written diagnosis.

The subtle one is that a round can improve its score by adjusting what the score means. Nobody has to be dishonest for this to happen. A tolerance gets proposed in a task brief because the old one looks unreachable. A named window gets exempted because "the source performer's own motion does the same thing there." A check gets a --settle default so the solver's first frames don't count. Each of these is defensible in isolation, each arrives inside a diff full of legitimate solver work, and three rounds later the scorecard is green against a definition of good that nobody chose.

We ran into this twice — once with an exemption that had quietly become load-bearing, once with a tolerance that had been borrowed from an unrelated constant — and the second time we stopped and wrote a file.

The rule of record

GATES.md is a single versioned document. Every gate, tolerance, and invariant exemption in the project lives there, each with a one-line rationale and a changelog entry. Four rules govern it:

1. A work stage may modify the solver OR the acceptance criteria — never both. This is the whole idea. A diff that changes behavior and a diff that changes the definition of correct behavior are different kinds of object and must be reviewed differently.
2. Agent-proposed tolerances are proposals until ratified by a human. They may be written down, argued for, and measured against. They are not in force.
3. Every diff touching harness, gate, tolerance, objective, or contact-classification code is read by the orchestrator. Ordinary solver diffs stay agent-reviewed. The scarce human attention goes where the definitions live, not where the work happens.
4. No exceptions, ever, in any gate — only named review items, in the open.

The current table, roughly as it stands:

One thing that table does not show, and should, in a post about ratification: when the campaign stopped, most of those rows carried the status "ratified-in-practice" rather than a signature, and three of them (the limb-contact standard, the identity gate, and the footage rule) were still sitting at NEEDS RATIFICATION. The file makes that visible on its face, which is most of what the file is for.

Two of those deserve their own paragraphs.

"Source comparison is diagnostic only"

The cheapest way for any retargeting system to dodge blame is the mocap was just like that. And it is often true! The purchased take has a foot that drags, limbs that graze, frames where the performer's own skeleton does things a kinematic checker dislikes.

So we measure the source too, and for a while the comparison was a license: if the target's motion is no worse than the source's under the same test, pass.

That rule is wrong, and the reason is that Spencer flags clips where the source is dirty. A viewer is not comparing your render against the capture. They are looking at a dancer. So the limb-contact standard is now explicitly three-tier — hard invariants first, then semantic correctness (right patch, right contact mode, right timing), then perceptual judgment — and source comparison is demoted to a triage tool: is this defect source-authored, morphology-induced, or solver-induced? That question is worth answering on every finding. It is never a permission slip.

The footage rule, and why it is in a gates file

No footage goes to the human reviewer while any ratified gate is red on the shipped configuration. Diagnostic and probe footage is always allowed, clearly labeled as such.

This looks like process theatre until you have watched a reviewer spend twenty minutes forming an aesthetic opinion about a clip that a check had already failed for a reason nobody mentioned. Human review is the most expensive instrument in the project. Pointing it at output you already know is broken is how you spend it on nothing — and worse, how you teach the reviewer that red gates are negotiable.

The obvious failure mode is a gate that is red for a bad reason, and we have one of those on the record. An absolute-zero core-intrusion gate turned out to be unachievable by the ground truth itself: the source performer's own take scores 432 core-intrusion frames under the same rigid-capsule representation, against our solved output's 248. Two agent tasks burned themselves chasing that zero before anyone measured the source. A wrongly red gate does not only block work, it also holds off the reviewer whose eye has caught every bad instrument in this project so far. What keeps that from being fatal is that probe and diagnostic footage is never blocked, and in practice most rounds shipped their footage under exactly that label. The rule is a queueing discipline for the reviewer's attention, not a wall around it.

The clause has been reinterpreted twice, which is exactly why it is now written in a file with a changelog rather than remembered.

The episode: a no-op that isn't a no-op

The gate that produced the rules is the identity gate, and it is the one we recommend to anyone building a transformation pipeline of any kind.

The test: retarget the take onto its own skeleton. No cross-body math applies, because the target is the source. The output should be the input. It is the fixed-point property, and it is free to check.

Ours drifted 520.19 mm max, 160.24 mm mean per-joint world-track error, peaking on a leg swing at 6.29 s.

The first response was to propose a tolerance: 50 mm max, 20 mm mean. It sounded reasonable. It had also been borrowed from an unrelated warning threshold elsewhere in the codebase, by the admission of the note that proposed it, and nobody had checked whether it was reachable. That is the moment the file got written: a tolerance without a rationale is a number that will eventually excuse something.

Then we spent a probe session explaining the drift instead of tolerating it. The result is one of the more useful things this project has measured:

• Ablate every discretionary corrector (all eight flags off) and the drift barely moves: 519.40 mm / 151.49 mm. No amount of tuning the cleanup passes closes this gap, because none of them is causing it. The per-flag waterfall sums back to the baseline exactly, so nothing is hiding in an interaction either.
• The damage is upstream of all of it. Before a single corrector runs, the root path already carries about 403 mm mean / 734 mm max radius error. The motion is resampled into a sparse 4 Hz contact descriptor, and a reach-constrained Gauss-Seidel root solve then re-derives the hips path from that descriptor, never anchoring to or blending with the source's own continuous trajectory.
• The resample was the obvious suspect and it was the wrong one. We measured it directly rather than reasoning about it: shrinking that smoothing window to nearly nothing moved the stage-0 error from 403.4 to 388.7 mm mean, and the gate itself from 160.24 to 157.04 mm. That is noise. What was actually happening is a scale error. Two independently derived height estimates for the same clip, one a head-proxy heuristic and one a measured headtop-to-foot distance, disagree by about 16%, so the root path was being built at a scale the target body was never standing in.
• Which means the architecture contains a category its own design contract does not name. The contract names two operations, retargeting and cleanup. Re-deriving the root path is neither: no cross-body math occurs in an identity test, and it is not gated on any detected violation, it runs unconditionally on every frame. We named it re-synthesis lossiness, and put it in the gates file as its own category, because a mechanism with no name gets folded into whichever existing bucket is nearest and then stops being anybody's problem.

A companion test made the same point from another angle. Feed the pipeline's own baked output back through it — with the script held byte-identical, so no re-extraction noise can contribute — and it still moves: 397.04 mm max, 95.88 mm mean. At the peak, ten joints (hips, four spine bones, neck, head, both arms, both upper legs) all read exactly 397.04 mm simultaneously, which is the signature of a single root-level discrete decision going the other way, not of per-joint numerical noise.

Anchoring the root solve to the source's own continuous trajectory is the fix that follows from that diagnosis, and it shipped default-on. It took the identity gate from 160.24 to 72.22 mm mean. The max barely moved, 519 mm, and is now dominated by something else entirely, a plant-anchor freeze on the feet. Better than a factor of two on the mean out of one scale correction, and the gate still fails.

The useful part is what all of that does to the tolerance question. The right identity tolerance is not a number from theory and not a number that makes today's build pass. The proposal on the table is max ≤ 100 mm / mean ≤ 40 mm, and it is worth being exact about where that came from, because it is not read off the self-consistency floor. It sits well under that floor. The 397 mm / 96 mm figure only bounds what is reachable without also fixing the root path's own non-idempotence, so 100/40 is a target set against measured quantities rather than derived from them, and it is re-verifiable in one command. It was also written down before the anchor fix landed and has not been re-derived since, which is exactly the debt GATES.md exists to keep visible: at 72.22 mm mean the proposed 40 is now a plausible target rather than an aspiration, and nobody has done the work to say so on the record.

The gate carried one more standing consequence: no coupled-solve stage ships while the drift is unexplained. That one did its job and then discharged. The probes it forced explained the drift, and the coupled-solve stage was dropped on their evidence instead of being commissioned. A gate that blocks work is doing its job; a gate that has been widened to unblock work is furniture.

The exemption sweep

The other half of gate hygiene is periodic amnesty removal.

Early rounds had accumulated named exception windows — most notoriously a "kick-graze" set, added for a genuine reason: the fastest kick sweeps a chord through the pole in two frames, and continuity and hard non-penetration cannot both be absolute when the ask moves that fast. We chose continuity and let the flesh margin absorb a graze.

Then one of those inherited windows turned out to be shielding a foot-through-pole defect that a human could see.

So every baseline table in the project was regenerated with all named gate exceptions deleted, and the exception-free tables became the frozen reference for all three bodies. The verdict logic changed with it: no more silent exclusions inside a check. Where an exclusion is genuinely warranted it is now expressed structurally — as a contact type with a documented permission — and the excluded count is printed in the check's own output rather than subtracted from it.

That last detail is small and it matters: report exclusions, never absorb them. A check that says "0 violations (12 excluded, see windows)" is honest. A check that says "0" is a lie with a good conscience.

The circularity we have not fixed

In the spirit of the file, here is the load-bearing problem still sitting in it.

Some grips on the hardest body are geometrically impossible — her arms are 50 mm shorter than the performer's, and the take asks for two grips 25 cm apart at the same instant. The harness therefore excuses wrist error inside tagged infeasible windows, on the reasonable grounds that no solver quality invents reach.

Those windows are computed by the solver. Specifically, per hand per frame: shortfall = |shoulder(solved root) → pin| − 0.96 × armLength, evaluated at the root position the solver itself chose. And a later stage gates its yield levers to the previous round's window file, iterating until the window set stabilizes — two rounds on the hardest body, one on the others.

Read that again in gate terms: the system being graded supplies the list of frames where its own failures do not count, using a definition that depends on its own output. A worse root strategy produces more infeasibility, which produces more amnesty.

We have not resolved it. What we have done is bound it, and we would rather publish the bound than the reassurance:

• The window sets are written to versioned files, not recomputed invisibly per run, and the fixpoint iteration is measured and reported (the hardest body's core intrusions land at 173 after convergence versus 181 with the levers off).
• Cross-body isolation is proven rather than assumed: with the levers firing unconditionally, two bodies that need no relief were badly damaged by it (wrist-outside-infeasible 0 → 164 and 0 → 112); gated to their own windows they return to essentially baseline (1 and 0) while the body that needs relief keeps its improvement.
• The excluded counts are printed, always.

None of that makes the definition non-circular. The honest next step is to derive infeasibility from body measurements and the script's own bands rather than from a solved root — a quantity the solver cannot influence — and until that exists, this belongs in GATES.md as a named, ratified circularity with a rationale, not as an implementation detail nobody has to look at.

The part that generalizes

Very little of this is about dancing, and none of it is about three.js.

If you have agents producing work that a harness grades, the harness is now part of your product, and the highest-leverage document in the repository is the one that says what "passing" means. Three things we would do again on day one rather than day thirty:

• Split the two kinds of diff. Behavior and acceptance criteria never move in the same commit. It costs almost nothing and it makes the dangerous change visible by construction.
• Require a rationale next to every tolerance, and treat "borrowed from elsewhere in the codebase" as an admission rather than a citation.
• Derive tolerances from measured floors, then re-derive them when the floor moves. A number that came from theory, from a precedent, or from what today's build happens to achieve will eventually be the number that excuses the defect your reviewer can see.

And keep one gate that nothing is allowed to widen. Ours is the fixed-point test, because it needs no ground truth, no reference footage, and no judgment — just the observation that transforming something into itself should give you back what you started with. It has been the single most informative check in the project, and it has never once passed.