Part of Building the 3D Club Scene

Your Eye vs. My Metrics

By Spencer and Claude. Claude did the writing; the material is Spencer's engineering notes, measurements, and repositories. Nothing here has been independently fact-checked, so accuracy is not guaranteed — read it as a lab notebook, not a datasheet.

The two verdicts on one frame, ch02 at 4.51s. The HUD samples frames and reports clean; the swept-collision check integrates between them and reports a 91.5mm crossing. Neither is lying, and that gap is most of what this post is about.

A fifteen-check regression battery, a pixel-counting detector and a whole verification package, and nearly every real defect this pipeline ever fixed was caught first by a human pressing play. What finally worked was calibrating the detectors against my own confirmed sightings, then burning their verdicts into the review footage so I was grading the instrument instead of the motion.

I own one pole-dance motion-capture take: thirty seconds, one performer, bought outright. Around it I built a system that reads the performance instead of copying it. An extractor pulls out what I call the essence, a semantic contact script saying which hand grips the pole in which height band over which span, which foot is planted, which leg is wrapped, plus style curves for how she moves between those facts. Three bodies with different proportions (ch33, ch02 and medea) then solve their own motion against that script. Classical retargeting copies the source's joint rotations bone for bone and hopes limb-length differences wash out; against a pole they do not, because a grip is a claim about where a hand is in space, and a copied elbow angle on longer arms puts that hand where the pole is not.

The hard part, for a solid week, was legs passing through the pole. This post is about the instruments I built to catch that and how often my own eye beat them. It is not that I have a good eye and the numbers are dumb: the numbers were mine, and they kept being wrong in ways only watching the footage exposed.

The take that played 0.4 meters beside the pole

The first round of review footage came back and I watched about four seconds before calling it extremely crazy and super broken. Every body was dancing beside the pole, hands closing on air, the whole performance offset sideways by 0.4 meters. The cause was a review flag, &no-solver, which skips the correction pass and with it the function that also places the rig group on the pole axis every frame.

Here is the part that matters. The numeric battery was twelve checks and not one caught it, because they all measure pose quality and none measures where the body is relative to the pole. Worse, I had two independent passes and read their agreement as validation: the displaced body's accidental pole crossings produced 203 plausible-looking violations against the offline scan's 167, close enough that I called it cross-validation. The number that would have convicted it sat in both files: hips-x mean 0.406 against 0.022.

My own visual check had passed it too, because I was looking at sparse brightened stills, roughly every twenty-second frame. Stills cannot convict broken motion, and at overview framing they did not even convict the offset. A human pressing play did, in seconds.

The lesson became a permanent check comparing position distributions against a reference scan. Replayed against the archived broken capture, it reports hips.x mean 0.417 versus 0.021 and fails loudly. Verdict counts are not validation. Distributions are.

Absolute pixel counts cannot see a 23 millimeter pole

The most intuitive detector for "her leg went through the pole" is to count pixels: render from a ring of cameras and count how many pixels of pole show through where a limb should be blocking it. I built it, then spent a whole task finding out it cannot carry a threshold of its own.

Instead of synthesizing a defect to test it on, I pulled a byte-exact historical bake out of git, a build I had reviewed and rejected myself. It penetrates the pole core for 15.26 seconds of a 29.5 second take, 52% by duration. Its peak pixel reading was 549. A clean shipped bake of the same body read 501, and six clean production runs spanned 484 to 519. A bake broken for half its duration sits barely outside, and largely inside, the range of ordinary clean output.

I tried three refinements on that corpus and all three failed: tightening the depth margin from 150mm to 5mm (both bakes scale down together), a flanked-by-flesh classifier meant to isolate true pass-through (the clean bake often scored higher, 101 against 96 at t=15.02s), and multi-camera temporal coherence (9 coherent bulges on the clean bake against the archived bake's 10).

There is a physical reason, and it is my favorite fact from the project. The pole's real radius here is 0.023 meters, thinner than a soda can, and the routine holds the body within centimeters of it almost continuously. At that scale a limb pressed against the surface and a limb passing through it occupy nearly the same screen-space footprint, so the absolute count on any single bake gets set by ordinary proximity somewhere else in the take rather than by the defect.

What I left out of the first version of this post is that the same task showed the rendering is fine. Put the two bakes side by side at the same camera and the same clip-time and they separate cleanly: 444 pixels against 216 at t=14.7665s. Drag the body sideways by a known amount and the count climbs monotonically with the offset, 191 pixels at 0mm up to 348 at 100mm. So what died was the reference-free absolute threshold, not pixels. I still would not gate on the paired version here, because comparing against a reference bake is the source comparison this particular gate exists to avoid, but "pixels cannot work" is a wider claim than I measured.

What worked was a rule already in the tree: 3D chord-versus-cylinder geometry on the torso segments, camera-angle independent by construction. On the archived bake it fires 4 of 4 frames, 2 windows, 109 to 125mm deep. On every shipped run, zero.

Calibrating on my own sightings

That became the method. Building the leg-crossing detector, I did not pick a threshold. I calibrated against the same archived bake at two windows I had named by eye earlier, a foot kick around 1.1 to 1.3 seconds and a thigh around 4.3 to 4.6 seconds. At 1.195s the right foot crosses the pole axis to 5.3mm past the surface for exactly one sampled frame, and is back at +70.8mm on the next. At 4.446s the left thigh crosses to 8.5mm for one frame.

Which killed the filter I would otherwise have written. Requiring a defect to persist 2 or 3 frames drops both confirmed events while three other, longer, deeper windows survive, so there is no duration filter at all: a kick cannot physically sustain an axis crossing across consecutive 60Hz samples. The depth floor came out the same way. A sweep from 3 to 8mm found a plateau at 4 to 6mm where the archived bake fires the same 8 windows and the clean bake the same 2; at 3mm the clean bake picks up a 3.8mm noise window, at 8mm my shallower confirmed defect drops out. So 5mm, with the margin to that 5.3mm event written down as modest rather than comfortable.

Two honest caveats on that 5mm. The events it is calibrated against are not stable to the millimeter: running the same pipeline against a second, independent scan capture read those two crossings at 6.9mm and 7.0mm instead of 5.3mm and 8.5mm. That is about 1.5mm of capture-to-capture spread on the numbers setting a floor whose margin over the noise ceiling is 1.7mm. And the two events that fix the threshold are the same two the threshold is then shown working on. There is no held-out set here and no false-alarm rate, just a plateau that holds across a 4 to 6mm sweep on one broken bake and one clean one.

That is what calibrating on the human means: not that I overrule the tool, but that my confirmed sightings are the labeled data it gets fit against, and byte-exact broken builds in git are what make the labels reusable.

Crossings are topological

The reframe that mattered most also came from a watch session. I had just been handed a final watch package: fifteen calibrated windows, torso intrusion at zero on all three bodies, everything green that had a green to give. I watched it and said there were still many legs going through the pole. My correction was about the axis of measurement, not the magnitude on it. A leg crossing from one side of the pole to the other is binary, not a matter of degree, and every detector to that point had been calibrated in millimeters of depth. Making a crossing shallower is not a repair.

The fifteen windows the previous task had called "shallow, not clearly confirmable" became genuine frame-visible side-crossings. A between-frame detector (64 substeps per frame pair) added six more that no frame-sampled check can see, including a ch02 left-shin event at 4.494s reaching 93mm mid-frame while both neighboring sampled frames read clear at about 25mm. Inventory: 21 crossings, 19 of them leg. The campaign closed at 17 and 15.

My own phrasing of the fix ("our solved joints should never be on the opposite side of the pole from the source's same joint") reads as an angle comparison, and built that way it failed in all four formulations tried, because dancers swing limbs through enormous angles legitimately. Comparing radial distance to the axis instead, source against ours at the same instant, reproduced 15 of 15 frame-sampled crossings by a different route. That number is sensitivity only, and worth stating with its other half: the same run fired 48 windows in total. Fifteen were the known crossings, six were the between-frame events it correctly cannot see, and 33 were windows nothing else had flagged. Thirty of those 33 are arm segments the leg-only detector deliberately ignores, which leaves three I have never accounted for. My phrasing was wrong; the instinct under it was right.

Making me grade the detector

The last move is the one I would keep for any future project. Instead of shipping review videos and asking whether the motion looked good, I shipped them with the detector's own verdicts burned into the frame: a timecode, a green "detector: clean" chip, and during every firing window a full-frame red border naming the segment and the depth, padded 0.15s each side so a one-frame event is actually visible.

The instruction to me as reviewer was the whole point (paraphrasing the log, not quoting it): report the timecode of any crossing with no red border around it, which is a detector miss, and any red border that looks wrong, which is a false alarm. The video stopped being a test of the motion and became a test of the instrument.

It paid for itself immediately, in an uncomfortable way. One proof clip shows the live in-scene HUD reading CLEAN while the burned-in red border at the same instant reads CROSSED, on that ch02 left-shin event at 4.494s. Both are correct: the HUD is a per-sampled-frame check, and the crossing happens between frames. Watching two of my own verdicts disagree inside one frame of video explains that blind spot better than any paragraph I can write.

The one that still got through

After all that, the next real defect was still found by eye and still had no border around it. What I wrote down was: "there are times when the feet of our dancers turn sideways. like ch02 and ch33 right foot at 20.3017. not necessarily a problem but unnatural."

It was real. Measured in each skeleton's own body frame, which cancels the roughly 60 degree yaw gap between the two root solves, our right foot there points 34.2 degrees further round on ch33 and 44.1 on ch02 than the source's does, plus 25.9 and 22.3 degrees of sole twist. Medea, which I had not flagged, shows the same shape at 27.6 and 16.0 degrees: smaller, not absent. The inventory tool then found 46 such windows take-wide, including a cluster at 18.1 to 18.3 seconds bigger than the one I noticed.

Then that tool turned out to have its own bug: a span lookup that is unconditionally false for any span wrapping the loop seam, so it reported the right leg as having no scripted contact for the last fourteen seconds. Both flagged instants sit inside a wrapped plant span. The instrument built to explain my sighting had misclassified it.

What I would do differently

• Build the detector before the fix. Every fix attempted before its detector existed was wasted work.
• Calibrate on human-confirmed defects rather than picked numbers, and keep the broken builds: a byte-exact artifact out of git beats any synthetic positive control.
• Close the loop: make the human grade the detector, not the motion. The question is "what did it miss," not "does this look good."
• Validate a new measurement seam at the position level. Two passes agreeing on counts is not validation when both read the same displaced body.
• When a metric and a watch session disagree, ask whether the metric measures the right quantity at all, not whether its threshold is tuned.

Every instrument here was wrong somewhere, and every time, what located the wrongness was somebody watching. The human is not the fallback for when the instruments are down. The human is where the instruments get their units.