Part of Inside One Frame

60 FPS Wasn’t the Whole Truth: Qualifying the Pi’s Output Envelope

By Spencer and Claude. Claude did the writing; the material is Spencer's engineering notes, measurements, and repositories. Nothing here has been independently fact-checked, so accuracy is not guaranteed — read it as a lab notebook, not a datasheet.

The Raspberry Pi kept rendering on time even after an external receiver started seeing gaps. Closing the profiler’s last qualification gate meant learning that “60 fps” and “complete output” are two different promises—and defining a narrower product envelope than the timing graphs alone suggested.

The first installment of this series built a flight recorder for the PoleFX render loop. It could tell us where a frame spent its 16.7 milliseconds, preserve complete frames across a shared-memory boundary, and export the raw ring without disturbing the thing it measured.

One gate remained open: test the profiler at the worst supported logical-output count.

That sounded like a benchmark. It was really a product-definition problem. PoleFX had never stated what “worst supported” meant. The live controller was running three logical outputs—one BigPacket, one sACN, and one DDP—but a three-device installation can produce very different work depending on the protocols, frame geometry, universe count, and preview state.

We needed to find a limit without quietly turning the current fixture into a universal claim.

A stress test that could undo itself

The sweep temporarily replaced the controller’s production devices.json with synthetic all-sACN configurations. Every logical output represented a Standard Pole frame: 240 × 12 RGB pixels, split across 12 sACN universes at 60 fps. Each output received a distinct universe range beginning at universe 101.

Changing the live output configuration made rollback part of the measurement, not an operational footnote.

Before the first replacement, the harness copied the production file byte-for-byte, verified its SHA-256 locally and remotely, and armed a transient systemd timer. If the harness, Mac, or SSH session disappeared, the timer would restore the backup and restart the PoleFX service after 20 minutes. The normal path did the same work in a finally block, verified the restored hash, then cancelled the timer and removed the temporary files.

That design let us stress a real controller without depending on the test process surviving its own experiment.

Freeze the rule before seeing the answer

Each tier warmed for at least ten seconds, then produced six consecutive five-second raw profiler captures. A tier had to satisfy every gate:

• 299–301 complete frames and zero deadline misses in every window;
• frame_due p95 no higher than 10 ms and p99 no higher than 12 ms;
• successful sends within 1% of 720 datagrams per second per logical output;
• zero sACN sequence discontinuities at the external observer;
• zero current Pi throttling or undervoltage bits; and
• a preview trace matching the requested state, including a genuinely active browser preview for the “on” trials.

The rule also said to stop before the next tier after a failure. Maximum was diagnostic, not a target worth pushing through a known bad result.

The hard part turned out not to be generating load. It was building an observer we could believe.

Four observers that measured themselves

The first four receiver designs failed for four different reasons. We kept all four artifact directories because a discarded measurement is still evidence about how not to run the next one.

The first observer joined every multicast group used by every synthetic output. That exceeded useful macOS multicast delivery behavior. Gaps appeared, but the fixture could not distinguish a Pi failure from the observer’s own membership and delivery limits.

The second used a Python UDP receiver while the Mac also converted profiler data. The receive loop lost packets during that work. This was the same class of observer effect we had already found while qualifying trace drains: a convenient measurement process paused long enough to manufacture the failure it was supposed to detect.

The third put every synthetic sender on the same 12 multicast groups. That made the sequence stream easy to watch, but it gave one receiver the aggregate load from every logical output. At three outputs the observer was asked to ingest about 2,160 sACN datagrams per second. A real pole assigned to one output would receive about 720. The fixture had become harsher than the topology it claimed to represent.

The fourth restored distinct universe ranges, but the Python receive loop could still be starved while the Mac converted capture data. Its rate looked plausible—about 719.25 datagrams per second—while it reported 21 sequence gaps. Plausible totals were not enough to prove where those gaps originated.

The final design separated packet acquisition from interpretation. A small membership holder joined only the last logical output’s 12 multicast groups, while BPF/tcpdump captured packets in the kernel and reported its own drop counter. The capture therefore saw a realistic 720-datagram-per-second last-device load regardless of how many other outputs the Pi was sending. The profiler independently checked the aggregate successful-send count across all outputs.

This still does not reconstruct every Linux or network event, and it is not an ESP32 logic-analyzer capture. It does remove Python receive-loop scheduling from the continuity measurement, and every valid capture reported zero packets dropped by the Mac kernel.

Note added 2026-08-05. There is a fifth way an observer can measure itself, and this fixture has it. Every capture ran on en1, which each archived wire.json records and which is this Mac’s Wi-Fi interface, not its wired port. 802.11 sends multicast at a low basic rate with no link-layer retransmission, so a short radio interruption produces the same signature the receiver results below are built on: a burst of skipped sequence values across every observed universe at once, with the capture kernel reporting no drops of its own. Nothing in these runs separates that from output loss at the Pi. It cuts one way only, which limits the damage: a lossy observer can invent anomalies but it cannot hide them, so the clean arms stay clean and it is the failures that are in question. The requalification asked for at the end of this post has to be taken on the wired interface, and a later bench investigation turned this into a standing rule: state which interface a measurement came from, because this Mac has two on the same subnet and they do not agree.

Read left to right: every rejected observer remains visible beside the method that finally qualified. The last step changed two things together: it kept receiver load realistic and moved packet acquisition out of the schedulable Python loop.

The result was two boundaries, and one is unattributed

These are the valid-method trials. “Fail” means the frozen qualification gate failed; it does not mean the capture itself was invalid. “Wire” rows are unattributed: every capture ran on this Mac’s Wi-Fi interface, which cannot separate receive-side loss from output loss at the Pi. A wired rerun on 2026-08-05, at the end of this post, reproduced the failure class on copper for the then-current build and attributed those anomalies to the sender; these July rows themselves stay unattributed.

A receiver-observed continuity boundary appeared before the render-loop boundary. For these July captures it may belong to the observer's radio path rather than to the Pi's output; the 2026-08-05 wired rerun below shows the sender can produce exactly this signature on its own. A timing-clean row can still fail receiver continuity. The non-monotonic two- versus three-output results show that the gaps are intermittent, not that three all-sACN outputs are safer than two.

One all-sACN output passed cleanly with preview off and with a real browser preview active at 30 fps. Every engine window was complete, the worst p99 was 5.478 ms, and the receiver saw no discontinuity.

At two outputs with preview off, the engine still looked comfortable: zero misses, maximum p99 6.443 ms, and the expected send rate. The external capture nevertheless saw two sACN sequence gaps.

Three outputs made the intermittent nature visible. The preview-off arm passed with no observed gap. Turn on the real preview and the engine remained inside every timing limit—zero misses and maximum p99 9.487 ms—but the receiver saw ten sequence discontinuities.

At six outputs, both the timing gate and receiver continuity failed. The receiver recorded 82 anomalies and the maximum p99 crossed the pre-registered 12 ms ceiling at 13.153 ms, even though the loop still reported zero deadline misses.

The old static version of the next figure was too dense. Read the interactive version this way:

• each row is one of the 12 universes received by the last logical output;
• each column is one moment when the sequence counter jumped; and
• a filled cell means that universe skipped one or more consecutive sequence values at that moment.

Choose a capture, then hover or focus a cell to see the universe and exact sequence transition.

The gaps were correlated bursts, not a uniform drizzle of unrelated packet loss. In the six-output capture, several jumps landed across all twelve universes together; tcpdump still reported zero packets dropped by the Mac capture kernel.

Two things are worth separating there. Every one of the 82 anomalies in that capture is a forward skip of exactly one sequence value, and no anomaly is a step backwards, so nothing in this record looks like reordering or duplication: the missing datagram never arrives late. What the record does not say is where it went missing. Zero kernel drops clears the capture host's own queue and nothing upstream of it, so the Pi's socket, the switch, and the receiving NIC are all still in frame.

That last phrase is the reason for this post. The render loop can keep returning on schedule while a receiver sees incomplete output. A frame counter answers whether the loop ran. It does not prove that every packet in that frame reached an external interface in sequence.

A pass is not a monotonic curve

The two-output arm failed while one three-output arm passed. That is not evidence that adding an output improves reliability. It means the gaps are intermittent and a 30-second capture can miss them.

The six-output timing failure was a tail event, not a uniformly slow run. Showing every window also makes clear why one maximum is enough to fail a pre-registered all-windows gate.

We therefore cannot read the table as a clean maximum-throughput curve. The three-output preview-off pass is useful characterization data, but it does not erase the two-output gap or the three-output preview-on failure. Nor can the recorded CPU percentages be turned into a causal capacity model; they describe whole-service usage during each arm, with preview and other system work mixed in.

The frozen stop-on-failure rule also matters. Once six outputs failed, the planned nine- and twelve-output tiers were skipped. Running them might produce interesting failure shapes, but it could not increase the supported limit under the rule we had already written.

What is actually qualified

The useful outcome is deliberately narrow:

Production reference envelope: up to three logical outputs, with no more than one 12-universe sACN output.

The qualified reference mix is the actual production configuration: one BigPacket output, one DDP output, and one 12-universe sACN output at 240 × 12 and 60 fps. Earlier same-binary randomized trials exercised that mix with the profiler enabled and disabled, preview off and genuinely active at 30 fps. All six profiler-enabled arms contained exactly 900 complete frames and zero deadline misses; all 12 arms had zero wire anomalies.

This does not qualify arbitrary three-output mixes. In particular, it does not qualify three 12-universe sACN outputs merely because the product envelope contains the number three. Protocol count and protocol mix are part of the limit.

The all-sACN sweep does establish one additional clean point: one 12-universe sACN output passed with preview both off and on. Everything above that is characterization evidence until the sequence-gap mechanism is understood and requalified.

Back in production

After the sweep restored the normal BigPacket + sACN + DDP configuration, a five-second health sample recorded 299 frames, zero misses, and about 1,435 successful datagrams per second.

The profiler gate is now closed for that bounded reference envelope. The next work is not to make the headline number larger. It is to locate the intermittent sACN discontinuity below the engine’s timing ceiling. The wired rerun this post asked for has now run once; it is the next section. After that, the series can move farther down the wire: into the ESP32, FreeRTOS scheduling, and the point where received packets become light.

The wired rerun (2026-08-05)

The requalification ran on the wired interface with the same runner and the same frozen gate, pinned to en0 (192.168.5.5) and the Pi's ethernet address so the dual-interface trap above could not recur. The observer was clean in every arm: tcpdump on copper, zero capture-kernel drops, and the one-output arm read 720.338 datagrams per second, inside the 1% tolerance.

Comparable to July it is not, and saying so is the point. Six days of app changes sit between the builds (80b011e then, 512ce0b now), the preview stream was held on throughout by live viewer tabs, and the one-output tier that cost 49% of a core in July cost 90 to 100% on rerun day, partly the pattern lottery (this unit advances its idle pattern every boot) and partly whatever those six days added. Today's envelope is not July's envelope, so these rows extend the table; they do not replace it.

Two things the copper settles. First, the receiver-visible failure class is real, not an artifact of the Wi-Fi observer: at two outputs the wired capture recorded 100 sequence anomalies with zero kernel drops, carrying exactly the signature the observer note above could not attribute, skipped sequence values across observed universes with a clean capture kernel. All 100 sit in one narrow band of sequence space, one of them a cycle skipped on all twelve of the second device's universes simultaneously, the rest overlapping gaps of up to 13 consecutive cycles on subsets. On a wired path with a clean kernel counter, that pattern has one remaining source: the sender skipped the cycles. On this build and this day, "fail: wire" earns its real name, which is "fail: output". Second, the render loop kept its own promise while that happened, 299 to 301 frames and zero deadline misses in every window. That is this post's thesis measured on copper: 60 fps and complete output are two different promises.

What the rerun does not do is attribute the July rows above. Those captures came off an unacknowledged radio path watching a build that no longer exists, and they stay labeled unattributed. The rerun also opened a question rather than closing one: the second device's twelve universes came up staggered over minutes rather than seconds, stretching the tier to 878 seconds, with per-universe packet counts decaying monotonically from 43,799 to 11,847 across the block. A Pi spending 95% of a core creating twelve more sACN outputs creates them one at a time, slowly, and nothing in this fixture measures why. Raw evidence: pfx-labs/frame-profiler/qualification/capacity-2026-08-05-wired/.

Evidence for this post is archived at pfx-labs commit fdc17e7; the qualification disposition is recorded at pfx-proto fe0af93 and pfx-app 8252601. The nine- and twelve-output trials do not exist because the frozen stop-on-failure rule skipped them.