The playground scenarios
The scenario picker splits in two. Mechanisms are guided tours of one crate feature each: pools, recovery, gated bring-up, demand-start, leases. Systems are architectures of real embedded products at close to real scale, grounded in public documentation and shipped source, cited below.
Every system carries a differently shaped contended resource (a serialized bus, a bounded spool, a divisible power budget, a severity-arbitrated annunciator), because no single mechanism covers them all.
Cellular asset tracker
Section titled “Cellular asset tracker”The archetype where energy, not time, is the scarce resource: a battery-powered tracker that sleeps most of its life, wakes on motion or a schedule, and uploads.
The sleep/wake cycle runs the README’s power-coordinator recipe. POWER has
no task:. The application spawns it, because only the app holds the
Spawner that respawn_terminate takes; it detaches itself so its own
teardown() skips it. Sleep is a reverse-order teardown(): the Pause
sensors park keeping their state, and the modem’s consume runner is dropped
for good. Wake is resume_pausable() (parked tasks resume in place) plus
respawn_terminate() (stateless services respawn in dependency order).
Wake without rebuilding the modem runner fails closed on the empty consume
slot; Rebuild modem runner re-provides it. GNSS carries ready_on_write:
it is ready while a fix is actually being produced, not merely because the
task started. The disabled FOTA latch and the detached run-once self-test
both survive the cycle.
Sources: Telit on tracker design choices · 1NCE on PSM and eDRX · DigiKey on NB-IoT / Cat-M power saving · LwM2M firmware update object · Memfault on OTA for IoT · GNSS/cellular coexistence blanking
Industrial edge gateway
Section titled “Industrial edge gateway”A protocol-translating gateway in the four-plane split real products use: field drivers, a tag database, a filter stage, and a cloud session northbound.
An RS-485 bus owner (Pause), an elastic pool of Modbus pollers (one per
device group, all contending for the one half-duplex bus BUS485: shared),
a tag database gated on the pool’s floor member, a deadband filter, a bounded
store-and-forward spool (drop_oldest: the uplink is opportunistic, the field
bus is clock-driven), and a singleton Sparkplug session
(deps: [MODEM ready bound]).
The pooling is deliberately southbound: a real MQTT client is one ordered session, and a pool of those is a shape nobody should copy. The elasticity real gateways have lives in driver instances.
Try: drop the uplink. The session bound-stops, polling continues, the spool
climbs and caps at capacity; restore it and the backlog drains. Turn the
device dial up and the poll pool grows; turn it down and DeferredShrink
folds it back.
Sources: MIGS modular edge gateway · TI Sitara IIoT gateway (Modbus acquisition cycle) · Modbus polling optimization · Diagnosing Modbus degradation · RS-485 multi-slave performance · Modbus exception responses · Sparkplug topic namespace and state management · HiveMQ on Sparkplug session management · OPC UA MonitoredItem queue overflow · reconnect backoff with jitter · MQTT persistent sessions
Battery management system
Section titled “Battery management system”The safety ladder certified BMS firmware runs: warn, limit charge, limit discharge, derate, open the contactors. Protection sits on its own executor tier, the way products separate a safety base layer from the application layer.
The safety tier carries the samplers and the protection loop under tight
liveness budgets. PRECHARGE provides HV_BUS only once the DC link reaches
threshold inside its slot_timeout:. Closing a contactor early welds it,
and a timeout is a latched fault. CONTACTORS sits behind
deps: [PRECHARGE ready, PROTECT ready bound].
The escalation policy is the point. Stall the protection loop and the
application withdraws its readiness (clear_ready), which opens the
contactors through the ready bound edge (the safe state) instead of
restarting a stalled loop with the pack live. Stall the SoC estimator instead
and the policy activates the disabled LIMP limiter, a second writer of the
same limits signal.
Cell balancing is a min: 0 pool under a thermal budget: at most four
channels bleed at once, one stays warm at rest, and the pool follows the
imbalance dial.
Sources: Battery Design on BMS architecture · Lithium Balance n3-BMS (segregated safety layer) · TI TIDA-020076 HV BMS reference design · Prohelion precharge sequence · precharge failure and contactor welding · SoC estimation methods · LTC6811 isoSPI daisy chain
Smart energy meter
Section titled “Smart energy meter”The dual-core split every solid meter uses: metrology sampling on its own tier, billing registers above, and the local load profile as the authoritative record.
SAMPLER carries ready_on_write: the meter is ready when it is actually
metering. Client associations are a min: 0 pool of OnDemand members bound
to the PLC carrier and capped at max: 3, the session limit a real meter
enforces with BadTooManySessions.
Drop the carrier and the associations bound-stop while the tariff chain and the load profile keep running: the uplink is a convenience, the billing record is the product. Restore it and the pool regrows on demand.
Sources: Microchip metrology firmware · Microchip smart meter SDK · NXP AN13338 three-phase meter design · NXP AN13742 secure metering · DLMS/COSEM and European metering standards
Robot cell controller
Section titled “Robot cell controller”The dual-plane architecture real robot controllers document: non-real-time planning, hard-RT motion, plus the safety plane that certification adds.
All three overflow policies appear side by side, each correct for its direction: the planner-to-RT segment queue back-pressures (dropping a segment means moving through unplanned geometry), the telemetry ring drops oldest (blocking the RT tier is worse than losing a sample), and the vision frame pool rejects (the camera cannot wait, and a late frame is worthless).
ECAT_MASTER holds its interface as consume (reaching OP again means a
full bus rebuild) and must reach OP before any setpoint is written. Each
servo carries ready_on_write, so its first command equals its first
measured position.
Try: crash the pendant. Segments starve but the interpolator holds last-good and setpoints keep flowing. Push the camera rate past what the frame pool drains and it rejects; stop the logger and the telemetry ring fills in a blink, dropping oldest. Stall a safety channel and STO asserts as a bound cascade: readiness withdrawn, safety IO stops, the limit enforcer follows, the servos stop. Restart the channel and the same edges bring the plane back in order.
Sources: ROS + EtherCAT real-time control architecture · ros2_control controller manager · per-component update rates · ethercat_driver_ros2 · EtherCAT multi-axis synchronization · CiA 402 drive profile · cascaded loops and increment streams · safe torque off · EN 61800-5-2 safety sub-functions · SS1/SS2/SOS explained · LinuxCNC code notes (RT / non-RT boundary) · Klipper code overview · Klipper’s 1024-slot move pool · Klipper MCU command flow control · Klipper toolhead look-ahead · grbl planner ring
Edge A/V streaming head
Section titled “Edge A/V streaming head”A 4K IP camera: dual-stream video, two-way audio, on-device recording and network serving.
The contended resource is a refcounted DMA frame pool whose teardown must
unblock its waiters before it frees. FRAMES is a Leased signal: stopping
the allocator refuses new leases, waits for the count to reach zero, then
frees. A naive drop would deadlock; a leaked guard degrades to an ordinary
ShutdownTimeout naming the producer, instead of a use-after-free.
All three resource kinds are load-bearing at once: the 3A loop lends the sensor I2C (borrow per frame, restored on exit); the capture pipe and encoder channel are consume (a rebuild loses reference-frame state and forces a fresh IDR); the frame pool is shared across every stage. Restarting the pool is rest_for_one, and the encoder respawn fails closed on its spent channel until you rebuild it.
The substream is demand-started (OnDemand, brought up by start_node when a
subscriber joins); per-client RTP sessions are a min: 0 pool that drops its
own frames and never back-pressures the shared encoder. And PTP_SERVO
carries the one dependency kind nothing else in the set has: value
freshness rather than liveness. Stall the timestamps and the streams stay
smooth while the servo holds last-good and drifts; a heartbeat cannot catch
it.
Sources: GStreamer queue leaky modes · GStreamer buffer-pool design (deactivate-to-unblock) · GStreamer latency design · appsink max-buffers · rtpjitterbuffer · gst-rtsp-server media factory · CVITEK MPI video encoding API · CVITEK VENC design overview · OpenIPC open Hi35xx SDK · V4L2 buffer queueing · libcamera IPA (frame N stats -> N+1 params) · RFC 3550 RTP/RTCP · RFC 4585 AVPF · RFC 5104 CCM (PLI vs FIR) · RFC 7273 reference clocks · Google congestion control draft · ONVIF Media2 (SetSynchronizationPoint, encoder instances) · ONVIF recording control · Axis on shared encoded streams · ALSA PCM ring, periods and XRUN · AES67 clocking · SMPTE ST 2110-10 · Dante latency guide · Infineon ASRC (drift compensation)
Substation protection IED
Section titled “Substation protection IED”An IEC 61850 relay across the process bus and the station bus: two networks
that fail asymmetrically, which makes it the clearest ready bound
demonstration in the set.
SV_ALIGN carries ready_on_write: protection must not evaluate before a
full aligned sample window exists, or the relay reacts to filter garbage.
PROT_87 (differential) needs matching time sync across merging units, so it
carries deps: [PTP_SLAVE ready bound]: losing PTP bound-stops
differential while overcurrent, which needs only magnitude, keeps
protecting. Drop the station bus instead and nothing in the protection
plane moves: MMS stops, SCADA goes blind, the relay keeps tripping. Two link
failures, two different blast radii.
Breaker failure (PROT_50BF) is OnDemand, armed by the first trip, as a
real relay arms it. Autoreclose (PROT_79) is Pause, a state machine that
must survive across cycles. TRIP is a veto write from both protection
functions: a VetoGate where each writer holds its own contributor bit, any
bit forces the safe state and none owns it. A stopped writer’s bit stays up:
trip on differential, then lose PTP, and the bound-stopped function keeps the
breaker open until it runs again and re-evaluates. Fail-safe by construction,
not by convention.
Sources: sampled values explained · GOOSE retransmission profile · SEL on IEC 61850 transfer-time classes · IEC/IEEE 61850-9-3 power profile · corrupted SV and protection blocking · time-sync degradation in digital substations · GOOSE vs sampled values · process / bay / station levels · backup SV subscription for differential · OMICRON on utility time sync
Multi-parameter patient monitor
Section titled “Multi-parameter patient monitor”The only system that changes shape at runtime, and the only one whose contended resource is arbitrated by severity rather than arrival.
Parameter modules are OnDemand subtrees inserted and removed from the
device buttons. Remove one (a single stop at the acquisition) and its DSP
and alarm stages follow it down through ready bound edges: the absence is
announced, not silent. An absent module is a state the arbiter knows
about, not a fault to respawn.
The gates are hazard analysis. PATIENT_CONTEXT must publish before anything
runs (ready_on_write): adult versus neonate changes every alarm limit, so a
detector started on defaults is a real hazard. Every alarm detector carries
deps: [ALARM_ARBITER ready]: a detector firing into a void is a
silent-alarm hazard. The arrhythmia analyzer must complete its learning
phase before it may publish a rhythm at all.
The audio codec is a Leased handle the arbiter holds per alarm burst: a
truncated melody is not a conformant signal, so it cannot be re-lent
mid-burst. The NIBP pneumatics are consume: the measurement cycle runs to
completion and the next one fails closed until the cuff is re-armed, while
an always-on safety monitor watches regardless.
Sources: IEC 60601-1-8 alarm systems · PUI Audio 60601-1-8 application guide · alarm priority assignment · EN IEC 80601-2-49 multifunction monitors · IEC 80601-2-30 NIBP overpressure cutoffs · IntelliVue plug-and-play module detection · ASTM F2761 ICE architecture · OpenICE · alarm fatigue data · Same Sky on 60601-1-8 · IntelliVue MMX module · ISO/IEEE 11073-10101 nomenclature · modular monitor docking patent · Philips ST/AR learn and relearn phases · IEC 62304 safety classes
CubeSat flight software
Section titled “CubeSat flight software”The cFS shape: the best-documented flight-software roster there is. Core
services (TIME, SCH, EVS, HS, the software-bus pipe, …) around
mission apps: the ADCS chain, EPS, thermal, payload, CFDP, file manager,
stored commands.
SCH waits out its first major-frame sync behind a generous slot_timeout,
like the real SCH app waits for its 1 Hz tone. The attitude chain will not
act on a garbage quaternion: the estimator is ready_on_write and the
controller gates on it. The software-bus pipe rejects on overflow: the
router must never block on a slow subscriber.
Demand-start is native here. FM, CF and the payload are OnDemand,
started by ground command; TO parks bound until the radio comes up for a
pass. And the purest demand-start edge in the set: the limit checker’s gated
read opens the sequence signal, which starts the disabled stored-command
app through the real control queue: a data access starting a task, with no
deps: on that edge. Health services answer a stalled app with the first
rung of the real ladder: an automatic restart.
Sources: cFE application developer’s guide · NASA cFS project · cFS app catalog · SCH platform config (100 slots, startup sync) · SCH readme · HS internal config (watchdog, restart budget) · HS app · SC · LC · DS · NOS3 cFS scenario · CubeSat flight software case study · NASA SWE on initialization and safe mode · NASA SWE on fault detection · JPL hierarchical fault protection
EV charging site controller
Section titled “EV charging site controller”The only scenario built on a divisible shared resource: one site power limit, continuously re-divided across the active charging sessions. This is the EVerest / OCPP smart-charging shape.
When a claimant joins, every grant shrinks instantly; when one leaves, grants grow back slowly, never in one jump. Sessions are a pool re-dividing the budget as it scales; the RCD and thermal monitors bypass the allocator entirely: a ground fault or an overheat is not negotiable.
The OCPP side shows the offline contract: drop the CSMS link and the transmitter bound-stops while transaction events queue in a back-pressured store-and-forward. A start precedes its meter values precedes its stop, and that order must survive the outage. Reconnect and the backlog drains.
The site limit is a divisible resource: ENERGY_MGR provides the
Budget, each session’s Claimant states its want while a car is connected,
and the allocator re-divides under ShrinkFastGrowSlow: a cut lands at once,
an increase moves at most 4 A per period. Stop a session from the outside and
the supervisor releases its share on the shutdown ack, the worker never touches
its claim, so a dead session never strands its amps. A derate is a re-provide
with less.
Sources: EVerest (LF Energy) · EVerest framework · everest-core · EVerest OCPP module interfaces · EVerest energy tree as deployed · OCPP 1.6J flows and offline ordering · OCPP 2.0.1 smart charging · Open Charge Alliance · AMPECO on dynamic load management · IEC 61851-1 control pilot · Wolfspeed on DC fast charger architecture · J.P. Morgan store-and-forward contract