Your Perception Stack Is a Snapshot Machine

Most perception stacks reason one frame at a time. The network detects, tracking is bolted on afterwards, and the system never really carries the world forward. It is a snapshot machine. And a snapshot machine cannot validate cleanly, because the thing that would make its output trustworthy is the thing it discards between frames: continuity.

I see this constantly when I evaluate architectures under ISO 26262 and SOTIF. The neural network is rarely the weak point. The weak point is that a brief disturbance, a dropped lidar return, a reflection, a spray of noise, is treated as ground truth for that frame, because the system has no physical prior that says this cannot be real. That is how you get phantom braking, and how you get an object that disappears for exactly long enough to matter.

There is no labeled ground truth in the open world. But there is physics. Objects cannot teleport, cannot accelerate without bound, cannot be seen through an opaque one. A system that carries those constraints forward in time can verify its own perception implicitly, because a reading that violates physics is an error, not a detection, and it can know that without a label.

This does not require a physics engine running at sensor rate. It requires a physically constrained, continuous world model. That is cheap. Leaving it out is expensive, in stalled release and residual risk. And the moment you add it, it becomes a safety mechanism that has to be traced to your safety goals, which is exactly the seam where most programs break.

Read the full piece on Medium → [[medium link]]

Run the check on your own perception function → https://felixschallercom.framer.website/tools/perception-autonomy-tool