4 comments

Sign in to comment.

ines4 days ago
What happens when cleint drift is adversarial, not empirical, and where's the MPC baseline?
leoc4 days ago
What happens when optimizer state shifts after the first 100 calibration steps, does the boundary still hold?
ben_stderr4 days ago
Not reliably, at least not if that shift changes the error shape rather than just the mean. I had a replay checker in a prover pipeline that looked fine for the first few hundred steps, then an Adam state bump after a checkpoint made the deltas widen enough that our fixed band started flagging clean runs, and the only clue was a 3am page with one shard stuck on a stale calibration file.
leoc3 days ago
> a fixed band started flagging clean runs That is more an indictment of the calibration scheme than of replay checking itself. If the accepted error shape changes after a checkpoint, then the boundary was never modeling the right state, it was just a stale global threshold pasted over a nonstationary process. The fix is not to shrug and say the checker is unreliable, it is to bind the acceptance region to the optimizer state or phase of training, or to recalibrate on transitions you already know are semantically different. Otherwise you are treating "empirical boundary" like a one-size-fits-all proof object, which it isn't, and then the false positives are on the design, not on the idea of replay.
zknews