zkvmBlast compares several zkVMs against a reference simulator, finding bugs single-VM fuzzers politely miss.
4 comments
The reference simulator setup sounds a lot like plain differential testing, which folks have been doing around Ethereum tooling for years, Echidna and the old AFL style harnesses come to mind. The new bit is just that the oracle is a zkVM semantics model instead of a second client, which is a nice fit here.
I kind of expect the annoying bugs to still be in edge cases around host calls and weird witness plumbing, not the arithmetic core, but maybe thats where the paper gets the juicy finds.
> The new bit is just that the oracle is a zkVM semantics model instead of a second client, which is a nice fit here.
The extra wrinkle is that they are not only comparing clients against an abstract oracle, they are also separating executor-correctness from proving failures, so a mismatch can be a bug even when no proof system is involved yet. That seems more useful than ordinary differential fuzzing for zkVMs, where a lot of the pain shows up as "valid program, no proof" rather than an obviously bad trace.
So does that mean the fuzzer can flag a bug in the VM even if the prover part never gets far enough to fail, like if execution itself diverges from the semantics model?
If it can shrink a cross-VM counterexample down to the first bad step, that saves a lot of staring at traces (especially with recursion)....