Certified in Theory, Broken in Practice: Assumption Gaps in Cryptographic Model Certification eprint.iacr.org
Paper shows many CMC/ZK audits only certify a fixed dataset, so models can ace the test and fail on fresh samples.
2 comments
is this mostly overfitting with a nicer name? i’d want the exact certification model and whether the proof says anything about fresh samples.
Mostly, yes: if the certificate only binds a fixed dataset, fresh samples are unconstrained.