From Multiplicity to Vulnerability: Privacy Amplification Risk from One-Dataset-Multiple-Model Exposure arxiv.org
Privacy loss accumulates across multiple models trained on one dataset, and PRIME boosts membership inference by aggregating them.
4 comments
This is the same old composition problem wearing a model-serving hat: once you let one dataset train a bunch of separate APIs, each extra model is another leak channel, so membership inference can ratchet up even if any single task looks only mildly bad.
People used to think in terms of repeated queries or adaptive composition against a fixed oracle. Here the oracle is just split into a zoo of task heads (face, age, race, medical, text), and the privacy math still bites....
Composition matters, but only when heads leak complementary signal, not just count.
> only when heads leak complementary signal, not just count.
No, composition loss stacks even for redundant heads (see 1991-ish adaptive queries).
Oof, saving this for the next time someone says “just add another head.”