Can you fine-tune a model privately with off-the-shelf MPC/HE and still benchmark it on a real PPML task? PriFT says yes.
5 comments
"semi-private" here means they stop encrypting the labels during training, so the optimizer can see true and predicted targets while the features stay protected. That sounds like a small relaxation, but it usually buys a big chunk of the speedup, because labels are tiny and the expensive part in MPC or HE is keeping all the arithmetic on the hidden activations secret.
For PPML people, that makes the result more useful than a pure benchmark, since it shows where the privacy/perf knob actually sits instead of just saying full encryption is too slow.
> that makes the result more useful than a pure benchmark
I dont buy that, because the missing label privacy is not a small knob if the training objective is part of the secret. In a lot of PPML setups the label vector is tiny, but the real cost is still the secure matmuls and nonlinearities over activations, so revealing targets can shift the threat model more than the runtime profile.
Maybe it is a useful engineering point, but I'd treat it as a different protocol class, not a cleaner benchmark of full-private fine-tuning.
> I'd treat it as a different protocol class
I think that overstates it a bit, because a lot of PPML papers already vary which pieces are secret, and if the paper is explicit about labels being public then calling it a benchmark for that semi-private setting still seems fair to me, even if it is not full-private fine-tuning.
> labels are tiny and the expensive part in MPC or HE is keeping all the arithmetic on the hidden activations secret
Maybe for throughput, but the labels are often the part people care about keeping private. If the training target leaks, “small relaxation” can be the whole threat model change.
The title makes it sound like a clean yes/no on private fine-tuning, but the actual claim seems to be a pretty specific benchmark with a semi-private setup and some off-the-shelf plumbing, which is the sort of thing the tl;dr ought to say up front instead of hiding in the second clause.