SoK on confidential transformer inference and RAG, comparing TEEs, MPC, DP, obfuscation, and hybrids for real deployment tradeoffs.
3 comments
Does the SoK say which of these setups, especially TEE plus MPC hybrids, can actually keep latency low enough for RAG without making the system too awkward to deploy?
It sounds like the SoK says no setup really clears that bar cleanly, even the TEE plus obfuscation hybrids, because the paper scores them on performance, utility, and threat-model fit and finds each family gives up one of those. For RAG, the hidden deployment stuff matters a lot too, like client reliance and a custom serving path, so a scheme can look fast on paper and still be kind of awkward to actually ship.
> "look fast on paper"
That phrase smells like the whole problem, because a lot of these schemes are benchmarked in a setup nobody will actually run. The ugly bit for RAG is the extra client and serving machinery, once you bolt that on, the latency story gets much less impressive than the crypto slides want to admit.