Anybody know anything about this? Sounds serious. https://blog.trailofbits.com/2024/01/16/leftoverlocals-listening-to-llm-responses-through-leaked-gpu-local-memory/
It is serious, but it requires the malicious app to be running on the same computer/GPU as the app from which it is trying to exfiltrate data.
On your personal devices, it shouldn’t be a problem unless you’ve already got malware running - in which case you may have a much bigger problem to deal with.
If you are using a public server (e.g. cloud services) for your ML work, then you may have more reason for concern, because you don’t know who else may be running apps on the physical hardware where your app is running.
1 Like