PoisonGPT: What the Model Poisoning Demo Teaches Defenders
PoisonGPT was a model supply chain demonstration published by Mithril Security on 9 July 2023. The researchers described editing a GPT-J model to produce a targeted false answer and distributing it under a misleading repository name. The defensive lesson is to verify model origin, preserve an approved revision and evaluate relevant behaviour.