We propose PatchZero, a task-agnostic defense that is compatible with multiple downstream models. Specifically, our defense detects the adversarial pixels and “zeros out” the patch region by repainting with mean pixel values. We formulate the patch detection problem as a semantic segmentation task such that our model can generalize to patches of any size and shape. We further design a two-stage adversarial training scheme to defend against the stronger adaptive attacks. Our method achieves state-of-the-art robust ac-curacy without any degrade in the benign performance.
TMLR
WACV 2023