motivation
BBN 对于处理长尾问题非常有效, 我在想, 能不能类似地用在鲁棒问题上.
思想很简单, 就是上面用干净数据, 下面用对抗样本(其用(alpha=0.5)的eval mode 生成), 但是结果非常差.
settings
- batch_size: 128
- beta1: 0.9
- beta2: 0.999
- dataset: cifar10
- epochs: 200
- epsilon: 0.03137254901960784
- eva_alpha: 0.5
- learning_policy: AT
- loss: cross_entropy
- lr: 0.1
- model: resnet32
- momentum: 0.9
- norm_cls: True | False
- optimizer: sgd
- progress: False
- resume: False
- seed: 1
- steps: 10
- stepsize: 0.25
- transform: default
- weight_decay: 0.0002
results
| Loss | Accuracy | Robustness | |
|---|---|---|---|
| parabolic decay; norm_cls=False |  |
 |
 |
| fixed_alpha=0.5; norm_cls=False |  |
 |
 |
| parabolic decay; norm_cls=True |  |
 |
 |
| fixed_alpha=0.5; norm_cls=True |  |
 |
 |
可以看得出来, 强行糅合在一起效果不好, 显然干净的features或者对抗的features占主导的时候精度能上去.