Get Started¶

Common Setting
Add Noise
Metrics
Training Pipeline

Common Setting¶

If you want to use this project to train or evaluate model(s), you can choose to create a work directory for saving config, checkpoints, scripts etc.

We have put some example for trainging or evlaluate. You can use it as follows

cd exprs/exp/imagenet-a_o-loop
bash run.sh

Add Noise¶

You can use the AddNoise’s add_noise function to add multiple noise for one image or a batch of images The supported noise list is: [‘imagenet-s’, ‘imagenet-c’, ‘pgd_linf’, ‘pgd_l2’, ‘fgsm’, ‘autoattack_linf’, ‘mim_linf’, ‘pgd_l1’]

Example of adding ImageNet-C noise for image

from RobustART.noise import AddNoise
NoiseClass = AddNoise(noise_type='imagenet-c')
# set the config of one kind of noise
NoiseClass.setconfig(corruption_name='gaussian_noise')
image_addnoise = NoiseClass.add_noise(image='test_input.jpeg')

Training Pipeline¶

We provided cls_solver solver to train a model with a specific config

Example of using base config to train a resnet50

cd exprs/robust_baseline_exp/resnet/resnet50
#Change the python path to the root path
PYTHONPATH=$PYTHONPATH:../../../../
srun -n8 --gpu "python -u -m RobustART.training.cls_solver --config config.yaml"

Evaluation Pipeline¶

We evaluate model(s) of different dataset, we provides several solver to evaluate the model on one or some specific dataset(s)

Example of evaluation on ImageNet-A and ImageNet-O dataset

cd exprs/exp/imagenet-a_0-loop
#Change the python path to the root path
PYTHONPATH=$PYTHONPATH:../../../
srun -n8 --gpu "python -u -m RobustART.training.cls_solver --config config.yaml"

Metrics¶

We provided metrics APIs, so that you can use these APIs to evaluate results for ImageNet-A,O,P,C,S and Adv noise.

from RobustART.metrics import ImageNetAEvaluator
metric = ImageNetAEvaluator()
metric.eval(res_file)