Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement

We suggest Dataset Reinforcement, a technique to enhance a dataset as soon as such that the accuracy of any mannequin structure skilled on the bolstered dataset is improved at no extra coaching price for customers. We suggest a Dataset Reinforcement technique based mostly on information augmentation and information distillation. Our generic technique is designed based mostly on intensive evaluation throughout CNN- and transformer-based fashions and performing large-scale research of distillation with state-of-the-art fashions with numerous information augmentations. We create a bolstered model of the ImageNet coaching dataset, known as ImageNet+, in addition to bolstered datasets CIFAR-100+, Flowers-102+, and Meals-101+. Fashions skilled with ImageNet+ are extra correct, sturdy, and calibrated, and switch nicely to downstream duties (e.g., segmentation and detection). For instance, the accuracy of ResNet-50 improves by 1.7% on the ImageNet validation set, 3.5% on ImageNetV2, and 10.0% on ImageNet-R. Anticipated Calibration Error (ECE) on the ImageNet validation set can be decreased by 9.9%. Utilizing this spine with Masks-RCNN for object detection on MS-COCO, the imply common precision improves by 0.8%. We attain comparable features for MobileNets, ViTs, and Swin-Transformers. For MobileNetV3 and Swin-Tiny, we observe important enhancements on ImageNet-R/A/C of as much as 20% improved robustness. Fashions pretrained on ImageNet+ and fine-tuned on CIFAR-100+, Flowers-102+, and Meals-101+, attain as much as 3.4% improved accuracy.