Bootstrap Your Own Variance – Apple Machine Learning Research

This paper was accepted on the workshop Self-Supervised Studying – Concept and Follow at NeurIPS 2023.

*=Equal Contributors

Understanding mannequin uncertainty is essential for a lot of functions. We suggest Bootstrap Your Personal Variance (BYOV), combining Bootstrap Your Personal Latent (BYOL), a negative-free Self-Supervised Studying (SSL) algorithm, with Bayes by Backprop (BBB), a Bayesian technique for estimating mannequin posteriors. We discover that the realized predictive std of BYOV vs. a supervised BBB mannequin is effectively captured by a Gaussian distribution, offering preliminary proof that the realized parameter posterior is beneficial for label free uncertainty estimation. BYOV improves upon the deterministic BYOL baseline (+2.83% check ECE, +1.03% check Brier) and presents higher calibration and reliability when examined with numerous augmentations (eg: +2.4% check ECE, +1.2% check Brier for Salt & Pepper noise).