Authors: Xinlei Chen , Haoqi Fan , Ross B. Girshick , Kaiming He - arXiv 2020 cited by 2,941
Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.
✨ Checking sign-in… PDF Cited by View BibTeX Hide BibTeX View BibTeX Cite
Microsoft COCO: Common Objects in Context Microsoft COCO: Common Objects in Context Faster R-CNN: Towards Real-Time Object… Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Deep Residual Learning for Image Recognition Deep Residual Learning for Image Recognition Unsupervised Feature Learning via… Unsupervised Feature Learning via Non-Parametric Instance Discrimination Representation Learning with Contrastive… Representation Learning with Contrastive Predictive Coding Learning Representations by Maximizing Mutual… Learning Representations by Maximizing Mutual Information Across Views Data-Efficient Image Recognition with… Data-Efficient Image Recognition with Contrastive Predictive Coding Learning deep representations by… Learning deep representations by mutual information estimation and maximization Momentum Contrast for Unsupervised Visual… Momentum Contrast for Unsupervised Visual Representation Learning A Simple Framework for Contrastive Learning of… A Simple Framework for Contrastive Learning of Visual Representations Contrastive Multiview Coding Contrastive Multiview Coding Self-Supervised Learning of Pretext-Invariant… Self-Supervised Learning of Pretext-Invariant Representations Self-supervised learning through the eyes of a… Self-supervised learning through the eyes of a child Multimodal Contrastive Training for Visual… Multimodal Contrastive Training for Visual Representation Learning Divide and Contrast: Self-supervised Learnin… Divide and Contrast: Self-supervised Learning from Uncurated Data Review on self-supervised image… Review on self-supervised image recognition using deep neural networks A survey on Semi-, Self- and Unsupervised… A survey on Semi-, Self- and Unsupervised Techniques in Image Classification Unsupervised Pre-Training for Person… Unsupervised Pre-Training for Person Re-Identification Towards Good Practices for Efficiently… Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets Multi-Label Contrastive Learning for Abstract… Multi-Label Contrastive Learning for Abstract Visual Reasoning Test-Agnostic Long-Tailed Recognition… Test-Agnostic Long-Tailed Recognition by Test-Time Aggregating Diverse Experts with Self-Supervision Self-Supervised Learning from Images with a… Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture Multi-Mode Online Knowledge Distillation… Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning Exploring the Coordination of… Exploring the Coordination of Frequency and Attention in Masked Image Modeling Improved Baselines with Momentum Contrastive… Improved Baselines with Momentum Contrastive Learning Earlier references Focus paper Citing papers Older Newer Click a node to pin it, click the empty canvas to go back to this paper, or hover to preview. Open a node’s page from its title.
