litlas
Log inCreate account

Momentum Contrast for Unsupervised Visual Representation Learning

Authors: Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross B. Girshick - IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020 cited by 12,039

We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.

Deep Residual Learningfor Image RecognitionDeep Residual Learning for Image RecognitionUnsupervised Learning ofVisual Representations…Unsupervised Learning of Visual Representations by Solving Jigsaw PuzzlesSplit-BrainAutoencoders…Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel PredictionUnsupervised FeatureLearning via…Unsupervised Feature Learning via Non-Parametric Instance DiscriminationUnsupervisedRepresentation Learning…Unsupervised Representation Learning by Predicting Image RotationsUnsupervised FeatureLearning via…Unsupervised Feature Learning via Non-Parametric Instance-level DiscriminationRevisitingSelf-Supervised Visual…Revisiting Self-Supervised Visual Representation LearningData-Efficient ImageRecognition with…Data-Efficient Image Recognition with Contrastive Predictive CodingLearning Representationsby Maximizing Mutual…Learning Representations by Maximizing Mutual Information Across ViewsA Simple Framework forContrastive Learning of…A Simple Framework for Contrastive Learning of Visual RepresentationsImproved Baselines withMomentum Contrastive…Improved Baselines with Momentum Contrastive LearningContrastive MultiviewCodingContrastive Multiview CodingImproving ObjectDetection with Selectiv…Improving Object Detection with Selective Self-supervised Self-trainingRobust Pre-Training byAdversarial Contrastive…Robust Pre-Training by Adversarial Contrastive LearningAn Overview of DeepSemi-Supervised LearningAn Overview of Deep Semi-Supervised LearningSelf-SupervisedLearning: Generative or…Self-Supervised Learning: Generative or ContrastiveTrain a One-Million-WayInstance Classifier for…Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation LearningImproving TransformationInvariance in…Improving Transformation Invariance in Contrastive Representation LearningSelf-supervised VisualAttribute Learning for…Self-supervised Visual Attribute Learning for Fashion CompatibilitySelf-Supervised Learningfor Videos: A SurveySelf-Supervised Learning for Videos: A SurveySelf-Supervised Learningfrom Images with a…Self-Supervised Learning from Images with a Joint-Embedding Predictive ArchitectureText-DIAE: ASelf-Supervised…Text-DIAE: A Self-Supervised Degradation Invariant Autoencoder for Text Recognition and Document EnhancementContrastive encoderpre-training-based…Contrastive encoder pre-training-based clustered federated learning for heterogeneous dataA Closer Look atBenchmarking…A Closer Look at Benchmarking Self-Supervised Pre-training with Image ClassificationMomentum Contrast forUnsupervised Visual…Momentum Contrast for Unsupervised Visual Representation LearningEarlier referencesFocus paperCiting papersOlderNewer

Click a node to pin it, click the empty canvas to go back to this paper, or hover to preview. Open a node’s page from its title.