litlas
ログインアカウント作成

Self-Supervised Models are Continual Learners

著者: Enrico Fini, Victor G. Turrisi da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, Julien Mairal - IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 被引用: 133

Self-supervised models have been shown to produce comparable or better visual representations than their su-pervised counterparts when trained offline on unlabeled data at scale. However, their efficacy is catastrophically reduced in a Continual Learning (CL) scenario where data is presented to the model sequentially. In this paper, we show that self-supervised loss functions can be seamlessly converted into distillation mechanisms for CL by adding a predictor network that maps the current state of the repre-sentations to their past state. This enables us to devise a framework for Continual self-supervised visual representation Learning that (i) significantly improves the quality of the learned representations, (ii) is compatible with several state-of-the-art self-supervised objectives, and (iii) needs little to no hyperparameter tuning. We demonstrate the ef-fectiveness of our approach empirically by training six pop-ular self-supervised models in various CL settings. Code: github.com/DonkeyShot21/cassle.

End-to-End IncrementalLearningEnd-to-End Incremental LearningRiemannian Walk forIncremental Learning…Riemannian Walk for Incremental Learning: Understanding Forgetting and IntransigenceUnsupervised Learning ofVisual Features by…Unsupervised Learning of Visual Features by Contrasting Cluster AssignmentsBootstrap Your OwnLatent - A New Approach…Bootstrap Your Own Latent - A New Approach to Self-Supervised LearningGDumb: A Simple Approachthat Questions Our…GDumb: A Simple Approach that Questions Our Progress in Continual LearningSelf-Supervised TrainingEnhances Online…Self-Supervised Training Enhances Online Continual LearningSPeCiaL: Self-SupervisedPretraining for…SPeCiaL: Self-Supervised Pretraining for Continual LearningEmerging Properties inSelf-Supervised Vision…Emerging Properties in Self-Supervised Vision TransformersWith a Little Help fromMy Friends…With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual RepresentationsContinual ContrastiveSelf-supervised Learnin…Continual Contrastive Self-supervised Learning for Image ClassificationRethinking theRepresentational…Rethinking the Representational Continuity: Towards Unsupervised Continual LearningVICReg:Variance-Invariance-Cov…VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised LearningSCALE: OnlineSelf-Supervised Lifelon…SCALE: Online Self-Supervised Lifelong Learning without Prior KnowledgeA soft nearest-neighborframework for continual…A soft nearest-neighbor framework for continual semi-supervised learningContinual Learning, Fastand SlowContinual Learning, Fast and SlowPrototype Reminiscenceand Augmented Asymmetri…Prototype Reminiscence and Augmented Asymmetric Knowledge Aggregation for Non-Exemplar Class-Incremental LearningSemi-supervised learningmade simple with…Semi-supervised learning made simple with self-supervised clusteringKaizen: Practicalself-supervised…Kaizen: Practical self-supervised continual learning with continual fine-tuningEvolve: EnhancingUnsupervised Continual…Evolve: Enhancing Unsupervised Continual Learning with Multiple ExpertsA Comprehensive Surveyof Continual Learning…A Comprehensive Survey of Continual Learning: Theory, Method and ApplicationClass IncrementalLearning with…Class Incremental Learning with Self-Supervised Pre-Training and Prototype LearningPlasticity-OptimizedComplementary Networks…Plasticity-Optimized Complementary Networks for Unsupervised Continual LearningContrastive ContinualLearning with Importanc…Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation DistillationExemplar-Free ContinualRepresentation Learning…Exemplar-Free Continual Representation Learning via Learnable Drift CompensationSelf-Supervised Modelsare Continual LearnersSelf-Supervised Models are Continual Learners過去の参考文献中心の論文この論文を引用する論文古い新しい

ノードをクリックするとフォーカスを固定、空白をクリックすると本論文に戻ります。ホバーで一時的にプレビューできます。各ノードのページはタイトルから開けます。