著者: Elahe Arani , Fahad Sarfraz , Bahram Zonooz - International Conference on Learning Representations, ICLR 2022 被引用: 177
Humans excel at continually learning from an ever-changing environment whereas it remains a challenge for deep neural networks which exhibit catastrophic forgetting. The complementary learning system (CLS) theory suggests that the interplay between rapid instance-based learning and slow structured learning in the brain is crucial for accumulating and retaining knowledge. Here, we propose CLS-ER, a novel dual memory experience replay (ER) method which maintains short-term and long-term semantic memories that interact with the episodic memory. Our method employs an effective replay mechanism whereby new knowledge is acquired while aligning the decision boundaries with the semantic memories. CLS-ER does not utilize the task boundaries or make any assumption about the distribution of the data which makes it versatile and suited for "general continual learning". Our approach achieves state-of-the-art performance on standard benchmarks as well as more realistic general continual learning settings.
✨ ログイン状態を確認しています… PDF 被引用 BibTeX を表示 BibTeX を閉じる BibTeX を表示 引用
Progressive Neural Networks Progressive Neural Networks Mean teachers are better role models… Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results On Large-Batch Training for Deep Learning… On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima Deep Generative Dual Memory Network for… Deep Generative Dual Memory Network for Continual Learning Towards Robust Evaluations of Continua… Towards Robust Evaluations of Continual Learning Lifelong Learning with Dynamically Expandable… Lifelong Learning with Dynamically Expandable Networks Efficient Lifelong Learning with A-GEM Efficient Lifelong Learning with A-GEM Three scenarios for continual learning Three scenarios for continual learning Continual learning: A comparative study on ho… Continual learning: A comparative study on how to defy forgetting in classification tasks Measuring and regularizing networks i… Measuring and regularizing networks in function space Learning to Learn without Forgetting By… Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference Dark Experience for General Continual… Dark Experience for General Continual Learning: a Strong, Simple Baseline Sparse Coding in a Dual Memory System for… Sparse Coding in a Dual Memory System for Lifelong Learning A Unified Approach to Domain Incremental… A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm Look-Ahead Selective Plasticity for Continua… Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks Task-Aware Information Routing from Common… Task-Aware Information Routing from Common Representation Space in Lifelong Learning Overcoming Catastrophic Forgetting in Continual… Overcoming Catastrophic Forgetting in Continual Learning by Exploring Eigenvalues of Hessian Matrix DualHSIC: HSIC-Bottleneck and… DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning Incremental Learning of Structured Memory via… Incremental Learning of Structured Memory via Closed-Loop Transcription Label-Efficient Online Continual Object… Label-Efficient Online Continual Object Detection in Streaming Video Decoupling Learning and Remembering: a Bilevel… Decoupling Learning and Remembering: a Bilevel Memory Framework with Knowledge Projection for Task-Incremental Learning Gradual Divergence for Seamless Adaptation: A… Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method Continual Learning and Catastrophic Forgetting Continual Learning and Catastrophic Forgetting MIND: Multi-Task Incremental Network… MIND: Multi-Task Incremental Network Distillation Learning Fast, Learning Slow: A General… Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System 過去の参考文献 中心の論文 この論文を引用する論文 古い 新しい ノードをクリックするとフォーカスを固定、空白をクリックすると本論文に戻ります。ホバーで一時的にプレビューできます。各ノードのページはタイトルから開けます。
