Incremental and Adaptive Machine Learning

As humans we learn continually throughout our lifetime, we are adaptive to ever-changing environments, and can exploit obtained knowledge to swiftly pick up novel or more advanced concepts. This is in stark contrast to the standard machine learning paradigm, where learning a model occurs strictly once before using its knowledge in practice. Limitations arise from a static model, as our world is ever-changing, pushing the need for a more flexible paradigm that enables continual learning and adaptation to the environment. Overcoming such limitations is central to the field of continual learning, and is the main topic of this thesis. In this highly interdisciplinary field, we maintain a feasible scope by focusing on artificial neural networks that learn to classify from labeled images. The thesis starts with a brief introduction to static machine learning and its limitations, and contrasts this to desirable aspects of an ideal adaptive learner. The reader is introduced to the field of continual learning, including the common performance metrics and datasets, typical use cases for continual learning, and the categorization into four learning scenarios. The central problem to learn continually in neural networks is that learning new tasks induces forgetting of those previously learned. The introduction establishes a background for the following chapters that engage more deeply in scientific contributions to the field. First, as progress in continual learning is often reported on disparate benchmarks and assumptions, we conduct a comprehensive study of continual learning methods that allows fair comparison. Here, we focus on the task incremental scenario, where for a given sample, the task is always assumed to be known, hence simplifying to task-specific classification problems. To increase the autonomy of the continual learner, we introduce the continual hyperparameter selection framework that algorithmically determines the trade-off between learning the new task and remembering the previous tasks. Subsequently, we investigate the more challenging scenario of class incremental learning, where tasks are still sequentially observed but classification combines the classes of all observed tasks. In this setup, revisiting tasks by storing small portions of their data in a buffer, is known to be highly effective in avoiding forgetting of previous tasks. This approach is called replay and despite its wide use in continual learning due to its simplicity and efficacy, its learning behavior is still poorly understood. Therefore, we provide further insights into the learning trajectory from the perspective of loss landscapes. As class incremental learning still assumes a task-based structure in the data stream, we subsequently consider the data incremental scenario where no such structure is presumed. We look at the intersection with representation learning and devise a method with continually evolving prototypes that enables online learning of the stream, and is robust to high class-imbalance. Finally, we shift the focus from continual learning to continual evaluation with a focus on worst-case performance analysis of continual learners. This reveals the unexpected phenomenon of the stability gap, where a sharp performance drop occurs when learning new tasks, followed by a phase of knowledge recovery. Moreover, we find that the phenomenon holds over a range of established continual learning methods.

Jaar van publicatie:2023

Toegankelijkheid:Open