Progressive Prompts: Continual Learning for Language Models

Original Paper: https://arxiv.org/abs/2301.12314

By: Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Amjad Almahairi

Abstract:

We introduce Progressive Prompts - a simple and efficient approach for continual learning in language models. Our method allows forward transfer and resists catastrophic forgetting, without relying on data replay or a large number of task-specific parameters. Progressive Prompts learns a new soft prompt for each task and sequentially concatenates it with the previously learned prompts, while keeping the base model frozen. Experiments on standard continual learning benchmarks show that our approach outperforms state-of-the-art methods, with an improvement >20% in average test accuracy over the previous best-preforming method on T5 model. We also explore a more challenging continual learning setup with longer sequences of tasks and show that Progressive Prompts significantly outperforms prior methods.

Summary Notes

Blog Post: Simplifying Continual Learning with Progressive Prompts for Language Models

The rapidly evolving field of Natural Language Processing (NLP) requires language models to learn new information continuously without losing what they've already learned.

However, the challenges of catastrophic forgetting and difficulty in forward transfer make this a tough goal to achieve.

A promising solution to this problem is the use of Progressive Prompts, a method that simplifies and improves memory efficiency in continual learning (CL) for language models.

Challenges in Continual Learning

Limitations of Traditional Finetuning

• Finetuning is commonly used to adapt language models for new tasks by updating all the model's parameters with new data. This method works well for learning single tasks but struggles in a continual learning setup where a model needs to learn multiple tasks sequentially without forgetting the old ones.

Advantages of Prompt Tuning

• Prompt tuning advances this by training a set of soft prompts (additional input tokens) while keeping the base model's parameters unchanged. It significantly reduces the number of trainable parameters, making the model more adaptable with less effort.

The Continual Learning Challenge

• Achieving effective continual learning, where a model sequentially learns tasks without forgetting previous ones, remains difficult. Existing solutions often involve complex structures or many task-specific parameters, causing inefficiencies and scalability problems.

Progressive Prompts: A Novel Solution

Progressive Prompts innovate by applying the concept of prompt tuning specifically for continual learning, through:

• Sequential Learning of New Prompts: Learning a new prompt for each task without changing the previously learned prompts or the base model parameters.
• Concatenating Learned Prompts: Combining new prompts with those from past tasks prevents interference with previous learning (avoiding catastrophic forgetting) and allows for knowledge reuse (aiding forward transfer).
• Prompt Embedding Reparameterization: A technique using a residual Multi-Layer Perceptron (MLP) that stabilizes the learning process and ensures the addition of new prompts doesn’t negatively impact performance on earlier tasks.

Insights from Experiments

Experiment Setup

• The effectiveness of Progressive Prompts was tested using a variety of text classification tasks (e.g., AG News, Amazon Reviews, Yelp Reviews) and compared against traditional finetuning and other continual learning approaches.

Results

• Progressive Prompts showed a remarkable improvement, with over 20% better average test accuracy on continual learning benchmarks using the T5 model. It was especially effective for long sequences of tasks and tasks with limited training data, showcasing its efficiency and versatility.

Looking Ahead

The development of Progressive Prompts is a significant advancement in the continual learning space for language models. This method addresses the major issues of catastrophic forgetting and forward transfer with exceptional efficiency and versatility.

With the need for far fewer parameters than traditional methods, Progressive Prompts set a new standard for scalable and practical continual learning in NLP applications.

As we progress, the impact of this research on creating more intelligent, adaptable, and efficient NLP systems is immense.

Progressive Prompts represent a breakthrough in overcoming the challenges of continual learning, promising to influence the future direction of NLP research and applications.

In summary, Progressive Prompts bring us closer to achieving language models that can learn and adapt over time without the limitations of previous methods.

This approach not only enhances the capabilities of existing models but also establishes a new benchmark for efficiency and scalability in NLP.