Universal and Transferable Adversarial Attacks on Aligned Language Models

Original Paper: https://arxiv.org/abs/2307.15043

By: Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, Matt Fredrikson

Abstract:

Because "out-of-the-box" large language models are capable of generating a great deal of objectionable content, recent work has focused on aligning these models in an attempt to prevent undesirable generation. While there has been some success at circumventing these measures -- so-called "jailbreaks" against LLMs -- these attacks have required significant human ingenuity and are brittle in practice. In this paper, we propose a simple and effective attack method that causes aligned language models to generate objectionable behaviors. Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer). However, instead of relying on manual engineering, our approach automatically produces these adversarial suffixes by a combination of greedy and gradient-based search techniques, and also improves over past automatic prompt generation methods.

Surprisingly, we find that the adversarial prompts generated by our approach are quite transferable, including to black-box, publicly released LLMs. Specifically, we train an adversarial attack suffix on multiple prompts (i.e., queries asking for many different types of objectionable content), as well as multiple models (in our case, Vicuna-7B and 13B). When doing so, the resulting attack suffix is able to induce objectionable content in the public interfaces to ChatGPT, Bard, and Claude, as well as open source LLMs such as LLaMA-2-Chat, Pythia, Falcon, and others. In total, this work significantly advances the state-of-the-art in adversarial attacks against aligned language models, raising important questions about how such systems can be prevented from producing objectionable information. Code is available at this http URL.

Summary

The realm of artificial intelligence (AI) faces ongoing challenges in securing and aligning language models (LLMs), which are becoming increasingly integral to various sectors. A recent study introduces an innovative method for creating universal and transferable adversarial attacks aimed at aligned LLMs, potentially circumventing their safeguards against inappropriate content. This blog post explores the study's findings, methodologies, contributions, and their implications for AI professionals in the corporate world.

Key Insights and Contributions

The study unveils a method for appending a specially designed suffix to user inputs, prompting LLMs to generate undesirable outputs. Highlights include:

• Universally Effective Attacks: The attacks work across a wide range of queries and can be applied to various LLMs, even those not considered during the attack development.
• Automated Creation of Adversarial Suffixes: This marks a significant improvement over older methods by automating the creation of these suffixes.
• Impressive Success Rates: Success rates are notable, with up to 84% effectiveness against GPT-3.5 and GPT-4, and 66% against PaLM-2.
• Ethical Approach: The researchers engaged in responsible disclosure and discussed the ethical implications and potential misuse of their work.

Methodology Simplified

The approach to generating these adversarial attacks involves three main strategies:

1. Prompting with Affirmative Responses: Starting responses with an affirmative phrase increases the likelihood of the model generating inappropriate content.
2. Optimization Techniques: A new optimization method combines greedy and gradient-based techniques to find the best tokens for eliciting the desired response.
3. Reliable Attacks Across Models and Prompts: Developing a single adversarial suffix that is effective across various prompts and models enhances the attack's reliability and transferability.

AdvBench Benchmark

The study introduces the AdvBench benchmark, a tool for systematically assessing how well adversarial attacks can induce LLMs to produce harmful content. It focuses on two aspects: harmful strings and behaviors, testing the models' ability to control outputs finely and bypass safety measures.

Results and Their Significance

The attack method far surpasses previous techniques in triggering inappropriate responses from LLMs, underscoring the need for stronger alignment and safety measures. It points to the difficulties in ensuring LLMs adhere to human values and safety standards, posing significant questions for future alignment strategies.

For AI engineers in enterprise settings, this underscores the ongoing need for enhancing LLM security and alignment. As adversarial techniques grow more complex, countermeasures must evolve. The study suggests exploring better alignment mechanisms and new adversarial methods.

Conclusion

This research marks a substantial step forward in understanding adversarial attacks on LLMs, emphasizing the challenges of aligning these models with human values and safety protocols. It calls for a joint effort among AI researchers, developers, and ethicists to tackle these challenges, highlighting the technical and ethical importance of maintaining the safety and alignment of LLMs in our increasingly digital society.