A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Original Paper: https://arxiv.org/abs/2302.11382

By: Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, Douglas C. Schmidt

Abstract:

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

Summary Notes

Blog Post: Simplifying Conversations with ChatGPT through Prompt Engineering

Prompt engineering is a key skill in artificial intelligence, especially for optimizing the use of Large Language Models (LLMs) like ChatGPT.

By creating effective prompts, developers can greatly improve the AI's responses, leading to more valuable conversations.

However, prompt engineering can be quite complex, requiring a systematic approach to achieve the best results from these advanced models.

A Structured Approach to Prompt Engineering

Taking cues from software engineering, where pattern cataloging is a common practice, we can apply a similar approach to prompt engineering for LLMs.

This method involves documenting patterns that offer standardized solutions to frequent prompt design challenges.

These patterns are detailed, covering aspects such as their purpose, motivation, structure, and examples, providing a valuable toolkit for AI Engineers.

Types of Prompt Patterns

Prompt engineering patterns can be grouped into several key categories:

• Input Semantics: Focuses on crafting prompts that the LLM can understand and process effectively.
• Output Customization: Helps guide the LLM to produce responses that meet specific user needs, including format, tone, and content.
• Error Identification: Aids in spotting and correcting errors or unwanted elements in the LLM's outputs.
• Prompt Improvement: Involves tweaking prompts to improve the quality and relevance of the LLM's responses.
• Interaction: Manages how users and the LLM communicate, ensuring smooth and natural exchanges.
• Context Control: Ensures the LLM's responses are relevant and coherent by managing the contextual information it uses.

Exploring the Patterns

Input Semantics

This ensures prompts are clear and structured in a way the LLM can easily interpret.

Output Customization

These patterns help in tailoring responses to fit user requirements, adjusting aspects like detail level and tone.

Error Identification

Critical for maintaining interaction integrity, these patterns help identify and correct inaccuracies in LLM outputs.

Prompt Improvement

Refining prompts based on previous responses helps enhance future interactions.

Interaction

Ensures conversations flow logically and engagingly, keeping context over multiple exchanges.

Context Control

Manages the information backdrop, ensuring the LLM's responses remain relevant throughout the conversation.

Practical Examples

• Output Automater: Automates script or code generation based on specific LLM outputs.
• Meta Language Creation: Allows for the definition of custom languages or terminologies, making the LLM understand and respond to specialized inputs effectively.

Impact and Future Directions

Adopting a pattern-based approach in prompt engineering can significantly improve the development of conversational AI.

It provides a structured way for AI Engineers to enhance system functionality and user experience with LLMs like ChatGPT.

However, as LLM capabilities evolve, the catalog of prompt patterns will also need to update, highlighting the importance of continuous research and adaptation in this area.

Conclusion

Creating a prompt pattern catalog is a crucial step forward in conversational AI. It equips AI Engineers with a comprehensive set of tools for prompt engineering, leading to more advanced, efficient, and user-friendly interactions with models like ChatGPT.

As we refine and grow this catalog, the possibilities for improving AI-human communication continue to expand, opening up new avenues for innovation in artificial intelligence.