research-papers

Prompt-Engineering and Transformer-based Question Generation and Evaluation

Original Paper: https://arxiv.org/abs/2310.18867 By: Rubaba Amyeen Abstract: Question generation has numerous applications in the educational context. Question generation can prove helpful for students when reviewing content and testing themselves. Furthermore, a question generation model can aid teachers by lessening the burden of creating assessments and

Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering

Original Paper: https://arxiv.org/abs/2308.13259 By: Keheng Wang, Feiyu Duan, Sirui Wang, Peiguang Li, Yunsen Xian, Chuantao Yin, Wenge Rong, Zhang Xiong Abstract: Equipped with Chain-of-Thought (CoT), Large language models (LLMs) have shown impressive reasoning ability in various downstream tasks. Even so, suffering from hallucinations and the

ResearDesign Guidelines for Prompt Engineering Text-to-Image Generative Models

Original Paper: https://arxiv.org/abs/2109.06977 By: Vivian Liu, Lydia B. Chilton Abstract: Text-to-image generative models are a new and powerful way to generate visual artwork. However, the open-ended nature of text as interaction is double-edged; while users can input anything and have access to an infinite range

Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review

Original Paper: https://arxiv.org/pdf/2310.14735 By: Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, Shengxin Zhu Abstract: This paper delves into the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs). Prompt engineering is the process of structuring input text for LLMs and is

Batch Prompting: Efficient Inference with Large Language Model APIs

Original Paper: https://arxiv.org/abs/2301.08721 By: Zhoujun Cheng, Jungo Kasai, Tao Yu Abstract: Performing inference on large volumes of samples with large language models (LLMs) can be computationally and financially costly in industry and real-world use. We propose batch prompting, a simple yet effective prompting approach that

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic

Original Paper: https://arxiv.org/abs/2309.13339 By: Xufeng Zhao, Mengdi Li, Wenhao Lu, Cornelius Weber, Jae Hee Lee, Kun Chu, Stefan Wermter Abstract: Recent advancements in large language models have showcased their remarkable generalizability across various domains. However, their reasoning abilities still have significant room for improvement, especially

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

Original Paper: https://arxiv.org/abs/2310.13345 By: Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, Mohan Kankanhalli Abstract: The wide-ranging applications of large language models (LLMs), especially in safety-critical domains, necessitate the proper evaluation of the LLM's adversarial robustness. This paper proposes

Text Summarization: LLM Failure Cases and Detection Methods

Large Language Models (LLMs) excel at sifting lengthy text to extract key insights, transforming documents into concise summaries. An increasing number of companies are using LLMs for text summarization—be it to condense massive datasets or enhance data quality. Yet, ensuring the reliability and consistency of LLMs in production remains

The Foundation Model Transparency Index

Original Paper: https://arxiv.org/abs/2310.12941 By: Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, Percy Liang Abstract: Foundation models have rapidly permeated society, catalyzing a wave of generative AI applications spanning enterprise and consumer-facing contexts. While the societal impact of foundation

research-papers

NLPBench: Evaluating Large Language Models on Solving NLP Problems

Original Paper: https://arxiv.org/abs/2309.15630 By: Linxin Song, Jieyu Zhang, Lechao Cheng, Pengyuan Zhou, Tianyi Zhou, Irene Li Abstract: Recent developments in large language models (LLMs) have shown promise in enhancing the capabilities of natural language processing (NLP). Despite these successes, there remains a dearth of research

Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

Original Paper: https://arxiv.org/abs/2310.11324 By: Melanie Sclar, Yejin Choi, Yulia Tsvetkov, Alane Suhr Abstract: As large language models (LLMs) are adopted as a fundamental component of language technologies, it is crucial to accurately characterize their performance. Because choices in prompt design can strongly influence model behavior,

Last One Standing: A Comparative Analysis of Security and Privacy of Soft Prompt Tuning, LoRA, and In-Context Learning

Original Paper: https://arxiv.org/abs/2310.11397 By: Rui Wen, Tianhao Wang, Michael Backes, Yang Zhang, Ahmed Salem Abstract: Large Language Models (LLMs) are powerful tools for natural language processing, enabling novel applications and user experiences. However, to achieve optimal performance, LLMs often require adaptation with private data, which