research-papers

Gemma: Open Models Based on Gemini, Research and Technology

Original Paper: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf By: Gemma Team, Google DeepMind1 Abstract: This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding,

research-papers

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Original Paper: https://arxiv.org/abs/2312.02119 By: Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, Amin Karbasi Abstract: While Large Language Models (LLMs) display versatile functionality, they continue to generate harmful, biased, and toxic content, as demonstrated by the prevalence of human-designed jailbreaks. In

research-papers

Prompt Engineering a Prompt Engineer

Original Paper: https://arxiv.org/abs/2311.05661 By: Qinyuan Ye, Maxamed Axmed, Reid Pryzant, Fereshte Khani Abstract: Prompt engineering is a challenging yet crucial task for optimizing the performance of large language models on customized tasks. It requires complex reasoning to examine the model's errors, hypothesize what

research-papers

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations

Original Paper: https://arxiv.org/abs/2402.12348 By: Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, Kaidi Xu Abstract: As Large Language Models (LLMs) are integrated into critical real-world applications, their strategic and logical reasoning abilities are increasingly crucial. This paper

research-papers

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information

Original Paper: https://arxiv.org/abs/2311.11509 By: Zhengmian Hu, Gang Wu, Saayan Mitra, Ruiyi Zhang, Tong Sun, Heng Huang, Viswanathan Swaminathan Abstract: In recent years, Large Language Models (LLM) have emerged as pivotal tools in various applications. However, these models are susceptible to adversarial prompt attacks, where attackers

research-papers

DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning

Original Paper: https://arxiv.org/abs/2309.05173 By: Zhengxiang Shi, Aldo Lipani Abstract: Prompt tuning (PT), where a small amount of trainable soft (continuous) prompt vectors is affixed to the input of language models (LM), has shown promising results across various tasks and models for parameter-efficient fine-tuning (PEFT). PT

research-papers

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

Original Paper: https://arxiv.org/abs/2402.11140 By: Sijia Chen, Baochun Li, Di Niu Abstract: The reasoning performance of Large Language Models (LLMs) on a wide range of problems critically relies on chain-of-thought prompting, which involves providing a few chain of thought demonstrations as exemplars in prompts. Recent work,

research-papers

LLM evaluation too expensive? Here's how we solve this.

According to the latest and greatest research, the SoTA eval techniques use LLMs as evaluators. It's a little meta, but it works better than anything else. But sometimes, they are too expensive to run in production. Here's how Athina solves this. How to run LLM-graded evals

research-papers

Retrieval-Augmented Thought Process as Sequential Decision Making

Original Paper: https://arxiv.org/abs/2402.07812 By: Thomas Pouplin, Hao Sun, Samuel Holt, Mihaela van der Schaar Abstract: Large Language Models (LLMs) have demonstrated their strong ability to assist people and show "sparks of intelligence". However, several open challenges hinder their wider application: such as concerns

research-papers

Large Language Models are Few-shot Generators: Proposing Hybrid Prompt Algorithm To Generate Webshell Escape Samples

Original Paper: https://arxiv.org/abs/2402.07408 By: Mingrui Ma, Lansheng Han, Chunjie Zhou Abstract: The frequent occurrence of cyber-attacks has made webshell attacks and defense gradually become a research hotspot in the field of network security. However, the lack of publicly available benchmark datasets and the over-reliance on

research-papers

Exploring EFL students' prompt engineering in human-AI story writing: an Activity Theory perspective

Original Paper: https://arxiv.org/abs/2306.01798 By: David James Woo, Kai Guo, Hengky Susanto Abstract: This study applies Activity Theory to investigate how English as a foreign language (EFL) students prompt generative artificial intelligence (AI) tools during short story writing. Sixty-seven Hong Kong secondary school students created generative-AI

research-papers

EntGPT: Linking Generative Large Language Models with Knowledge Bases

Original Paper: https://arxiv.org/abs/2402.06738 By: Yifan Ding, Amrit Poudel, Qingkai Zeng, Tim Weninger, Balaji Veeramani, Sanmitra Bhattacharya Abstract: The ability of Large Language Models (LLMs) to generate factually correct output remains relatively unexplored due to the lack of fact-checking and knowledge grounding during training and inference.