research-papers

Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models

Original Paper: https://arxiv.org/abs/2307.08303 By: Zhiyuan Peng, Xuyang Wu, Qifan Wang, Yi Fang Abstract: Dense retrieval (DR) converts queries and documents into dense embeddings and measures the similarity between queries and documents in vector space. One of the challenges in DR is the lack of domain-specific

research-papers

PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers

Original Paper: https://arxiv.org/abs/2406.12430 By: Myeonghwa Lee, Seonho An, Min-Soo Kim Abstract: In this paper, we conduct a study to utilize LLMs as a solution for decision making that requires complex data analysis. We define Decision QA as the task of answering the best decision, dbest,

research-papers

Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models

Original Paper: https://arxiv.org/abs/2307.08303 By: Zhiyuan Peng, Xuyang Wu, Qifan Wang, Yi Fang Abstract: Dense retrieval (DR) converts queries and documents into dense embeddings and measures the similarity between queries and documents in vector space. One of the challenges in DR is the lack of domain-specific

research-papers

Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching

Original Paper: https://arxiv.org/abs/2406.06326 By: Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Yipeng Zhang, Haitao Mi, Helen Meng Abstract: Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs

research-papers

On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey

Original Paper: https://arxiv.org/abs/2406.15126 By: Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, Haobo Wang Abstract: Within the evolving landscape of deep learning, the dilemma of data quantity and quality has been a long-standing problem. The recent advent of Large Language Models

research-papers

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

Original Paper: https://arxiv.org/abs/2406.10209 By: Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, Tom Goldstein Abstract: Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization,

research-papers

Discovering Preference Optimization Algorithms with and for Large Language Models

Original Paper: https://arxiv.org/abs/2406.08414 By: Chris Lu, Samuel Holt, Claudio Fanconi, Alex J. Chan, Jakob Foerster, Mihaela van der Schaar, Robert Tjarko Lange Abstract: Offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. Typically, preference optimization

research-papers

Extreme Compression of Large Language Models via Additive Quantization

Original Paper: https://arxiv.org/abs/2401.06118v3 By: Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, Dan Alistarh Abstract: The emergence of accurate open large language models (LLMs) has led to a race towards performant quantization techniques which can enable their execution on end-user devices. In this

research-papers

SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals

Original Paper: https://arxiv.org/abs/2406.04784 By: Ruihan Yang, Jiangjie Chen, Yikai Zhang, Siyu Yuan, Aili Chen, Kyle Richardson, Yanghua Xiao, Deqing Yang Abstract: Language agents powered by large language models (LLMs) are increasingly valuable as decision-making tools in domains such as gaming and programming. However, these agents

research-papers

Mixture-of-Agents Enhances Large Language Model Capabilities

Original Paper: https://arxiv.org/abs/2406.04692 By: Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou Abstract: Recent advances in large language models (LLMs) demonstrate substantial capabilities in natural language understanding and generation tasks. With the growing number of LLMs, how to harness the collective expertise of

research-papers

MedPromptExtract (Medical Data Extraction Tool): Anonymization and Hi-fidelity Automated data extraction using NLP and prompt engineering

Original Paper: https://arxiv.org/abs/2405.02664 By: Roomani Srivastava, Suraj Prasad, Lipika Bhat, Sarvesh Deshpande, Barnali Das, Kshitij Jadhav Abstract: A major roadblock in the seamless digitization of medical records remains the lack of interoperability of existing records. Extracting relevant medical information required for further treatment planning or

research-papers

RAFT: Adapting Language Model to Domain Specific RAG

Original Paper: https://arxiv.org/abs/2403.10131 By: Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, Joseph E. Gonzalez Abstract: Pretraining Large Language Models (LLMs) on large corpora of textual data is now a standard paradigm. When using these LLMs for many downstream applications,