research-papers

research-papers

Beyond Preferences in AI Alignment

Original Paper: https://arxiv.org/abs/2408.16984 By: Tan Zhi-Xuan, Micah Carroll, Matija Franklin, Hal Ashton Abstract: The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences

research-papers

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework

Original Paper: https://arxiv.org/abs/2408.01262 By: Kunlun Zhu, Yifan Luo, Dingling Xu, Ruobing Wang, Shi Yu, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, Zhiyuan Liu, Maosong Sun Abstract: Retrieval-Augmented Generation (RAG) systems have demonstrated their advantages in alleviating the hallucination of Large Language Models (LLMs). Existing

research-papers

LLM Pruning and Distillation in Practice: The Minitron Approach

Original Paper: https://arxiv.org/pdf/2408.11796 By: Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov Abstract We present a comprehensive report on compressing the Llama 3.1 8B and Mistral NeMo 12B models to 4B and 8B

research-papers

Controllable Text Generation for Large Language Models: A Survey

Original Paper: https://arxiv.org/abs/2408.12599 By: Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, Zhiyu Li Abstract: In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world

research-papers

RAG-Fusion (Fusion Retrieval RAG)

Original Paper: https://arxiv.org/pdf/2402.03367 Code Sample: https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/fusion_retrieval.ipynb RAG-Fusion or Fusion-retrieval RAG, is an advanced technique that enhances the traditional Retrieval Augmented Generation (RAG) approach used in AI and natural language processing. This method

research-papers

Query Transformations: Rewriting, Step-back Prompting, and Sub-query Decomposition

Original Papers: * Query Rewriting: https://arxiv.org/pdf/2305.14283 * Step-back Prompting: https://arxiv.org/abs/2310.06117 * Sub-query Decomposition: https://arxiv.org/pdf/2404.00610 Code Sample: https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/query_transformations.ipynb Query transformations are advanced techniques used to enhance

research-papers

Re-ranking methods

Code Sample: https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/reranking.ipynb Reranking is a powerful technique used in Retrieval-Augmented Generation (RAG) systems to refine and improve the relevance of retrieved documents. Here's a detailed explanation of reranking methods in RAG systems, along with their

research-papers

Challenges and Responses in the Practice of Large Language Models

Original Paper: https://arxiv.org/abs/2408.09416 By: Hongyin Zhu Abstract: This paper carefully summarizes extensive and profound questions from all walks of life, focusing on the current high-profile AI field, covering multiple dimensions such as industry trends, academic research, technological innovation and business applications. This paper meticulously curates

research-papers

Enhancing Robustness in Large Language Models: Prompting for Mitigating the Impact of Irrelevant Information

Original Paper: https://arxiv.org/abs/2408.10615 By: Ming Jiang, Tingting Huang, Biao Guo, Yao Lu, Feng Zhang Abstract: In recent years, Large language models (LLMs) have garnered significant attention due to their superior performance in complex reasoning tasks. However, recent studies may diminish their reasoning capabilities markedly when

research-papers

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework

Original Paper: https://arxiv.org/abs/2408.01262 By: Kunlun Zhu, Yifan Luo, Dingling Xu, Ruobing Wang, Shi Yu, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, Zhiyuan Liu, Maosong Sun Abstract: Retrieval-Augmented Generation (RAG) systems have demonstrated their advantages in alleviating the hallucination of Large Language Models (LLMs). Existing

research-papers

RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation

Original Paper: https://arxiv.org/abs/2408.08067 By: Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Cheng Jiayang, Cunxiang Wang, Shichao Sun, Huanyu Li, Zizhao Zhang, Binjie Wang, Jiarong Jiang, Tong He, Zhiguo Wang, Pengfei Liu, Yue Zhang, Zheng Zhang Abstract Despite Retrieval-Augmented Generation (RAG)

research-papers

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

Original Paper: https://arxiv.org/abs/2408.02479 By: Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen Abstract:  One of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used