RAG Fusion: Elevating the Power of Retrieval-Augmented Generation

Original Paper: https://arxiv.org/abs/2402.03367

Code Sample: https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/fusion_retrieval.ipynb

Fusion Retrieval in document search combines multiple retrieval techniques to improve the overall quality and relevance of search results.

The two main techniques used in this approach are vector-based similarity search and keyword-based BM25 retrieval. Let's explore each technique and their fusion in detail:

Vector-based Similarity Search

Vector-based similarity search, also known as dense retrieval, uses dense vector representations (embeddings) of documents and queries to find relevant matches.

Advantages:

• Captures semantic meaning and contextual relationships
• Handles synonyms and related concepts well
• Effective for understanding the overall intent of queries

Limitations:

• May miss exact keyword matches or rare terms
• Computationally intensive for large document collections
• Requires training on domain-specific data for best performance

Keyword-based BM25 Retrieval

BM25 (Best Matching 25) is a probabilistic retrieval model that ranks documents based on the appearance of query terms.

Advantages:

• Excellent at exact keyword matching
• Handles rare terms and proper nouns effectively
• Computationally efficient and easy to implement

Limitations:

• Lacks semantic understanding
• Struggles with synonyms and related concepts
• May miss relevant documents that use different terminology

Fusion Retrieval

Fusion retrieval combines the results of vector-based and keyword-based searches to leverage the strengths of both approaches.

Process:

1. Perform vector similarity search on document embeddings
2. Conduct BM25 keyword search on the same document set
3. Combine the results using a fusion method

Fusion Methods:

1. Linear Combination: A weighted sum of the vector similarity and BM25 scores[1][3].
Copy

H = (1-α) * K + α * V

Where:

1. H is the hybrid score
2. K is the keyword (BM25) score
3. V is the vector similarity score
4. α is a weighting parameter (0 ≤ α ≤ 1)
2. Reciprocal Rank Fusion (RRF): Combines rankings based on the reciprocal of each document's rank in both result sets[5][6].
Copy

RRF(d) = Σ 1 / (k + rank_i(d))

Where:

1. d is a document
2. k is a constant (often set to 60)
3. rank_i(d) is the rank of document d in the i-th result list

Advantages of Fusion Retrieval:

• Balances semantic understanding with exact matching
• Improves overall retrieval performance
• Handles a wider range of query types effectively

Limitations:

• Increased complexity in implementation and tuning
• May require additional computational resources
• Determining optimal fusion parameters can be challenging

Practical Implementation

To implement fusion retrieval:

1. Index documents using both vector embeddings and inverted indexes for BM25.
2. For each query:
• Generate a query embedding
• Perform vector similarity search
• Conduct BM25 keyword search
• Apply the chosen fusion method to combine results
3. Return the final ranked list of documents

Many modern search systems, including those used in Retrieval Augmented Generation (RAG) applications, employ hybrid search techniques to enhance performance[1][2].

By combining the strengths of different retrieval methods, fusion retrieval can significantly improve the accuracy and relevance of document search results across various domains and use cases.

This is an AI generated summary by Athina AI

Relevant Links:

1. https://superlinked.com/vectorhub/articles/optimizing-rag-with-hybrid-search-reranking
2. https://zilliz.com/learn/what-are-rerankers-enhance-information-retrieval
3. https://rockset.com/whitepapers/hybrid-search-architecture/
4. https://www.reddit.com/r/MachineLearning/comments/18bhlsj/d_you_do_not_need_a_vector_database/
5. https://community.pinecone.io/t/understanding-bm25-parameters-and-hybrid-search-logic-with-sparse-dense-vectors-in-pinecone/4906
6. https://docs.databricks.com/en/generative-ai/vector-search.html
7. https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/fusion_retrieval.ipynb
8. https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf