Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek-r1-0528· DeepSeekHugging Face LMArena Human Preference 140K

whats the current bets practice for rag

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-235b-a22b-no-thinking’s answer.
  1. User

    whats the current bets practice for rag

  2. DeepSeek Assistant

    Current Best Practices for RAG (Retrieval-Augmented Generation)

    RAG systems combine retrieval of relevant information with generative AI to produce accurate, context-aware responses. Here are the latest best practices (as of 2024):


    1. Optimized Retrieval

    • Hybrid Retrieval:
      • Combine dense vectors (e.g., embeddings from models like text-embedding-ada-002 or bge-large) with sparse retrieval (BM25/keyword-based) for better recall.
      • Why? Dense vectors capture semantic meaning; sparse methods handle exact keyword matches.
    • Re-Ranking:
      • Use cross-encoders (e.g., bge-reranker-large, Cohere Rerank) to refine top-k retrieved documents.
      • Tip: Retrieve 50-100 documents initially, then re-rank to 3-10 most relevant passages.
    • Small Chunks + Metadata:
      • Split documents into small chunks (256-512 tokens) for precision. Attach metadata (e.g., titles, dates) to maintain context.

    2. Advanced Embedding & Indexing

    • Embedding Models:
      • Prefer instructor-XL, bge-large, or OpenAI embeddings optimized for retrieval tasks. Fine-tune embeddings on domain data if possible.
    • Vector Databases:
      • Use Pinecone, Weaviate, or Chroma for low-latency search. FAISS (open-source) is ideal for on-prem deployments.
      • Optimization: Apply HNSW or IVF-PQ indexing for faster search in large datasets.

    3. Context-Aware Generation

    • Prompt Engineering:
      • Structure prompts to:
        • Clearly separate retrieved context from instructions.
        • Include fallback mechanisms (e.g., "If context is irrelevant, say 'I don't know'").
        • Example:
          Context: {retrieved_text}  
          Question: {query}  
          Answer based ONLY on context above:  
          
    • LLM Selection:
      • Use GPT-4-turbo, Claude 3, or open-source models (e.g., Llama 3, Mixtral) for balanced cost/performance.
    • Chunk Expansion:
      • Add surrounding sentences/chunks to retrieved text to avoid missing key details.

    4. Query Optimization

    • Query Expansion/Rewriting:
      • Generate multiple query variants using an LLM (e.g., "paraphrase this question in 3 ways").
      • Use HyDE (Hypothetical Document Embeddings): Generate a hypothetical answer first, then retrieve based on it.
    • Step-Back Prompting:
      • Start with high-level conceptual queries to retrieve foundational context before answering specifics.

    5. Evaluation & Iteration

    • Metrics:
      • Retrieval: Precision@k, Recall@k, MRR (Mean Reciprocal Rank).
      • Generation: ROUGE, BLEU, or LLM-based eval (e.g., GPT-4 as a judge for faithfulness).
    • A/B Testing:
      • Compare different embedding models, chunk sizes, or prompts via user feedback.
    • Hallucination Mitigation:
      • Use self-consistency (generate multiple answers and vote) or RAGAS framework for automated quality checks.

    6. Advanced Techniques

    • Recursive Retrieval:
      • Iteratively retrieve, generate, then retrieve again for complex multi-step queries.
    • Fine-Tuning:
      • Fine-tune the retriever or generator on domain data (e.g., LoRA adapters for efficiency).
    • Agents:
      • Use RAG within agent frameworks (e.g., LangChain, LlamaIndex) for dynamic tool/task routing.

    7. Security & Scalability

    • Access Control:
      • Implement document-level permissions (e.g., using metadata filters in vector DBs).
    • Caching:
      • Cache frequent queries/retrieved results to reduce latency and costs.
    • Cost Optimization:
      • Use smaller models for retrieval (e.g., bge-small) and reserve large LLMs for final generation.

    Tools & Frameworks

    Key Trends for 2024

    1. Smaller Specialized Embeddings (e.g., jina-embeddings).
    2. End-to-End RAG Fine-Tuning (e.g., RA-DIT, PROD).
    3. Multi-Modal RAG (retrieving images/tables alongside text).

    By integrating these practices, RAG systems achieve ~15-30% higher accuracy in benchmarks like HotpotQA and Natural Questions. Always validate against domain-specific data!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.