Skip to content
TheChatArchiveThe Chat Archive
Qwenqwen3-235b-a22b-no-thinking· Alibaba (Qwen)Hugging Face LMArena Human Preference 140K

whats the current bets practice for rag

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with DeepSeekdeepseek-r1-0528’s answer.
  1. User

    whats the current bets practice for rag

  2. Qwen Assistant

    As of 2024, Retrieval-Augmented Generation (RAG) has become a widely adopted practice for enhancing the capabilities of large language models (LLMs) by grounding their responses in external knowledge sources. Below is a summary of current best practices for building and deploying RAG systems:


    🔍 1. Data Preparation & Ingestion

    ✅ Best Practices:

    • Chunking Strategy:

      • Use semantic chunking or recursive splitting rather than fixed-size chunking.
      • Consider using libraries like LangChain, LlamaIndex, or custom logic to split documents into meaningful segments.
    • Metadata Enrichment:

      • Add metadata (e.g., source, date, document type) to chunks for filtering and provenance tracking.
    • Embedding Models:

      • Use high-quality, domain-specific embedding models (e.g., BAAI/bge-large-en-v1.5, sentence-transformers).
      • Normalize embeddings before storage for better cosine similarity performance.
    • Vector Stores:

      • Store embeddings in efficient vector databases such as:
        • Pinecone, Weaviate, FAISS, Chroma, Qdrant, or Elasticsearch (for hybrid search).
      • Choose based on scalability, speed, and hybrid search needs.

    🧠 2. Retrieval Process

    ✅ Best Practices:

    • Hybrid Search:

      • Combine dense retrieval (vector search) with sparse keyword search (e.g., BM25) for better recall and precision.
    • Re-ranking:

      • Use re-ranking models like Cohere’s rerank, Sentence-BERT-based rankers, or cross-encoders to improve relevance after initial retrieval.
    • Search Depth:

      • Retrieve more documents than needed (e.g., top 20), then filter down during re-ranking or generation.
    • Filtering:

      • Filter retrieved documents by metadata (e.g., time range, source, domain) to ensure relevance and compliance.

    🤖 3. Prompt Engineering & Generation

    ✅ Best Practices:

    • Prompt Design:

      • Construct prompts that clearly instruct the LLM to use only the provided context.
      • Include instructions for citing sources and handling unknown information.
    • Guardrails:

      • Prevent hallucinations by including directives like:

        "Answer based only on the provided context. If the answer isn't in the context, say 'I don’t have enough information.'"

    • Chain-of-Thought (CoT) Prompting:

      • Encourage step-by-step reasoning from the context to improve interpretability and accuracy.
    • Context Length Management:

      • Truncate or summarize long contexts to fit within model token limits while preserving key info.

    🛠️ 4. Evaluation & Monitoring

    ✅ Best Practices:

    • Evaluation Metrics:

      • Retrieval: Precision@K, Recall@K, MRR
      • Generation: BLEU, ROUGE, METEOR, BERTScore, or human evaluation
      • End-to-end: QA Accuracy, Faithfulness, Relevance
    • Logging & Feedback Loop:

      • Log queries, retrieved docs, and responses for auditing and improvement.
      • Implement user feedback loops to refine retrieval and ranking.
    • Monitoring:

      • Track drift in data, retrieval quality, and response correctness over time.

    📦 5. Tooling & Frameworks

    ✅ Popular Tools:

    CategoryTools
    FrameworksLangChain, LlamaIndex, Haystack
    EmbeddingsSentence Transformers, HuggingFace, OpenAI, Cohere
    Vector DBsPinecone, Weaviate, FAISS, Chroma, Qdrant
    MonitoringArize, WhyLogs, Prometheus + Grafana

    🧪 6. Advanced Techniques (Optional but Powerful)

    • Self-RAG: Use the LLM to evaluate its own retrieved context and decide whether to continue searching or generate.
    • Adaptive Retrieval: Dynamically adjust number of documents retrieved based on query complexity.
    • Fine-tuned Models: Fine-tune retrievers or rerankers on domain-specific data for better performance.
    • Multi-hop RAG: Chain multiple retrieval steps to answer complex questions.

    🚫 Common Pitfalls to Avoid

    • Overloading prompts with irrelevant context.
    • Using outdated or low-quality documents.
    • Not evaluating both retrieval and generation components separately.
    • Ignoring latency in production deployments.
    • Failing to cite sources or provide traceability.

    🧩 Example Architecture

    User Query → Retriever → Re-ranker → LLM Generator → Final Answer
                ↖ Context ↖ Retrieved Docs
    

    ✅ Summary: RAG Best Practices Checklist

    StepPractice
    Data PrepClean, chunk, enrich, embed
    StorageUse vector DB with metadata
    RetrievalHybrid search + re-ranking
    PromptingClear instructions, guardrails
    EvalMeasure retrieval + generation
    OpsMonitor, log, update regularly

    Would you like a code example using LangChain or LlamaIndex? Or are you looking to implement RAG in a specific domain (like healthcare, legal, finance)?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.