Skip to content
TheChatArchiveThe Chat Archive
Googlegemma-3-27b-it· GoogleHugging Face LMArena Human Preference 140K

you are a prompt engineer and you have to design a prompt for a llm model that user will…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with DeepSeekdeepseek-r1-0528’s answer.
  1. User

    you are a prompt engineer and you have to design a prompt for a llm model that user will give it query and it have to guess like it is relevant to which of the following field finance,educational,traval for eg something like this- val selectionPrompt = """Your task is to identify the most relevant knowledge base for the given user query. The available knowledge bases are: "Finance", "Educational", "Travel".

    Based on the user's query below, choose EXACTLY ONE of these knowledge base names. Your entire response MUST be ONLY one of these three words:

    • Finance
    • Educational
    • Travel

    Do NOT add any other text, explanation, or punctuation.

    User Query: "$userQuery"

    Most Relevant Knowledge Base (respond with one word only):""""

  2. Google Assistant

    Okay, that's a good starting point! Here are a few iterations of the prompt, building on your foundation, with explanations of the changes and why they might improve performance. I'll provide a few options, ranging from more concise to more robust. I'll also include a "Rationale" section explaining the thinking behind each change.

    Option 1: Concise & Direct (Similar to yours, slightly refined)

    val selectionPrompt = """You are a knowledge base router.  Your task is to categorize user queries.
    
    Available categories: Finance, Educational, Travel.
    
    For the following query, respond with *only* the single most relevant category.  Do not include any explanation.
    
    User Query: "{userQuery}"
    
    Category:"""
    

    Rationale:

    • "You are a knowledge base router": Gives the LLM a clear role. This can improve consistency.
    • "categorize user queries": More active and direct instruction.
    • "Available categories": Clearer than "knowledge bases".
    • "only the single most relevant category": Emphasizes the strict one-word response requirement. The asterisk adds a bit of visual weight.
    • "Do not include any explanation": Reinforces the constraint.
    • "Category:": Provides a clear cue for the LLM to complete the response. This can help with formatting.
    • Using f-strings: More modern and readable string formatting.

    Option 2: More Robust with Examples (Few-Shot Learning)

    val selectionPrompt = """You are a knowledge base router.  Your task is to categorize user queries into one of the following categories: Finance, Educational, Travel.
    
    Here are some examples:
    
    User Query: "What is the current interest rate on a mortgage?"
    Category: Finance
    
    User Query: "How do I solve quadratic equations?"
    Category: Educational
    
    User Query: "Best hotels in Paris with a view of the Eiffel Tower?"
    Category: Travel
    
    User Query: "What are the benefits of a Roth IRA?"
    Category: Finance
    
    User Query: "Explain the theory of relativity."
    Category: Educational
    
    User Query: "Cheap flights to Bangkok."
    Category: Travel
    
    Now, categorize the following user query:
    
    User Query: "{userQuery}"
    
    Category:"""
    

    Rationale:

    • Few-Shot Learning: Providing examples dramatically improves the LLM's ability to understand the task and generalize to new queries. The examples show the style of response expected.
    • Diverse Examples: The examples cover a range of queries within each category.
    • Clear Separation: The examples are clearly separated from the final query.
    • "Now, categorize...": Signals the transition to the actual task.

    Option 3: With Explicit Negative Constraints (For stubborn LLMs)

    val selectionPrompt = """You are a knowledge base router.  Your task is to categorize user queries.
    
    Available categories: Finance, Educational, Travel.
    
    For the following query, respond with *only* one of the categories listed above.  Do *not* respond with anything else.  Specifically, do *not* include:
    
    *   Any explanations
    *   Any punctuation
    *   Any introductory phrases (e.g., "The category is...")
    *   Any text other than the category name.
    
    User Query: "{userQuery}"
    
    Category:"""
    

    Rationale:

    • Explicit Negative Constraints: Some LLMs are prone to adding extra text. This version explicitly lists what not to do. The use of "Do not" and bullet points emphasizes these constraints.
    • Reinforces Single-Word Response: Repeats the requirement for a single category name.

    Option 4: Using a JSON format (Good for programmatic parsing)

    val selectionPrompt = """You are a knowledge base router. Your task is to categorize user queries.
    
    Available categories: Finance, Educational, Travel.
    
    For the following query, respond with a JSON object containing the 'category' key.  The value of the 'category' key should be *one* of the available categories.  Do not include any other keys or values.
    
    User Query: "{userQuery}"
    
    Response:"""
    

    Rationale:

    • JSON Output: This forces the LLM to output a structured JSON object. This is much easier to parse programmatically than free-form text. You'd then parse the JSON to get the category.
    • Clear Instructions: Specifies the exact JSON format required.

    Important Considerations & Testing:

    • LLM Specifics: The best prompt will depend on the specific LLM you are using (e.g., GPT-3.5, GPT-4, Llama 2, Gemini). You'll need to experiment.
    • Temperature: Lower temperature settings (e.g., 0.0 to 0.2) will make the LLM more deterministic and less likely to deviate from the instructions.
    • Testing: Thoroughly test the prompt with a wide variety of user queries to ensure it consistently produces the correct results. Create a test set with known correct answers.
    • Error Handling: If you're using the JSON output, include error handling in your code to gracefully handle cases where the LLM doesn't produce valid JSON.
    • Prompt Length: Keep the prompt as concise as possible while still providing clear instructions. Longer prompts can be more expensive and may not always improve performance.

    To help me refine the prompt further, could you tell me:

    • Which LLM are you using?
    • What kind of user queries are you expecting? (e.g., short questions, long paragraphs, specific jargon)
    • How will you be using the output? (e.g., programmatically, manually)
    • Are you experiencing any specific issues with the current prompt? (e.g., the LLM is adding extra text, misclassifying queries)

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.