Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

I want an LLM to extract knowledge insights from an existing conversation between a user…

  1. User

    I want an LLM to extract knowledge insights from an existing conversation between a user and its chat session with its psychologist/buddy. The idea is to summarize the most important ideas from the conversation so that later we can use them to provide more personalized and contextually relevant responses in future interactions. This is where the doc-LLM comes in.

    The doc-LLM, or document-focused Language Model, is a specialized AI system designed to analyze conversations and extract key information. Here's how it works:

    1. Input: The doc-LLM receives the entire conversation history between the user and the AI psychologist/buddy.

    2. Analysis: It carefully reads through the conversation, identifying important themes, emotions, personal details, and potential issues or goals mentioned by the user.

    3. Categorization: The doc-LLM organizes this information into predefined categories such as "Personal Relationships," "Emotional Patterns," "Goals and Aspirations," etc.

    4. Document Creation: For each piece of significant information, the doc-LLM creates or updates a "document" in our knowledge base. These documents are structured with keys (like a title or topic) and values (the actual information).

    5. Contextual Linking: The doc-LLM also identifies related persons for each piece of information, helping to build a network of relationships and contexts.

    6. Relevance Scoring: Although we've decided to drop explicit relevance scoring, the doc-LLM still prioritizes information based on its importance to understanding the user.

    7. Output: The result is a set of well-organized, concise documents that summarize the key insights from the conversation.

    These documents can then be used by our main system to provide more informed, empathetic, and contextually appropriate responses in future interactions. The doc-LLM helps bridge the gap between raw conversation data and actionable insights, allowing our AI psychologist/buddy to build a deeper understanding of the user over time.

    By periodically analyzing conversations with the doc-LLM, we can continuously update and refine our knowledge about the user, leading to more personalized and effective therapeutic interactions.


    However, in a more lower level, in the implementation, i was using structured outputs from the LLM which doesn't work, because i need a flexible number of context cues, and the current schema always outputs one per conversation. For that reason, I want to migrate to a function-calling based approach. I need you to suggest a function name, description, and parameters to make it as easy as possible for the doc-llm to understand the task and take notes. The doc-llm won't type the 'key' itself so you can ignore that attribute. here is the previous code for reference about the parameters and stuff.

    def extract_context(gemini: Gemini, messages):
        """
        Extract context from a conversation.
        """
        
        response_schema = content.Schema(
            type = content.Type.OBJECT,
            required = ["category", "subcategory", "content"],
            properties = {
                "category": content.Schema(
                    type = content.Type.STRING
                ),
                "subcategory": content.Schema(
                    type = content.Type.STRING
                ),
                "content": content.Schema(
                    type = content.Type.STRING
                ),
                "related_persons": content.Schema(
                    type = content.Type.ARRAY,
                    items = content.Schema(
                        type = content.Type.STRING
                    )
                )
            }
        )
        
        generation_config = {
            "temperature": 0.4,
            "top_p": 0.95,
            "top_k": 64,
            "max_output_tokens": 8192,
            "response_schema": response_schema,
            "response_mime_type": "application/json",
        }
        
        system_instruction = """
    Analyze the following conversation and create or update information based on the conversation provided.
    For each piece of significant information:
    1. Determine the most appropriate category from the list: [Personal Relationships, Professional Life, Health and Wellness, Hobbies and Interests, Goals and Aspirations, Emotional Patterns, Significant Life Events]
    2. Assign or create an appropriate subcategory. Existing subcategories: []
    3. Provide a concise content string that summarizes the information
    4. Identify any persons related to this information. Always specify some qualifier (e.g. "Janice (mother)", "Bob (long-time friend)"). If none, leave blank
    """
    
        response = gemini.chat(system_instruction=system_instruction,
                                        gen_config=generation_config,
                                        content=messages)
        
        try:
            context_updates = json.loads(response.text)
        except Exception as e:
            # TODO manage error
            raise Exception(f"Failed to extract context updates from Gemini response: {e}")
        
        return context_updates
    
  2. Assistant

    Certainly! To help you migrate to a function-calling approach with a flexible number of context cues, I'll provide a function definition that aligns with OpenAI's function-calling schema. This function will allow the doc-LLM to extract multiple pieces of significant information from the conversation and structure them appropriately.

    Here's the suggested function:

    {
      "name": "update_context_entries",
      "description": "Extract significant information from the conversation to update the user's context.",
      "parameters": {
        "type": "object",
        "properties": {
          "entries": {
            "type": "array",
            "description": "A list of context entries extracted from the conversation.",
            "items": {
              "type": "object",
              "properties": {
                "category": {
                  "type": "string",
                  "description": "The most appropriate category for the information. Choose from: [Personal Relationships, Professional Life, Health and Wellness, Hobbies and Interests, Goals and Aspirations, Emotional Patterns, Significant Life Events]."
                },
                "subcategory": {
                  "type": "string",
                  "description": "An appropriate subcategory for the information."
                },
                "content": {
                  "type": "string",
                  "description": "A concise summary of the significant information."
                },
                "related_persons": {
                  "type": "array",
                  "description": "List of related persons with qualifiers (e.g., 'Janice (mother)', 'Bob (long-time friend)'). Leave empty if none.",
                  "items": {
                    "type": "string"
                  }
                }
              },
              "required": ["category", "subcategory", "content"]
            }
          }
        },
        "required": ["entries"]
      }
    }
    

    Explanation:

    • Function Name: update_context_entries

      • This name clearly indicates that the function updates the context with multiple entries.
    • Description:

      • Provides a brief explanation of the function's purpose, which is to extract significant information from the conversation and update the user's context.
    • Parameters:

      • Type: An object with a single property entries.
      • entries:
        • Type: An array, allowing for a flexible number of context cues.
        • Description: Explains that it's a list of context entries extracted from the conversation.
        • Items:
          • Type: An object representing each context entry.
          • Properties:
            • category (string): The main category for the information.
            • subcategory (string): A more specific category.
            • content (string): A concise summary.
            • related_persons (array of strings): List of related persons with qualifiers.
          • Required: category, subcategory, and content are required for each entry.

    Usage in Your Code:

    You can integrate this function into your existing codebase by defining it according to the OpenAI function-calling conventions. Here's how you might adjust your extract_context function:

    def extract_context(gemini, messages):
        """
        Extract context entries from a conversation using function calling.
        """
    
        function_definitions = [
            {
                "name": "update_context_entries",
                "description": "Extract significant information from the conversation to update the user's context.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "entries": {
                            "type": "array",
                            "description": "A list of context entries extracted from the conversation.",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "category": {
                                        "type": "string",
                                        "description": "The most appropriate category for the information. Choose from: [Personal Relationships, Professional Life, Health and Wellness, Hobbies and Interests, Goals and Aspirations, Emotional Patterns, Significant Life Events]."
                                    },
                                    "subcategory": {
                                        "type": "string",
                                        "description": "An appropriate subcategory for the information."
                                    },
                                    "content": {
                                        "type": "string",
                                        "description": "A concise summary of the significant information."
                                    },
                                    "related_persons": {
                                        "type": "array",
                                        "description": "List of related persons with qualifiers (e.g., 'Janice (mother)', 'Bob (long-time friend)'). Leave empty if none.",
                                        "items": {
                                            "type": "string"
                                        }
                                    }
                                },
                                "required": ["category", "subcategory", "content"]
                            }
                        }
                    },
                    "required": ["entries"]
                }
            }
        ]
    
        system_instruction = """
    You are an AI language model tasked with analyzing conversations to extract key insights. Analyze the following conversation and for each piece of significant information:
    1. Determine the most appropriate category from the list: [Personal Relationships, Professional Life, Health and Wellness, Hobbies and Interests, Goals and Aspirations, Emotional Patterns, Significant Life Events].
    2. Assign or create an appropriate subcategory.
    3. Provide a concise content string that summarizes the information.
    4. Identify any related persons with qualifiers (e.g., "Janice (mother)", "Bob (long-time friend)"). If none, leave the list empty.
    
    Use the `update_context_entries` function to output the extracted information.
    """
    
        # Your chat function should accept the function definitions and handle function calling.
        response = gemini.chat(
            system_instruction=system_instruction,
            content=messages,
            functions=function_definitions,
            function_call={"name": "update_context_entries"}
        )
    
        # Extract the function arguments from the response
        if response.get("function_call", {}).get("arguments"):
            function_args = json.loads(response["function_call"]["arguments"])
            context_updates = function_args.get("entries", [])
        else:
            # Handle the case where no function call was made
            context_updates = []
    
        return context_updates
    

    Notes:

    • Function Definitions: The function_definitions variable contains the schema for your function, which is passed to the LLM.
    • System Instruction: Updated to guide the LLM to use the update_context_entries function.
    • Function Call: Specified in the gemini.chat request to prompt the LLM to use the function.

    Benefits:

    • Flexible Number of Context Cues: By allowing the function to accept an array of entries, the LLM can extract and return multiple pieces of significant information from the conversation.
    • Structured Output: Using function calling ensures that the data is returned in a structured format, making it easier to parse and use in your application.
    • Clear Guidance: The function parameters provide clear instructions to the LLM on how to structure the extracted information.

    Additional Recommendations:

    • Temperature Settings: You might want to experiment with the generation settings to balance creativity and adherence to instructions. For extraction tasks, a lower temperature (e.g., 0.2) can help maintain consistency.
    • Error Handling: Ensure your code gracefully handles cases where the LLM does not return the expected function calls or arguments.
    • Testing: Test the new function with different conversation samples to ensure it extracts the information correctly.

    Example Function Call Response:

    Here's how the LLM might respond using the update_context_entries function:

    {
      "function_call": {
        "name": "update_context_entries",
        "arguments": "{\n  \"entries\": [\n    {\n      \"category\": \"Personal Relationships\",\n      \"subcategory\": \"Family\",\n      \"content\": \"The user discussed feeling distant from their sister after a recent argument.\",\n      \"related_persons\": [\"Emily (sister)\"]\n    },\n    {\n      \"category\": \"Professional Life\",\n      \"subcategory\": \"Job Satisfaction\",\n      \"content\": \"The user is considering a career change due to dissatisfaction with their current role.\",\n      \"related_persons\": []\n    }\n  ]\n}"
      }
    }
    

    Parsing the Response:

    After receiving the response, you can parse the function_call.arguments to extract the entries list and update your knowledge base accordingly.


    By adopting this function-calling approach, you enable the doc-LLM to extract multiple pieces of information flexibly, improving the personalization and contextual relevance of future interactions.