Skip to content
TheChatArchiveThe Chat Archive
Coherecommand-a-03-2025· CohereHugging Face LMArena Human Preference 140K

How can I connect Azure Document Intelligence to an Azure AI Foundry agent?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with MiniMaxminimax-m1’s answer.
  1. User

    How can I connect Azure Document Intelligence to an Azure AI Foundry agent?

  2. Cohere Assistant

    Connecting Azure Document Intelligence (formerly known as Azure Form Recognizer) to an Azure AI Foundry agent involves integrating the document processing capabilities of Document Intelligence with the workflow automation and AI orchestration capabilities of AI Foundry. Below is a step-by-step guide to achieve this integration:


    1. Understand the Components

    • Azure Document Intelligence: Used for extracting structured data from documents (e.g., forms, invoices, receipts).
    • Azure AI Foundry: A platform for building and deploying AI-driven workflows and agents.

    2. Set Up Azure Document Intelligence

    1. Create a Document Intelligence Resource:

      • Go to the Azure Portal.
      • Create a new Document Intelligence resource (or use an existing one).
      • Note down the endpoint and API key.
    2. Train a Custom Model (if needed):

      • If you’re processing specific document types, train a custom model using labeled data.
      • Alternatively, use prebuilt models for common document types (e.g., invoices, receipts).
    3. Test the API:

      • Use the Document Intelligence API to ensure it’s working as expected. You can use the REST API, SDK, or tools like Postman.

    3. Set Up Azure AI Foundry

    1. Create an AI Foundry Instance:

      • Provision an AI Foundry instance in the Azure Portal.
      • Set up the necessary compute resources and storage.
    2. Create an Agent:

      • Define an agent in AI Foundry that will handle document processing tasks.
      • Configure the agent to accept document inputs (e.g., via API, blob storage, or event triggers).

    4. Integrate Document Intelligence with AI Foundry

    Option 1: Use Azure Logic Apps as a Middleware

    1. Create a Logic App:

      • Set up a Logic App to act as a bridge between AI Foundry and Document Intelligence.
      • Use triggers like HTTP requests, Azure Blob Storage, or AI Foundry events.
    2. Call Document Intelligence API:

      • Add a step in the Logic App to call the Document Intelligence API using the HTTP action.
      • Pass the document content (e.g., a file URL or base64-encoded data) to the API.
    3. Process the Response:

      • Parse the response from Document Intelligence to extract structured data.
      • Pass the extracted data to the AI Foundry agent for further processing.
    4. Trigger AI Foundry Agent:

      • Use the AI Foundry API or SDK to invoke the agent with the processed data.

    Option 2: Direct Integration via Custom Code

    1. Develop a Custom Connector:

      • Write a custom connector or function (e.g., using Azure Functions) to call the Document Intelligence API.
      • Deploy the function to Azure and expose it as an API.
    2. Invoke the Connector from AI Foundry:

      • Configure the AI Foundry agent to call the custom connector API.
      • Pass the document data to the connector and receive the processed output.
    3. Handle Workflow Logic:

      • Use AI Foundry’s workflow orchestration capabilities to handle the end-to-end process, including error handling and retries.

    5. Configure Storage and Triggers

    • Input Documents:

      • Store input documents in Azure Blob Storage or another accessible location.
      • Use event-driven triggers (e.g., Azure Event Grid) to notify the AI Foundry agent when new documents arrive.
    • Output Data:

      • Store the structured output data in a database (e.g., Azure SQL Database) or another storage solution.

    6. Test and Deploy

    1. Test the Integration:

      • Upload sample documents and verify that the data is extracted correctly and processed by the AI Foundry agent.
    2. Monitor and Optimize:

      • Use Azure Monitor and AI Foundry’s logging capabilities to track performance and errors.
      • Optimize the workflow based on feedback and usage patterns.

    Example Workflow

    1. A document is uploaded to Azure Blob Storage.
    2. An Event Grid trigger notifies the AI Foundry agent.
    3. The agent calls a Logic App or custom connector.
    4. The connector invokes the Document Intelligence API to extract data.
    5. The extracted data is passed back to the AI Foundry agent for further processing (e.g., validation, enrichment).
    6. The final output is stored in a database or sent to another system.

    Tools and SDKs

    • Azure SDKs: Use Python, .NET, or other SDKs for easier integration.
    • Azure Logic Apps: For low-code/no-code integration.
    • Azure Functions: For custom code and serverless integration.

    By following these steps, you can effectively connect Azure Document Intelligence to an Azure AI Foundry agent, enabling automated document processing within your AI workflows.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.