Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0314· OpenAIAi2 WildChat-4.8M (AI2)

Give complete solution with code for the below project stepwise in docker and docker…

  1. User

    Give complete solution with code for the below project stepwise in docker and docker compose (Automate everything) As a user, I should be able to access APIs associated with any model (Please tell me how it is done in solution):

    Project: AI Toolchain is a collection of tools for quickly building and deploying machine learning models for various use cases. Currently, the toolchain includes a text translation model, and more models may be added in the future. It abstracts the dirty details of how a model works similar to Huggingface and gives a clean API that you can orchestrate at aenter code here BFF level.

    Features to be implemented Abstract the layer of deployment for AI Tools. Anyone should be easily add a new model to the stack without thinking about deployments. We should be able to deploy AI Tools in such a way where each model (every model can be packaged as a container) should be independently scalable. As a user, I should be able to access APIs associated with any model.

    Product Set Up https://github.com/Samagra-Development/ai-tools#setup

    Github Information: ReadME: AI Toolchain AI Toolchain is a collection of tools for quickly building and deploying machine learning models for various use cases. Currently, the toolchain includes a text translation model, and more models may be added in the future.

    How to Run To deploy all models, simply execute the deploy.sh script located in the root folder. This script calls the deployment files of each model. Note that the toolchain may switch to using Docker in the future for deployment.

    To create a new model class, use the template_batch_model.py file as a starting point. Your new model class should implement the method mentioned in the template file.

    To create a new request class, use the template_model_request.py file as a starting point. This class is used to map the incoming request to the data needed by the model.

    To add your new model and request to the API, modify the repository dictionary in api.py.

    Repository The repository is structured as follows

    Setup To set up the AI Toolchain environment, follow these steps:

    python3 -m venv venv source venv/bin/activate pip install poetry poetry install quart --app api --debug run Poetry Fixes poetry lock --no-update Contributing Contributions to AI Toolchain are welcome! To contribute, please follow these guidelines:

    Fork the repository and create a new branch for your feature or bug fix. Write tests for your changes. Submit a pull request describing your changes and why they are needed. Thank you for considering contributing to AI Toolchain!

    Files: .github/workflows Lower case repo name for GH Packages 2 months ago benchmarks Fix add test 3 weeks ago flake8 Fix: module/folder names 2 months ago src Merge pull request #100 from rishav-eulb/rishav-eulb-patch-2 last week .flake8 Fixes: #29 2 months ago .gitignore Initial Commit 2 months ago .gitpod.Dockerfile Included installation of Git last week .gitpod.yml Corrected .gitpod.yml last week Dockerfile Added Gitpod last week README.md docs: readme updated last month api.py Moved restart and watch functions to separate file last week contribution.md Corrected Gitpod button last week deploy.sh deploy.sh_error last month docker-compose.yml Added Gitpod last week poetry.lock Feat: Reload debug server on changes to src folder last week prometheus.yml add prometheus.yml 2 weeks ago pyproject.toml Feat: Reload debug server on changes to src folder last week repository_data.json Feat: Azure translation 3 weeks ago sample.env Fix: Add sample env 2 months ago template_batch_model.py added repo structure and translation model 2 months ago template_model_request.py added repo structure and translation model 2 months ago test.py Fix add test 3 weeks ago watch_folder.py Moved resta

    Dockercompose file: version: '3'

    services: app: build: context: . dockerfile: Dockerfile ports: - "8000:8000" restart: always environment: - PYTHONUNBUFFERED=1 - PYTHONDONTWRITEBYTECODE=1 - OPENAI_API_KEY=OPENAIAPIKEY−AUTHHEADER={OPENAI_API_KEY} - AUTH_HEADER={AUTH_HEADER} - AUTH_HEADER_KEY=${AUTH_HEADER_KEY} prometheus: image: prom/prometheus ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml gitpod: image: gitpod/workspace-full ports: - "8888:8888" volumes: - ./workspace:/workspace

  2. Assistant

    To achieve the complete solution using Docker and Docker Compose, follow these steps:

    1. Modify the project's Dockerfile Add all necessary setup and installation commands in the Dockerfile to properly configure the environment for the AI Toolchain project.

    Here's a modified Dockerfile:

    FROM python:3.8-slim
    
    WORKDIR /app
    
    COPY . /app
    
    RUN pip install --upgrade pip && \
        pip install --no-cache-dir virtualenv && \
        python3 -m venv venv && \
        . venv/bin/activate && \
        pip install --no-cache-dir poetry && \
        poetry config virtualenvs.create false && \
        poetry install --no-dev --no-root
    
    ENV PYTHONUNBUFFERED=1
    ENV PYTHONDONTWRITEBYTECODE=1
    
    EXPOSE 8000
    
    CMD ["venv/bin/quart", "--app", "api", "--debug", "run", "--host", "0.0.0.0"]
    
    1. Create a docker-compose.yml file

    Use Docker Compose to manage multiple container configurations. In this case, we'll have containers for the app, prometheus, and gitpod.

    Here's the new docker-compose.yml file:

    version: '3'
    
    services:
      app:
        build: .
        ports:
          - "8000:8000"
        restart: always
        environment:
          - OPENAI_API_KEY=${OPENAI_API_KEY}
          - AUTH_HEADER=${AUTH_HEADER}
          - AUTH_HEADER_KEY=${AUTH_HEADER_KEY}
      prometheus:
        image: prom/prometheus
        ports:
          - "9090:9090"
        volumes:
          - ./prometheus.yml:/etc/prometheus/prometheus.yml
      gitpod:
        image: gitpod/workspace-full
        ports:
          - "8888:8888"
        volumes:
          - ./workspace:/workspace
    
    1. Add .env file for environment variables

    Create a .env file in the project root directory, and provide values for the required environment variables:

    OPENAI_API_KEY=<your_openai_api_key>
    AUTH_HEADER=<your_auth_header>
    AUTH_HEADER_KEY=<your_auth_header_key>
    

    Remember to add the .env file to the .gitignore file to avoid sharing sensitive information on public repositories.

    1. Run the services using Docker Compose

    Use the following command to run the services defined in the docker-compose.yml file:

    docker-compose up -d
    

    This command builds and starts the app, prometheus, and gitpod containers. You can now access the app at http://localhost:8000 and prometheus at http://localhost:9090.

    1. Add a new AI model to the stack

    To add a new model, follow these steps:

    • Create a new model class using the template_batch_model.py file as a starting point
    • Create a new request class using the template_model_request.py file as a starting point
    • Modify the repository dictionary in api.py to add the new model and request classes

    The Docker and Docker Compose configurations will ensure that any new models added to the stack are automatically included, without any need for manual deployment.

  3. User

    In above :

    1. Dockerfile:

    Use an official Python runtime as a parent image

    FROM python:3.9-slim

    Set the working directory to /app

    WORKDIR /app

    Copy the poetry.lock and pyproject.toml files to the working directory

    COPY poetry.lock pyproject.toml /app/

    Install dependencies using Poetry

    RUN pip install poetry &&
    poetry config virtualenvs.create false &&
    poetry install --no-dev --no-root

    Copy the rest of the application code to the working directory

    COPY . /app/

    Set the entrypoint for the container

    ENTRYPOINT ["hypercorn", "api", "-b", "0.0.0.0"]

    Expose additional ports for Gitpod

    EXPOSE 8888

    1. api.py: import importlib

    from quart import Quart, g, request, jsonify, abort, Response from markupsafe import escape import json import aiohttp import os from functools import wraps from dotenv import load_dotenv from quart_compress import Compress import time from watch_folder import watch_src_folder import asyncio

    load_dotenv()

    app = Quart(name) Compress(app)

    watch_dir = "src" debug_mode = app.debug

    async def create_client_session(): """ Create an aiohttp ClientSession. """ async with aiohttp.ClientSession() as session: return session

    extra_dirs = ['src'] extra_files = extra_dirs[:] for extra_dir in extra_dirs: for dirname, dirs, files in os.walk(extra_dir): for filename in files: filename = os.path.join(dirname, filename) if os.path.isfile(filename): extra_files.append(filename)

    app.config.update(extra_files=extra_files)

    with open('repository_data.json') as f: repository_data = json.load(f)

    AUTH_HEADER = os.getenv("AUTH_HEADER") AUTH_HEADER_KEY = os.getenv("AUTH_HEADER_KEY")

    def verify_auth_header(auth_header_key, expected_value): def decorator(f): @wraps(f) async def decorated_function(*args, **kwargs): auth_header = request.headers.get(auth_header_key) if not auth_header or auth_header != expected_value: print("Unauthorized access"); abort(401) # Unauthorized return await f(*args, **kwargs)

        return decorated_function
    
    return decorator
    

    @app.route("/") def welcome(): return "Welcome!"

    @app.route("/repository") def repository(): """ Returns the repository data, which contains the available models and their configurations""" return jsonify(repository_data)

    def json_to_object(request_class, json_str): """Converts a JSON string to an object of the given class at level 1.""" data = json.loads(json_str) return request_class(**data)

    def get_model_config(use_case, provider, mode): """ Returns the model config for the given use case, provider and mode """ use_case_data = repository_data.get('use_cases').get(use_case) if use_case_data is None: return f'{escape(use_case)} Use case is not available', 400

    provider_data = use_case_data.get(provider)
    if provider_data is None:
        return f'{escape(provider)} Provider is not available', 400
    
    mode = provider_data.get(mode)
    if mode is None:
        return f'{escape(mode)} Mode is not available', 400
    
    return mode, 200
    

    @app.route("/<use_case>//", methods=['POST']) @verify_auth_header(AUTH_HEADER_KEY, AUTH_HEADER) async def transformer(use_case, provider, mode): """ Returns the translation for the given tex; provider and mode are as mentioned in the repository""" start_time = time.time()

    model_config = get_model_config(use_case, provider, mode)
    if model_config[1] != 200:
        return model_config
    
    model_class_name = model_config[0].get('model_class')
    model_request_class_name = model_config[0].get('request_class')
    module = importlib.import_module("src" + "." + use_case + "." + provider + "." + mode)
    model = getattr(module, model_class_name)(app)
    model_request = getattr(module, model_request_class_name)
    request_class = json_to_object(model_request, json.dumps(await request.json))
    if model_config[0].get("__is_async"):
        response = await model.inference(request_class)
    else:
        response = model.inference(request_class)
    
    end_time = time.time()
    response_time_ms = int((end_time - start_time) * 1000)
    headers = {"ai-tools-response-time": str(response_time_ms)}
    return Response(response, headers=headers)
    

    @app.before_serving async def startup(): """ Startup function called before serving requests. """ app.client = await create_client_session() # monitor src if in debug mode if debug_mode: asyncio.get_event_loop().create_task(watch_src_folder(app, watch_dir))

    quart --app api --debug run

    hypercorn api -b 0.0.0.0:8000

    3)template_batch_model.py: class AI4BharatBatchModel(): def new(cls): if not hasattr(cls, 'instance'): cls.instance = super(AI4BharatBatchModel, cls).new(cls) cls.load_model(cls) return cls.instance

    def load_model(self):
        """ Loads the model. This method is called only once when the model is first loaded."""
        pass
    
    def inference(self, request):
        """ Performs inference on the given request. This method is called for every request."""
        pass
    

    4)template_model_request.py class BatchModelRequest(): def init(self): """ Initializes the request object with the given parameters""" pass

    def to_json(self):
        """ Returns the json representation of the object"""
        import json
        return json.dumps(self, default=lambda o: o.__dict__, 
                          sort_keys=True, indent=4)
    
  4. Assistant

    Here are the changes in the Dockerfile and Python scripts that you provided:

    1. Dockerfile:

    Keep the Dockerfile as provided in the first answer, which installs dependencies using poetry and sets the entrypoint using quart.

    FROM python:3.8-slim
    
    WORKDIR /app
    
    COPY . /app
    
    RUN pip install --upgrade pip && \
        pip install --no-cache-dir virtualenv && \
        python3 -m venv venv && \
        . venv/bin/activate && \
        pip install --no-cache-dir poetry && \
        poetry config virtualenvs.create false && \
        poetry install --no-dev --no-root
    
    ENV PYTHONUNBUFFERED=1
    ENV PYTHONDONTWRITEBYTECODE=1
    
    EXPOSE 8000
    
    CMD ["venv/bin/quart", "--app", "api", "--debug", "run", "--host", "0.0.0.0"]
    
    1. Modify environment variable usage:

    In api.py, use os.environ instead of os.getenv to avoid confusion about expected environment variable names.

    AUTH_HEADER = os.environ["AUTH_HEADER"]
    AUTH_HEADER_KEY = os.environ["AUTH_HEADER_KEY"]
    
    1. Other files:

    Since the code for the other files (template_batch_model.py, template_model_request.py) does not need any changes related to Docker or Docker Compose, you can keep them as they are. These files provide a template for adding new AI models and requests.

    Now, your project structure should look like this:

    project/
    ├── Dockerfile
    ├── docker-compose.yml
    ├── .env
    ├── .gitignore
    ├── api.py
    ├── template_batch_model.py
    ├── template_model_request.py
    ├── repository_data.json
    └── src/
        └── your_model_package/
            ├── your_model.py
            └── your_model_request.py
    

    Follow these steps to run your project:

    1. Build and run your Docker containers using Docker Compose:
    docker-compose up -d
    
    1. Access your API using the given port (for example, http://localhost:8000) and call the necessary routes as defined in your api.py.

    Remember to modify the repository_data.json file whenever you add new AI models to your project, so the API can properly access and use them.