Give complete solution with code for the below project stepwise in docker and docker…
- User
Give complete solution with code for the below project stepwise in docker and docker compose (Automate everything) As a user, I should be able to access APIs associated with any model (Please tell me how it is done in solution):
Project: AI Toolchain is a collection of tools for quickly building and deploying machine learning models for various use cases. Currently, the toolchain includes a text translation model, and more models may be added in the future. It abstracts the dirty details of how a model works similar to Huggingface and gives a clean API that you can orchestrate at aenter code here BFF level.
Features to be implemented Abstract the layer of deployment for AI Tools. Anyone should be easily add a new model to the stack without thinking about deployments. We should be able to deploy AI Tools in such a way where each model (every model can be packaged as a container) should be independently scalable. As a user, I should be able to access APIs associated with any model.
Product Set Up https://github.com/Samagra-Development/ai-tools#setup
Github Information: ReadME: AI Toolchain AI Toolchain is a collection of tools for quickly building and deploying machine learning models for various use cases. Currently, the toolchain includes a text translation model, and more models may be added in the future.
How to Run To deploy all models, simply execute the deploy.sh script located in the root folder. This script calls the deployment files of each model. Note that the toolchain may switch to using Docker in the future for deployment.
To create a new model class, use the template_batch_model.py file as a starting point. Your new model class should implement the method mentioned in the template file.
To create a new request class, use the template_model_request.py file as a starting point. This class is used to map the incoming request to the data needed by the model.
To add your new model and request to the API, modify the repository dictionary in api.py.
Repository The repository is structured as follows
Setup To set up the AI Toolchain environment, follow these steps:
python3 -m venv venv source venv/bin/activate pip install poetry poetry install quart --app api --debug run Poetry Fixes poetry lock --no-update Contributing Contributions to AI Toolchain are welcome! To contribute, please follow these guidelines:
Fork the repository and create a new branch for your feature or bug fix. Write tests for your changes. Submit a pull request describing your changes and why they are needed. Thank you for considering contributing to AI Toolchain!
Files: .github/workflows Lower case repo name for GH Packages 2 months ago benchmarks Fix add test 3 weeks ago flake8 Fix: module/folder names 2 months ago src Merge pull request #100 from rishav-eulb/rishav-eulb-patch-2 last week .flake8 Fixes: #29 2 months ago .gitignore Initial Commit 2 months ago .gitpod.Dockerfile Included installation of Git last week .gitpod.yml Corrected .gitpod.yml last week Dockerfile Added Gitpod last week README.md docs: readme updated last month api.py Moved restart and watch functions to separate file last week contribution.md Corrected Gitpod button last week deploy.sh deploy.sh_error last month docker-compose.yml Added Gitpod last week poetry.lock Feat: Reload debug server on changes to src folder last week prometheus.yml add prometheus.yml 2 weeks ago pyproject.toml Feat: Reload debug server on changes to src folder last week repository_data.json Feat: Azure translation 3 weeks ago sample.env Fix: Add sample env 2 months ago template_batch_model.py added repo structure and translation model 2 months ago template_model_request.py added repo structure and translation model 2 months ago test.py Fix add test 3 weeks ago watch_folder.py Moved resta
Dockercompose file: version: '3'
services: app: build: context: . dockerfile: Dockerfile ports: - "8000:8000" restart: always environment: - PYTHONUNBUFFERED=1 - PYTHONDONTWRITEBYTECODE=1 - OPENAI_API_KEY={AUTH_HEADER} - AUTH_HEADER_KEY=${AUTH_HEADER_KEY} prometheus: image: prom/prometheus ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml gitpod: image: gitpod/workspace-full ports: - "8888:8888" volumes: - ./workspace:/workspace
- Assistant
To achieve the complete solution using Docker and Docker Compose, follow these steps:
- Modify the project's Dockerfile Add all necessary setup and installation commands in the Dockerfile to properly configure the environment for the AI Toolchain project.
Here's a modified Dockerfile:
FROM python:3.8-slim WORKDIR /app COPY . /app RUN pip install --upgrade pip && \ pip install --no-cache-dir virtualenv && \ python3 -m venv venv && \ . venv/bin/activate && \ pip install --no-cache-dir poetry && \ poetry config virtualenvs.create false && \ poetry install --no-dev --no-root ENV PYTHONUNBUFFERED=1 ENV PYTHONDONTWRITEBYTECODE=1 EXPOSE 8000 CMD ["venv/bin/quart", "--app", "api", "--debug", "run", "--host", "0.0.0.0"]- Create a docker-compose.yml file
Use Docker Compose to manage multiple container configurations. In this case, we'll have containers for the app, prometheus, and gitpod.
Here's the new docker-compose.yml file:
version: '3' services: app: build: . ports: - "8000:8000" restart: always environment: - OPENAI_API_KEY=${OPENAI_API_KEY} - AUTH_HEADER=${AUTH_HEADER} - AUTH_HEADER_KEY=${AUTH_HEADER_KEY} prometheus: image: prom/prometheus ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml gitpod: image: gitpod/workspace-full ports: - "8888:8888" volumes: - ./workspace:/workspace- Add .env file for environment variables
Create a .env file in the project root directory, and provide values for the required environment variables:
OPENAI_API_KEY=<your_openai_api_key> AUTH_HEADER=<your_auth_header> AUTH_HEADER_KEY=<your_auth_header_key>Remember to add the .env file to the .gitignore file to avoid sharing sensitive information on public repositories.
- Run the services using Docker Compose
Use the following command to run the services defined in the docker-compose.yml file:
docker-compose up -dThis command builds and starts the app, prometheus, and gitpod containers. You can now access the app at http://localhost:8000 and prometheus at http://localhost:9090.
- Add a new AI model to the stack
To add a new model, follow these steps:
- Create a new model class using the template_batch_model.py file as a starting point
- Create a new request class using the template_model_request.py file as a starting point
- Modify the repository dictionary in api.py to add the new model and request classes
The Docker and Docker Compose configurations will ensure that any new models added to the stack are automatically included, without any need for manual deployment.
- User
In above :
- Dockerfile:
Use an official Python runtime as a parent image
FROM python:3.9-slim
Set the working directory to /app
WORKDIR /app
Copy the poetry.lock and pyproject.toml files to the working directory
COPY poetry.lock pyproject.toml /app/
Install dependencies using Poetry
RUN pip install poetry &&
poetry config virtualenvs.create false &&
poetry install --no-dev --no-rootCopy the rest of the application code to the working directory
COPY . /app/
Set the entrypoint for the container
ENTRYPOINT ["hypercorn", "api", "-b", "0.0.0.0"]
Expose additional ports for Gitpod
EXPOSE 8888
- api.py: import importlib
from quart import Quart, g, request, jsonify, abort, Response from markupsafe import escape import json import aiohttp import os from functools import wraps from dotenv import load_dotenv from quart_compress import Compress import time from watch_folder import watch_src_folder import asyncio
load_dotenv()
app = Quart(name) Compress(app)
watch_dir = "src" debug_mode = app.debug
async def create_client_session(): """ Create an aiohttp ClientSession. """ async with aiohttp.ClientSession() as session: return session
extra_dirs = ['src'] extra_files = extra_dirs[:] for extra_dir in extra_dirs: for dirname, dirs, files in os.walk(extra_dir): for filename in files: filename = os.path.join(dirname, filename) if os.path.isfile(filename): extra_files.append(filename)
app.config.update(extra_files=extra_files)
with open('repository_data.json') as f: repository_data = json.load(f)
AUTH_HEADER = os.getenv("AUTH_HEADER") AUTH_HEADER_KEY = os.getenv("AUTH_HEADER_KEY")
def verify_auth_header(auth_header_key, expected_value): def decorator(f): @wraps(f) async def decorated_function(*args, **kwargs): auth_header = request.headers.get(auth_header_key) if not auth_header or auth_header != expected_value: print("Unauthorized access"); abort(401) # Unauthorized return await f(*args, **kwargs)
return decorated_function return decorator@app.route("/") def welcome(): return "Welcome!"
@app.route("/repository") def repository(): """ Returns the repository data, which contains the available models and their configurations""" return jsonify(repository_data)
def json_to_object(request_class, json_str): """Converts a JSON string to an object of the given class at level 1.""" data = json.loads(json_str) return request_class(**data)
def get_model_config(use_case, provider, mode): """ Returns the model config for the given use case, provider and mode """ use_case_data = repository_data.get('use_cases').get(use_case) if use_case_data is None: return f'{escape(use_case)} Use case is not available', 400
provider_data = use_case_data.get(provider) if provider_data is None: return f'{escape(provider)} Provider is not available', 400 mode = provider_data.get(mode) if mode is None: return f'{escape(mode)} Mode is not available', 400 return mode, 200@app.route("/<use_case>//", methods=['POST']) @verify_auth_header(AUTH_HEADER_KEY, AUTH_HEADER) async def transformer(use_case, provider, mode): """ Returns the translation for the given tex; provider and mode are as mentioned in the repository""" start_time = time.time()
model_config = get_model_config(use_case, provider, mode) if model_config[1] != 200: return model_config model_class_name = model_config[0].get('model_class') model_request_class_name = model_config[0].get('request_class') module = importlib.import_module("src" + "." + use_case + "." + provider + "." + mode) model = getattr(module, model_class_name)(app) model_request = getattr(module, model_request_class_name) request_class = json_to_object(model_request, json.dumps(await request.json)) if model_config[0].get("__is_async"): response = await model.inference(request_class) else: response = model.inference(request_class) end_time = time.time() response_time_ms = int((end_time - start_time) * 1000) headers = {"ai-tools-response-time": str(response_time_ms)} return Response(response, headers=headers)@app.before_serving async def startup(): """ Startup function called before serving requests. """ app.client = await create_client_session() # monitor src if in debug mode if debug_mode: asyncio.get_event_loop().create_task(watch_src_folder(app, watch_dir))
quart --app api --debug run
hypercorn api -b 0.0.0.0:8000
3)template_batch_model.py: class AI4BharatBatchModel(): def new(cls): if not hasattr(cls, 'instance'): cls.instance = super(AI4BharatBatchModel, cls).new(cls) cls.load_model(cls) return cls.instance
def load_model(self): """ Loads the model. This method is called only once when the model is first loaded.""" pass def inference(self, request): """ Performs inference on the given request. This method is called for every request.""" pass4)template_model_request.py class BatchModelRequest(): def init(self): """ Initializes the request object with the given parameters""" pass
def to_json(self): """ Returns the json representation of the object""" import json return json.dumps(self, default=lambda o: o.__dict__, sort_keys=True, indent=4) - Assistant
Here are the changes in the Dockerfile and Python scripts that you provided:
- Dockerfile:
Keep the Dockerfile as provided in the first answer, which installs dependencies using poetry and sets the entrypoint using quart.
FROM python:3.8-slim WORKDIR /app COPY . /app RUN pip install --upgrade pip && \ pip install --no-cache-dir virtualenv && \ python3 -m venv venv && \ . venv/bin/activate && \ pip install --no-cache-dir poetry && \ poetry config virtualenvs.create false && \ poetry install --no-dev --no-root ENV PYTHONUNBUFFERED=1 ENV PYTHONDONTWRITEBYTECODE=1 EXPOSE 8000 CMD ["venv/bin/quart", "--app", "api", "--debug", "run", "--host", "0.0.0.0"]- Modify environment variable usage:
In api.py, use
os.environinstead ofos.getenvto avoid confusion about expected environment variable names.AUTH_HEADER = os.environ["AUTH_HEADER"] AUTH_HEADER_KEY = os.environ["AUTH_HEADER_KEY"]- Other files:
Since the code for the other files (template_batch_model.py, template_model_request.py) does not need any changes related to Docker or Docker Compose, you can keep them as they are. These files provide a template for adding new AI models and requests.
Now, your project structure should look like this:
project/ ├── Dockerfile ├── docker-compose.yml ├── .env ├── .gitignore ├── api.py ├── template_batch_model.py ├── template_model_request.py ├── repository_data.json └── src/ └── your_model_package/ ├── your_model.py └── your_model_request.pyFollow these steps to run your project:
- Build and run your Docker containers using Docker Compose:
docker-compose up -d- Access your API using the given port (for example, http://localhost:8000) and call the necessary routes as defined in your api.py.
Remember to modify the repository_data.json file whenever you add new AI models to your project, so the API can properly access and use them.