You're given an existing project you need to analyze and continue developing. To do…
- User
You're given an existing project you need to analyze and continue developing. To do this, you'll need to determine the project architecture, technologies used (platform, libraries, etc) and reverse-engineer the technical and functional spec.
Here is the list of all the files in the project:
api.py- This file defines a FastAPI-based web service that provides an endpoint for handling chat completions using an external AI service (presumably OpenAI). The service is designed to stream responses back to the client in real-time using Server-Sent Events (SSE). The file imports necessary modules from FastAPI, Pydantic, and an external OpenAI client library. The main functionality revolves around the '/v1/chat/completions' POST endpoint, which accepts a request body containing a model name, a list of messages, and an optional stream flag. The request body is validated using Pydantic's BaseModel (ChatBody class). The endpoint returns a streaming response using FastAPI's StreamingResponse, which streams data from the external AI service. The file also configures CORS middleware to allow cross-origin requests from any domain. Additionally, the file includes a main block to run the FastAPI app using Uvicorn when executed directly.providers.py- This file defines a FastAPI application that interacts with an external API to generate chat completions using different models (e.g., 'gpt-4o', 'gpt-4', 'gpt-3.5-turbo'). The file imports necessary libraries for asynchronous HTTP requests (aiohttp), data validation (pydantic), and FastAPI for building the web service. The main functionality includes defining data models for chat messages and requests, making asynchronous HTTP POST requests to an external API, and handling the responses. The 'make_request' function sends a prompt to the external API and processes the response, while 'extract_messages' extracts the message content from the API's JSON response. The file also defines HTTP headers for proxied requests, although they are not directly used in the 'make_request' function.test.py- The file implements a series of mathematical functions designed to perform basic arithmetic operations, such as addition, subtraction, multiplication, and division. Additionally, it contains utility functions for more complex calculations, including exponentiation and square roots. The main functionality is encapsulated in classes that ensure organized and modular code, making it easy to extend and maintain.README.md- Empty file
Here's the full content of interesting files that may help you to determine the specification:
api.py:from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import StreamingResponse from pydantic import BaseModel, Field from typing import List import json import asyncio import uuid from providers import make_request, make_toolbaz_request app = FastAPI() app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) class Message(BaseModel): role: str content: str class ChatCompletionRequest(BaseModel): model: str = Field(default="gpt-4o-mini") messages: List[Message] stream: bool = False async def custom_stream_response(response_text, model): chunk_size = 5 # Adjust this value to control the size of each chunk response_id = f"chatcmpl-{uuid.uuid4().hex[:12]}" created = int(asyncio.get_event_loop().time()) # Initial response yield f"data: {json.dumps({'id': response_id, 'object': 'chat.completion.chunk', 'created': created, 'model': model, 'choices': [{'delta': {'role': 'assistant'}, 'index': 0}]})}\n\n" for i in range(0, len(response_text), chunk_size): chunk = response_text[i:i+chunk_size] yield f"data: {json.dumps({'id': response_id, 'object': 'chat.completion.chunk', 'created': created, 'model': model, 'choices': [{'delta': {'content': chunk}, 'index': 0}]})}\n\n" await asyncio.sleep(0.1) # Add a small delay between chunks # Final response yield f"data: {json.dumps({'id': response_id, 'object': 'chat.completion.chunk', 'created': created, 'model': model, 'choices': [{'delta': {}, 'finish_reason': 'stop', 'index': 0}]})}\n\n" yield "data: [DONE]\n\n" @app.post("/v1/chat/completions") async def chat_completions(request: ChatCompletionRequest): latest_message = request.messages[-1].content if request.model in ['gpt-4o', 'gpt-4', 'gpt-3.5-turbo']: response_json = await make_request(latest_message, model=request.model) else: response_json = await make_toolbaz_request(latest_message, request.model, 'session_id') response_text = response_json.get('choices', [{}])[0].get('message', {}).get('content', '') if request.stream: return StreamingResponse(custom_stream_response(response_text, request.model), media_type='text/event-stream') else: return { "id": f"chatcmpl-{uuid.uuid4().hex[:12]}", "object": "chat.completion", "created": int(asyncio.get_event_loop().time()), "model": request.model, "choices": [{ "index": 0, "message": { "role": "assistant", "content": response_text }, "finish_reason": "stop" }], "usage": { "prompt_tokens": len(latest_message), "completion_tokens": len(response_text), "total_tokens": len(latest_message) + len(response_text) } } if __name__ == "__main__": import uvicorn uvicorn.run(app, host="0.0.0.0", port=8000)providers.py:import aiohttp import json HEADERS = { 'Accept': '/', 'Accept-Language': 'en-US,en;q=0.9', 'Cache-Control': 'no-cache', 'Connection': 'keep-alive', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Origin': 'https://toolbaz.com/', 'Pragma': 'no-cache', 'Referer': 'https://toolbaz.com/', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-site', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36', 'sec-ch-ua': '"Google Chrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', } async def make_request(prompt, model='gpt-4o'): headers = { 'Content-Type': 'application/json' } data = { 'messages': [{'role': 'user', 'content': prompt}], 'model': model } url = { 'gpt-4o': "https://nexra.aryahcr.cc/api/chat/complements", 'gpt-4': "https://nexra.aryahcr.cc/api/chat/gpt", 'gpt-3.5-turbo': "https://nexra.aryahcr.cc/api/chat/gpt" }.get(model, "https://nexra.aryahcr.cc/api/chat/gpt") async with aiohttp.ClientSession() as session: try: async with session.post(url, headers=headers, json=data) as response: response.raise_for_status() content_type = response.headers.get('Content-Type', '') if 'application/json' in content_type: json_response = await response.json() else: text_response = await response.text() json_response = json.loads(text_response) return {"choices": [{"message": {"content": json_response.get('message', '')}}]} except Exception as e: print(f"Error in make_request: {e}") return {"error": str(e)} async def make_toolbaz_request(prompt, model, session_id): payload = { 'text': prompt, 'model': model, 'sessionid': session_id } async with aiohttp.ClientSession() as session: try: async with session.post('https://data.toolbaz.com/writer.php', headers=HEADERS, data=payload) as response: response.raise_for_status() text_response = await response.text() return {"choices": [{"message": {"content": text_response}}]} except Exception as e: print(f"Error in make_toolbaz_request: {e}") return {"error": f"Error in external service: {str(e)}"}test.py:import asyncio import aiohttp import json async def make_request(prompt, model): url = 'http://localhost:8000/v1/chat/completions' payload = { 'model': model, 'messages': [ {'role': 'user', 'content': prompt} ], 'stream': True # Enable streaming } async with aiohttp.ClientSession() as session: async with session.post(url, json=payload) as response: async for line in response.content.iter_any(): if line: line = line.decode('utf-8').strip() if line.startswith("data:"): data = line[5:].strip() # Remove the "data: " prefix if data == "[DONE]": break try: json_data = json.loads(data) print('Received:', json_data) except json.JSONDecodeError: print('Received (non-JSON):', data) async def test_reverse_api(): models = ['gpt-4o', 'gpt-4', 'gpt-3.5-turbo', 'gpt-4o-mini'] prompt = 'Hello, how are you?' for model in models: print(f"\nTesting model: {model}") try: await make_request(prompt, model) except Exception as e: print('Error calling reverse API:', str(e)) if __name__ == '__main__': asyncio.run(test_reverse_api())README.md:Based on this information, please provide detailed specification for the project. Here is an example specification format:
---START_OF_EXAMPLE_SPEC---
The application is a simple ToDo app built using React. Its primary function is to allow users to manage a list of tasks (todos). Each task has a description and a state (open or completed, with the default state being open). The application is frontend-only, with no user sign-up or authentication process. The goal is to provide a straightforward and user-friendly interface for task management.
Features:
- Display of Todos: A list that displays all todo items. Each item shows its description and a checkbox to indicate its state (open or completed).
- Add New Todo: A button to add a new todo item. Clicking this button will prompt the user to enter a description for the new todo.
- Toggle State: Each todo item includes a checkbox. Checking/unchecking this box toggles the todo's state between open and completed.
- Local Storage: The application will use the browser's local storage to persist todos between sessions, ensuring that users do not lose their data upon reloading the application.
Functional Specification:
- Upon loading the application, it fetches existing todos from the local storage and displays them in a list.
- Each todo item in the list displays a checkbox and a description. The checkbox reflects the todo's current state (checked for completed, unchecked for open).
- When the user checks or unchecks a checkbox, the application updates the state of the corresponding todo item and saves the updated list to local storage.
- Clicking the "Add New Todo" button prompts the user to enter a description for the new todo. Upon confirmation, the application adds the new todo (with the default state of open) to the list and updates local storage.
- The application does not support deleting or editing todo items to keep the interface and interactions simple.
- Todos persist between sessions using the browser's local storage. The application saves any changes to the todo list (additions or state changes) in local storage and retrieves this data when the application is reloaded.
Technical Specification:
- Platform/Technologies: The application is a web application developed using React on frontend and Express on the backend, using SQLite as the database.
- Styling: Use Bootstrap 5 for a simple and functional interface. Load Boostrap from the CDN (don't install it locally):
- State Management: Directly in the React component
- make sure to initialize the state from the local storage as default (... = useState(JSON.parse(localStorage.getItem('todos')) || []) to avoid race conditions
- Data Persistence: Using the SQLite database on the backend via a REST API.
---END_OF_EXAMPLE_SPEC---
IMPORTANT: In the specification, you must include the following sections:
- Project Description: A detailed description of what the project is about.
- Features: A list of features that the project has implemented. Each feature should be described in detail.
- Technical Specification: Detailed description of how the project works, including any important technical details.
- Assistant
---START_OF_SPEC---
Project Description:
The project is a chat completion API implemented using FastAPI, designed to interact with external AI models like OpenAI's GPT-4 and custom-defined models via a service called Toolbaz. The primary goal of this API is to provide asynchronous chat completion services with real-time streaming capabilities using server-sent events (SSE). The API processes user messages, requests AI completions, and streams the results back to clients, making it well-suited for applications requiring conversational AI features.
Features:
-
Chat Completion Endpoint:
- Provides a POST endpoint (
/v1/chat/completions) that accepts a model identifier, a list of chat messages, and a streaming option. - Validates incoming requests using Pydantic models to ensure correct data structure (ensuring data integrity before processing).
- Provides a POST endpoint (
-
Model Selection:
- Supports multiple AI models such as 'gpt-3.5-turbo', 'gpt-4', 'gpt-4o', and a custom model 'gpt-4o-mini'.
- Automatically routes requests to the appropriate external API based on the model specified.
-
Streaming Responses:
- Implements server-sent events (SSE) to provide real-time, chunked responses when the streaming option is enabled.
- Responses are streamed in small chunks to the client, allowing for lower latency interactions.
-
CORS Configuration:
- Configures CORS middleware to enable cross-origin requests from any domain, facilitating integration with various frontend clients.
-
Error Handling:
- Incorporates try-except blocks around external API calls to handle errors gracefully, preventing application crashes.
-
Testing and Utility Functions:
- Provides a script (
test.py) for simulating requests to the API, enabling testing of different models and verifying streaming functionality.
- Provides a script (
Technical Specification:
-
Platform/Technologies:
- The project is built using FastAPI for developing the backend API.
- Uses Pydantic for data validation, ensuring that incoming requests match expected formats before processing.
- Asynchronous HTTP requests are handled via
aiohttp, which allows for non-blocking interactions with external APIs.
-
Asynchronous Operations:
- Ensures efficient handling of multiple concurrent chats via asynchronous functions, leveraging Python's
asynciolibrary. - Supports high concurrency, crucial for real-time chat applications, by using non-blocking network calls.
- Ensures efficient handling of multiple concurrent chats via asynchronous functions, leveraging Python's
-
External API Integration:
- Communicates with external AI services using standard HTTP requests, routing to specific endpoints based on the selected model.
- Utilizes a combination of JSON payloads and form data, depending on the external service requirements.
-
Custom Streaming Protocol:
- Implements a custom function
custom_stream_responsefor generating and sending SSE-compliant messages to the client. - Includes logic to divide responses into chunks, creating an interactive experience for the end-user by providing partial data as it becomes available.
- Implements a custom function
-
Deployment and Execution:
- Can be run locally using Uvicorn, a lightning-fast ASGI server, enabling simple local testing and development.
- Contains configurations for deployment (e.g., using Uvicorn's
0.0.0.0host setting makes it accessible to other machines on the local network).
-
Security and Headers:
- Example headers provided in
providers.pyfor Toolbaz requests illustrate how to manage HTTP headers for specific API requirements.
- Example headers provided in
This detailed specification should guide the understanding of the project, highlighting its technical infrastructure and scope of features to support further development or assessment.
---END_OF_SPEC---
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →
- gpt-4o-2024-08-06Compare side by side →