whats the command to run this right? import io from fastapi import FastAPI, File,…
- User
whats the command to run this right?
import io from fastapi import FastAPI, File, UploadFile, Form, HTTPException from fastapi.middleware.cors import CORSMiddleware from transformers import pipeline import torch import soundfile as sf import numpy as np
app = FastAPI( title="Fast Whisper Turbo HTTP API", description="Ultra-fast Whisper V3 Turbo inference HTTP server" )
Allow CORS for testing from WinForms if needed
app.add_middleware( CORSMiddleware, allow_origins=[""], allow_credentials=True, allow_methods=[""], allow_headers=["*"], )
Enable cudnn autotuner to find best algorithm for your hardware & input sizes
torch.backends.cudnn.benchmark = True
print("Loading model... This may take some time.")
Load model once on GPU with float16 for speed
pipe = pipeline( "automatic-speech-recognition", model="openai/whisper-large-v3-turbo", device=0, # device=0 is same as "cuda:0" torch_dtype=torch.float16, model_kwargs={"attn_implementation": "flash_attention_2"}, )
print("Model loaded on GPU.")
@app.post("/transcribe") async def transcribe( audio_file: UploadFile = File(...), task: str = Form("transcribe") ): """ Transcribe or translate audio file.
- audio_file: audio file (wav, mp3, etc) - task: "transcribe" or "translate" """ if task not in ("transcribe", "translate"): raise HTTPException(status_code=400, detail="Invalid task parameter") contents = await audio_file.read() try: audio_bytes = io.BytesIO(contents) waveform, sample_rate = sf.read(audio_bytes, dtype='float32') except Exception as e: raise HTTPException(status_code=400, detail=f"Could not read audio file: {e}") if waveform.ndim > 1: # Convert to mono by averaging channels if needed waveform = waveform.mean(axis=1) # Whisper expected sample rate is 16000 Hz; resample if necessary target_sr = 16000 if sample_rate != target_sr: import torchaudio.transforms as T waveform_tensor = torch.from_numpy(waveform).unsqueeze(0) # shape [1, n_samples] resampler = T.Resample(orig_freq=sample_rate, new_freq=target_sr) waveform = resampler(waveform_tensor).squeeze(0).numpy() sample_rate = target_sr # The transformer pipeline accepts numpy array audio in waveform form print(f"Starting {task} task for file: {audio_file.filename}") try: with torch.inference_mode(): outputs = pipe( waveform, chunk_length_s=30, generate_kwargs={"task": task}, return_timestamps=False, ) except Exception as e: raise HTTPException(status_code=500, detail=f"Inference failed: {e}") text = outputs.get('text', '').strip() print("Finished transcription task") return {"text": text} - Assistant
To run this FastAPI app, save it to a Python file, for example
app.py, then run it with an ASGI server such as uvicorn:uvicorn app:app --host 0.0.0.0 --port 8000 --reloadExplanation:
app:app— means "import theappobject from theapp.pyfile"--host 0.0.0.0— makes the server accessible from other machines on your network (optional)--port 8000— the port to listen on (optional, default is 8000)--reload— automatically reload server on code changes (only for development)
If you do not have uvicorn installed, install it first:
pip install "uvicorn[standard]"Then run the command above. Your FastAPI server should start and load the Whisper model.
Summary
- Save your code in
app.py - Install uvicorn if needed:
pip install "uvicorn[standard]" - Run the app:
uvicorn app:app --host 0.0.0.0 --port 8000 --reloadYou can then send requests to
http://localhost:8000/transcribe. - User
based on how I have it in visual studio change the command
public async Task WhisperTurbo(string fileToTranscribe) { var url = "http://localhost:21421/transcribe";
try { using var multipart = new MultipartFormDataContent(); var fileStream = File.OpenRead(fileToTranscribe); var streamContent = new StreamContent(fileStream); streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream"); multipart.Add(streamContent, "audio_file", Path.GetFileName(fileToTranscribe)); multipart.Add(new StringContent("transcribe"), "task"); var request = new HttpRequestMessage(HttpMethod.Post, url) { Content = multipart }; using var response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false); response.EnsureSuccessStatusCode(); using var responseStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false); using var jsonDoc = await JsonDocument.ParseAsync(responseStream).ConfigureAwait(false); if (jsonDoc.RootElement.TryGetProperty("text", out var textElement)) { return textElement.GetString() ?? string.Empty; } else { throw new Exception("Response JSON does not contain 'text' property."); } } catch (Exception ex) { throw new InvalidOperationException("WhisperTurbo request failed", ex); }}
- Assistant
Given your C# code is trying to send a POST request with a multipart form to the FastAPI endpoint at:
var url = "http://localhost:21421/transcribe";That means your FastAPI server needs to be running and listening on port 21421.
So to match your C# code, change your uvicorn run command to use port 21421:
uvicorn app:app --host 0.0.0.0 --port 21421 --reload
Full steps for your setup:
- Save your Python FastAPI code in
app.py - Install uvicorn if you haven't:
pip install "uvicorn[standard]" - Run FastAPI server on port 21421:
uvicorn app:app --host 0.0.0.0 --port 21421 --reload - Your C# method can now successfully POST to
http://localhost:21421/transcribeas it currently does.
Optional Extra:
- If you want to make the audio file content type more appropriate (FastAPI expects typical audio files), you might want to adapt your C# code’s content type to something like
"audio/wav"or"audio/mpeg"depending on your file. But"application/octet-stream"usually also works fine.
TL;DR
Run FastAPI app with:
uvicorn app:app --host 0.0.0.0 --port 21421 --reloadand your existing C#
WhisperTurbofunction will post to the matching server URL. - Save your Python FastAPI code in