i have python with openai-whisper instlled locally. I need a setup to take microphone…
- User
i have python with openai-whisper instlled locally. I need a setup to take microphone input and do speech to text on it (in english only)
Assistant
To set up a Python script that uses OpenAI Whisper for real-time microphone input and speech-to-text in English, follow these steps:
Step 1: Install Required Packages
Ensure you have the necessary Python packages installed. Open your terminal and run:
pip install pyaudio whisperOn Ubuntu/Debian, you might need to install
portaudio19-devfirst:sudo apt-get install python3-pyaudio portaudio19-devStep 2: Create the Python Script
Save the following code to a file (e.g.,
speech_to_text.py):import pyaudio import wave import whisper def record_audio(output_filename="temp_audio.wav"): FORMAT = pyaudio.paInt16 CHANNELS = 1 RATE = 16000 # Sample rate (matches Whisper's requirement) CHUNK = 1024 audio = pyaudio.PyAudio() print("Recording. Press Ctrl+C to stop.") frames = [] try: stream = audio.open( format=FORMAT, channels=CHANNELS, rate=RATE, input=True, frames_per_buffer=CHUNK ) while True: data = stream.read(CHUNK) frames.append(data) except KeyboardInterrupt: print("Stopping recording...") finally: stream.stop_stream() stream.close() audio.terminate() # Save the recording to a WAV file with wave.open(output_filename, 'wb') as wf: wf.setnchannels(CHANNELS) wf.setsampwidth(audio.get_sample_size(FORMAT)) wf.setframerate(RATE) wf.writeframes(b''.join(frames)) def transcribe_audio(audio_file="temp_audio.wav"): model = whisper.load_model("base") # Use "small" for faster processing result = model.transcribe(audio_file, language="english") return result["text"] if __name__ == "__main__": record_audio() transcription = transcribe_audio() print("\nTranscription:") print(transcription)Step 3: Run the Script
Execute the script:
python speech_to_text.pyHow to Use:
- When the script runs, it will start recording audio from your microphone.
- Press
Ctrl+Cto stop recording. - The script will then transcribe the recorded audio and display the text.
Notes:
- Dependencies: Ensure your microphone is working and recognized by your system.
- Model Choice: The
basemodel is used by default. For faster processing, trywhisper.load_model("small"). - Language: The transcription is forced to English via
language="english".
Optional Improvements:
- Add error handling for microphone access.
- Use a GUI to start/stop recording instead of
Ctrl+C. - Delete the temporary WAV file after transcription to save disk space.
- User
ok, but this need to be performed in real time - not a batch listhen then transcribe, but rather transcribe in real time
Assistant
To achieve real-time transcription using OpenAI Whisper, you need to process audio chunks incrementally as they are recorded. Here's how to modify the previous setup for real-time transcription:
Step 1: Install Required Packages (if not already done)
pip install pyaudio whisper numpy
Step 2: Create the Real-Time Script (
realtime_speech_to_text.py)import pyaudio import numpy as np import whisper def real_time_transcribe(model_name="base", chunk_duration_seconds=1): model = whisper.load_model(model_name) FORMAT = pyaudio.paInt16 CHANNELS = 1 RATE = 16000 # Whisper expects 16kHz audio CHUNK = int(RATE * chunk_duration_seconds) # Chunk size in samples audio = pyaudio.PyAudio() print("Recording. Press Ctrl+C to stop.") try: stream = audio.open( format=FORMAT, channels=CHANNELS, rate=RATE, input=True, frames_per_buffer=CHUNK ) while True: # Read audio chunk data = stream.read(CHUNK) audio_array = np.frombuffer(data, dtype=np.int16) # Transcribe the chunk result = model.transcribe(audio_array, language="english") print(f"\rTranscription: {result['text']}", end="") except KeyboardInterrupt: print("\nStopping...") finally: stream.stop_stream() stream.close() audio.terminate() if __name__ == "__main__": real_time_transcribe(model_name="base", chunk_duration_seconds=1)
Step 3: Run the Script
python realtime_speech_to_text.py
How It Works:
-
Real-Time Audio Capture:
- The script records audio in chunks of
chunk_duration_seconds(default 1 second). - Each chunk is converted to a numpy array for processing.
- The script records audio in chunks of
-
Incremental Transcription:
- The Whisper model processes each chunk immediately after recording.
- The transcription is printed in real-time, updating every second.
-
Stopping:
- Press
Ctrl+Cto exit the loop and stop recording.
- Press
Key Parameters:
-
model_name:- Use smaller models (
tiny,base) for faster inference. - Larger models (
medium,large) improve accuracy but increase latency.
- Use smaller models (
-
chunk_duration_seconds:- Adjust this value to control how often the transcription updates (e.g.,
0.5for updates every 0.5 seconds).
- Adjust this value to control how often the transcription updates (e.g.,
Notes:
-
Latency:
- Shorter chunks reduce latency but may produce less accurate results (due to less context).
- Larger models take longer to process each chunk, increasing latency.
-
Performance:
- For faster processing, use the
tinymodel and smaller chunk durations (e.g.,0.5seconds).
- For faster processing, use the
-
Continuous Output:
- The transcription updates in-place (using
\rto overwrite the same line).
- The transcription updates in-place (using
Example Output:
Recording. Press Ctrl+C to stop. Transcription: Hello, how are you today? Transcription: I'm doing well, thanks for asking. Transcription: What's the plan for the weekend? ^CStopping...
This setup provides real-time transcription by processing audio in chunks, balancing speed and accuracy based on your model choice and chunk size. Adjust the parameters to optimize for your use case!
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.