import sounddevice as sd import soundfile as sf import queue import threading import…
- User
import sounddevice as sd import soundfile as sf import queue import threading import tempfile import os from faster_whisper import WhisperModel import requests
def record_audio(filename, stop_event): q = queue.Queue()
def callback(indata, frames, time, status): if status: print(f"Recording error: {status}") q.put(indata.copy()) sample_rate = 16000 channels = 1 with sf.SoundFile(filename, mode='w', samplerate=sample_rate, channels=channels) as file: with sd.InputStream(samplerate=sample_rate, channels=channels, callback=callback): print("Recording... Press ENTER to stop.") while not stop_event.is_set(): try: data = q.get(timeout=0.1) file.write(data) except queue.Empty: continue print("Recording stopped.")def transcribe_audio(model, audio_path): print("Transcribing audio...") segments, info = model.transcribe(audio_path, beam_size=5) transcription = " ".join([segment.text for segment in segments]) return transcription
def send_to_api(api_url, text): try: response = requests.post(api_url, json={"text": text}) response.raise_for_status() print("Successfully sent data to the API.") return response.json() except requests.RequestException as e: print(f"Failed to send data to the API: {e}") return None
def main(): API_URL = "" # replace with API url
with tempfile.NamedTemporaryFile(suffix='.wav', delete=False) as temp_audio_file: audio_path = temp_audio_file.name stop_event = threading.Event() recording_thread = threading.Thread(target=record_audio, args=(audio_path, stop_event)) recording_thread.start() # Wait for the user to press ENTER to stop recording input() stop_event.set() recording_thread.join() print("Loading Whisper model...") model_size = "small" # Options: tiny, base, small, medium, large device = "cpu" # Use "cuda" if you have a GPU compute_type = "int8" # Options: int8, int16, float16 model = WhisperModel(model_size, device=device, compute_type=compute_type) transcription = transcribe_audio(model, audio_path) print("\n--- Transcribed Text ---") print(transcription) print("------------------------\n") api_response = send_to_api(API_URL, transcription) if api_response: print("\n--- API Response ---") print(api_response) print("---------------------") else: print("No response received from the API.") os.remove(audio_path) print("Temporary audio file deleted.")if name == "main": main()
explain all the main parts of this code, what the functions used in here do
- Assistant
This Python script is a complete program that records audio from the user's microphone, transcribes the audio into text using OpenAI's Whisper model (via the
faster-whisperimplementation), and sends the transcribed text to a specified API endpoint. Below is a detailed explanation of the main parts of the code and the functions used.
Overview of the Code
- Imports necessary libraries for audio recording, transcription, threading, and HTTP requests.
- Defines functions to handle audio recording, transcription, and API communication.
- Uses multithreading to handle recording without blocking the main thread.
- Implements a main function that orchestrates the recording, transcription, and API interaction.
Imports and Library Functions
import sounddevice as sd import soundfile as sf import queue import threading import tempfile import os from faster_whisper import WhisperModel import requestssounddevice(sd): Provides real-time audio input/output (I/O) from the microphone.soundfile(sf): Reads and writes sound files in various formats.queue: Creates a thread-safe FIFO queue for inter-thread communication.threading: Enables the use of threads for concurrent execution.tempfile: Creates temporary files and directories.os: Provides a way of using operating system-dependent functionality.faster_whisper: A faster implementation of the Whisper model for transcription.requests: Allows sending HTTP requests to interact with web APIs.
Function Definitions
1.
record_audio(filename, stop_event)Purpose: Records audio from the microphone and saves it to a WAV file until a stop event is triggered.
Parameters:
filename: The name of the file where the recorded audio will be saved.stop_event: Athreading.Eventobject used to signal when to stop recording.
Function Workflow:
-
Initialize a Queue:
- A
queue.Queue()objectqis used to safely pass audio data between the audio callback and the file-writing loop.
- A
-
Define the Audio Callback:
- The
callbackfunction is called bysd.InputStreamfor each audio chunk captured. - It checks for any
statusmessages and places the audio data (indata.copy()) into the queueq.
- The
-
Set Recording Parameters:
sample_rateis set to 16000 Hz (standard for speech recognition).channelsis set to 1 for mono audio.
-
Create a Sound File for Writing:
- Uses
sf.SoundFileto open a file in write mode with the specified sample rate and channels.
- Uses
-
Open an Input Stream:
- Uses
sd.InputStreamto capture audio data using the specified sample rate and channels. - The
callbackfunction is registered to handle incoming audio data.
- Uses
-
Recording Loop:
- Prints a message indicating recording has started.
- Enters a loop that continues until
stop_event.is_set()returnsTrue. - Inside the loop:
- Tries to get audio data from the queue with a timeout of 0.1 seconds.
- If data is available, it writes it to the audio file.
- If the queue is empty (
queue.Empty), it continues without writing.
-
Cleanup:
- Once
stop_eventis set, exits the loop. - Closes the audio stream and file.
- Prints "Recording stopped."
- Once
Key Functions Used:
sf.SoundFile: Opens a file for reading or writing audio data.sd.InputStream: Creates an input stream to read audio data from the microphone.queue.Queue: Thread-safe data exchange between the audio callback and the main thread.threading.Event: Used to signal the recording thread to stop.
2.
transcribe_audio(model, audio_path)Purpose: Transcribes the audio file at
audio_pathusing the provided Whisper model and returns the transcribed text.Parameters:
model: An instance of the Whisper model loaded viafaster_whisper.audio_path: The file path to the audio file to transcribe.
Function Workflow:
-
Prints "Transcribing audio..." to indicate the process has started.
-
Calls the
transcribeMethod:- Uses
model.transcribe(audio_path, beam_size=5)to perform the transcription. beam_size=5sets the beam search width for decoding, improving accuracy at the cost of speed.
- Uses
-
Processes the Output:
segments: A list of transcribed text segments with timestamps.info: Additional information about the transcription process.
-
Concatenates Text Segments:
- Uses a list comprehension to extract
segment.textfrom each segment. - Joins all the text segments into a single string separated by spaces.
- Uses a list comprehension to extract
-
Returns the Transcribed Text.
Key Functions Used:
model.transcribe: Transcribes audio data into text.beam_size: A parameter that controls the beam search width; higher values can improve accuracy.
3.
send_to_api(api_url, text)Purpose: Sends the transcribed text to a specified API endpoint using an HTTP POST request.
Parameters:
api_url: The URL of the API endpoint where the text will be sent.text: The transcribed text to be sent.
Function Workflow:
-
Tries to Send a POST Request:
- Uses
requests.post(api_url, json={"text": text})to send the data. - Sets the
jsonparameter to pass the data in JSON format.
- Uses
-
Checks the Response:
- Calls
response.raise_for_status()to raise an exception if the HTTP request returned an unsuccessful status code.
- Calls
-
Prints Success Message:
- If no exception is raised, prints "Successfully sent data to the API."
-
Returns the JSON Response:
- Uses
response.json()to parse and return the JSON content from the response.
- Uses
-
Handles Exceptions:
- If a
requests.RequestExceptionis caught, it prints an error message and returnsNone.
- If a
Key Functions Used:
requests.post: Sends an HTTP POST request.response.raise_for_status(): Checks for HTTP errors.response.json(): Parses the response content as JSON.
4.
main()Purpose: The main function orchestrates the entire process: recording audio, transcribing it, and sending the transcription to the API.
Function Workflow:
-
Set the API URL:
API_URL = ""is a placeholder. The user should replace it with the actual API endpoint.
-
Create a Temporary Audio File:
- Uses
tempfile.NamedTemporaryFileto create a temporary WAV file. - The
delete=Falseparameter ensures the file is not deleted upon closing (it will be deleted manually later). audio_pathstores the path to the temporary audio file.
- Uses
-
Set Up a Stop Event:
stop_event = threading.Event()creates an event object used to signal when to stop recording.
-
Start the Recording Thread:
- Creates a new thread
recording_threadthat targets therecord_audiofunction withaudio_pathandstop_eventas arguments. - Starts the recording thread with
recording_thread.start().
- Creates a new thread
-
Wait for User Input to Stop Recording:
input()waits for the user to press ENTER.- Once ENTER is pressed,
stop_event.set()signals the recording thread to stop. recording_thread.join()waits for the recording thread to finish execution.
-
Load the Whisper Model:
- Prints "Loading Whisper model...".
- Sets
model_sizeto "small". Other options include "tiny", "base", "medium", "large". - Sets
deviceto "cpu". If the system has a compatible GPU, "cuda" can be used for faster processing. - Sets
compute_typeto "int8". Options include "int8", "int16", "float16"; lower precision can speed up inference with some trade-offs in accuracy. - Creates an instance of
WhisperModelwith the specified parameters.
-
Transcribe the Audio:
- Calls
transcribe_audio(model, audio_path)to transcribe the recorded audio. - Stores the result in
transcription. - Prints the transcribed text within decorative separators.
- Calls
-
Send Transcription to API:
- Calls
send_to_api(API_URL, transcription)to send the transcribed text to the API. - Stores the response in
api_response. - If a response is received, it prints the response within decorative separators.
- If no response is received, it prints "No response received from the API."
- Calls
-
Clean Up:
- Deletes the temporary audio file using
os.remove(audio_path). - Prints "Temporary audio file deleted."
- Deletes the temporary audio file using
-
Run the Main Function:
- The
if __name__ == "__main__":block ensuresmain()is called when the script is executed directly.
- The
Key Functions and Concepts Used:
-
Threading:
threading.Thread: Creates a new thread.threading.Event: Manages a stop/start flag between threads.
-
Temporary Files:
tempfile.NamedTemporaryFile: Manages temporary files that can be deleted after use.
-
Model Loading and Configuration:
- Model Size: Determines the size and capability of the Whisper model (trade-off between speed and accuracy).
- Device: CPU or GPU for model inference.
- Compute Type: Precision of computations, affecting speed and memory usage.
Additional Notes
-
User Interaction:
- The program waits for the user to press ENTER to stop recording, allowing flexible recording duration.
-
API URL Placeholder:
- The
API_URLvariable must be set to a valid endpoint for thesend_to_apifunction to work properly.
- The
-
Error Handling:
- The
record_audiofunction prints any recording errors encountered in the callback. - The
send_to_apifunction handles exceptions related to the HTTP request and provides feedback.
- The
-
Precision and Performance Trade-offs:
beam_sizein transcription affects accuracy and speed; higher values may yield better transcriptions but take longer.compute_typeaffects inference speed and resource usage; lower precision types likeint8can speed up processing on CPUs.
Summary of Functions and Their Roles
record_audio(): Captures audio from the microphone asynchronously and saves it to a temporary WAV file.transcribe_audio(): Uses the Whisper model to convert the recorded audio into a text transcription.send_to_api(): Sends the transcribed text to a web API via an HTTP POST request.main(): Coordinates the overall workflow by initializing components, handling user input, and managing threads.
Explanation of Key Libraries and Functions
-
sounddevice(sd):InputStream: Streams audio data from the input device (e.g., microphone).callback: A function called periodically with chunks of audio data.
-
soundfile(sf):SoundFile: Reads from or writes audio data to a file.write: Writes audio frames to the file.
-
queue.Queue:- Provides a thread-safe FIFO queue for communication between the audio callback thread and the main thread.
-
threading.Thread:- Allows the
record_audiofunction to run in a separate thread, enabling concurrent execution without blocking the main program.
- Allows the
-
tempfile.NamedTemporaryFile:- Generates a temporary file that can be used to store the recorded audio and is deleted after use to free resources.
-
WhisperModelfromfaster_whisper:- Loads a version of the Whisper speech recognition model optimized for faster inference.
transcribe: Converts audio data to text using the model.
-
requests:post: Sends HTTP POST requests to communicate with web APIs.raise_for_status: Checks for HTTP request errors.json: Parses JSON content from responses.
Execution Flow
-
Start of the Program:
- The
main()function is called when the script is executed.
- The
-
Recording Phase:
- A temporary WAV file is created to store the audio.
- The recording thread is started, and audio is captured asynchronously.
- The program waits for the user to press ENTER to stop recording.
-
Transcription Phase:
- The recorded audio is transcribed into text using the Whisper model.
- The transcribed text is printed to the console.
-
API Communication Phase:
- The transcribed text is sent to the specified API endpoint.
- The API response is printed to the console if available.
-
Cleanup Phase:
- The temporary audio file is deleted.
- The program exits.
Customization and Usage
-
Setting the API URL:
- Replace
API_URL = ""with the actual endpoint you wish to send the transcription to.
- Replace
-
Adjusting Model Settings:
- Model Size:
- Options:
"tiny","base","small","medium","large". - Larger models may provide better accuracy but require more computational resources.
- Options:
- Device Configuration:
- If you have a compatible GPU, set
device = "cuda"to leverage faster computation.
- If you have a compatible GPU, set
- Compute Type:
- Adjust
compute_typebased on the desired balance between speed and precision.
- Adjust
- Model Size:
-
Handling Long Recordings:
- The program waits indefinitely for the user to press ENTER, allowing for recordings of any length.
-
Error Handling Enhancements:
- You can add more robust error handling, such as checking for microphone availability or validating API responses.
Conclusion
This script provides a practical example of combining audio processing, machine learning (speech recognition), and web communication in Python. By understanding each component and function, you can customize and extend the script for various applications, such as voice-controlled interfaces, transcription services, or automated captioning systems.
Feel free to ask if you have any questions or need further clarification on any part of the code.