I found this super code on the internet. I want to implement it to my script. The script…
- User
I found this super code on the internet. I want to implement it to my script. The script i want to make is a TTS that stream play audio smoothly. Code I found on internet :
ing callback, which allows manipulating stream data while its being played in realtime.
This solution plays audio based on last line of log continuously until log data changes.
This solution also eliminated popping / cracking sound that occurs when merging two tones.
Inspiration from here.
import pyaudio import numpy as np from time import time,sleep import os
CHANNELS = 2 RATE = 44100
TT = time() freq = 100 newfreq = 100 phase = 0 log_file = “probemon.log”
def callback(in_data, frame_count, time_info, status): global TT,phase,freq,newfreq if newfreq != freq: phase = 2np.piTT*(freq-newfreq)+phase freq=newfreq left = (np.sin(phase+2np.pifreq*(TT+np.arange(frame_count)/float(RATE)))) data = np.zeros((left.shape[0]2,),np.float32) data[0::2] = left #left data data[1::2] = left #right data TT+=frame_count/float(RATE) return (data, pyaudio.paContinue)
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paFloat32, channels=CHANNELS, rate=RATE, output=True, stream_callback=callback)
stream.start_stream() tmphold = “” try: while True: line = os.popen(‘tail -n 1 {}’.format(log_file)).read() try: key, val = line.split() except: key, val = “default”, 0.0
f = abs(int(val)) newfreq = f * 10 #update freq per log if newfreq != tmphold: tmphold = newfreq print "mac:{} , rssi:{} , freq:{} finally: stream.stop_stream() stream.close() p.terminate()
My script, it plays audio but there is void between chunks of audio which is not good :
import asyncio import edge_tts import pyaudio from io import BytesIO from pydub import AudioSegment
TEXT = “aHello World!” VOICE = “en-GB-SoniaNeural”
p = pyaudio.PyAudio()
async def stream_tts(text: str, voice: str) -> None:
We’re assuming a certain format, channels, and rate
This will need to be dynamic based on the actual audio data from TTS
stream = p.open( format=pyaudio.paInt16, channels=1, rate=26000, output=True )
communicate = edge_tts.Communicate(text, voice)
This will act as our chunk size
chunk_size = 1024
Process and play audio chunks as they arrive
async for chunk in communicate.stream(): if chunk[“type”] == “audio”: try: audio_segment = AudioSegment.from_file(BytesIO(chunk[“data”]), format=“mp3”)
Write data to the stream directly without extra buffering
stream.write(audio_segment.raw_data)
If this is the last chunk, break after playing
if chunk.get(‘end’, False): break except Exception as e: print(“Error processing audio chunk:”, e)
Cleanup
stream.stop_stream() stream.close() p.terminate()
if name == “main”:
Run the asyncio event loop
asyncio.run(stream_tts(TEXT100, VOICE))
- Assistant
Your current implementation using
pyaudioandasyncioto stream TTS audio has a gap between chunks because it doesn't seem to consider the need for contiguous playback without any intervening silence. To implement the TTS solution using the streaming concept from the code you found, you will need to adapt its streaming callback approach.The code you found uses a callback function to continuously feed the audio stream with data, avoiding gaps because the callback function gets called by the
pyaudiostream whenever it needs more data. You can use a similar approach for your TTS solution.Here is a modified example of your script with a streaming callback to play text-to-speech audio smoothly:
import asyncio import pyaudio from io import BytesIO from pydub import AudioSegment import edge_tts TEXT = "Hello World!" VOICE = "en-GB-SoniaNeural" p = pyaudio.PyAudio() # Define a stream to push audio data to stream = p.open( format=pyaudio.paInt16, channels=1, rate=26000, # Use the correct rate provided by the TTS engine output=True, stream_callback=None # We will set this later ) buffer = asyncio.Queue() async def fetch_and_buffer_audio(text: str, voice: str): communicate = edge_tts.Communicate(text, voice) async for chunk in communicate.stream(): if chunk["type"] == "audio": try: audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3") await buffer.put(audio_segment.raw_data) if chunk.get('end', False): await buffer.put(None) # Signal the end of stream except Exception as e: print("Error processing audio chunk:", e) def callback(in_data, frame_count, time_info, status): frames = [] while True: try: # Attempt to retrieve data from the buffer data = buffer.get_nowait() if data is None: # End of stream signal return (bytes(), pyaudio.paComplete) else: frames.append(data) if len(b"".join(frames)) >= frame_count * 2: # 2 bytes per frame break except asyncio.QueueEmpty: # Buffer underrun, send silent frames and wait for more data return (bytes(frame_count * 2), pyaudio.paContinue) # Concatenate all the frames and trim to the required frame_count output_data = b"".join(frames)[:frame_count * 2] return (output_data, pyaudio.paContinue) # Set the callback function to the stream stream._stream_callback = callback async def main(text: str, voice: str): stream.start_stream() # Start the coroutine that fetches and buffers audio data buffer_coro = fetch_and_buffer_audio(text, voice) await buffer_coro # Will complete when "None" is placed indicating end of audio # Cleanup while stream.is_active(): # Wait for the stream callback to finish processing await asyncio.sleep(0.1) stream.stop_stream() stream.close() p.terminate() if __name__ == "__main__": # Run the asyncio event loop asyncio.run(main(TEXT * 100, VOICE))With this approach, the
fetch_and_buffer_audiocoroutine fetches audio data from the TTS service, converts each chunk of audio as it arrives, and places it in anasyncio.Queue. Thecallbackfunction, when called bypyaudio, retrieves data from this queue without waiting, providing silent frames if necessary (to avoid gaps) until more data arrives. When the end of the TTS audio is detected, aNoneis placed in the queue to indicate the end of the stream.Please remember to adjust the audio parameters (such as rate) to match the output of your TTS engine, and handle the potential for underrun in the callback as needed. The sample code might need further adjustments depending on the specifics of the
edge_ttslibrary and the format of the TTS data you receive. - User
I launched the script but i hear nothing
- Assistant
I apologize for the inconvenience. We will need to adjust the script to ensure audio is played correctly.
The issue might be due to incorrect handling of the queue or audio parameters. Let's modify the script to use the callback method properly. Here is an updated version of the script that should work:
import asyncio import pyaudio from io import BytesIO from pydub import AudioSegment import edge_tts TEXT = "Hello World!" VOICE = "en-US-JennyNeural" # We're assuming a certain format, channels, and rate FORMAT = pyaudio.paInt16 CHANNELS = 1 RATE = 24000 # Adjust based on the TTS output format p = pyaudio.PyAudio() # Use an asyncio Queue to handle audio data between the producer and consumer buffer = asyncio.Queue() async def produce_audio(text: str, voice: str): communicate = edge_tts.Communicate(text, voice) async for chunk in communicate.stream(): if chunk["type"] == "audio": audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3") raw_data = audio_segment.set_frame_rate(RATE).set_channels(CHANNELS).set_sample_width(p.get_sample_size(FORMAT)).raw_data await buffer.put(raw_data) if chunk.get('end', False): await buffer.put(None) # Signal the end of stream break def callback(in_data, frame_count, time_info, status): # Get data from the queue try: data = buffer.get_nowait() if data is None: return (None, pyaudio.paComplete) except asyncio.QueueEmpty: data = b'\x00' * frame_count * CHANNELS * 2 # silence if no data return (data, pyaudio.paContinue) # Create an audio stream with the callback stream = p.open( format=FORMAT, channels=CHANNELS, rate=RATE, output=True, stream_callback=callback ) async def main(text, voice): # Start the audio stream stream.start_stream() # Start the producer coroutine producer_task = asyncio.create_task(produce_audio(text, voice)) # Run the producer coroutine until it finishes await producer_task # Wait for the stream to finish playing while stream.is_active(): await asyncio.sleep(0.1) # Stop and close the stream stream.stop_stream() stream.close() # Terminate the PyAudio object p.terminate() if __name__ == "__main__": asyncio.run(main(TEXT * 100, VOICE)) # Play the text-to-speech 100 timesThis script is designed to use the
asynciolibrary and a PyAudio stream to play audio frames without gaps continuously. Theproduce_audiocoroutine fetches the text-to-speech audio and puts the data into anasyncio.Queue. It also signals when the end of the stream is reached. The callback function pulls data from this queue and returns it to the stream for playback. If the queue is empty, it returns a buffer filled with silence to prevent underrun.Please make sure that the
RATE,CHANNELS, andFORMATmatch the output specifications of your TTS engine. The script assumes that the TTS engine output is MP3. You might need to adjust parameters accordingly.Additionally, there are necessary installations such as
pyduband any dependencies that might be missing:pip install pydub asyncioRemember that
edge_ttsmay also have its specific installation or setup requirements that must be satisfied for it to function correctly. - User
Still nothing, no audio, no sound...
- Assistant
I apologize for the issues you're experiencing. To troubleshoot, let's start by verifying a few things:
- Ensure your speakers or headphones are connected, powered on, and the volume is up.
- Ensure
pyaudioandedge_ttsare installed correctly. - Verify that
edge_ttsis receiving text and generating audio as expected. - Make sure the audio sample rate provided by
edge_ttsmatches theRATEin the script. - Ensure there are no exceptions or errors being raised that might be silently caught.
For debugging, you should add print statements or logging throughout your code to understand where it might be failing. Here's an example of how you could modify the
produce_audiocoroutine and thecallbackfunction:async def produce_audio(text: str, voice: str): communicate = edge_tts.Communicate(text, voice) # Insert a print statement here to make sure this part is reached. print("Fetching audio content from TTS engine...") async for chunk in communicate.stream(): if chunk["type"] == "audio": audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3") # Print the duration of the audio segment for reference. print(f"Received audio chunk with duration: {audio_segment.duration_seconds} seconds.") raw_data = audio_segment.raw_data await buffer.put(raw_data) if chunk.get('end', False): print("End of audio stream reached.") await buffer.put(None) # Signal the end of stream break def callback(in_data, frame_count, time_info, status): # Print the status for debugging. print(f"Callback called with frame_count: {frame_count} - status: {status}") try: data = buffer.get_nowait() if data is None: print("End of data received. Finishing playback.") return (None, pyaudio.paComplete) else: print(f"Playing audio chunk of size: {len(data)}") return (data, pyaudio.paContinue) except asyncio.QueueEmpty: print("Buffer underrun. Sending silence.") data = b'\x00' * frame_count * CHANNELS * p.get_sample_size(FORMAT) # silence if no data return (data, pyaudio.paContinue)If the
print()statements show that audio chunks are being fetched and processed but you still hear no output, the issue could be in the PyAudio configuration or in the system audio settings.If you suspect it's a system-related issue, try playing a local audio file using pyaudio or another library to verify that audio output works correctly with your current setup.
Here's a simple example to test your system's audio output with
pydubandpyaudio:from pydub import AudioSegment from pydub.playback import play # Load a local mp3 file audio = AudioSegment.from_file("your_audio_file.mp3") # Play the audio play(audio)Make sure to replace
"your_audio_file.mp3"with the path to an actual MP3 file on your system.Running this snippet will help confirm whether your system and your installation of
pydubcan successfully play audio. If this does not work, you'll need to diagnose and fix your audio output before attempting to stream TTS audio.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.