Skip to content
TheChatArchiveThe Chat Archive
gpt-4-1106-preview· openaiHugging Face Suzhen/CodeChat

I found this super code on the internet. I want to implement it to my script. The script…

  1. User

    I found this super code on the internet. I want to implement it to my script. The script i want to make is a TTS that stream play audio smoothly. Code I found on internet :

    ing callback, which allows manipulating stream data while its being played in realtime.

    This solution plays audio based on last line of log continuously until log data changes.

    This solution also eliminated popping / cracking sound that occurs when merging two tones.

    Inspiration from here.

    import pyaudio import numpy as np from time import time,sleep import os

    CHANNELS = 2 RATE = 44100

    TT = time() freq = 100 newfreq = 100 phase = 0 log_file = “probemon.log”

    def callback(in_data, frame_count, time_info, status): global TT,phase,freq,newfreq if newfreq != freq: phase = 2np.piTT*(freq-newfreq)+phase freq=newfreq left = (np.sin(phase+2np.pifreq*(TT+np.arange(frame_count)/float(RATE)))) data = np.zeros((left.shape[0]2,),np.float32) data[0::2] = left #left data data[1::2] = left #right data TT+=frame_count/float(RATE) return (data, pyaudio.paContinue)

    p = pyaudio.PyAudio()

    stream = p.open(format=pyaudio.paFloat32, channels=CHANNELS, rate=RATE, output=True, stream_callback=callback)

    stream.start_stream() tmphold = “” try: while True: line = os.popen(‘tail -n 1 {}’.format(log_file)).read() try: key, val = line.split() except: key, val = “default”, 0.0

    f = abs(int(val)) newfreq = f * 10 #update freq per log if newfreq != tmphold: tmphold = newfreq print "mac:{} , rssi:{} , freq:{} finally: stream.stop_stream() stream.close() p.terminate()

    My script, it plays audio but there is void between chunks of audio which is not good :

    import asyncio import edge_tts import pyaudio from io import BytesIO from pydub import AudioSegment

    TEXT = “aHello World!” VOICE = “en-GB-SoniaNeural”

    p = pyaudio.PyAudio()

    async def stream_tts(text: str, voice: str) -> None:

    We’re assuming a certain format, channels, and rate

    This will need to be dynamic based on the actual audio data from TTS

    stream = p.open( format=pyaudio.paInt16, channels=1, rate=26000, output=True )

    communicate = edge_tts.Communicate(text, voice)

    This will act as our chunk size

    chunk_size = 1024

    Process and play audio chunks as they arrive

    async for chunk in communicate.stream(): if chunk[“type”] == “audio”: try: audio_segment = AudioSegment.from_file(BytesIO(chunk[“data”]), format=“mp3”)

    Write data to the stream directly without extra buffering

    stream.write(audio_segment.raw_data)

    If this is the last chunk, break after playing

    if chunk.get(‘end’, False): break except Exception as e: print(“Error processing audio chunk:”, e)

    Cleanup

    stream.stop_stream() stream.close() p.terminate()

    if name == “main”:

    Run the asyncio event loop

    asyncio.run(stream_tts(TEXT100, VOICE))

  2. Assistant

    Your current implementation using pyaudio and asyncio to stream TTS audio has a gap between chunks because it doesn't seem to consider the need for contiguous playback without any intervening silence. To implement the TTS solution using the streaming concept from the code you found, you will need to adapt its streaming callback approach.

    The code you found uses a callback function to continuously feed the audio stream with data, avoiding gaps because the callback function gets called by the pyaudio stream whenever it needs more data. You can use a similar approach for your TTS solution.

    Here is a modified example of your script with a streaming callback to play text-to-speech audio smoothly:

    import asyncio
    import pyaudio
    from io import BytesIO
    from pydub import AudioSegment
    import edge_tts
    
    TEXT = "Hello World!"
    VOICE = "en-GB-SoniaNeural"
    
    p = pyaudio.PyAudio()
    
    # Define a stream to push audio data to
    stream = p.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=26000,  # Use the correct rate provided by the TTS engine
        output=True,
        stream_callback=None  # We will set this later
    )
    
    buffer = asyncio.Queue()
    
    async def fetch_and_buffer_audio(text: str, voice: str):
        communicate = edge_tts.Communicate(text, voice)
        async for chunk in communicate.stream():
            if chunk["type"] == "audio":
                try:
                    audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3")
                    await buffer.put(audio_segment.raw_data)
                    if chunk.get('end', False):
                        await buffer.put(None)  # Signal the end of stream
                except Exception as e:
                    print("Error processing audio chunk:", e)
    
    def callback(in_data, frame_count, time_info, status):
        frames = []
        while True:
            try:
                # Attempt to retrieve data from the buffer
                data = buffer.get_nowait()
                if data is None:
                    # End of stream signal
                    return (bytes(), pyaudio.paComplete)
                else:
                    frames.append(data)
                if len(b"".join(frames)) >= frame_count * 2:  # 2 bytes per frame
                    break
            except asyncio.QueueEmpty:
                # Buffer underrun, send silent frames and wait for more data
                return (bytes(frame_count * 2), pyaudio.paContinue)
        
        # Concatenate all the frames and trim to the required frame_count
        output_data = b"".join(frames)[:frame_count * 2]
        return (output_data, pyaudio.paContinue)
    
    # Set the callback function to the stream
    stream._stream_callback = callback
    
    async def main(text: str, voice: str):
        stream.start_stream()
    
        # Start the coroutine that fetches and buffers audio data
        buffer_coro = fetch_and_buffer_audio(text, voice)
        await buffer_coro  # Will complete when "None" is placed indicating end of audio
    
        # Cleanup
        while stream.is_active():
            # Wait for the stream callback to finish processing
            await asyncio.sleep(0.1)
    
        stream.stop_stream()
        stream.close()
        p.terminate()
    
    if __name__ == "__main__":
        # Run the asyncio event loop
        asyncio.run(main(TEXT * 100, VOICE))
    

    With this approach, the fetch_and_buffer_audio coroutine fetches audio data from the TTS service, converts each chunk of audio as it arrives, and places it in an asyncio.Queue. The callback function, when called by pyaudio, retrieves data from this queue without waiting, providing silent frames if necessary (to avoid gaps) until more data arrives. When the end of the TTS audio is detected, a None is placed in the queue to indicate the end of the stream.

    Please remember to adjust the audio parameters (such as rate) to match the output of your TTS engine, and handle the potential for underrun in the callback as needed. The sample code might need further adjustments depending on the specifics of the edge_tts library and the format of the TTS data you receive.

  3. User

    I launched the script but i hear nothing

  4. Assistant

    I apologize for the inconvenience. We will need to adjust the script to ensure audio is played correctly.

    The issue might be due to incorrect handling of the queue or audio parameters. Let's modify the script to use the callback method properly. Here is an updated version of the script that should work:

    import asyncio
    import pyaudio
    from io import BytesIO
    from pydub import AudioSegment
    import edge_tts
    
    TEXT = "Hello World!"
    VOICE = "en-US-JennyNeural"
    
    # We're assuming a certain format, channels, and rate
    FORMAT = pyaudio.paInt16
    CHANNELS = 1
    RATE = 24000  # Adjust based on the TTS output format
    
    p = pyaudio.PyAudio()
    
    # Use an asyncio Queue to handle audio data between the producer and consumer
    buffer = asyncio.Queue()
    
    async def produce_audio(text: str, voice: str):
        communicate = edge_tts.Communicate(text, voice)
        async for chunk in communicate.stream():
            if chunk["type"] == "audio":
                audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3")
                raw_data = audio_segment.set_frame_rate(RATE).set_channels(CHANNELS).set_sample_width(p.get_sample_size(FORMAT)).raw_data
                await buffer.put(raw_data)
            if chunk.get('end', False):
                await buffer.put(None)  # Signal the end of stream
                break
    
    def callback(in_data, frame_count, time_info, status):
        # Get data from the queue
        try:
            data = buffer.get_nowait()
            if data is None:
                return (None, pyaudio.paComplete)
        except asyncio.QueueEmpty:
            data = b'\x00' * frame_count * CHANNELS * 2  # silence if no data
        
        return (data, pyaudio.paContinue)
    
    # Create an audio stream with the callback
    stream = p.open(
        format=FORMAT,
        channels=CHANNELS,
        rate=RATE,
        output=True,
        stream_callback=callback
    )
    
    async def main(text, voice):
        # Start the audio stream
        stream.start_stream()
    
        # Start the producer coroutine
        producer_task = asyncio.create_task(produce_audio(text, voice))
    
        # Run the producer coroutine until it finishes
        await producer_task
    
        # Wait for the stream to finish playing
        while stream.is_active():
            await asyncio.sleep(0.1)
    
        # Stop and close the stream
        stream.stop_stream()
        stream.close()
    
        # Terminate the PyAudio object
        p.terminate()
    
    if __name__ == "__main__":
        asyncio.run(main(TEXT * 100, VOICE))  # Play the text-to-speech 100 times
    

    This script is designed to use the asyncio library and a PyAudio stream to play audio frames without gaps continuously. The produce_audio coroutine fetches the text-to-speech audio and puts the data into an asyncio.Queue. It also signals when the end of the stream is reached. The callback function pulls data from this queue and returns it to the stream for playback. If the queue is empty, it returns a buffer filled with silence to prevent underrun.

    Please make sure that the RATE, CHANNELS, and FORMAT match the output specifications of your TTS engine. The script assumes that the TTS engine output is MP3. You might need to adjust parameters accordingly.

    Additionally, there are necessary installations such as pydub and any dependencies that might be missing:

    pip install pydub asyncio
    

    Remember that edge_tts may also have its specific installation or setup requirements that must be satisfied for it to function correctly.

  5. User

    Still nothing, no audio, no sound...

  6. Assistant

    I apologize for the issues you're experiencing. To troubleshoot, let's start by verifying a few things:

    1. Ensure your speakers or headphones are connected, powered on, and the volume is up.
    2. Ensure pyaudio and edge_tts are installed correctly.
    3. Verify that edge_tts is receiving text and generating audio as expected.
    4. Make sure the audio sample rate provided by edge_tts matches the RATE in the script.
    5. Ensure there are no exceptions or errors being raised that might be silently caught.

    For debugging, you should add print statements or logging throughout your code to understand where it might be failing. Here's an example of how you could modify the produce_audio coroutine and the callback function:

    async def produce_audio(text: str, voice: str):
        communicate = edge_tts.Communicate(text, voice)
        # Insert a print statement here to make sure this part is reached.
        print("Fetching audio content from TTS engine...")
    
        async for chunk in communicate.stream():
            if chunk["type"] == "audio":
                audio_segment = AudioSegment.from_file(BytesIO(chunk["data"]), format="mp3")
                # Print the duration of the audio segment for reference.
                print(f"Received audio chunk with duration: {audio_segment.duration_seconds} seconds.")
                raw_data = audio_segment.raw_data
                await buffer.put(raw_data)
            
            if chunk.get('end', False):
                print("End of audio stream reached.")
                await buffer.put(None)  # Signal the end of stream
                break
    
    def callback(in_data, frame_count, time_info, status):
        # Print the status for debugging.
        print(f"Callback called with frame_count: {frame_count} - status: {status}")
        
        try:
            data = buffer.get_nowait()
            if data is None:
                print("End of data received. Finishing playback.")
                return (None, pyaudio.paComplete)
            else:
                print(f"Playing audio chunk of size: {len(data)}")
                return (data, pyaudio.paContinue)
        except asyncio.QueueEmpty:
            print("Buffer underrun. Sending silence.")
            data = b'\x00' * frame_count * CHANNELS * p.get_sample_size(FORMAT)  # silence if no data
            return (data, pyaudio.paContinue)
    

    If the print() statements show that audio chunks are being fetched and processed but you still hear no output, the issue could be in the PyAudio configuration or in the system audio settings.

    If you suspect it's a system-related issue, try playing a local audio file using pyaudio or another library to verify that audio output works correctly with your current setup.

    Here's a simple example to test your system's audio output with pydub and pyaudio:

    from pydub import AudioSegment
    from pydub.playback import play
    
    # Load a local mp3 file
    audio = AudioSegment.from_file("your_audio_file.mp3")
    
    # Play the audio
    play(audio)
    

    Make sure to replace "your_audio_file.mp3" with the path to an actual MP3 file on your system.

    Running this snippet will help confirm whether your system and your installation of pydub can successfully play audio. If this does not work, you'll need to diagnose and fix your audio output before attempting to stream TTS audio.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.