Skip to content
TheChatArchiveThe Chat Archive
Amazon Novaamazon.nova-pro-v1:0· AmazonHugging Face LMArena Human Preference 140K

实时语音识别 from funasr import AutoModel chunksize = [0, 10, 5] #[0, 10, 5] 600ms, [0, 8, 4]…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-sonnet-4-20250514’s answer.
  1. User

    实时语音识别 from funasr import AutoModel

    chunk_size = [0, 10, 5] #[0, 10, 5] 600ms, [0, 8, 4] 480ms encoder_chunk_look_back = 4 #number of chunks to lookback for encoder self-attention decoder_chunk_look_back = 1 #number of encoder chunks to lookback for decoder cross-attention

    model = AutoModel(model="paraformer-zh-streaming", model_revision="v2.0.4")

    import soundfile import os

    wav_file = os.path.join(model.model_path, "example/asr_example.wav") speech, sample_rate = soundfile.read(wav_file) chunk_stride = chunk_size[1] * 960 # 600ms

    cache = {} total_chunk_num = int(len((speech)-1)/chunk_stride+1) for i in range(total_chunk_num): speech_chunk = speech[ichunk_stride:(i+1)chunk_stride] is_final = i == total_chunk_num - 1 res = model.generate(input=speech_chunk, cache=cache, is_final=is_final, chunk_size=chunk_size, encoder_chunk_look_back=encoder_chunk_look_back, decoder_chunk_look_back=decoder_chunk_look_back) print(res) 注:chunk_size为流式延时配置,[0,10,5]表示上屏实时出字粒度为1060=600ms,未来信息为560=300ms。每次推理输入为600ms(采样点数为16000*0.6=960),输出为对应文字,最后一个语音片段输入需要设置is_final=True来强制输出最后一个字。

    能否添加vad_model和punc_model?

  2. Amazon Nova Assistant

    可以在实时语音识别中添加语音活动检测(VAD)和标点符号预测模型(Punc)来提高识别的准确性和流畅性。以下是如何集成这两个模型的示例代码:

    from funasr import AutoModel
    import soundfile
    import os
    
    # VAD and Punc model initialization
    vad_model = AutoModel(model="vad-zh", model_revision="v1.0.0")
    punc_model = AutoModel(model="punc-zh", model_revision="v1.0.0")
    
    # ASR model initialization
    chunk_size = [0, 10, 5]  # [0, 10, 5] 600ms, [0, 8, 4] 480ms
    encoder_chunk_look_back = 4  # number of chunks to lookback for encoder self-attention
    decoder_chunk_look_back = 1  # number of encoder chunks to lookback for decoder cross-attention
    
    asr_model = AutoModel(model="paraformer-zh-streaming", model_revision="v2.0.4")
    
    # Load audio file
    wav_file = os.path.join(asr_model.model_path, "example/asr_example.wav")
    speech, sample_rate = soundfile.read(wav_file)
    chunk_stride = chunk_size[1] * 960  # 600ms
    
    cache = {}
    total_chunk_num = int((len(speech) - 1) / chunk_stride + 1)
    final_text = ""
    
    for i in range(total_chunk_num):
        speech_chunk = speech[i * chunk_stride:(i + 1) * chunk_stride]
        is_final = i == total_chunk_num - 1
        
        # VAD to check if the chunk contains speech
        vad_res = vad_model.generate(input=speech_chunk)
        if vad_res['vad']:  # If speech is detected
            asr_res = asr_model.generate(
                input=speech_chunk, 
                cache=cache, 
                is_final=is_final, 
                chunk_size=chunk_size, 
                encoder_chunk_look_back=encoder_chunk_look_back, 
                decoder_chunk_look_back=decoder_chunk_look_back
            )
            # Punc to add punctuation to the ASR result
            punc_res = punc_model.generate(input=asr_res['text'])
            final_text += punc_res['text']
            print(punc_res['text'])
    
    print("Final text:", final_text)
    

    解释

    1. VAD模型:用于检测音频片段中是否包含语音。只有当VAD检测到语音时,才会进行ASR处理。这可以减少无效的计算和提高识别的准确性。
    2. Punc模型:用于在ASR结果中添加标点符号,使得输出文本更加流畅和易读。

    注意事项

    • 确保VAD和Punc模型的加载和使用与ASR模型类似。
    • 根据实际需求调整VAD和Punc模型的参数和配置。

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.