ubuntu@ubuntu-virtual-machine:~/Downloads$ python3 transcribe.py…
- User
ubuntu@ubuntu-virtual-machine:~/Downloads$ python3 transcribe.py /home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/core/io.py:43: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. torchaudio.set_audio_backend("soundfile") /home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/pipelines/speaker_verification.py:43: UserWarning: torchaudio._backend.get_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. backend = torchaudio.get_audio_backend() INFO:speechbrain.utils.quirks:Applied quirks (see
speechbrain.utils.quirks): [allow_tf32, disable_jit_profiling] INFO:speechbrain.utils.quirks:Excluded quirks specified by theSB_DISABLE_QUIRKSenvironment (comma-separated list): [] /home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/pipelines/speaker_verification.py:45: UserWarning: Module 'speechbrain.pretrained' was deprecated, redirecting to 'speechbrain.inference'. Please update your script. This is a change from SpeechBrain 1.0. See: https://github.com/speechbrain/speechbrain/releases/tag/v1.0.0 from speechbrain.pretrained import ( /home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/pipelines/speaker_verification.py:53: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. torchaudio.set_audio_backend(backend) /home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/tasks/segmentation/mixins.py:37: UserWarning:torchaudio.backend.common.AudioMetaDatahas been moved totorchaudio.AudioMetaData. Please update the import path. from torchaudio.backend.common import AudioMetaDataProcessing with base model... No language specified, language will be first be detected for each audio file (increases inference time). /home/ubuntu/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/cloud_io.py:47: You are using
torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. Model was trained with pyannote.audio 0.0.1, yours is 3.1.1. Bad things might happen unless you revert pyannote.audio to 0.x. Model was trained with torch 1.10.0+cu102, yours is 2.5.1. Bad things might happen unless you revert torch to 1.x. Detected language: en (0.72) in first 30s of audio... Error processing base: string indices must be integersProcessing with base.en model... /home/ubuntu/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/cloud_io.py:47: You are using
torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. Model was trained with pyannote.audio 0.0.1, yours is 3.1.1. Bad things might happen unless you revert pyannote.audio to 0.x. Model was trained with torch 1.10.0+cu102, yours is 2.5.1. Bad things might happen unless you revert torch to 1.x. Error processing base.en: string indices must be integersProcessing with small model... No language specified, language will be first be detected for each audio file (increases inference time). /home/ubuntu/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/cloud_io.py:47: You are using
torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. Model was trained with pyannote.audio 0.0.1, yours is 3.1.1. Bad things might happen unless you revert pyannote.audio to 0.x. Model was trained with torch 1.10.0+cu102, yours is 2.5.1. Bad things might happen unless you revert torch to 1.x. Detected language: en (0.81) in first 30s of audio... ^CError processing small: string indices must be integersProcessing with small.en model... /home/ubuntu/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/cloud_io.py:47: You are using
torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. Model was trained with pyannote.audio 0.0.1, yours is 3.1.1. Bad things might happen unless you revert pyannote.audio to 0.x. Model was trained with torch 1.10.0+cu102, yours is 2.5.1. Bad things might happen unless you revert torch to 1.x.transcribe.py
import whisperx import os from datetime import datetime import time import torch
Path to your audio file
audio_path = os.path.expanduser("~/Downloads/audio.mp3")
Create main output directory
output_base_dir = "whisperx_outputs" os.makedirs(output_base_dir, exist_ok=True)
Function to transcribe and save results
def transcribe_with_model(model_name, device="cpu"): print(f"\nProcessing with {model_name} model...") start_time = time.time()
# Create model-specific directory model_dir = os.path.join(output_base_dir, model_name) os.makedirs(model_dir, exist_ok=True) # Load model with compute_type set to "float32" model = whisperx.load_model(model_name, device=device, compute_type="float32") # Transcribe the audio segments, info = model.transcribe(audio_path) # Extract the full transcription text full_text = ' '.join([segment['text'] for segment in segments]) # Save output timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") output_file = os.path.join(model_dir, f"transcript_{timestamp}.txt") with open(output_file, 'w', encoding='utf-8') as f: f.write(full_text) end_time = time.time() print(f"{model_name} processing time: {end_time - start_time:.2f} seconds") print(f"Output saved to: {output_file}")Process with all models including medium and medium.en
models = ["base", "base.en", "small", "small.en", "medium", "medium.en"]
for model_name in models: try: transcribe_with_model(model_name) except Exception as e: print(f"Error processing {model_name}: {e}")
print("\nAll processing complete! Check the whisperx_outputs directory for results.")
- Assistant
The errors and warnings you're encountering are primarily due to version incompatibilities between the libraries used to train your models and the versions currently installed in your environment. Let's break down the issues and provide a comprehensive solution to resolve them.
1. Understanding the Issues
a. Version Mismatches
-
Pyannote.audio:
- Model Training Version:
0.0.1 - Current Installed Version:
3.1.1
- Model Training Version:
-
Torch:
- Model Training Version:
1.10.0+cu102 - Current Installed Version:
2.5.1
- Model Training Version:
These mismatches can lead to unexpected behavior, as newer versions of libraries often introduce changes that are not backward compatible.
b. Deprecation Warnings
-
Torchaudio Backend Warnings:
- Functions like
torchaudio.set_audio_backendandtorchaudio.get_audio_backendhave been deprecated.
- Functions like
-
SpeechBrain Module Warning:
- The module
speechbrain.pretrainedhas been deprecated and redirected tospeechbrain.inference.
- The module
These warnings indicate that the code is using outdated API calls, which may not be supported in future releases.
c. Runtime Error
- Error Message:
Error processing base: string indices must be integers
This suggests that the code is trying to access elements in a data structure (likely expecting a dictionary) using string keys, but it's actually encountering a string or improperly structured data.
2. Solutions
Option 1: Downgrade Libraries to Compatible Versions
To align the library versions with those used during model training, you can downgrade
pyannote.audioandtorchto the specified versions.a. Create and Activate a New Virtual Environment
It's best practice to create a separate virtual environment to avoid conflicts with other projects.
# Create a new virtual environment named 'transcribe_env' python3 -m venv transcribe_env # Activate the virtual environment source transcribe_env/bin/activateb. Downgrade PyTorch
You need to install the specific versions of PyTorch that are compatible with your model.
# Install PyTorch 1.10.0 with CUDA 10.2 (if you have GPU support) pip install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio==0.10.0+cu102 -f https://download.pytorch.org/whl/torch_stable.html # If you're using CPU only, install the CPU version pip install torch==1.10.0 torchvision==0.11.1 torchaudio==0.10.0c. Downgrade Pyannote.audio
pip install pyannote.audio==0.0.1Note: Pyannote.audio
0.0.1is an extremely old version and might not be available on PyPI. If it's unavailable, you might need to find the source repository or consider other options.d. Install Other Dependencies
Ensure you have all other required libraries installed at compatible versions.
pip install whisperx==version # Replace 'version' with the compatible version if necessary pip install speechbrain==version # Replace 'version' as neededCaution: Downgrading to very old versions might introduce security vulnerabilities or other issues. Proceed with caution and ensure that the environment is secured.
Option 2: Update the Code to Be Compatible with Newer Library Versions
If downgrading libraries is not feasible or you prefer to use the latest versions, you'll need to modify your code to be compatible with the current library APIs.
a. Address Deprecation Warnings
-
Remove Deprecated Torchaudio Backend Settings:
The warnings indicate that setting the torchaudio backend is now a no-op. You can safely remove these lines from your code or any library code if you have access.
# Remove or comment out lines like: torchaudio.set_audio_backend("soundfile") -
Update SpeechBrain Imports:
Replace deprecated
speechbrain.pretrainedimports withspeechbrain.inference.# Old import from speechbrain.pretrained import some_module # New import from speechbrain.inference import some_module
b. Investigate the Runtime Error
The error
string indices must be integerssuggests thatsegmentsis not a list of dictionaries as expected. To debug:# After transcribing, check the type and content of 'segments' segments, info = model.transcribe(audio_path) print(type(segments)) print(segments)Depending on the structure, adjust the code accordingly. For example, if
segmentsis a dictionary with a specific key containing the list of segments:# Example adjustment full_text = ' '.join([segment['text'] for segment in segments['segments']])Ensure that the structure aligns with the current version's expected output.
Option 3: Use Docker or Containerization
If managing library versions becomes too cumbersome, consider using Docker to create a containerized environment with the exact dependencies required.
a. Create a Dockerfile
FROM python:3.10 # Install specific versions RUN pip install torch==1.10.0 torchvision==0.11.1 torchaudio==0.10.0 RUN pip install pyannote.audio==0.0.1 whisperx==<version> speechbrain==<version> # Copy your script COPY transcribe.py /app/transcribe.py WORKDIR /app # Set the entrypoint ENTRYPOINT ["python3", "transcribe.py"]b. Build and Run the Docker Container
# Build the Docker image docker build -t transcribe_env . # Run the container docker run --rm -v ~/Downloads:/Downloads transcribe_envThis approach ensures that your environment remains consistent and isolated from other projects.
3. Recommendations
-
Check Model Compatibility: Ensure that the models you're using are compatible with the versions of the libraries you intend to install. Sometimes, retraining or updating the models with newer library versions might be necessary.
-
Update Libraries Gradually: Instead of making drastic version changes, consider incrementally updating libraries and testing at each step to identify where incompatibilities arise.
-
Consult Documentation and Release Notes:
- Pyannote.audio: Documentation
- Torch: PyTorch Releases
- WhisperX: Refer to the WhisperX GitHub Repository for specific version compatibilities and updates.
-
Handle Deprecations Proactively: Regularly update your codebase to accommodate library updates, reducing the accumulation of deprecated code sections.
-
Security Considerations: Be mindful of the security warnings related to
torch.load. Ensure that you only load trusted models to prevent potential security vulnerabilities.You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly...To address this, you can set
weights_only=Trueif it aligns with your use case:model = torch.load('model_path', weights_only=True)However, ensure that this change is compatible with how your models are saved and loaded.
4. Final Thoughts
Resolving version incompatibilities can be challenging, especially when dealing with models trained on significantly older library versions. Assess the effort required to downgrade libraries versus updating your codebase. Sometimes, migrating to newer versions with updated code provides longer-term benefits, despite the initial investment in modifying the code.
If you continue to face issues after attempting the above solutions, consider reaching out to the maintainers of the specific libraries or seeking assistance on platforms like GitHub Issues related to those projects.
By addressing the version mismatches and updating your code to align with the current library APIs, you should be able to resolve the errors and successfully transcribe your audio files.
-