tem erro isto: /content/VistaDream…
- User
tem erro isto: /content/VistaDream /usr/local/lib/python3.11/dist-packages/natten/functional.py:48: FutureWarning:
torch.cuda.amp.custom_fwd(args...)is deprecated. Please usetorch.amp.custom_fwd(args..., device_type='cuda')instead. @custom_fwd(cast_inputs=torch.float16) /usr/local/lib/python3.11/dist-packages/natten/functional.py:60: FutureWarning:torch.cuda.amp.custom_bwd(args...)is deprecated. Please usetorch.amp.custom_bwd(args..., device_type='cuda')instead. @custom_bwd /usr/local/lib/python3.11/dist-packages/natten/functional.py:82: FutureWarning:torch.cuda.amp.custom_fwd(args...)is deprecated. Please usetorch.amp.custom_fwd(args..., device_type='cuda')instead. @custom_fwd(cast_inputs=torch.float16) /usr/local/lib/python3.11/dist-packages/natten/functional.py:93: FutureWarning:torch.cuda.amp.custom_bwd(args...)is deprecated. Please usetorch.amp.custom_bwd(args..., device_type='cuda')instead. @custom_bwd /usr/local/lib/python3.11/dist-packages/natten/functional.py:115: FutureWarning:torch.cuda.amp.custom_fwd(args...)is deprecated. Please usetorch.amp.custom_fwd(args..., device_type='cuda')instead. @custom_fwd(cast_inputs=torch.float16) /usr/local/lib/python3.11/dist-packages/natten/functional.py:127: FutureWarning:torch.cuda.amp.custom_bwd(args...)is deprecated. Please usetorch.amp.custom_bwd(args..., device_type='cuda')instead. @custom_bwd /usr/local/lib/python3.11/dist-packages/natten/functional.py:149: FutureWarning:torch.cuda.amp.custom_fwd(args...)is deprecated. Please usetorch.amp.custom_fwd(args..., device_type='cuda')instead. @custom_fwd(cast_inputs=torch.float16) /usr/local/lib/python3.11/dist-packages/natten/functional.py:160: FutureWarning:torch.cuda.amp.custom_bwd(args...)is deprecated. Please usetorch.amp.custom_bwd(args..., device_type='cuda')instead. @custom_bwd /content/VistaDream/tools/OneFormer/oneformer/modeling/pixel_decoder/msdeformattn.py:318: FutureWarning:torch.cuda.amp.autocast(args...)is deprecated. Please usetorch.amp.autocast('cuda', args...)instead. @autocast(enabled=False) 2025-02-13 19:00:00.386431: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered WARNING: All log messages before absl::InitializeLog() is called are written to STDERR E0000 00:00:1739473200.682484 16660 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered E0000 00:00:1739473200.764153 16660 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2025-02-13 19:00:01.335600: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. Loading config tools/OneFormer/configs/ade20k/dinat/coco_pretrain_oneformer_dinat_large_bs16_160k_1280x1280.yaml with yaml.unsafe_load. Your machine may be at risk if the file contains malicious content. Loading config tools/OneFormer/configs/ade20k/dinat/../Base-ADE20K-UnifiedSegmentation.yaml with yaml.unsafe_load. Your machine may be at risk if the file contains malicious content. /usr/local/lib/python3.11/dist-packages/fvcore/common/checkpoint.py:252: FutureWarning: You are usingtorch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. return torch.load(f, map_location=torch.device("cpu")) The checkpoint state_dict contains keys that are not used by the model: text_encoder.positional_embedding text_encoder.transformer.resblocks.0.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.0.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.0.ln_1.{bias, weight} text_encoder.transformer.resblocks.0.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.0.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.0.ln_2.{bias, weight} text_encoder.transformer.resblocks.1.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.1.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.1.ln_1.{bias, weight} text_encoder.transformer.resblocks.1.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.1.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.1.ln_2.{bias, weight} text_encoder.transformer.resblocks.2.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.2.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.2.ln_1.{bias, weight} text_encoder.transformer.resblocks.2.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.2.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.2.ln_2.{bias, weight} text_encoder.transformer.resblocks.3.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.3.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.3.ln_1.{bias, weight} text_encoder.transformer.resblocks.3.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.3.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.3.ln_2.{bias, weight} text_encoder.transformer.resblocks.4.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.4.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.4.ln_1.{bias, weight} text_encoder.transformer.resblocks.4.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.4.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.4.ln_2.{bias, weight} text_encoder.transformer.resblocks.5.attn.{in_proj_bias, in_proj_weight} text_encoder.transformer.resblocks.5.attn.out_proj.{bias, weight} text_encoder.transformer.resblocks.5.ln_1.{bias, weight} text_encoder.transformer.resblocks.5.mlp.c_fc.{bias, weight} text_encoder.transformer.resblocks.5.mlp.c_proj.{bias, weight} text_encoder.transformer.resblocks.5.ln_2.{bias, weight} text_encoder.ln_final.{bias, weight} text_encoder.token_embedding.weight text_projector.layers.0.{bias, weight} text_projector.layers.1.{bias, weight} prompt_ctx.weight Load default preset failed. [Errno 2] No such file or directory: '/content/VistaDream/presets/default.json' No presets found. No presets found. Total VRAM 15095 MB, total RAM 12979 MB Set vram state to: NORMAL_VRAM Always offload VRAM Device: cuda:0 Tesla T4 : native VAE dtype: torch.float32 Using pytorch cross attention Refiner unloaded. model_type EPS UNet ADM Dimension 2816 Using pytorch attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using pytorch attention in VAE extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'} left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids']) Base model loaded: /content/VistaDream/tools/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors VAE loaded: None Request to load LoRAs [] for model [/content/VistaDream/tools/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors]. Fooocus V2 Expansion: Vocab with 642 words. Fooocus Expansion engine loaded for cuda:0, use_fp16 = True. Requested to load SDXLClipModel Requested to load GPT2LMHeadModel Loading 2 new models [Fooocus Model Management] Moving model(s) has taken 0.58 seconds Started worker with PID 16660 Loading checkpoint shards: 100% 4/4 [00:09<00:00, 2.50s/it] /content/VistaDream/tools/Fooocus/modules/patch.py:465: FutureWarning: You are usingtorch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user viatorch.serialization.add_safe_globals. We recommend you start settingweights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. result = original_loader(*args, **kwargs) ^C - Assistant
As you provided a log output from a Python environment, it appears to contain multiple warnings, errors, and notices related to the use of libraries like PyTorch and TensorFlow, along with other components of your project. Below is a breakdown of the key items in the output and possible actions to address them:
Key Points from the Log:
-
FutureWarnings from PyTorch:
- You're seeing several
FutureWarningmessages indicating that functions liketorch.cuda.amp.custom_fwdandtorch.cuda.amp.custom_bwdare deprecated, and you should switch to usingtorch.amp.custom_fwdandtorch.amp.custom_bwdinstead. - Action: Update the code in the library (if you maintain it) or check if there's an updated version of the library that has already made these changes.
- You're seeing several
-
CUDNN and CUFFT Registration Errors:
- Messages like
Unable to register cuDNN factorysuggest there are issues with loading specific CUDA libraries. - Action: Ensure that your CUDA installation is correct and compatible with the current version of TensorFlow/PyTorch you are using. Sometimes, updating NVIDIA drivers or CUDA installations helps resolve these issues.
- Messages like
-
Configure Loading Warnings:
- Loading configurations with
yaml.unsafe_loadflags a potential security risk if the YAML files could contain malicious content. - Action: To mitigate risks, either review these YAML files thoroughly or avoid using
unsafe_loadif not necessary.
- Loading configurations with
-
Checkpoint Loading Warnings:
- Messages about
torch.loadandweights_only=Falseindicate security risks when unpickling data and that in future versions it may restrict functionality. - Action: For safety, adopt the practice of using
weights_only=Truefor loading models when feasible.
- Messages about
-
Checkpoint State Dictionary Keys Warning:
- This indicates that the state dictionary of the model being loaded contains keys that are not used by the model structure.
- Action: Check your model definitions and the checkpoints. If the keys are unnecessary, you may consider a model architecture change or checkpoint adjustment.
-
Missing File Error:
- The error
No such file or directory: '/content/VistaDream/presets/default.json'indicates a missing configuration file. - Action: Make sure that all necessary files are present in your directory, or create the missing
default.json.
- The error
-
General Environment Information:
- Your setup is functioning with a Tesla T4 GPU and outlines VRAM and RAM statuses. It's good to ensure that you are monitoring GPU utilization and memory efficiently.
Final Recommendations:
Make sure your libraries are updated to the latest versions compatible with your CUDA installation. Also, verify your code against the deprecation warnings and adjust where necessary. Note any paths to files that are missing, and ensure all necessary configurations and dependencies are appropriately set up.
If you continue to experience issues, providing more specifics about the project setup, including library versions and the execution environment, might help in diagnosing the problem further.
-
- User
fix erros
- Assistant
To resolve the errors and warnings you’re encountering, follow these steps systematically. They focus on addressing deprecated function usage, missing files, and configuration issues. Some of these corrections may require changes in your code, configurations, or updates to your libraries.
Step 1: Update Code for Deprecated Functions
-
Replace Deprecated Functions:
- Search the source code or the libraries that are causing the warnings and replace instances of the deprecated functions. For example, you need to change calls like:
@custom_fwd(cast_inputs=torch.float16)
to:
@custom_fwd(cast_inputs=torch.float16, device_type='cuda')Similarly for backward functions, switch from:
@custom_bwdto:
@custom_bwd(device_type='cuda')If you're using libraries that you do not control, check if there's an updated version available that includes these changes. If not, you may need to clone and modify the library manually as a temporary fix.
- Search the source code or the libraries that are causing the warnings and replace instances of the deprecated functions. For example, you need to change calls like:
Step 2: Update CUDA and TensorFlow/PyTorch
-
Ensure Compatibility of CUDA and Libraries:
- Check your installation of CUDA and ensure it matches the versions required by PyTorch and TensorFlow you are using. You can find the compatible versions on the PyTorch website and TensorFlow website.
-
Update Libraries: You can update your Python libraries using pip. You can do this in your terminal or Jupyter notebook:
pip install --upgrade torch torchvision torchaudio tensorflow
Step 3: Modify Configuration Loaders
- Change
yaml.unsafe_load:- When loading YAML files, if feasible, switch to a safer loading method if you are controlling the content or were informed that the files are safe:
import yaml with open('your_config_file.yaml', 'r') as file: config = yaml.safe_load(file)
Step 4: Address Missing Configuration Files
- Ensure Default JSON Exists:
- Locate
default.jsonor create a new one if it is missing. Ensure the path/content/VistaDream/presets/default.jsonis valid and that the file is present. - Here’s a basic example structure for
default.jsonyou might consider:
{ "some_setting": "value", "another_setting": 123 } - Locate
Step 5: Use Weights Loading with Caution
- Update Weights Loading:
- Wherever you use
torch.load, specifyweights_only=Trueif you can either by modifying the library code or your own scripts. Here’s a sample that reflects this adjustment:
model.load_state_dict(torch.load('path_to_your_model.pth', map_location='cpu', weights_only=True)) - Wherever you use
Step 6: Test Your Changes
- Run the Code:
- After making the above changes, rerun your code to ensure that the errors and warnings are resolved. Carefully inspect the output for any remaining or new issues.
Additional Considerations
- Check for Updates: It's always a good idea to check if any of the libraries you are using have releases that specifically address issues you are encountering.
- Report Issues: If you are using a library and encounter persistent issues, consider opening an issue on the library’s GitHub page for community assistance or to alert the maintainers.
These steps should help you clear out the current warnings and preparatory steps before your final model deployment or further development. If errors persist after following these corrections, please provide additional context, and I can assist further.
-
- User
from pipe.cfgs import load_cfg from pipe.c2f_recons import Pipeline
cfg = load_cfg(f'pipe/cfgs/basic.yaml') cfg.scene.input.rgb = 'data/sd_readingroom/color.png' vistadream = Pipeline(cfg) vistadream()
- Assistant
Your code snippet initializes a configuration for a pipeline, presumably in a deep learning or computer vision context, using a YAML configuration file. However, the context you provided doesn't specify any explicit errors, so let's cover a few potential issues you might encounter when running this code, along with suggested fixes:
Common Issues and Fixes
-
File Not Found Errors:
- If you encounter an error indicating that
pipe/cfgs/basic.yamlordata/sd_readingroom/color.pngcannot be found, ensure these paths are correct relative to your current working directory. - Fix: Check if the files exist in the specified locations. You can also print the current working directory in Python:
import os print(os.getcwd())
- If you encounter an error indicating that
-
Invalid Configuration in YAML:
- If the
load_cfgfunction encounters an error loading the YAML file, check for syntax errors or misconfigurations in thebasic.yamlfile. - Fix: Ensure the YAML structure is correct. Here’s a simple YAML structure as a reference:
scene: input: rgb: 'data/sd_readingroom/color.png'
- If the
-
Import Errors:
- If there’s an import error (e.g.,
ImportError: cannot import name 'load_cfg' from 'pipe.cfgs'), make sure that thepipemodule and itscfgsandc2f_reconsmodules can be found based on your Python path. - Fix: Ensure that the
pipedirectory is correctly structured (with an__init__.pyfile where necessary) and that your Python environment has access to this path. You may need to adjust your PYTHONPATH or set it up as a package.
- If there’s an import error (e.g.,
-
Pipeline Execution Issues:
- When calling
vistadream(), ensure that the__call__method is properly defined within thePipelineclass. If it raises exceptions, carefully examine the error message for insights. - Fix: If you encounter an issue while executing
vistadream(), inspect thePipelineclass implementation to see if the input needs to be formatted in a specific way or if any required parameters are missing.
- When calling
-
Check for Dependencies:
- If
pipe.c2f_recons.Pipelineorload_cfgdepends on certain Python libraries (e.g.,torch,opencv, etc.), make sure all dependencies are installed and compatible. - Fix: Use
pipto install any missing libraries. For instance:pip install torch torchvision # Include any other libraries as needed
- If
Example Usage
Make sure your script looks something like this, ensuring all parts are correct and any adjustments from above suggestions are made:
import os from pipe.cfgs import load_cfg from pipe.c2f_recons import Pipeline # Check the current working directory print("Current Working Directory:", os.getcwd()) # Load the configuration cfg = load_cfg('pipe/cfgs/basic.yaml') # Ensure this path is correct cfg.scene.input.rgb = 'data/sd_readingroom/color.png' # Ensure this path is correct # Initialize the pipeline vistadream = Pipeline(cfg) # Execute the pipeline result = vistadream() # Check if this runs without errorsDebugging Tips
- Debugging Output: If any errors occur, print relevant variables or states before the line causing the error.
- Error Logs: Capture full error messages including stack traces for better diagnosis.
- Documentation: If
pipe.cfgsorpipe.c2f_reconsare custom libraries, refer to their documentation (if available) for appropriate usage patterns.
If you encounter specific errors while running this code, please share the error messages, and I can help troubleshoot further.
-
- User
name: basic
model: sky: value: 1e5 # to update oneformer: ckpt: 'tools/OneFormer/checkpoints/coco_pretrain_1280x1280_150_16_dinat_l_oneformer_ade20k_160k.pth' yaml: 'tools/OneFormer/configs/ade20k/dinat/coco_pretrain_oneformer_dinat_large_bs16_160k_1280x1280.yaml' vlm: llava: ckpt: 'llava-hf/bakLlava-v1-hf' # downloaded from hugging face mde: dpt_pro: ckpt: 'tools/DepthPro/checkpoints/depth_pro.pt' paint: fooocus: ckpts: 'tools/Fooocus/models' optimize: sd: 'sd-legacy/stable-diffusion-v1-5' # downloaded from hugging face lcm: 'tools/StableDiffusion/lcm_ckpt/pytorch_lora_weights.safetensors'
scene: input: rgb: 'data/sd_readingroom/color.png' resize_long_edge: 512
outpaint: outpaint_selections: ['Left','Right','Top','Bottom'] outpaint_extend_times: 0.45 seed: 746805015404516437
traj: n_sample: 10 traj_type: 'spiral' # for 'rot' we suggest large n_sample like 25. near_percentage: 5 far_percentage: 95 traj_forward_ratio: 0.3 traj_backward_ratio: 0.7
gaussian: opt_iters_per_frame: 512
mcs: steps: 10 # among 50 total steps n_view: 8 rect_w: 0.7 gsopt_iters: 256
- Assistant
Your YAML configuration file appears to be for a pipeline that utilizes various models and settings required for processing images, particularly in the context of generative tasks or image manipulation. Below, I'll outline some best practices, potential issues, and clarifications that might help you ensure everything is configured correctly.
General Overview of the Configuration
-
Model Configuration:
- The
modelsection appears to configure multiple models involved in the process:- sky: References a checkpoint and YAML config for a model related to sky processing.
- vlm: Sets up a vision-language model (VLM) and specifies its checkpoint path.
- mde: Configures a model for depth estimation (MDE) with a specific checkpoint.
- paint: Sets a directory for the Fooocus model checkpoints.
- optimize: Specifies checkpoints for optimization tasks.
- The
-
Scene Configuration:
- This section specifies how the input image is handled and how the output should be generated.
- Input: Defines the input image to be processed, with a resizing configuration.
- Outpaint: Specifies options for outpainting the image, including selections for extensions and seed values.
- Trajectory (traj): Parameters related to the sampling trajectory for processing.
- Gaussian and MCS configurations: Settings for additional processing techniques.
Key Points to Ensure Configuration Works
-
File Paths:
- Ensure that all specified paths in the configuration file (
ckpt,yaml, and image paths) are correct and that the files exist. - You can use the following Python code snippet to quickly check if paths are valid:
import os paths_to_check = [ 'tools/OneFormer/checkpoints/coco_pretrain_1280x1280_150_16_dinat_l_oneformer_ade20k_160k.pth', 'tools/OneFormer/configs/ade20k/dinat/coco_pretrain_oneformer_dinat_large_bs16_160k_1280x1280.yaml', 'llava-hf/bakLlava-v1-hf', # Verify by downloading the model from Hugging Face 'tools/DepthPro/checkpoints/depth_pro.pt', 'tools/Fooocus/models', 'sd-legacy/stable-diffusion-v1-5', # Verify by downloading the model from Hugging Face 'tools/StableDiffusion/lcm_ckpt/pytorch_lora_weights.safetensors', 'data/sd_readingroom/color.png' ] for path in paths_to_check: if not os.path.exists(path): print(f"Path does not exist: {path}")
- Ensure that all specified paths in the configuration file (
-
Parameter Validity:
- Ensure that values are appropriate for their context, e.g.,:
resize_long_edgeshould be a positive integer to scale images correctly.outpaint_extend_timesshould typically be between 0 and 1 for percentage-related metrics.
- Verify that the values you set for trajectory types, optimization steps, and others are appropriate for your models and dataset.
- Ensure that values are appropriate for their context, e.g.,:
-
Dependencies:
- If you're using libraries specific to any of the models, ensure they are installed and up to date. E.g., PyTorch, transformers, etc. Use the following commands:
pip install torch torchvision transformers
- If you're using libraries specific to any of the models, ensure they are installed and up to date. E.g., PyTorch, transformers, etc. Use the following commands:
-
Read Documentation:
- Check the documentation for each model to understand the impact of each configuration option and any default settings that may affect performance or outcomes.
-
Testing:
- Once you've double-checked everything, run your pipeline with this configuration and monitor the output for any runtime errors or inconsistencies.
- If errors arise, capturing the full stack trace will be useful for diagnosing issues.
Example of Loading This Configuration in Code
If you've set up the above YAML correctly, you might load it into your pipeline as follows (extend this from the earlier example):
from pipe.cfgs import load_cfg from pipe.c2f_recons import Pipeline # Load the YAML configuration cfg = load_cfg('pipe/cfgs/basic.yaml') # Make sure this path is correct # Check for the input image cfg.scene.input.rgb = 'data/sd_readingroom/color.png' # Verify it's present # Initialize the pipeline with the loaded config vistadream = Pipeline(cfg) # Run the pipeline result = vistadream() # Monitor for errors during executionThis should effectively prepare your pipeline according to the configurations you've detailed. If you face any specific issues or errors during the execution, please provide those error messages for more precise troubleshooting help.
-