Following this instruction : 1. Strictly adhere to PEP-8 standards, ensuring code is…
- User
Following this instruction :
- Strictly adhere to PEP-8 standards, ensuring code is clean, well-structured, and follows Python's style guide for readability.
- Use appropriate modules like
typingfor type hints, ensuring code clarity, maintainability, and leveraging Python's standard libraries effectively. - Focus on time, space, robustness, scalability; write optimized, efficient code that can handle large datasets and scale seamlessly.
- Implement comprehensive error handling, catching exceptions gracefully, providing meaningful error messages, and ensuring code robustness under unexpected conditions.
- Ensure the developer agent writes advanced, maintainable Python code, balancing performance and readability while adhering to best practices.
TASK : use we have advanced develop model visualization tools save in directory(all image all .html format) each and every 1d,2d,3d,4d,
generalised any model , nlp or computer vision properly legend , labelling , colouring plots , very professional ,resuable code
reference code :
import torch import torch.nn as nn from transformers import GPT2LMHeadModel, GPT2Tokenizer import plotly.graph_objs as go from plotly.subplots import make_subplots from typing import Any, Dict import logging
Configure logging for debugging and error tracking
logging.basicConfig(level=logging.INFO) logger = logging.getLogger(name)
class DynamicTensorModel(nn.Module): """ A PyTorch model that accepts a recursive key input and returns a tensor of varying dimensions. It also visualizes the tensor using Plotly based on its dimensionality. """
def __init__(self) -> None: super(DynamicTensorModel, self).__init__() def forward(self, key_input: Dict[str, Any]) -> torch.Tensor: """ Processes the recursive key input to generate a tensor. Args: key_input (Dict[str, Any]): A nested dictionary input. Returns: torch.Tensor: A tensor with dimensions based on the key input. """ try: tensor = self.generate_tensor(key_input) self.visualize_tensor(tensor, key_input) return tensor except Exception as e: logger.error(f"Error in forward pass: {e}") raise def generate_tensor(self, key_input: Dict[str, Any], depth: int = 1) -> torch.Tensor: """ Recursively processes the key input to generate a tensor with varying dimensions. Args: key_input (Dict[str, Any]): A nested dictionary input. depth (int, optional): Current depth of recursion. Defaults to 1. Returns: torch.Tensor: A tensor with dimensions based on recursion depth. """ if not isinstance(key_input, dict): raise ValueError("Key input must be a dictionary.") if not key_input: raise ValueError("Key input dictionary is empty.") # Base case: when the value is not a dictionary for key, value in key_input.items(): if isinstance(value, dict): return self.generate_tensor(value, depth + 1) else: # Create a tensor with dimension based on depth size = 10 # arbitrary size for each dimension dims = min(depth, 4) # Limit to 4D shape = tuple([size] * dims) logger.info(f"Generating a {dims}D tensor with shape {shape} based on key '{key}'.") return torch.randn(shape) # If all values are dictionaries, increase depth size = 10 dims = min(depth, 4) shape = tuple([size] * dims) logger.info(f"Generating a {dims}D tensor with shape {shape} based on nested keys.") return torch.randn(shape) def visualize_tensor(self, tensor: torch.Tensor, key_input: Dict[str, Any]) -> None: """ Visualizes the tensor using Plotly based on its dimensionality. Args: tensor (torch.Tensor): The tensor to visualize. key_input (Dict[str, Any]): The original key input for dynamic naming. """ dim = tensor.dim() logger.info(f"Visualizing a tensor with {dim} dimensions.") if dim == 1: self._plot_1d(tensor, key_input) elif dim == 2: self._plot_2d(tensor, key_input) elif dim == 3: self._plot_3d(tensor, key_input) elif dim == 4: self._plot_4d(tensor, key_input) else: logger.warning(f"Tensors with {dim} dimensions are not supported for visualization.") def _plot_1d(self, tensor: torch.Tensor, key_input: Dict[str, Any]) -> None: """Generates a simple 1D scatter plot.""" try: x = torch.arange(tensor.size(0)).numpy() y = tensor.numpy() fig = go.Figure(data=go.Scatter(x=x, y=y, mode='markers', name='1D Scatter')) title = self._generate_title(key_input, "1D Scatter Plot") fig.update_layout(title=title, xaxis_title='Index', yaxis_title='Value') fig.show() logger.info("1D scatter plot generated successfully.") except Exception as e: logger.error(f"Error plotting 1D tensor: {e}") def _plot_2d(self, tensor: torch.Tensor, key_input: Dict[str, Any]) -> None: """Generates a standard 2D scatter plot.""" try: if tensor.size(1) < 2: raise ValueError("2D tensor must have at least two features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() fig = go.Figure(data=go.Scatter(x=x, y=y, mode='markers', name='2D Scatter')) title = self._generate_title(key_input, "2D Scatter Plot") fig.update_layout(title=title, xaxis_title='Feature 1', yaxis_title='Feature 2') fig.show() logger.info("2D scatter plot generated successfully.") except Exception as e: logger.error(f"Error plotting 2D tensor: {e}") def _plot_3d(self, tensor: torch.Tensor, key_input: Dict[str, Any]) -> None: """Generates an interactive 3D scatter plot.""" try: if tensor.size(1) < 3: raise ValueError("3D tensor must have at least three features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() z = tensor[:, 2].numpy() fig = go.Figure(data=go.Scatter3d( x=x, y=y, z=z, mode='markers', marker=dict(size=5), name='3D Scatter' )) title = self._generate_title(key_input, "3D Scatter Plot") fig.update_layout(title=title, scene=dict( xaxis_title='Feature 1', yaxis_title='Feature 2', zaxis_title='Feature 3' )) fig.show() logger.info("3D scatter plot generated successfully.") except Exception as e: logger.error(f"Error plotting 3D tensor: {e}") def _plot_4d(self, tensor: torch.Tensor, key_input: Dict[str, Any]) -> None: """Generates an advanced 3D scatter plot with a dynamic legend based on the key input.""" try: if tensor.size(1) < 4: raise ValueError("4D tensor must have at least four features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() z = tensor[:, 2].numpy() color_feature = tensor[:, 3].numpy() # Create a color scale based on the fourth dimension colorscale = 'Viridis' fig = go.Figure(data=go.Scatter3d( x=x, y=y, z=z, mode='markers', marker=dict( size=5, color=color_feature, colorscale=colorscale, opacity=0.8, colorbar=dict(title='Feature 4') ), name='4D Scatter' )) title = self._generate_title(key_input, "4D Scatter Plot with Dynamic Legend") fig.update_layout(title=title, scene=dict( xaxis_title='Feature 1', yaxis_title='Feature 2', zaxis_title='Feature 3' )) fig.show() logger.info("4D scatter plot generated successfully.") except Exception as e: logger.error(f"Error plotting 4D tensor: {e}") def _generate_title(self, key_input: Dict[str, Any], plot_type: str) -> str: """ Generates a descriptive title based on the key input. Args: key_input (Dict[str, Any]): The original key input. plot_type (str): The type of plot. Returns: str: A descriptive title. """ flat_keys = self._flatten_keys(key_input) key_str = "_".join(flat_keys) title = f"{plot_type} - Keys: {key_str}" return title def _flatten_keys(self, d: Dict[str, Any], parent_key: str = '', sep: str = '_') -> list: """ Flattens nested dictionary keys. Args: d (Dict[str, Any]): The dictionary to flatten. parent_key (str, optional): The base key string. Defaults to ''. sep (str, optional): Separator between keys. Defaults to '_'. Returns: list: A list of flattened key strings. """ items = [] for k, v in d.items(): new_key = f"{parent_key}{sep}{k}" if parent_key else k if isinstance(v, dict): items.extend(self._flatten_keys(v, new_key, sep=sep)) else: items.append(new_key) return items - Assistant
import os import logging from typing import Any, Dict, Optional, Union import torch import torch.nn as nn import plotly.graph_objs as go from plotly.subplots import make_subplots # Configure logging for debugging and error tracking logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) class ModelVisualizer: """ A utility class for visualizing model outputs in 1D, 2D, 3D, and 4D. Supports saving visualizations as image files or HTML files. """ def __init__(self, save_dir: str = "visualizations") -> None: """ Initializes the ModelVisualizer. Args: save_dir (str, optional): Directory to save the visualizations. Defaults to "visualizations". """ self.save_dir = save_dir os.makedirs(self.save_dir, exist_ok=True) logger.info(f"Visualization directory set to: {self.save_dir}") def visualize( self, data: Union[torch.Tensor, Any], key_input: Optional[Dict[str, Any]] = None, file_format: str = "html", ) -> None: """ Determines the dimensionality of the data and generates the appropriate plot. Args: data (Union[torch.Tensor, Any]): The model output to visualize. key_input (Optional[Dict[str, Any]], optional): Metadata for dynamic naming. Defaults to None. file_format (str, optional): Format to save the visualization ('html' or 'png'). Defaults to "html". """ try: tensor = self._validate_and_convert_data(data) dim = tensor.dim() logger.info(f"Data has {dim} dimensions.") if dim == 1: self._plot_1d(tensor, key_input, file_format) elif dim == 2: self._plot_2d(tensor, key_input, file_format) elif dim == 3: self._plot_3d(tensor, key_input, file_format) elif dim == 4: self._plot_4d(tensor, key_input, file_format) else: logger.warning(f"Visualization for {dim}D data is not supported.") except Exception as e: logger.error(f"Failed to visualize data: {e}") def _validate_and_convert_data(self, data: Union[torch.Tensor, Any]) -> torch.Tensor: """ Validates and converts input data to a torch.Tensor. Args: data (Union[torch.Tensor, Any]): The input data. Raises: TypeError: If data cannot be converted to a torch.Tensor. Returns: torch.Tensor: Converted tensor. """ if isinstance(data, torch.Tensor): return data try: tensor = torch.tensor(data) logger.info("Data successfully converted to torch.Tensor.") return tensor except Exception as e: logger.error("Input data must be a torch.Tensor or convertible to torch.Tensor.") raise TypeError("Invalid data type for visualization.") from e def _plot_1d( self, tensor: torch.Tensor, key_input: Optional[Dict[str, Any]], file_format: str ) -> None: """Generates and saves a 1D scatter plot.""" try: x = torch.arange(tensor.size(0)).numpy() y = tensor.numpy() fig = go.Figure(data=go.Scatter(x=x, y=y, mode='markers', name='1D Scatter')) title = self._generate_title(key_input, "1D Scatter Plot") fig.update_layout(title=title, xaxis_title='Index', yaxis_title='Value') self._save_figure(fig, title, file_format) logger.info("1D scatter plot generated and saved successfully.") except Exception as e: logger.error(f"Error plotting 1D tensor: {e}") def _plot_2d( self, tensor: torch.Tensor, key_input: Optional[Dict[str, Any]], file_format: str ) -> None: """Generates and saves a 2D scatter plot.""" try: if tensor.size(1) < 2: raise ValueError("2D tensor must have at least two features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() fig = go.Figure(data=go.Scatter(x=x, y=y, mode='markers', name='2D Scatter')) title = self._generate_title(key_input, "2D Scatter Plot") fig.update_layout(title=title, xaxis_title='Feature 1', yaxis_title='Feature 2') self._save_figure(fig, title, file_format) logger.info("2D scatter plot generated and saved successfully.") except Exception as e: logger.error(f"Error plotting 2D tensor: {e}") def _plot_3d( self, tensor: torch.Tensor, key_input: Optional[Dict[str, Any]], file_format: str ) -> None: """Generates and saves a 3D scatter plot.""" try: if tensor.size(1) < 3: raise ValueError("3D tensor must have at least three features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() z = tensor[:, 2].numpy() fig = go.Figure(data=go.Scatter3d( x=x, y=y, z=z, mode='markers', marker=dict(size=5, color='blue'), name='3D Scatter' )) title = self._generate_title(key_input, "3D Scatter Plot") fig.update_layout(title=title, scene=dict( xaxis_title='Feature 1', yaxis_title='Feature 2', zaxis_title='Feature 3' )) self._save_figure(fig, title, file_format) logger.info("3D scatter plot generated and saved successfully.") except Exception as e: logger.error(f"Error plotting 3D tensor: {e}") def _plot_4d( self, tensor: torch.Tensor, key_input: Optional[Dict[str, Any]], file_format: str ) -> None: """Generates and saves a 4D scatter plot with dynamic coloring.""" try: if tensor.size(1) < 4: raise ValueError("4D tensor must have at least four features for plotting.") x = tensor[:, 0].numpy() y = tensor[:, 1].numpy() z = tensor[:, 2].numpy() color_feature = tensor[:, 3].numpy() fig = go.Figure(data=go.Scatter3d( x=x, y=y, z=z, mode='markers', marker=dict( size=5, color=color_feature, colorscale='Viridis', opacity=0.8, colorbar=dict(title='Feature 4') ), name='4D Scatter' )) title = self._generate_title(key_input, "4D Scatter Plot") fig.update_layout(title=title, scene=dict( xaxis_title='Feature 1', yaxis_title='Feature 2', zaxis_title='Feature 3' )) self._save_figure(fig, title, file_format) logger.info("4D scatter plot generated and saved successfully.") except Exception as e: logger.error(f"Error plotting 4D tensor: {e}") def _generate_title(self, key_input: Optional[Dict[str, Any]], plot_type: str) -> str: """ Generates a descriptive title based on the key input. Args: key_input (Optional[Dict[str, Any]]): Metadata for the plot. plot_type (str): The type of plot. Returns: str: A descriptive title. """ if key_input: flat_keys = self._flatten_keys(key_input) key_str = "_".join(flat_keys) title = f"{plot_type} - Keys: {key_str}" else: title = plot_type return title def _flatten_keys( self, d: Dict[str, Any], parent_key: str = '', sep: str = '_' ) -> list: """ Flattens nested dictionary keys. Args: d (Dict[str, Any]): The dictionary to flatten. parent_key (str, optional): The base key string. Defaults to ''. sep (str, optional): Separator between keys. Defaults to '_'. Returns: list: A list of flattened key strings. """ items = [] for k, v in d.items(): new_key = f"{parent_key}{sep}{k}" if parent_key else k if isinstance(v, dict): items.extend(self._flatten_keys(v, new_key, sep=sep)) else: items.append(new_key) return items def _save_figure(self, fig: go.Figure, title: str, file_format: str) -> None: """ Saves the Plotly figure in the specified format. Args: fig (go.Figure): The Plotly figure to save. title (str): The title of the plot, used for the filename. file_format (str): The format to save ('html' or 'png'). Raises: ValueError: If an unsupported file format is specified. """ safe_title = "".join(c if c.isalnum() else "_" for c in title) filename = os.path.join(self.save_dir, f"{safe_title}.{file_format}") if file_format.lower() == "html": fig.write_html(filename) logger.info(f"Figure saved as HTML: {filename}") elif file_format.lower() in {"png", "jpeg", "jpg"}: fig.write_image(filename) logger.info(f"Figure saved as image: {filename}") else: logger.error(f"Unsupported file format: {file_format}") raise ValueError("Unsupported file format. Choose 'html' or an image format like 'png'.") # Example usage if __name__ == "__main__": # Initialize the visualizer visualizer = ModelVisualizer(save_dir="model_visualizations") # Example 1D tensor tensor_1d = torch.randn(50) visualizer.visualize(tensor_1d, key_input={"model": "ExampleModel", "layer": "output"}, file_format="html") # Example 2D tensor tensor_2d = torch.randn(100, 2) visualizer.visualize(tensor_2d, key_input={"model": "ExampleModel", "layer": "hidden"}, file_format="html") # Example 3D tensor tensor_3d = torch.randn(200, 3) visualizer.visualize(tensor_3d, key_input={"model": "ExampleModel", "layer": "intermediate"}, file_format="html") # Example 4D tensor tensor_4d = torch.randn(300, 4) visualizer.visualize(tensor_4d, key_input={"model": "ExampleModel", "layer": "embedding"}, file_format="html")Explanation
The provided
ModelVisualizerclass is designed to handle visualization of model outputs with varying dimensions (1D to 4D). Here's a breakdown of its features and functionalities:-
Initialization:
- The class initializes with a
save_dirparameter that specifies where the visualizations will be saved. It ensures the directory exists usingos.makedirs.
- The class initializes with a
-
Visualization Method:
visualize: This is the primary method that accepts model output data, optional metadata (key_input), and the desired file format (htmlor image formats likepng). It determines the tensor's dimensionality and calls the appropriate plotting method.
-
Data Validation:
_validate_and_convert_data: Ensures that the input data is atorch.Tensor. If not, it attempts to convert it. If conversion fails, it raises aTypeError.
-
Plotting Methods:
_plot_1dto_plot_4d: Each of these methods handles plotting for tensors with respective dimensions using Plotly. They generate scatter plots with appropriate labels, titles, and color schemes.- Each plotting method generates a title based on the provided
key_inputfor better traceability and saves the figure using the_save_figuremethod.
-
Title Generation:
_generate_title: Creates a descriptive title incorporating metadata fromkey_inputif provided._flatten_keys: Helper method to flatten nested dictionaries into a list of key strings for dynamic title generation.
-
Saving Figures:
_save_figure: Saves the generated Plotly figure in the specified format (htmlor image formats likepng). It sanitizes the title to create a safe filename.
-
Error Handling:
- Comprehensive error handling is implemented across methods to catch and log exceptions, ensuring robustness.
-
Example Usage:
- The
__main__section provides example usage of theModelVisualizerwith tensors of different dimensions. It demonstrates how to visualize and save outputs from various layers of a hypothetical model.
- The
This class adheres to PEP-8 standards, includes type hints for better clarity and maintainability, utilizes Python's standard libraries effectively, and ensures that visualizations are both professional and reusable.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.