Skip to content
TheChatArchiveThe Chat Archive
Metallama-3.3-70b-instruct· MetaHugging Face LMArena Human Preference 140K

Write script to to plot bar chart for metrics in yaml files for different models input:…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-pro’s answer.
  1. User

    Write script to to plot bar chart for metrics in yaml files for different models

    input:

    • list of yaml files to compare

    output:

    • png image of pretty bar chart

    Example of yaml file:

    PR AUC: 0.727
    PR AUC macro prod weights: 0.723
    PR AUC macro train weights: 0.673
    
    
  2. Meta Assistant

    To accomplish the task of plotting a bar chart for metrics in YAML files for different models, we will follow these steps:

    1. Read YAML Files: Load the data from each YAML file into a Python dictionary.
    2. Extract Metrics: Extract the metrics from each dictionary.
    3. Plot Bar Chart: Use a plotting library (like matplotlib) to create a bar chart comparing the metrics across different models.

    Below is a Python script that performs these steps. This script assumes you have matplotlib and pyyaml installed. If not, you can install them using pip:

    pip install matplotlib pyyaml
    

    Here's the script:

    import yaml
    import matplotlib.pyplot as plt
    import numpy as np
    
    def load_yaml_file(file_path):
        """Loads a YAML file into a Python dictionary."""
        with open(file_path, 'r') as file:
            return yaml.safe_load(file)
    
    def extract_metrics(yaml_data):
        """Extracts metrics from a dictionary loaded from a YAML file."""
        return yaml_data
    
    def plot_bar_chart(models, metrics):
        """Plots a bar chart comparing metrics across different models."""
        # Assuming all models have the same metrics
        metric_names = list(metrics[0].keys())
        
        # Prepare data for plotting
        data = [[metrics[model][metric] for model in models] for metric in metric_names]
        
        # Set up the figure and axis
        fig, ax = plt.subplots(figsize=(10, 6))
        
        # Set the x-axis ticks
        x = np.arange(len(models))
        width = 0.8 / len(metric_names)
        
        # Plot bars for each metric
        for i, metric in enumerate(metric_names):
            ax.bar(x + i * width, [metrics[model][metric] for model in models], width, label=metric)
        
        # Set labels and title
        ax.set_ylabel('Value')
        ax.set_title('Comparison of Model Metrics')
        ax.set_xticks(x + width * (len(metric_names) - 1) / 2)
        ax.set_xticklabels(models)
        ax.legend()
        
        # Save the plot to a PNG file
        plt.savefig('model_metrics_comparison.png', bbox_inches='tight')
        plt.show()
    
    def main(yaml_files):
        """Main function to load YAML files, extract metrics, and plot a bar chart."""
        models = [file.split('.')[0] for file in yaml_files]  # Assuming model name is the file name without extension
        metrics = {model: load_yaml_file(file) for model, file in zip(models, yaml_files)}
        plot_bar_chart(models, metrics)
    
    if __name__ == "__main__":
        yaml_files = ['model1.yaml', 'model2.yaml', 'model3.yaml']  # Replace with your YAML file names
        main(yaml_files)
    

    Replace ['model1.yaml', 'model2.yaml', 'model3.yaml'] with the paths to your actual YAML files. This script assumes that the model name can be derived from the YAML file name (without the extension), and it plots a bar chart where each group of bars represents a model, and each bar within a group represents a metric.

    Please note, the script above assumes that all YAML files have the same structure and metrics. If the metrics vary across files, you might need to adjust the script to handle missing metrics or to dynamically determine which metrics to plot.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.