Skip to content
TheChatArchiveThe Chat Archive
Coherecommand-a-03-2025· CohereHugging Face LMArena Human Preference 140K

Make this script onnx campatible at least schema import numpy as np import pandas as pd…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-235b-a22b’s answer.
  1. User

    Make this script onnx campatible at least schema

    import numpy as np import pandas as pd from sklearn.base import BaseEstimator, TransformerMixin from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.preprocessing import StandardScaler === Transformer qui reshape en (n_samples, 2, size) ===

    class ReshapeBack(BaseEstimator, TransformerMixin): def init(self, n_samples): self.n_samples = n_samples # on passe n_samples à l'init

    def fit(self, X, y=None): self.n_features = X.shape[1] # devrait être 2 self.size = X.shape[0] // self.n_samples return self

    def transform(self, X): reshaped = X.reshape(self.n_samples, self.size, self.n_features) return reshaped.transpose(0, 2, 1) # (n_samples, 2, size)

    === Exemple de données ===

    df = pd.DataFrame({ 't10': [[1, 2, 3], [4, 5, 6], [7, 8, 9]], 'temp': [[10, 20, 30], [40, 50, 60], [70, 80, 90]], }) === Pipeline ===

    pipe = Pipeline([ ('flatten_and_scale', Pipeline([ ('flatten', ColumnTransformer([ ('feature', FlattenListsPaired(), ['t10', 'temp']), ])), ('scaler', StandardScaler()), ])), ('reshape_back', ReshapeBack(n_samples=len(df))) ]) === Exécution ===

    result = pipe.fit_transform(df)

    print(result.shape) # (3, 2, 3) print(result)

  2. Cohere Assistant

    To make the provided script ONNX-compatible, we need to address several key aspects:

    1. Custom Transformers: The ReshapeBack class needs a custom ONNX converter.
    2. Input Data Handling: The initial DataFrame contains lists, which are not directly compatible with ONNX. We'll separate the flattening step.
    3. ONNX Schema Compliance: Ensure all operations can be represented in the ONNX schema.

    Here’s the revised script with ONNX compatibility:

    import numpy as np
    import pandas as pd
    from sklearn.base import BaseEstimator, TransformerMixin
    from sklearn.pipeline import Pipeline
    from sklearn.compose import ColumnTransformer
    from sklearn.preprocessing import StandardScaler
    from skl2onnx import convert_sklearn, update_registered_converter
    from skl2onnx.common.data_types import FloatTensorType
    from skl2onnx.algebra.onnx_ops import OnnxReshape, OnnxTranspose
    from onnx import TensorProto
    
    # === Custom Transformer: Flatten Lists (Python Preprocessing) ===
    class FlattenListsPaired(BaseEstimator, TransformerMixin):
        def fit(self, X, y=None):
            return self
    
        def transform(self, X):
            if isinstance(X, pd.DataFrame):
                X = X.values
            n_features = X.shape[1]
            flattened = np.concatenate([X[:, i].ravel() for i in range(n_features)], axis=0).reshape(-1, n_features)
            return flattened.astype(np.float32)
    
    # === Custom Transformer: ReshapeBack ===
    class ReshapeBack(BaseEstimator, TransformerMixin):
        def __init__(self, n_samples):
            self.n_samples = n_samples
    
        def fit(self, X, y=None):
            self.n_features = X.shape[1]
            self.size = X.shape[0] // self.n_samples
            return self
    
        def transform(self, X):
            return X.reshape(self.n_samples, self.size, self.n_features).transpose(0, 2, 1)
    
    # === ONNX Converter for ReshapeBack ===
    def reshape_back_converter(scope, operator, container):
        op = operator.raw_operator
        op_type = 'Reshape'
        op_name = scope.get_unique_operator_name(op_type)
        
        # Calculate target shape: (n_samples, size, n_features)
        target_shape = np.array([op.n_samples, op.size, op.n_features], dtype=np.int64)
        
        # Add target shape as initializer
        shape_name = scope.get_unique_variable_name('target_shape')
        container.add_initializer(shape_name, TensorProto.INT64, target_shape.shape, target_shape)
        
        # Reshape operation
        reshaped = scope.get_unique_variable_name('reshaped')
        container.add_node(op_type, [operator.input_full_names[0], shape_name], [reshaped], name=op_name)
        
        # Transpose operation (0, 2, 1)
        attrs = {'name': scope.get_unique_operator_name('Transpose'), 'perm': [0, 2, 1]}
        container.add_node('Transpose', reshaped, operator.output_full_names, **attrs)
    
    # Register the converter
    update_registered_converter(ReshapeBack, 'CustomReshapeBack', reshape_back_converter)
    
    # === Example Data ===
    df = pd.DataFrame({
        't10': [[1, 2, 3], [4, 5, 6], [7, 8, 9]],
        'temp': [[10, 20, 30], [40, 50, 60], [70, 80, 90]],
    })
    
    # === Separate Flattening (Python Preprocessing) ===
    flatten_transformer = ColumnTransformer([
        ('flatten', FlattenListsPaired(), ['t10', 'temp']),
    ])
    X_flattened = flatten_transformer.fit_transform(df)
    
    # === ONNX-Compatible Pipeline ===
    onnx_pipe = Pipeline([
        ('scaler', StandardScaler()),
        ('reshape_back', ReshapeBack(n_samples=len(df)))
    ])
    
    # Fit and transform using flattened data
    result_sklearn = onnx_pipe.fit_transform(X_flattened)
    
    # === ONNX Conversion ===
    initial_type = [('float_input', FloatTensorType([X_flattened.shape[0], X_flattened.shape[1]]))]
    onnx_model = convert_sklearn(onnx_pipe, initial_types=initial_type, target_opset=11)
    
    # Save the model
    with open('pipeline.onnx', 'wb') as f:
        f.write(onnx_model.SerializeToString())
    
    print("ONNX model saved to 'pipeline.onnx'")
    

    Key Changes and Explanations:

    1. FlattenListsPaired:
      • This transformer converts DataFrame columns containing lists into a flat NumPy array.
      • Note: This step must be done in Python as ONNX cannot handle DataFrames with list-like objects directly.
    2. ReshapeBack Converter:
      • A custom converter is registered using update_registered_converter.
      • The converter performs two ONNX operations:
        • Reshape: Transforms the input into (n_samples, size, n_features).
        • Transpose: Rearranges dimensions to (n_samples, n_features, size).
    3. Pipeline Separation:
      • The original pipeline is split into two parts:
        • Python Preprocessing: FlattenListsPaired (handled outside ONNX).
        • ONNX Pipeline: StandardScaler and ReshapeBack (converted to ONNX).
    4. ONNX Conversion:
      • The convert_sklearn function generates an ONNX model from the compatible pipeline.
      • The model is saved in the ONNX format.

    Usage Notes:

    • Input Data: The ONNX model expects flattened numerical data as input (shape: (total_elements, n_features)).
    • Inference: To use the ONNX model, load it with an ONNX runtime (e.g., onnxruntime) and pass the flattened data.

    This approach ensures the core processing pipeline is ONNX-compatible while handling the initial data transformation in Python.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.