Skip to content
TheChatArchiveThe Chat Archive
o4-mini-2025-04-16· OpenAIHugging Face LMArena Human Preference 140K

What would it take to implement an autoencoder in a less ml focused language like pascal…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwq-32b’s answer.
  1. User

    What would it take to implement an autoencoder in a less ml focused language like pascal or modula-2?

  2. Assistant

    Implementing an autoencoder in a language like Pascal or Modula‐2 boils down to re‐building from scratch all the “plumbing” that ML-focused frameworks (TensorFlow, PyTorch, etc.) give you for free:

    1. Linear algebra primitives
      • dynamic/static arrays or records of arrays for vectors/matrices
      • matrix‐vector and matrix‐matrix multiply routines
      • element‐wise vector ops (add, subtract, scale)

    2. Random initialization
      • uniform or Gaussian RNG for weights & biases

    3. Activation functions (and their derivatives)
      • sigmoid(x) = 1/(1+e^(−x)), tanh, ReLU, …
      • sigmoid′(x) = sigmoid(x)*(1−sigmoid(x)), etc.

    4. Forward propagation
      • hidden = σ(W1·input + b1)
      • output = σ(W2·hidden + b2)

    5. Loss / cost
      • mean squared error: ½‖output − input‖²

    6. Backpropagation
      • compute δ₂ = (output − input) ⊙ σ′(z₂)
      • compute δ₁ = (W2ᵀ·δ₂) ⊙ σ′(z₁)
      • accumulate gradients ∇W, ∇b

    7. Parameter update (gradient descent)
      • W ← W − η·∇W, b ← b − η·∇b

    8. Training loop
      • repeat forward/backprop on each training sample (or mini-batch) for N epochs


    What you’ll actually write in Pascal / Modula-2

    • Data structures:

    type
      TVector = array of Double;
      TMatrix = array of TVector;
    

    • Linear algebra routines:

    function MatVecMul(const M: TMatrix; const v: TVector): TVector;
    var i,j: Integer;
    begin
      SetLength(Result, Length(M));
      for i := 0 to High(M) do begin
        Result[i] := 0;
        for j := 0 to High(v) do
          Result[i] := Result[i] + M[i][j]*v[j];
      end;
    end;
    

    • Activation and derivative:

    function Sigmoid(x: Double): Double;
    begin
      Result := 1.0 / (1.0 + Exp(-x));
    end;
    
    function SigmoidPrime(x: Double): Double;
    var s: Double;
    begin
      s := Sigmoid(x);
      Result := s*(1 - s);
    end;
    

    • Forward pass:

    procedure Forward(const input: TVector; out hidden, output: TVector);
    var z1, z2: TVector;
    begin
      z1 := VecAdd(MatVecMul(W1, input), b1);
      hidden := Apply(z1, Sigmoid);
      z2 := VecAdd(MatVecMul(W2, hidden), b2);
      output := Apply(z2, Sigmoid);
    end;
    

    • Backprop & update (outline):

    // compute deltas
    delta2 := ElementWiseMul( VecSub(output, input), Apply(z2, SigmoidPrime) );
    delta1 := ElementWiseMul( MatVecMul(Transpose(W2), delta2), Apply(z1, SigmoidPrime) );
    
    // gradient step
    W2 := MatSub(W2, ScalarMul(eta, OuterProduct(delta2, hidden)));
    b2 := VecSub(b2, ScalarMul(eta, delta2));
    W1 := MatSub(W1, ScalarMul(eta, OuterProduct(delta1, input)));
    b1 := VecSub(b1, ScalarMul(eta, delta1));
    

    Practical tips

    • Start very small: 2–3 hidden nodes, tiny dataset.
    • Debug forward pass → numerical outputs match hand‐calculations.
    • Verify gradients via finite differences.
    • Use single‐precision or double‐precision Real as needed.
    • For performance: consider unrolling loops or calling a BLAS/C library via the language’s FFI.


    In summary

    There’s no magic: you need to re‐implement
    • matrix math,
    • random weight init,
    • activations + derivatives,
    • feed-forward & backprop loops,
    • parameter updates.

    Once you have those building blocks, an autoencoder (or any small neural net) in Pascal/Modula-2 becomes just a few hundred lines of code. 🚀

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.

Identical first prompt