Skip to content
TheChatArchiveThe Chat Archive
gpt-4.1-2025-04-14· OpenAIHugging Face LMArena Human Preference 140K

give me a Matlab code for windy grid world problem in reinforcement learning

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen-max-2025-01-25’s answer.
  1. User

    give me a Matlab code for windy grid world problem in reinforcement learning

  2. Assistant

    Certainly! The Windy Gridworld is a classic reinforcement learning problem introduced by Sutton and Barto in their book "Reinforcement Learning: An Introduction". Below is a simple MATLAB implementation using the SARSA algorithm.


    Windy Gridworld in MATLAB (SARSA)

    % Windy Gridworld using SARSA
    clc; clear; close all;
    
    % Gridworld parameters
    rows = 7;
    cols = 10;
    start = [4,1];
    goal = [4,8];
    
    % Wind strength for each column
    wind = [0 0 0 1 1 1 2 2 1 0];
    
    % RL parameters
    alpha = 0.5;      % Learning rate
    gamma = 1.0;      % Discount factor
    epsilon = 0.1;    % Epsilon-greedy
    episodes = 200;
    
    % Actions: up, down, left, right
    actions = [ -1 0; 1 0; 0 -1; 0 1 ];
    nA = size(actions,1);
    
    % Q-table: rows x cols x actions
    Q = zeros(rows, cols, nA);
    
    steps_per_episode = zeros(episodes,1);
    
    for ep = 1:episodes
        state = start;
        % Choose action using epsilon-greedy
        a = epsilon_greedy(Q, state, epsilon, nA);
        steps = 0;
        while ~isequal(state, goal)
            steps = steps + 1;
            % Take action, observe next state and reward
            next_state = state + actions(a,:);
            % Apply wind
            next_state(1) = next_state(1) - wind(state(2));
            % Boundaries
            next_state(1) = min(max(next_state(1),1),rows);
            next_state(2) = min(max(next_state(2),1),cols);
            % Reward
            reward = -1;
            % Next action
            next_a = epsilon_greedy(Q, next_state, epsilon, nA);
            % SARSA update
            Q(state(1),state(2),a) = Q(state(1),state(2),a) + ...
                alpha * (reward + gamma * Q(next_state(1),next_state(2),next_a) - Q(state(1),state(2),a));
            % Move to next state
            state = next_state;
            a = next_a;
        end
        steps_per_episode(ep) = steps;
    end
    
    % Plot learning curve
    figure;
    plot(1:episodes, steps_per_episode, 'LineWidth',2);
    xlabel('Episode');
    ylabel('Steps per Episode');
    title('Windy Gridworld: SARSA Learning Curve');
    grid on;
    
    % --- Helper function: epsilon-greedy policy ---
    function a = epsilon_greedy(Q, state, epsilon, nA)
        if rand < epsilon
            a = randi(nA);
        else
            [~, a] = max(Q(state(1), state(2), :));
            a = squeeze(a);
        end
    end
    

    How it works:

    • The agent starts at start and tries to reach goal.
    • The wind in each column pushes the agent upward.
    • The agent learns using the SARSA algorithm.
    • The plot shows how the number of steps per episode decreases as the agent learns.

    You can further modify this code to use Q-learning, add diagonal moves, or visualize the policy. Let me know if you want any of these features!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.