Skip to content
TheChatArchiveThe Chat Archive
Qwenqwen-max-2025-01-25· Alibaba (Qwen)Hugging Face LMArena Human Preference 140K

give me a Matlab code for windy grid world problem in reinforcement learning

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4.1-2025-04-14’s answer.
  1. User

    give me a Matlab code for windy grid world problem in reinforcement learning

  2. Qwen Assistant

    Certainly! Below is an implementation of the Windy Grid World problem in MATLAB. This is a classic reinforcement learning problem where an agent learns to navigate a grid while dealing with "wind" that pushes it upward. The goal is to find the optimal path from a start state to a goal state.

    The algorithm used here is Q-learning, which is a model-free reinforcement learning algorithm. The code includes comments for clarity.


    MATLAB Code: Windy Grid World with Q-Learning

    % Windy Grid World Problem using Q-Learning
    
    % Parameters
    gridSize = [7, 10]; % Grid dimensions (rows x columns)
    startState = [4, 1]; % Starting position (row, col)
    goalState = [4, 8];  % Goal position (row, col)
    wind = [0, 0, 0, 1, 1, 1, 2, 2, 1, 0]; % Wind strength for each column
    numActions = 4; % Up, Down, Left, Right
    actions = [-1, 0; 1, 0; 0, -1; 0, 1]; % Action effects on (row, col)
    
    % Hyperparameters
    alpha = 0.5; % Learning rate
    gamma = 0.9; % Discount factor
    epsilon = 0.1; % Exploration rate
    numEpisodes = 500; % Number of episodes for training
    
    % Initialize Q-table
    Q = zeros(gridSize(1), gridSize(2), numActions);
    
    % Helper function to check if a state is within bounds
    function valid = isValidState(state, gridSize)
        valid = state(1) >= 1 && state(1) <= gridSize(1) && ...
                state(2) >= 1 && state(2) <= gridSize(2);
    end
    
    % Training loop
    for episode = 1:numEpisodes
        % Initialize state
        state = startState;
        
        while ~isequal(state, goalState)
            % Epsilon-greedy policy for action selection
            if rand < epsilon
                % Explore: Choose random action
                action = randi(numActions);
            else
                % Exploit: Choose best action
                [~, action] = max(Q(state(1), state(2), :));
            end
            
            % Apply wind effect
            nextRow = state(1) + actions(action, 1) - wind(state(2));
            nextCol = state(2) + actions(action, 2);
            
            % Ensure the next state is within bounds
            nextRow = max(1, min(nextRow, gridSize(1)));
            nextCol = max(1, min(nextCol, gridSize(2)));
            nextState = [nextRow, nextCol];
            
            % Compute reward
            reward = -1; % Default reward for each step
            if isequal(nextState, goalState)
                reward = 0; % Reward for reaching the goal
            end
            
            % Update Q-value
            Q(state(1), state(2), action) = Q(state(1), state(2), action) + ...
                alpha * (reward + gamma * max(Q(nextState(1), nextState(2), :)) - ...
                Q(state(1), state(2), action));
            
            % Move to the next state
            state = nextState;
        end
    end
    
    % Testing the learned policy
    disp('Optimal Path:');
    state = startState;
    path = state;
    while ~isequal(state, goalState)
        [~, action] = max(Q(state(1), state(2), :)); % Choose best action
        nextRow = state(1) + actions(action, 1) - wind(state(2));
        nextCol = state(2) + actions(action, 2);
        
        % Ensure the next state is within bounds
        nextRow = max(1, min(nextRow, gridSize(1)));
        nextCol = max(1, min(nextCol, gridSize(2)));
        state = [nextRow, nextCol];
        path = [path; state];
    end
    disp(path);
    

    Explanation of the Code

    1. Grid and Wind Setup:

      • The grid is defined as a 7x10 matrix.
      • The wind array specifies how much the agent is pushed upward in each column.
    2. Q-Learning Algorithm:

      • The Q-table is initialized to zeros.
      • For each episode, the agent starts at the startState and follows an epsilon-greedy policy to balance exploration and exploitation.
      • The Q-values are updated using the Bellman equation: Q(s,a)←Q(s,a)+α⋅[r+γ⋅max⁡a′Q(s′,a′)−Q(s,a)]Q(s, a) \leftarrow Q(s, a) + \alpha \cdot \left[ r + \gamma \cdot \max_{a'} Q(s', a') - Q(s, a) \right]
    3. Wind Effect:

      • After taking an action, the agent is pushed upward by the wind corresponding to its current column.
    4. Testing the Policy:

      • After training, the agent follows the learned policy to find the optimal path from the start state to the goal state.

    Output

    The output will display the sequence of states (coordinates) that the agent follows to reach the goal. For example:

    Optimal Path:
        4    1
        3    1
        2    1
        ...
        4    8
    

    This represents the trajectory taken by the agent from the start to the goal.


    Notes

    • You can adjust the hyperparameters (alpha, gamma, epsilon, numEpisodes) to experiment with different learning behaviors.
    • The wind array can be modified to simulate different environments.
    • If you want to visualize the grid or the path, you can use MATLAB's plotting functions.

    Let me know if you need further clarification or enhancements!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.