gpt-4.1-2025-04-14· OpenAI
LMArena Human Preference 140K
give me a Matlab code for windy grid world problem in reinforcement learning
- User
give me a Matlab code for windy grid world problem in reinforcement learning
- Assistant
Certainly! The Windy Gridworld is a classic reinforcement learning problem introduced by Sutton and Barto in their book "Reinforcement Learning: An Introduction". Below is a simple MATLAB implementation using the SARSA algorithm.
Windy Gridworld in MATLAB (SARSA)
% Windy Gridworld using SARSA clc; clear; close all; % Gridworld parameters rows = 7; cols = 10; start = [4,1]; goal = [4,8]; % Wind strength for each column wind = [0 0 0 1 1 1 2 2 1 0]; % RL parameters alpha = 0.5; % Learning rate gamma = 1.0; % Discount factor epsilon = 0.1; % Epsilon-greedy episodes = 200; % Actions: up, down, left, right actions = [ -1 0; 1 0; 0 -1; 0 1 ]; nA = size(actions,1); % Q-table: rows x cols x actions Q = zeros(rows, cols, nA); steps_per_episode = zeros(episodes,1); for ep = 1:episodes state = start; % Choose action using epsilon-greedy a = epsilon_greedy(Q, state, epsilon, nA); steps = 0; while ~isequal(state, goal) steps = steps + 1; % Take action, observe next state and reward next_state = state + actions(a,:); % Apply wind next_state(1) = next_state(1) - wind(state(2)); % Boundaries next_state(1) = min(max(next_state(1),1),rows); next_state(2) = min(max(next_state(2),1),cols); % Reward reward = -1; % Next action next_a = epsilon_greedy(Q, next_state, epsilon, nA); % SARSA update Q(state(1),state(2),a) = Q(state(1),state(2),a) + ... alpha * (reward + gamma * Q(next_state(1),next_state(2),next_a) - Q(state(1),state(2),a)); % Move to next state state = next_state; a = next_a; end steps_per_episode(ep) = steps; end % Plot learning curve figure; plot(1:episodes, steps_per_episode, 'LineWidth',2); xlabel('Episode'); ylabel('Steps per Episode'); title('Windy Gridworld: SARSA Learning Curve'); grid on; % --- Helper function: epsilon-greedy policy --- function a = epsilon_greedy(Q, state, epsilon, nA) if rand < epsilon a = randi(nA); else [~, a] = max(Q(state(1), state(2), :)); a = squeeze(a); end end
How it works:
- The agent starts at
startand tries to reachgoal. - The wind in each column pushes the agent upward.
- The agent learns using the SARSA algorithm.
- The plot shows how the number of steps per episode decreases as the agent learns.
You can further modify this code to use Q-learning, add diagonal moves, or visualize the policy. Let me know if you want any of these features!
- The agent starts at
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.