Given this data structure: class PPOMemory: def init(self, numsteps, numenvs,…
- User
Given this data structure:
class PPOMemory: def init(self, num_steps, num_envs, state_shape, batch_size): self.states = np.zeros((num_steps, num_envs, *state_shape), dtype=np.float32) self.actions = np.zeros((num_steps, num_envs), dtype=np.int32) self.probs = np.zeros((num_steps, num_envs), dtype=np.float32) self.vals = np.zeros((num_steps, num_envs), dtype=np.float32) self.rewards = np.zeros((num_steps, num_envs), dtype=np.float32) self.dones = np.zeros((num_steps, num_envs), dtype=bool) self.batch_size = batch_size
I need a new array for each one, which has first dimension num_steps*num_envs and elements are sorted by the num_envs dimension
- Assistant
To rearrange your data so that each element is grouped by the
num_envsdimension when flattened, you can swap the first two axes of your arrays and then reshape them. This effectively orders the elements so that all steps for a particular environment are contiguous in the flattened array.Here's how you can do it for each array in your
PPOMemoryclass:# Swap axes 0 and 1, so shape becomes (num_envs, num_steps, ...) # Then reshape to combine the first two dimensions states_reordered = self.states.swapaxes(0, 1).reshape(-1, *self.states.shape[2:]) actions_reordered = self.actions.swapaxes(0, 1).reshape(-1, *self.actions.shape[2:]) probs_reordered = self.probs.swapaxes(0, 1).reshape(-1, *self.probs.shape[2:]) vals_reordered = self.vals.swapaxes(0, 1).reshape(-1, *self.vals.shape[2:]) rewards_reordered = self.rewards.swapaxes(0, 1).reshape(-1, *self.rewards.shape[2:]) dones_reordered = self.dones.swapaxes(0, 1).reshape(-1, *self.dones.shape[2:])This code does the following for each array:
- Swaps the first two axes (
swapaxes(0, 1)) to bring thenum_envsdimension to the first axis. - Reshapes the array to combine the first two dimensions (
reshape(-1, ...)), which flattens thenum_envsandnum_stepsdimensions into one.
As a result, the first dimension of each new array is
num_steps * num_envs, and the elements are ordered so that all steps for each environment are grouped together.Example:
If you have
num_steps = 3andnum_envs = 2, the original ordering inself.stateswould be:[ [state_0_env_0, state_0_env_1], [state_1_env_0, state_1_env_1], [state_2_env_0, state_2_env_1] ]After reordering:
[ state_0_env_0, state_1_env_0, state_2_env_0, state_0_env_1, state_1_env_1, state_2_env_1 ]This groups all the steps of
env_0together, followed by all the steps ofenv_1.Final Answer:
Swap the first two axes of each array and then reshape to flatten, so elements are grouped by
num_envs:reordered_array = original_array.swapaxes(0, 1).reshape(-1, ...) - Swaps the first two axes (