in unity's ML Agents do I need to modify my PlayerController which works with player…
- User
in unity's ML Agents do I need to modify my PlayerController which works with player input? or do I need to create a new script and make references in the code to each other?
- Assistant
In Unity's ML Agents, you typically don't modify your existing PlayerController script directly. Instead, you create a new script that inherits from the Agent class. This new script will control the agent's interaction with the environment, receive observations, and execute actions determined by the trained model.
You can reuse some code from your PlayerController in this new agent script if it's related to agent's movement, decision-making, etc. However, player input-related code should be separated as it doesn't apply to the agent.
Here's a high-level overview of the steps you need to follow:
-
Create a new C# script that inherits from the Agent class (from "using Unity.MLAgents;") and attach it to your game object that you want to train.
-
Implement the required methods such as
OnEpisodeBegin(),CollectObservations(), andOnActionReceived(). These methods are responsible for resetting the environment, collecting input observations for training, and executing the necessary actions based on the output of the trained model. -
If needed, you can keep your PlayerController script for player control during testing and development. However, make sure that the new agent script and PlayerController script do not interfere with each other. You may use conditional checks like "#if UNITY_EDITOR" in your PlayerController script to disable certain sections of the code when the build is running in the Unity Editor to avoid conflicts.
-
In your new agent script, you can reference and utilize any necessary components or functionality from your PlayerController script. This allows you to use the same logic for movement or other actions, reducing code duplication. However, you may need to make some adjustments to ensure compatibility with the Agent class.
-
After implementing the new agent script, set up your ML-Agent training environment following Unity's ML-Agents documentation, create your training configuration file, and start training your agent using the ML-Agents Python package.
Remember that your new agent script should focus on agent logic rather than player input. When you want to test your trained agent, simply enable the new agent script and disable the PlayerController script, so that the agent acts based on the trained model instead of player input.
-
- User
Well, in that case, make me an agent for my PlayerController. What is important to me is a minimally working code that can start training agent training. I repeat, the code is not supposed to be useful, but it must work and the training must run in ML Agents. Observation can be anything, be it the position of the player or anything, the output can be anything too. I just need to get the training going, even if it's crazy. Further, based on your code, I myself will write what I need.
Here is my PlayerController:
using System.Collections; using System.Collections.Generic; using UnityEngine; using UnityEngine.UI; using Unity.MLAgents; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators; using UnityEditor.U2D;
public enum CharacterGroup { Player, Ally, Enemy }
public enum ControlMode { Mouse, Joystick, AI }
public class PlayerController : MonoBehaviour { #region Public Variables
public CharacterGroup group; public ShootingAgent shootingAgent; public OpponentGenerator opponentGenerator; public VariableJoystick variableJoystick; public ControlMode currentControlMode = ControlMode.Mouse; // Default control mode is mouse public GameObject bulletPrefab; public string nickname = "lovekpek"; public string brawlerName = "Nita"; public int maxHealth = 3000; // Add a new variable for maximum health public int damage = 750; // Add a new variable for damage public bool damagePiercing = true; public float movementSpeed = 5f; public float aimingDistance = 5f; public float bulletSpeed = 10f; public int maxBullets = 3; public float reloadTime = 2f; public float shotDelay = 0.5f; // Delay between consecutive shots public float bulletWidth = 0.1f; // Bullet width [SerializeField] private HealthBar _healthbar; public Text healthText; // Reference to the UI text component for health display [SerializeField] private BulletBar _bulletbar; public Text bulletsText; public int currentHealth = 3000; public int currentBullets = 3; public Text speedText; #endregion #region Private Variables private GameObject sightLine; private bool isAiming; private bool canShoot = true; // Flag to allow shooting private Vector2 aimDirection; private Vector2 lineEndPosition; private float currentDistance; private float reloadTimer; private float shotTimer; // Timer for shot delay private bool hasFiredRecently; // New flag private Vector2 movementInput; private float currentSpeed; private bool isPlayerControlled = false; private Collider2D lastCollision; // Class-level variable to store the last collision // Joystick control variables private float horizontalInput; private float verticalInput; #endregion #region Unity Callbacks private void Start() { InitializePlayer(); } private void Update() { if (IsPlayerControlled()) { HandleAimingAndShooting(); HandleReloading(); HandlePlayerMovement(); UpdateUIElements(); } } private void OnTriggerEnter2D(Collider2D collision) { ProcessCollision(collision); } #endregion #region Initialization public void InitializePlayer() { currentBullets = maxBullets; UpdateBulletsText(); switch (group) { case CharacterGroup.Player: isPlayerControlled = true; break; case CharacterGroup.Ally: break; case CharacterGroup.Enemy: break; } } #endregion #region Movement and Aiming private void HandlePlayerMovement() { movementInput.x = Input.GetAxisRaw("Horizontal"); movementInput.y = Input.GetAxisRaw("Vertical"); movementInput.Normalize(); currentSpeed = movementSpeed * movementInput.magnitude; Vector3 movement = new Vector3(movementInput.x, movementInput.y, 0f) * currentSpeed * Time.deltaTime; transform.position += movement; } private void HandleAimingAndShooting() { switch (currentControlMode) { case ControlMode.Mouse: MouseControlLogic(); break; case ControlMode.Joystick: JoystickControlLogic(); break; case ControlMode.AI: AIControlLogic(); break; } if (isAiming) { UpdateSightLine(); } if (!canShoot) { shotTimer -= Time.deltaTime; if (shotTimer <= 0f) { canShoot = true; } } if (!canShoot && hasFiredRecently) { shotTimer -= Time.deltaTime; if (shotTimer <= 0f) { canShoot = true; hasFiredRecently = false; } } } private void MouseControlLogic() { if (Input.GetMouseButtonDown(0)) { StartAiming(); } else if (Input.GetMouseButtonUp(0)) { StopAiming(); } } private void JoystickControlLogic() { if (Mathf.Abs(variableJoystick.Horizontal) > 0.1f || Mathf.Abs(variableJoystick.Vertical) > 0.1f) { if (!isAiming) { StartAiming(); } } else { if (isAiming) { StopAiming(); } } } private void AIControlLogic() { if (shootingAgent != null) { // Get the joystick control values from the ML agent float horizontal = 0f; float vertical = 0f; //Debug.Log(horizontal); // Retrieve the joystick control values from the ML agent var behaviorParameters = shootingAgent.GetComponent<Unity.MLAgents.Policies.BehaviorParameters>(); if (behaviorParameters != null) { var actions = shootingAgent.GetStoredActionBuffers().ContinuousActions; if (actions.Length > 0) horizontal = actions[0]; if (actions.Length > 1) vertical = actions[1]; } // Check if the joystick control values exceed the threshold if (Mathf.Abs(horizontal) > 0.1f || Mathf.Abs(vertical) > 0.1f) { if (!isAiming) { StartAiming(); } } else { if (isAiming) { StopAiming(); } } } } public void ControlJoystick(float horizontal, float vertical) { currentControlMode = ControlMode.AI; // Switch to AI control mode // Set horizontal and vertical input values horizontalInput = horizontal; verticalInput = vertical; Debug.Log(horizontalInput); Debug.Log(verticalInput); } private void StartAiming() { isAiming = true; // Destroy any existing SightLine object if (sightLine != null) Destroy(sightLine); // Create and initialize sight line CreateSightLine(); currentDistance = aimingDistance; } private void StopAiming() { isAiming = false; Destroy(sightLine); if (canShoot && currentBullets > 0) { if (shotTimer <= 0f) // Check if shot delay has elapsed { FireBullet(); currentBullets--; UpdateBulletsText(); canShoot = false; // Set canShoot to false shotTimer = shotDelay; // Reset shot delay timer } } } private void CreateSightLine() { sightLine = new GameObject("SightLine"); LineRenderer lineRenderer = sightLine.AddComponent<LineRenderer>(); lineRenderer.positionCount = 2; lineRenderer.startWidth = bulletWidth; lineRenderer.endWidth = bulletWidth; // Create a new material and set its color to white Material lineMaterial = new Material(Shader.Find("Sprites/Default")); lineMaterial.color = Color.white; lineRenderer.material = lineMaterial; // Assign the material to the line renderer lineRenderer.sortingLayerName = "Background"; // Set the sorting layer lineRenderer.sortingOrder = -1; // Set the sorting order to render behind the player lineRenderer.SetPosition(0, transform.position); lineRenderer.SetPosition(1, transform.position); } private void UpdateSightLine() { switch (currentControlMode) { case ControlMode.Mouse: Vector2 mousePosition = Camera.main.ScreenToWorldPoint(Input.mousePosition); aimDirection = (mousePosition - (Vector2)transform.position).normalized; break; case ControlMode.Joystick: aimDirection = new Vector2(variableJoystick.Horizontal, variableJoystick.Vertical).normalized; break; default: aimDirection = Vector2.zero; break; } lineEndPosition = (Vector2)transform.position + aimDirection * currentDistance; LineRenderer lineRenderer = sightLine.GetComponent<LineRenderer>(); lineRenderer.SetPosition(0, transform.position); lineRenderer.SetPosition(1, lineEndPosition); } #endregion #region Reloading private void HandleReloading() { if (currentBullets < maxBullets) { reloadTimer += Time.deltaTime; if (reloadTimer >= reloadTime) { reloadTimer = 0f; currentBullets++; UpdateBulletsText(); } } else if (!isAiming && sightLine != null) { Destroy(sightLine); } } #endregion #region Shooting public void FireBullet() { GameObject bullet = Instantiate(bulletPrefab, transform.position, Quaternion.identity); BulletController bulletController = bullet.GetComponent<BulletController>(); bulletController.SetDirection(aimDirection); bulletController.SetSpeed(bulletSpeed); bulletController.SetMaxDistance(currentDistance); bulletController.SetGroup(group); bulletController.SetDamage(damage); bulletController.SetDamagePiercing(damagePiercing); } #endregion #region UI Updates private void UpdateUIElements() { healthText.text = currentHealth.ToString(); _healthbar.UpdateHealthBar(maxHealth, currentHealth); UpdateBulletsText(); } private void UpdateBulletsText() { bulletsText.text = currentBullets.ToString(); _bulletbar.UpdateHealthBar(maxBullets, currentBullets); } #endregion #region Collision Handling private void ProcessCollision(Collider2D collision) { if (group.Equals(CharacterGroup.Player) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Ally)) { // Ignore bullets from allies damaging the player return; } else if (group.Equals(CharacterGroup.Ally) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Player)) { // Ignore bullets from the player damaging allies return; } else if (group.Equals(CharacterGroup.Enemy) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Player)) { // Damage the enemy when hit by a player’s bullet int bulletDamage = collision.GetComponent<BulletController>().GetDamage(); bool bulletDamagePiercing = collision.GetComponent<BulletController>().GetDamagePiercing(); TakeDamage(bulletDamage); if (bulletDamagePiercing == false) { Destroy(collision.gameObject); } return; } } private void TakeDamage(int damageValue) { if (group == CharacterGroup.Enemy) { currentHealth -= damageValue; if (currentHealth <= 0) { Destroy(gameObject); } _healthbar.UpdateHealthBar(maxHealth, currentHealth); healthText.text = currentHealth.ToString(); } } #endregion #region Observations for ML public void CollectObservations(VectorSensor sensor) { // Add player position to observations sensor.AddObservation(transform.position); // Get enemy positions from OpponentGenerator var enemyPositions = opponentGenerator.GetEnemyPositions(); // Add enemy positions to observations foreach (var enemyPosition in enemyPositions) { sensor.AddObservation(enemyPosition); } } #endregion #region Utility Functions public void Shoot(float shootingAngle) { // Perform shooting logic based on the shooting angle // … } public void RenderToTexture(RenderTexture renderTexture) { // Render the game view to the specified render texture RenderTexture currentRT = RenderTexture.active; RenderTexture.active = renderTexture; Camera.main.targetTexture = renderTexture; Camera.main.Render(); // Reset the render texture and camera settings Camera.main.targetTexture = null; RenderTexture.active = currentRT; } #endregion // Helper Methods private bool IsPlayerControlled() { return isPlayerControlled; }}
- Assistant
Based on your
PlayerControllerscript, I have created a simple agent script that can start the training process in ML-Agents. Note that this is a minimal code just to start the training; you will need to modify it according to your requirements.Create a new C# script called
BasicAgentand replace the contents with the following code:using UnityEngine; using Unity.MLAgents; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators; public class BasicAgent : Agent { public PlayerController controller; private void Start() { if (controller == null) { controller = GetComponent<PlayerController>(); } } public override void OnEpisodeBegin() { // Reset any necessary environment components or agent properties here, if needed. } public override void CollectObservations(VectorSensor sensor) { // Add agent's position as observation sensor.AddObservation(transform.position); } public override void OnActionReceived(ActionBuffers actions) { // Get discrete action int action = actions.DiscreteActions[0]; // Execute actions based on the model's output switch (action) { case 0: // Do nothing break; case 1: // Start aiming controller.StartAiming(); break; case 2: // Stop aiming controller.StopAiming(); break; } } public override void Heuristic(in ActionBuffers actionsOut) { // Get actions from the PlayerController int action = 0; if (Input.GetKeyDown(KeyCode.A)) { action = 1; } else if (Input.GetKeyUp(KeyCode.A)) { action = 2; } actionsOut.DiscreteActions.Array[0] = action; } }In your scene, add the
BasicAgentscript to the same GameObject which has thePlayerControllerscript attached. Set thePlayerControllerreference in theBasicAgentscript by dragging the GameObject that has yourPlayerControllerto the controller field in the Inspector.Modify your
PlayerControllerscript to remove theshootingAgentreference and related logic.Next, follow the steps listed in Unity's ML-Agents documentation to set up your training environment, create a new training configuration file, and start training your agent using the ML-Agents Python package.
Keep in mind that the agent isn't doing anything meaningful yet, it's just a minimal example to start the training process. You'll need to expand the agent's script to handle your specific use case.
- User
I did everything as you said, in the Behavior Name I specified MyPlayerAgent. I write a command to the console, press Play in Unity, it seems to connect to each other, but my agent does nothing, as if the console cannot find the agent, then after a while the console gives an error about a long wait.
here's my playerAgent_config.yaml: behaviors: MyPlayerAgent: trainer_type: ppo hyperparameters: batch_size: 1024 buffer_size: 10240 learning_rate: 3.0e-4 beta: 5.0e-3 epsilon: 0.2 lambd: 0.95 num_epoch: 3 learning_rate_schedule: linear network_settings: normalize: false hidden_units: 128 num_layers: 2 vis_encode_type: simple memory: memory_size: 256 sequence_length: 64 reward_signals: extrinsic: gamma: 0.99 strength: 1.0 keep_checkpoints: 5 max_steps: 5.0e4 time_horizon: 64 summary_freq: 10000
(mlagents) C:\Projects\Unity\ml-agents-release_20>mlagents-learn playerAgent_config.yaml --run-id=player_agent_001 --force Version information: ml-agents: 0.30.0, ml-agents-envs: 0.30.0, Communicator API: 1.5.0, PyTorch: 2.0.1+cu117 [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor. [INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0
- Assistant
The error you are facing might be due to not using the same behavior name in both Unity and the training configuration file. Please follow these steps to ensure that the behavior name is set up correctly:
- In your Unity editor, click on the GameObject with your BasicAgent script.
- In the Inspector, look for the component called "Behavior Parameters" (which is added when you inherit from the Agent class).
- In the "Behavior Parameters" component, set the "Behavior Name" field to match the name specified in your YAML file. In your case, it should be
MyPlayerAgent. - In your YAML file, make sure that the behavior name is the same as the one specified in Unity. It should look like this:
behaviors: MyPlayerAgent: ...- Save and re-run the
mlagents-learncommand.
Make sure your Unity project is open and press the Play button in Unity once you start the training process in the console.
If your agent still doesn't do anything, ensure that the
Heuristic()orOnActionReceived()functions are being called in theBasicAgentscript by adding someDebug.Log()statements. If they are not being called, examine the agent setup in your scene.Please note that the example BasicAgent provided earlier is a minimal code setup for ML-Agents training. It does not have a sophisticated decision-making process, meaning the agent might not show any meaningful actions during training. Once you progress with your training, you would want to include more logic based on your game requirements.
- User
everything is done as you wrote, but functions like Heuristic and OnActionReceived are not called. In the Unity console only: Registered Communicator in Agent. UnityEngine.Debug:Log (object) Unity.MLAgents.Agent:Awake() (at C:/Projects/Unity/ml-agents-release_20/com.unity.ml-agents/Runtime/Agent.cs:373)
- Assistant
It seems that there might be some problem with the agent's setup in the scene. Here are some steps to check and fix the issue:
-
Make sure the GameObject with the
BasicAgentscript also has a "Decision Requester" component attached. If it doesn't, click "Add Component" and search for "Decision Requester" to add it to the GameObject.In the "Decision Requester" component, set the "Decision Period" to a non-zero value, like 5 or 10. This value determines how often the agent requests a decision.
-
In the "Behavior Parameters" component, make sure "Model" is set to "None (Heuristic Only)" during training. If you have finished training and are using the trained model, then you should load the model in the "Model" field.
If you have correctly set up the "Decision Requester" and "Behavior Parameters" components and are still facing the issue, verify if the agent's
OnActionReceived()method is not being called due to an incorrect scene setup or a conflicting script.Sometimes, due to specific implementations, agents may not work correctly. In this case, you can create a new and simple Unity project with a new agent and environment setup to test the ML-Agents training pipeline to ensure it's working as expected. Once you verify the pipeline is working, integrate the agent script and training setup into your project, making necessary adjustments as needed.
-
- User
I added a Decision Requester and some actions started in the code. I got a line in the terminal "[INFO] Connected new brain: MyPlayerAgent?team=0". then a copy of my yaml file, but the training did not start, after some time I got an error:
[WARNING] Restarting worker[0] after 'The Unity environment took too long to respond. Make sure that: The environment does not need user interaction to launch The Agents' Behavior Parameters > Behavior Type is set to "Default" The environment and the Python interface have compatible versions. If you're running on a headless server without graphics support, turn off display by either passing --no-graphics option or build your Unity executable as server build.'
- Assistant
The warning you received indicates that the training process is not receiving data (actions and observations) properly from the Unity environment. Since you added the Decision Requester to your agent and it started calling
Heuristic()andOnActionReceived()methods, it could be due to a slow scene or environment setup. Here are some suggestions to fix this issue:-
Make sure that the agents in the environment are set to Behavior Type "Default" in the "Behavior Parameters" component.
-
Try running the training with the
--no-graphicsflag in yourmlagents-learncommand:mlagents-learn playerAgent_config.yaml --run-id=player_agent_001 --force --no-graphicsThis flag tells Unity to not allocate resources for rendering graphics, which can improve performance during training.
-
If you are running the training with multiple agents or a big environment, try simplifying the problem to just one agent and a minimal environment during training to avoid overburdening the system.
-
In case you have created a custom learning environment, double-check all scripts and objects to make sure they are optimized and set up properly for ML-Agents. You can also try the training process with one of the example environments provided by Unity ML-Agents to ensure your installation and training pipeline are working as expected.
-
Finally, try restarting your computer and running the training process again to ensure that there are no issues with the runtime environment.
These suggestions should help you identify and resolve the issue. If the problem persists, consider sharing more details about your environment setup and the other components in your scene. This would help in debugging the issue more effectively.
-
- User
when i start training i get error in terminal: RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x4 and 3x128)
here's my BasicAgent.cs: using UnityEngine; using Unity.MLAgents; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators;
public class BasicAgent : Agent { public PlayerController playerController;
public override void OnEpisodeBegin() { Debug.Log("Episode Begin"); } public override void CollectObservations(VectorSensor sensor) { Debug.Log("Collect Observations"); // Target and Agent positions sensor.AddObservation(1); sensor.AddObservation(2); // Agent velocity sensor.AddObservation(3); sensor.AddObservation(4); } public override void OnActionReceived(ActionBuffers actionBuffers) { Debug.Log("On Action Received"); SetReward(1.0f); EndEpisode(); } public override void Heuristic(in ActionBuffers actionsOut) { Debug.Log("Heuristic - manual mode"); var continuousActionsOut = actionsOut.ContinuousActions; continuousActionsOut[0] = Input.GetAxis("Horizontal"); continuousActionsOut[1] = Input.GetAxis("Vertical"); }}
- Assistant
The error you received could be due to a mismatch between the expected input and output dimensions in your agent script and the ones set in the YAML configuration file.
There are two main components you need to check:
- Observations collected in your Agent script should match the expected input dimensions in your YAML file. Since you are collecting a total of 4 observations in the
CollectObservations()method, make sure the YAML file correctly sets the input dimensions. In yourplayerAgent_config.yamlfile, make sure you have setvector_observationdimensions under thenetwork_settingslike this:
network_settings: normalize: false hidden_units: 128 num_layers: 2 vis_encode_type: simple vector_observation: space_type: continuous space_size: 4- The number of actions specified in the agent script should match what's defined in the YAML configuration file. In your
BasicAgentscript, you have two continuous actions which are meant to control horizontal and vertical movement. In yourplayerAgent_config.yamlfile, set the continuous action size undernetwork_settingslike this:
network_settings: ... continuous_action: space_size: 2Make sure to modify your YAML file accordingly, and restart your training process.
For more information about configuring ML-Agents settings, you can check out the ML-Agents Configuration Settings documentation.
- Observations collected in your Agent script should match the expected input dimensions in your YAML file. Since you are collecting a total of 4 observations in the
- User
i change observations to 3, but i still get this error:
[INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor. [INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0 [INFO] Connected new brain: MyPlayerAgent?team=0 C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\onnx\symbolic_opset9.py:4476: UserWarning: Exporting a model to ONNX with a batch_size other than 1, with a variable length with LSTM can cause an error when running the ONNX model with a different batch size. Make sure to save the model with a batch size of 1, or define the initial states (h0/c0) as inputs of the model. warnings.warn( ============= Diagnostic Run torch.onnx.export version 2.0.1+cu117 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results\player_agent_001\MyPlayerAgent\MyPlayerAgent-494.onnx [INFO] Copied results\player_agent_001\MyPlayerAgent\MyPlayerAgent-494.onnx to results\player_agent_001\MyPlayerAgent.onnx. Traceback (most recent call last): File "C:\Users\mrogo\anaconda3\envs\mlagents\Scripts\mlagents-learn-script.py", line 33, in sys.exit(load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')()) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 264, in main run_cli(parse_command_line()) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 260, in run_cli run_training(run_seed, options, num_areas) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 136, in run_training tc.start_learning(env_manager) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\trainer_controller.py", line 175, in start_learning n_steps = self.advance(env_manager) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\trainer_controller.py", line 233, in advance new_step_infos = env_manager.get_steps() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\env_manager.py", line 124, in get_steps new_step_infos = self._step() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 424, in _step self._queue_steps() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 302, in _queue_steps env_action_info = self._take_step(env_worker.previous_step) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 543, in _take_step all_action_info[brain_name] = self.policies[brain_name].get_action( File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\policy\torch_policy.py", line 130, in get_action run_out = self.evaluate(decision_requests, global_agent_ids) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\policy\torch_policy.py", line 100, in evaluate action, run_out, memories = self.actor.get_action_and_stats( File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\networks.py", line 637, in get_action_and_stats encoding, memories = self.network_body( File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\networks.py", line 244, in forward encoding = self._body_endoder(encoded_self) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\layers.py", line 169, in forward return self.seq_layers(input_tensor) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\container.py", line 217, in forward input = module(input) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x3 and 4x128)
- Assistant
The error you mentioned indicates a mismatch in the dimensions of the input tensor in your agent script and the expected dimensions of the network. Make sure you have updated the YAML file with the correct number of observations you are collecting in your agent script:
network_settings: normalize: false hidden_units: 128 num_layers: 2 vis_encode_type: simple vector_observation: space_type: continuous space_size: 3Notice that I have changed
space_sizefrom 4 to 3, given that you mentioned you have updated the observations collected to 3 in your agent script.In the BasicAgent.cs script, please make sure that you are collecting exactly 3 observations:
public override void CollectObservations(VectorSensor sensor) { Debug.Log("Collect Observations"); // You should have exactly 3 observations here sensor.AddObservation(1); sensor.AddObservation(2); sensor.AddObservation(3); }After updating the YAML file, restart your training process. Ensure that you have saved any changes made to the agent script, the training configuration file, and the Unity environment before re-running the training.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.