Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0314· openaiHugging Face Suzhen/CodeChat

in unity's ML Agents do I need to modify my PlayerController which works with player…

  1. User

    in unity's ML Agents do I need to modify my PlayerController which works with player input? or do I need to create a new script and make references in the code to each other?

  2. Assistant

    In Unity's ML Agents, you typically don't modify your existing PlayerController script directly. Instead, you create a new script that inherits from the Agent class. This new script will control the agent's interaction with the environment, receive observations, and execute actions determined by the trained model.

    You can reuse some code from your PlayerController in this new agent script if it's related to agent's movement, decision-making, etc. However, player input-related code should be separated as it doesn't apply to the agent.

    Here's a high-level overview of the steps you need to follow:

    1. Create a new C# script that inherits from the Agent class (from "using Unity.MLAgents;") and attach it to your game object that you want to train.

    2. Implement the required methods such as OnEpisodeBegin(), CollectObservations(), and OnActionReceived(). These methods are responsible for resetting the environment, collecting input observations for training, and executing the necessary actions based on the output of the trained model.

    3. If needed, you can keep your PlayerController script for player control during testing and development. However, make sure that the new agent script and PlayerController script do not interfere with each other. You may use conditional checks like "#if UNITY_EDITOR" in your PlayerController script to disable certain sections of the code when the build is running in the Unity Editor to avoid conflicts.

    4. In your new agent script, you can reference and utilize any necessary components or functionality from your PlayerController script. This allows you to use the same logic for movement or other actions, reducing code duplication. However, you may need to make some adjustments to ensure compatibility with the Agent class.

    5. After implementing the new agent script, set up your ML-Agent training environment following Unity's ML-Agents documentation, create your training configuration file, and start training your agent using the ML-Agents Python package.

    Remember that your new agent script should focus on agent logic rather than player input. When you want to test your trained agent, simply enable the new agent script and disable the PlayerController script, so that the agent acts based on the trained model instead of player input.

  3. User

    Well, in that case, make me an agent for my PlayerController. What is important to me is a minimally working code that can start training agent training. I repeat, the code is not supposed to be useful, but it must work and the training must run in ML Agents. Observation can be anything, be it the position of the player or anything, the output can be anything too. I just need to get the training going, even if it's crazy. Further, based on your code, I myself will write what I need.

    Here is my PlayerController:

    using System.Collections; using System.Collections.Generic; using UnityEngine; using UnityEngine.UI; using Unity.MLAgents; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators; using UnityEditor.U2D;

    public enum CharacterGroup { Player, Ally, Enemy }

    public enum ControlMode { Mouse, Joystick, AI }

    public class PlayerController : MonoBehaviour { #region Public Variables

    public CharacterGroup group;
    public ShootingAgent shootingAgent;
    public OpponentGenerator opponentGenerator;
    public VariableJoystick variableJoystick;
    public ControlMode currentControlMode = ControlMode.Mouse; // Default control mode is mouse
    public GameObject bulletPrefab;
    public string nickname = "lovekpek";
    public string brawlerName = "Nita";
    public int maxHealth = 3000; // Add a new variable for maximum health
    public int damage = 750; // Add a new variable for damage
    public bool damagePiercing = true;
    public float movementSpeed = 5f;
    public float aimingDistance = 5f;
    public float bulletSpeed = 10f;
    public int maxBullets = 3;
    public float reloadTime = 2f;
    public float shotDelay = 0.5f; // Delay between consecutive shots
    public float bulletWidth = 0.1f; // Bullet width
    
    [SerializeField] private HealthBar _healthbar;
    public Text healthText; // Reference to the UI text component for health display
    [SerializeField] private BulletBar _bulletbar;
    public Text bulletsText;
    public int currentHealth = 3000;
    public int currentBullets = 3;
    public Text speedText;
    
    #endregion
    
    #region Private Variables
    
    private GameObject sightLine;
    private bool isAiming;
    private bool canShoot = true; // Flag to allow shooting
    private Vector2 aimDirection;
    private Vector2 lineEndPosition;
    private float currentDistance;
    private float reloadTimer;
    private float shotTimer; // Timer for shot delay
    private bool hasFiredRecently; // New flag
    private Vector2 movementInput;
    private float currentSpeed;
    
    private bool isPlayerControlled = false;
    
    private Collider2D lastCollision; // Class-level variable to store the last collision
    
    // Joystick control variables
    private float horizontalInput;
    private float verticalInput;
    
    #endregion
    
    #region Unity Callbacks
    
    private void Start()
    {
        InitializePlayer();
    }
    
    private void Update()
    {
        if (IsPlayerControlled())
        {
            HandleAimingAndShooting();
            HandleReloading();
            HandlePlayerMovement();
            UpdateUIElements();
        }
    }
    
    private void OnTriggerEnter2D(Collider2D collision)
    {
        ProcessCollision(collision);
    }
    
    #endregion
    
    #region Initialization
    
    public void InitializePlayer()
    {
        currentBullets = maxBullets;
        UpdateBulletsText();
    
        switch (group)
        {
            case CharacterGroup.Player:
                isPlayerControlled = true;
                break;
    
            case CharacterGroup.Ally:
    
                break;
    
            case CharacterGroup.Enemy:
    
                break;
        }
    }
    
    #endregion
    
    #region Movement and Aiming
    
    private void HandlePlayerMovement()
    {
        movementInput.x = Input.GetAxisRaw("Horizontal");
        movementInput.y = Input.GetAxisRaw("Vertical");
        movementInput.Normalize();
        currentSpeed = movementSpeed * movementInput.magnitude;
        Vector3 movement = new Vector3(movementInput.x, movementInput.y, 0f) * currentSpeed * Time.deltaTime;
        transform.position += movement;
    }
    
    private void HandleAimingAndShooting()
    {
        switch (currentControlMode)
        {
            case ControlMode.Mouse:
                MouseControlLogic();
                break;
            case ControlMode.Joystick:
                JoystickControlLogic();
                break;
            case ControlMode.AI:
                AIControlLogic();
                break;
        }
    
        if (isAiming)
        {
            UpdateSightLine();
        }
    
        if (!canShoot)
        {
            shotTimer -= Time.deltaTime;
            if (shotTimer <= 0f)
            {
                canShoot = true;
            }
        }
    
        if (!canShoot && hasFiredRecently)
        {
            shotTimer -= Time.deltaTime;
            if (shotTimer <= 0f)
            {
                canShoot = true;
                hasFiredRecently = false;
            }
        }
    }
    
    private void MouseControlLogic()
    {
        if (Input.GetMouseButtonDown(0))
        {
            StartAiming();
        }
        else if (Input.GetMouseButtonUp(0))
        {
            StopAiming();
        }
    }
    
    private void JoystickControlLogic()
    {
        if (Mathf.Abs(variableJoystick.Horizontal) > 0.1f || Mathf.Abs(variableJoystick.Vertical) > 0.1f)
        {
            if (!isAiming)
            {
                StartAiming();
            }
        }
        else
        {
            if (isAiming)
            {
                StopAiming();
            }
        }
    }
    
    private void AIControlLogic()
    {
        if (shootingAgent != null)
        {
            // Get the joystick control values from the ML agent
            float horizontal = 0f;
            float vertical = 0f;
            //Debug.Log(horizontal);
    
            // Retrieve the joystick control values from the ML agent
            var behaviorParameters = shootingAgent.GetComponent<Unity.MLAgents.Policies.BehaviorParameters>();
            if (behaviorParameters != null)
            {
                var actions = shootingAgent.GetStoredActionBuffers().ContinuousActions;
    
                if (actions.Length > 0)
                    horizontal = actions[0];
    
                if (actions.Length > 1)
                    vertical = actions[1];
            }
    
            // Check if the joystick control values exceed the threshold
            if (Mathf.Abs(horizontal) > 0.1f || Mathf.Abs(vertical) > 0.1f)
            {
                if (!isAiming)
                {
                    StartAiming();
                }
            }
            else
            {
                if (isAiming)
                {
                    StopAiming();
                }
            }
        }
    }
    
    public void ControlJoystick(float horizontal, float vertical)
    {
        currentControlMode = ControlMode.AI; // Switch to AI control mode
    
        // Set horizontal and vertical input values
        horizontalInput = horizontal;
        verticalInput = vertical;
        Debug.Log(horizontalInput);
        Debug.Log(verticalInput);
    }
    
    
    private void StartAiming()
    {
        isAiming = true;
    
        // Destroy any existing SightLine object
        if (sightLine != null)
            Destroy(sightLine);
    
        // Create and initialize sight line
        CreateSightLine();
    
        currentDistance = aimingDistance;
    }
    
    private void StopAiming()
    {
        isAiming = false;
        Destroy(sightLine);
    
        if (canShoot && currentBullets > 0)
        {
            if (shotTimer <= 0f) // Check if shot delay has elapsed
            {
                FireBullet();
    
                currentBullets--;
                UpdateBulletsText();
    
                canShoot = false; // Set canShoot to false
                shotTimer = shotDelay; // Reset shot delay timer
            }
        }
    }
    
    private void CreateSightLine()
    {
        sightLine = new GameObject("SightLine");
        LineRenderer lineRenderer = sightLine.AddComponent<LineRenderer>();
        lineRenderer.positionCount = 2;
    
        lineRenderer.startWidth = bulletWidth;
        lineRenderer.endWidth = bulletWidth;
    
        // Create a new material and set its color to white
        Material lineMaterial = new Material(Shader.Find("Sprites/Default"));
        lineMaterial.color = Color.white;
    
        lineRenderer.material = lineMaterial; // Assign the material to the line renderer
    
        lineRenderer.sortingLayerName = "Background"; // Set the sorting layer
        lineRenderer.sortingOrder = -1; // Set the sorting order to render behind the player
    
        lineRenderer.SetPosition(0, transform.position);
        lineRenderer.SetPosition(1, transform.position);
    }
    
    private void UpdateSightLine()
    {
        switch (currentControlMode)
        {
            case ControlMode.Mouse:
                Vector2 mousePosition = Camera.main.ScreenToWorldPoint(Input.mousePosition);
                aimDirection = (mousePosition - (Vector2)transform.position).normalized;
                break;
            case ControlMode.Joystick:
                aimDirection = new Vector2(variableJoystick.Horizontal, variableJoystick.Vertical).normalized;
                break;
            default:
                aimDirection = Vector2.zero;
                break;
        }
    
        lineEndPosition = (Vector2)transform.position + aimDirection * currentDistance;
    
        LineRenderer lineRenderer = sightLine.GetComponent<LineRenderer>();
        lineRenderer.SetPosition(0, transform.position);
        lineRenderer.SetPosition(1, lineEndPosition);
    }
    
    #endregion
    
    #region Reloading
    
    private void HandleReloading()
    {
        if (currentBullets < maxBullets)
        {
            reloadTimer += Time.deltaTime;
            if (reloadTimer >= reloadTime)
            {
                reloadTimer = 0f;
                currentBullets++;
                UpdateBulletsText();
            }
        }
        else if (!isAiming && sightLine != null)
        {
            Destroy(sightLine);
        }
    }
    
    #endregion
    
    #region Shooting
    
    public void FireBullet()
    {
        GameObject bullet = Instantiate(bulletPrefab, transform.position, Quaternion.identity);
    
        BulletController bulletController = bullet.GetComponent<BulletController>();
        bulletController.SetDirection(aimDirection);
        bulletController.SetSpeed(bulletSpeed);
        bulletController.SetMaxDistance(currentDistance);
        bulletController.SetGroup(group);
        bulletController.SetDamage(damage);
        bulletController.SetDamagePiercing(damagePiercing);
    }
    
    #endregion
    
    #region UI Updates
    
    private void UpdateUIElements()
    {
        healthText.text = currentHealth.ToString();
        _healthbar.UpdateHealthBar(maxHealth, currentHealth);
    
        UpdateBulletsText();
    }
    
    private void UpdateBulletsText()
    {
        bulletsText.text = currentBullets.ToString();
        _bulletbar.UpdateHealthBar(maxBullets, currentBullets);
    }
    
    #endregion
    
    #region Collision Handling
    
    private void ProcessCollision(Collider2D collision)
    {
        if (group.Equals(CharacterGroup.Player) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Ally))
        {
            // Ignore bullets from allies damaging the player
            return;
        }
        else if (group.Equals(CharacterGroup.Ally) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Player))
        {
            // Ignore bullets from the player damaging allies
            return;
        }
        else if (group.Equals(CharacterGroup.Enemy) && collision.CompareTag("Bullet") && collision.GetComponent<BulletController>().group.Equals(CharacterGroup.Player))
        {
            // Damage the enemy when hit by a player’s bullet
            int bulletDamage = collision.GetComponent<BulletController>().GetDamage();
            bool bulletDamagePiercing = collision.GetComponent<BulletController>().GetDamagePiercing();
            TakeDamage(bulletDamage);
    
            if (bulletDamagePiercing == false)
            {
                Destroy(collision.gameObject);
            }
            return;
        }
    }
    
    
    private void TakeDamage(int damageValue)
    {
        if (group == CharacterGroup.Enemy)
        {
            currentHealth -= damageValue;
            if (currentHealth <= 0)
            {
                Destroy(gameObject);
            }
    
            _healthbar.UpdateHealthBar(maxHealth, currentHealth);
            healthText.text = currentHealth.ToString();
        }
    }
    
    #endregion
    
    #region Observations for ML
    
    public void CollectObservations(VectorSensor sensor)
    {
        // Add player position to observations
        sensor.AddObservation(transform.position);
    
        // Get enemy positions from OpponentGenerator
        var enemyPositions = opponentGenerator.GetEnemyPositions();
    
        // Add enemy positions to observations
        foreach (var enemyPosition in enemyPositions)
        {
            sensor.AddObservation(enemyPosition);
        }
    }
    
    #endregion
    
    #region Utility Functions
    
    public void Shoot(float shootingAngle)
    {
        // Perform shooting logic based on the shooting angle
        // …
    }
    
    public void RenderToTexture(RenderTexture renderTexture)
    {
        // Render the game view to the specified render texture
        RenderTexture currentRT = RenderTexture.active;
        RenderTexture.active = renderTexture;
        Camera.main.targetTexture = renderTexture;
        Camera.main.Render();
    
        // Reset the render texture and camera settings
        Camera.main.targetTexture = null;
        RenderTexture.active = currentRT;
    }
    
    #endregion
    
    // Helper Methods
    
    private bool IsPlayerControlled()
    {
        return isPlayerControlled;
    }
    

    }

  4. Assistant

    Based on your PlayerController script, I have created a simple agent script that can start the training process in ML-Agents. Note that this is a minimal code just to start the training; you will need to modify it according to your requirements.

    Create a new C# script called BasicAgent and replace the contents with the following code:

    using UnityEngine;
    using Unity.MLAgents;
    using Unity.MLAgents.Sensors;
    using Unity.MLAgents.Actuators;
    
    public class BasicAgent : Agent
    {
        public PlayerController controller;
    
        private void Start()
        {
            if (controller == null)
            {
                controller = GetComponent<PlayerController>();
            }
        }
    
        public override void OnEpisodeBegin()
        {
            // Reset any necessary environment components or agent properties here, if needed.
        }
    
        public override void CollectObservations(VectorSensor sensor)
        {
            // Add agent's position as observation
            sensor.AddObservation(transform.position);
        }
    
        public override void OnActionReceived(ActionBuffers actions)
        {
            // Get discrete action
            int action = actions.DiscreteActions[0];
    
            // Execute actions based on the model's output
            switch (action)
            {
                case 0: // Do nothing
                    break;
                case 1: // Start aiming
                    controller.StartAiming();
                    break;
                case 2: // Stop aiming
                    controller.StopAiming();
                    break;
            }
        }
    
        public override void Heuristic(in ActionBuffers actionsOut)
        {
            // Get actions from the PlayerController
            int action = 0;
    
            if (Input.GetKeyDown(KeyCode.A))
            {
                action = 1;
            }
            else if (Input.GetKeyUp(KeyCode.A))
            {
                action = 2;
            }
    
            actionsOut.DiscreteActions.Array[0] = action;
        }
    }
    

    In your scene, add the BasicAgent script to the same GameObject which has the PlayerController script attached. Set the PlayerController reference in the BasicAgent script by dragging the GameObject that has your PlayerController to the controller field in the Inspector.

    Modify your PlayerController script to remove the shootingAgent reference and related logic.

    Next, follow the steps listed in Unity's ML-Agents documentation to set up your training environment, create a new training configuration file, and start training your agent using the ML-Agents Python package.

    Keep in mind that the agent isn't doing anything meaningful yet, it's just a minimal example to start the training process. You'll need to expand the agent's script to handle your specific use case.

  5. User

    I did everything as you said, in the Behavior Name I specified MyPlayerAgent. I write a command to the console, press Play in Unity, it seems to connect to each other, but my agent does nothing, as if the console cannot find the agent, then after a while the console gives an error about a long wait.

    here's my playerAgent_config.yaml: behaviors: MyPlayerAgent: trainer_type: ppo hyperparameters: batch_size: 1024 buffer_size: 10240 learning_rate: 3.0e-4 beta: 5.0e-3 epsilon: 0.2 lambd: 0.95 num_epoch: 3 learning_rate_schedule: linear network_settings: normalize: false hidden_units: 128 num_layers: 2 vis_encode_type: simple memory: memory_size: 256 sequence_length: 64 reward_signals: extrinsic: gamma: 0.99 strength: 1.0 keep_checkpoints: 5 max_steps: 5.0e4 time_horizon: 64 summary_freq: 10000

    (mlagents) C:\Projects\Unity\ml-agents-release_20>mlagents-learn playerAgent_config.yaml --run-id=player_agent_001 --force Version information: ml-agents: 0.30.0, ml-agents-envs: 0.30.0, Communicator API: 1.5.0, PyTorch: 2.0.1+cu117 [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor. [INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0

  6. Assistant

    The error you are facing might be due to not using the same behavior name in both Unity and the training configuration file. Please follow these steps to ensure that the behavior name is set up correctly:

    1. In your Unity editor, click on the GameObject with your BasicAgent script.
    2. In the Inspector, look for the component called "Behavior Parameters" (which is added when you inherit from the Agent class).
    3. In the "Behavior Parameters" component, set the "Behavior Name" field to match the name specified in your YAML file. In your case, it should be MyPlayerAgent.
    4. In your YAML file, make sure that the behavior name is the same as the one specified in Unity. It should look like this:
    behaviors:
      MyPlayerAgent:
        ...
    
    1. Save and re-run the mlagents-learn command.

    Make sure your Unity project is open and press the Play button in Unity once you start the training process in the console.

    If your agent still doesn't do anything, ensure that the Heuristic() or OnActionReceived() functions are being called in the BasicAgent script by adding some Debug.Log() statements. If they are not being called, examine the agent setup in your scene.

    Please note that the example BasicAgent provided earlier is a minimal code setup for ML-Agents training. It does not have a sophisticated decision-making process, meaning the agent might not show any meaningful actions during training. Once you progress with your training, you would want to include more logic based on your game requirements.

  7. User

    everything is done as you wrote, but functions like Heuristic and OnActionReceived are not called. In the Unity console only: Registered Communicator in Agent. UnityEngine.Debug:Log (object) Unity.MLAgents.Agent:Awake() (at C:/Projects/Unity/ml-agents-release_20/com.unity.ml-agents/Runtime/Agent.cs:373)

  8. Assistant

    It seems that there might be some problem with the agent's setup in the scene. Here are some steps to check and fix the issue:

    1. Make sure the GameObject with the BasicAgent script also has a "Decision Requester" component attached. If it doesn't, click "Add Component" and search for "Decision Requester" to add it to the GameObject.

      In the "Decision Requester" component, set the "Decision Period" to a non-zero value, like 5 or 10. This value determines how often the agent requests a decision.

    2. In the "Behavior Parameters" component, make sure "Model" is set to "None (Heuristic Only)" during training. If you have finished training and are using the trained model, then you should load the model in the "Model" field.

    If you have correctly set up the "Decision Requester" and "Behavior Parameters" components and are still facing the issue, verify if the agent's OnActionReceived() method is not being called due to an incorrect scene setup or a conflicting script.

    Sometimes, due to specific implementations, agents may not work correctly. In this case, you can create a new and simple Unity project with a new agent and environment setup to test the ML-Agents training pipeline to ensure it's working as expected. Once you verify the pipeline is working, integrate the agent script and training setup into your project, making necessary adjustments as needed.

  9. User

    I added a Decision Requester and some actions started in the code. I got a line in the terminal "[INFO] Connected new brain: MyPlayerAgent?team=0". then a copy of my yaml file, but the training did not start, after some time I got an error:

    [WARNING] Restarting worker[0] after 'The Unity environment took too long to respond. Make sure that: The environment does not need user interaction to launch The Agents' Behavior Parameters > Behavior Type is set to "Default" The environment and the Python interface have compatible versions. If you're running on a headless server without graphics support, turn off display by either passing --no-graphics option or build your Unity executable as server build.'

  10. Assistant

    The warning you received indicates that the training process is not receiving data (actions and observations) properly from the Unity environment. Since you added the Decision Requester to your agent and it started calling Heuristic() and OnActionReceived() methods, it could be due to a slow scene or environment setup. Here are some suggestions to fix this issue:

    1. Make sure that the agents in the environment are set to Behavior Type "Default" in the "Behavior Parameters" component.

    2. Try running the training with the --no-graphics flag in your mlagents-learn command:

      mlagents-learn playerAgent_config.yaml --run-id=player_agent_001 --force --no-graphics
      

      This flag tells Unity to not allocate resources for rendering graphics, which can improve performance during training.

    3. If you are running the training with multiple agents or a big environment, try simplifying the problem to just one agent and a minimal environment during training to avoid overburdening the system.

    4. In case you have created a custom learning environment, double-check all scripts and objects to make sure they are optimized and set up properly for ML-Agents. You can also try the training process with one of the example environments provided by Unity ML-Agents to ensure your installation and training pipeline are working as expected.

    5. Finally, try restarting your computer and running the training process again to ensure that there are no issues with the runtime environment.

    These suggestions should help you identify and resolve the issue. If the problem persists, consider sharing more details about your environment setup and the other components in your scene. This would help in debugging the issue more effectively.

  11. User

    when i start training i get error in terminal: RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x4 and 3x128)

    here's my BasicAgent.cs: using UnityEngine; using Unity.MLAgents; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators;

    public class BasicAgent : Agent { public PlayerController playerController;

    public override void OnEpisodeBegin()
    {
       Debug.Log("Episode Begin");
    }
    
    public override void CollectObservations(VectorSensor sensor)
    {
        Debug.Log("Collect Observations");
        // Target and Agent positions
        sensor.AddObservation(1);
        sensor.AddObservation(2);
    
        // Agent velocity
        sensor.AddObservation(3);
        sensor.AddObservation(4);
    }
    
    public override void OnActionReceived(ActionBuffers actionBuffers)
    {
        Debug.Log("On Action Received");
        SetReward(1.0f);
        EndEpisode();
    }
    
    public override void Heuristic(in ActionBuffers actionsOut)
    {
        Debug.Log("Heuristic - manual mode");
        var continuousActionsOut = actionsOut.ContinuousActions;
        continuousActionsOut[0] = Input.GetAxis("Horizontal");
        continuousActionsOut[1] = Input.GetAxis("Vertical");
    }
    

    }

  12. Assistant

    The error you received could be due to a mismatch between the expected input and output dimensions in your agent script and the ones set in the YAML configuration file.

    There are two main components you need to check:

    1. Observations collected in your Agent script should match the expected input dimensions in your YAML file. Since you are collecting a total of 4 observations in the CollectObservations() method, make sure the YAML file correctly sets the input dimensions. In your playerAgent_config.yaml file, make sure you have set vector_observation dimensions under the network_settings like this:
    network_settings:
      normalize: false
      hidden_units: 128
      num_layers: 2
      vis_encode_type: simple
      vector_observation:
        space_type: continuous
        space_size: 4
    
    1. The number of actions specified in the agent script should match what's defined in the YAML configuration file. In your BasicAgent script, you have two continuous actions which are meant to control horizontal and vertical movement. In your playerAgent_config.yaml file, set the continuous action size under network_settings like this:
    network_settings:
      ...
      continuous_action:
        space_size: 2
    

    Make sure to modify your YAML file accordingly, and restart your training process.

    For more information about configuring ML-Agents settings, you can check out the ML-Agents Configuration Settings documentation.

  13. User

    i change observations to 3, but i still get this error:

    [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor. [INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0 [INFO] Connected new brain: MyPlayerAgent?team=0 C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\onnx\symbolic_opset9.py:4476: UserWarning: Exporting a model to ONNX with a batch_size other than 1, with a variable length with LSTM can cause an error when running the ONNX model with a different batch size. Make sure to save the model with a batch size of 1, or define the initial states (h0/c0) as inputs of the model. warnings.warn( ============= Diagnostic Run torch.onnx.export version 2.0.1+cu117 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

    [INFO] Exported results\player_agent_001\MyPlayerAgent\MyPlayerAgent-494.onnx [INFO] Copied results\player_agent_001\MyPlayerAgent\MyPlayerAgent-494.onnx to results\player_agent_001\MyPlayerAgent.onnx. Traceback (most recent call last): File "C:\Users\mrogo\anaconda3\envs\mlagents\Scripts\mlagents-learn-script.py", line 33, in sys.exit(load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')()) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 264, in main run_cli(parse_command_line()) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 260, in run_cli run_training(run_seed, options, num_areas) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\learn.py", line 136, in run_training tc.start_learning(env_manager) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\trainer_controller.py", line 175, in start_learning n_steps = self.advance(env_manager) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\trainer_controller.py", line 233, in advance new_step_infos = env_manager.get_steps() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\env_manager.py", line 124, in get_steps new_step_infos = self._step() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 424, in _step self._queue_steps() File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 302, in _queue_steps env_action_info = self._take_step(env_worker.previous_step) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 543, in _take_step all_action_info[brain_name] = self.policies[brain_name].get_action( File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\policy\torch_policy.py", line 130, in get_action run_out = self.evaluate(decision_requests, global_agent_ids) File "c:\projects\unity\ml-agents-release_20\ml-agents-envs\mlagents_envs\timers.py", line 305, in wrapped return func(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\policy\torch_policy.py", line 100, in evaluate action, run_out, memories = self.actor.get_action_and_stats( File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\networks.py", line 637, in get_action_and_stats encoding, memories = self.network_body( File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\networks.py", line 244, in forward encoding = self._body_endoder(encoded_self) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "c:\projects\unity\ml-agents-release_20\ml-agents\mlagents\trainers\torch_entities\layers.py", line 169, in forward return self.seq_layers(input_tensor) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\container.py", line 217, in forward input = module(input) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\mrogo\anaconda3\envs\mlagents\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x3 and 4x128)

  14. Assistant

    The error you mentioned indicates a mismatch in the dimensions of the input tensor in your agent script and the expected dimensions of the network. Make sure you have updated the YAML file with the correct number of observations you are collecting in your agent script:

    network_settings:
      normalize: false
      hidden_units: 128
      num_layers: 2
      vis_encode_type: simple
      vector_observation:
        space_type: continuous
        space_size: 3
    

    Notice that I have changed space_size from 4 to 3, given that you mentioned you have updated the observations collected to 3 in your agent script.

    In the BasicAgent.cs script, please make sure that you are collecting exactly 3 observations:

    public override void CollectObservations(VectorSensor sensor)
    {
        Debug.Log("Collect Observations");
        // You should have exactly 3 observations here
        sensor.AddObservation(1);
        sensor.AddObservation(2);
        sensor.AddObservation(3);
    }
    

    After updating the YAML file, restart your training process. Ensure that you have saved any changes made to the agent script, the training configuration file, and the Unity environment before re-running the training.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.