Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential Memory leak when using model #6109

Open
noki00003 opened this issue May 7, 2024 · 1 comment
Open

Potential Memory leak when using model #6109

noki00003 opened this issue May 7, 2024 · 1 comment
Labels
bug Issue describes a potential bug in ml-agents. stale Issues that have been idle for a while. Automatically closed by a bot if idle for too long.

Comments

@noki00003
Copy link

noki00003 commented May 7, 2024

I am experiencing a bug, that when I have finished training my model and I want to use it then, I put the model into the Model Field in Unity. I do this for 16 Agents in a single environment. The first run there are no problems. Then I stop the run and change all models except for one to "Inference only" (the first run was "Default") and the one remaining agent to "Heuristic only", as I want to play myself against the other agents.
Then I get the warning:

"Found unreferenced, but undisposed BurstTensorData that potentially participates in an unfinished job and might lead to hazardous memory overwrites
UnityEngine.Debug:LogWarning (object)
Unity.Sentis.D:LogWarning (object) (at ./Library/PackageCache/com.unity.sentis@1.2.0-exp.2/Runtime/Core/Internals/Debug.cs:72)
Unity.Sentis.BurstTensorData:Finalize () (at ./Library/PackageCache/com.unity.sentis@1.2.0-exp.2/Runtime/Core/Backends/CPU/BurstTensorData.cs:145)".

When I then change all the agents back to "Default" as in the first run, the warning still occurs. I have looked for solutions for a long time on the Internet, but there is only one solution, where I do not understand where in the code I need to change something
( AlexRibard/Barracuda-U-2-NetTest#3 ).

To reproduce this error you need to follow these simple steps:

  1. open up the default PushBlock environment provided by mlagents
  2. hit run once and then everything should run as desired with the provided model asset
  3. stop running and change only the agent of the first environment to "heuristic only"
  4. hit run again and then there should be the warning in the console

Edit: You can simply hit run then stop and then run again, I just found out
Edit2: It does only happen in release 21 (3.0.0) and not in release 20 (2.3.0)

I am not really sure, whether I tested this excact scenario with Torch-CPU before, but right now I am using Torch-CUDA 12.1 with torch version 2.3.0.
My other specs:

  • Windows 11
  • Unity 2022.3.26f*
  • ml-agents 1.0.0
  • in unity mlagents 3.0.0 (stable release 21)
  • Python 3.10.12

memoryLeakIssueUnity

@noki00003 noki00003 added the bug Issue describes a potential bug in ml-agents. label May 7, 2024
Copy link

github-actions bot commented Jun 6, 2024

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Issues that have been idle for a while. Automatically closed by a bot if idle for too long. label Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue describes a potential bug in ml-agents. stale Issues that have been idle for a while. Automatically closed by a bot if idle for too long.
Projects
None yet
Development

No branches or pull requests

1 participant