Hi,
What's the best way to free-up memory while looping (say over spectral bands) in a script?
I've tried:
# trying to keep memory down on loop
collectGarbage()
layerVIS.doRemove()
and even
# taking out the boomstick...
boomstick()
but in both cases, the memory used just keeps increasing.
I'm a a PC, running mcidas-v version 1.4.
Thanks,
TJS
increasing memory
- tommy.jasmin
- Posts: 35
- Joined: Tue Aug 17, 2010 6:56 pm
Re: increasing memory
Tim - is the script not finishing as expected? Or are you just concerned that at some point as you add functionality there will be trouble?
Not knowing the details, it's a bit of a hack but if you can break the script up into multiple scripts, that's one way to be sure all resources are freed up each step.
Not knowing the details, it's a bit of a hack but if you can break the script up into multiple scripts, that's one way to be sure all resources are freed up each step.
Re: increasing memory
Hi Tim,
Tommy's first question is important here. If you're running out of memory, or if you're using a significantly larger amount of memory from the script than you do from the user interface, then we may have a problem in the code.
By default, McIDAS-V uses 80% of the available memory on your machine. Java will often allow the memory usage to approach that value before it attempts any cleanup that it can do. This is normal for Java, but very different than what we're used to with Fortran or C.
In a nutshell, if you're seeing other problems (actually running out of memory or using way too much memory), then we should investigate further. If you're not seeing other problems, then it's probably just Java being Java, and nothing to worry about.
I hope this helps!
Becky
Tommy's first question is important here. If you're running out of memory, or if you're using a significantly larger amount of memory from the script than you do from the user interface, then we may have a problem in the code.
By default, McIDAS-V uses 80% of the available memory on your machine. Java will often allow the memory usage to approach that value before it attempts any cleanup that it can do. This is normal for Java, but very different than what we're used to with Fortran or C.
In a nutshell, if you're seeing other problems (actually running out of memory or using way too much memory), then we should investigate further. If you're not seeing other problems, then it's probably just Java being Java, and nothing to worry about.
I hope this helps!
Becky
Re: increasing memory
Thanks. The script does finish, but it becomes real slow, along with any other application running on my PC.
Re: increasing memory
Plus, even when the script finishes, I don't get the memory back. So, breaking up the script that does 16 bands into 16 scripts won't help, unless I kill mcidas after each band. TJS
Re: increasing memory
Hi Tim,
After your script has completed, go to the task manager and let us know if there are any leftover mcservl.exe or javaw.exe processes running. Java won't release the memory until these processes have finished.
Also, if you have not done so, please try running your script from the jython shell. From the main display window menu bar, Tools->Formulas->Jython Shell. Generally it's easiest to paste your script into the shell. Select the 2 down arrows to the right of the evaluate button. Paste your script into the that same text box and select evaluate. From the main display window, watch the memory allocation. If the numbers are in red, you might run out of memory.
Let us know your results.
Rick
After your script has completed, go to the task manager and let us know if there are any leftover mcservl.exe or javaw.exe processes running. Java won't release the memory until these processes have finished.
Also, if you have not done so, please try running your script from the jython shell. From the main display window menu bar, Tools->Formulas->Jython Shell. Generally it's easiest to paste your script into the shell. Select the 2 down arrows to the right of the evaluate button. Paste your script into the that same text box and select evaluate. From the main display window, watch the memory allocation. If the numbers are in red, you might run out of memory.
Let us know your results.
Rick
Re: increasing memory
Hi,
javaw.exe stays around, but that might be since I'm running this in the jython shell.
That said, it takes up lots of memory (GBs) and so it seems like the 'remove all data' doesn't free the memory.
TJS
javaw.exe stays around, but that might be since I'm running this in the jython shell.
That said, it takes up lots of memory (GBs) and so it seems like the 'remove all data' doesn't free the memory.
TJS
Re: increasing memory
When you exit your McIDAS-V session check to see if javaw.exe is still running. If it is not, do your other applications again behave normally?
Please post your entire script so we can give it a try.
Sounds like Java is behaving properly by taking all the memory it can grab. Removing a layer should release some of the memory, but not all the memory will be freed up.
Please post your entire script so we can give it a try.
Sounds like Java is behaving properly by taking all the memory it can grab. Removing a layer should release some of the memory, but not all the memory will be freed up.