Jump to content
beefly

Memory deallocation bug

Recommended Posts

Hello!
It was found that the function Request Deallocation does not free memory after using an array of strings (or any other multi-dimensional array).
Description of the experiment:
There is a VI from which subVI is called by command. In the subVI only one-dimensional array (numbers or strings) generated. If the array element type is number (I32), then allocated memory is released (using the function Request Deallocation). If the array element type is string (or any other multi-dimensional array), the memory will not be released. Anyone faced this problem? If yes, what should we do?
Excuse for bad english.

main.vi

memoryusage.vi

Share this post


Link to post
Share on other sites

Why are you using that function?  The reason I ask is because no program that I have ever written has needed it.  Every time I see it used it didn't need to be.  I'm not saying it doesn't serve a purpose I am just trying to say you don't need to use it. 

 

That function as stated will release memory not being used.  How do you know which memory is not being used?  The compiler will decide what memory it needs and doesn't and will best optimize itself.  So you running that function may do nothing and that would be expected.  Or it may do something but what the something is, will likely be unpredictable.

 

My advice is to just trust the compiler and not use that function to release memory, unless someone at NI has said to for some fix to an issue.

 

EDIT: Okay after re-reading the function description I think it could be used and have some benefit in some programs that perform some work seldomly, and you want to discard the data before exiting the application.  One reason your code may not work is there is a race condition, where you are requesting a deallocation in parallel with creating your array.  Perform one then the other using some data flow or sequence structure. 

Share this post


Link to post
Share on other sites

I do recall a colleague having quite a bit of memory problems with large 2D string arrays. The memory would never de-allocate even after the VI had finished running. This problem was unique to strings, it did not occur when the data type was changed to something else.

 

This was LV2012, so perhaps it has been fixed.

Share this post


Link to post
Share on other sites

I also have described similar situations (I think on the NI forums).  The main takeaway was that the function is in fact named Request Deallocation, so I guess "Request denied!" is a valid response.  Actually I looked at it in some detail at the time and I seem to recall it worked like this:  string arrays are stored as an array of handles to the individual strings.  RD dutifully returns the memory used by the handle array immediately but the memory used by the individual strings is not freed.

 

I could troll AQ and discuss the LV garbage collector, but there is not one so that memory is not going to be freed in the normal sense.  I seem to recall that the memory is reusable by LV so it is not completely lost, but LV can be the RAM roach motel, bytes check in but they don't check out.

 

The quick and dirty solution I have used the few times this has been an issue is to put the offending code into an executable which I can then run dynamically in its own context.   Either as a one-shot process or as a daemon.  Reclaiming memory was as simple as shutting down the process and restarting.

  • Like 2

Share this post


Link to post
Share on other sites
...but LV can be the RAM roach motel, bytes check in but they don't check out.

:lol:

 

Request deallocation can also be a huge time hit.  I've tried it several times on 2D arrays of strings in different situations, and it's... well, not ideal.  I ended up writing my string array to a file, and storing an array of mapped integer file offsets.

 

The deallocation problem first showed up with some changes in LV6.1... apparently, it's either not soluble or not a priority.

  • Like 1

Share this post


Link to post
Share on other sites
Hi everyone, thanks for the replies.

 

It is worth to note that the problem is not incorrectly using the Request Deallocation, and generally is not in this function. Even the subVI is run through the reference (see attachment), and then stoped and the reference is being closed in the main program, the memory is not released for any, any multidimensional array (it may be a 1d array of strings or 2d array of numbers)! It should be said that LabVIEW reuses this memory, but the  resources are not returned to the system. By the way in LabVIEW 2013 this problem remained. If this "feature" are known from the earliest versions LabVIEW, then why it is not in the official list of bugs? I understand, to leave this problem out, it is necessary either to avoide using any complex arrays (above 1st order) or work with multidimensional arrays in a separate executable.

main_reference.vi

memoryusage_reference.vi

Edited by beefly

Share this post


Link to post
Share on other sites

I don't want to insult you, but you are generally a text based programmer aren't you?  I say that because at least from my perspective I don't care what LabVIEW does with memory, but I've seen other text based developers get hung up on trying to understand the low level of what is going on.  The compiler is better equipped to do as it pleases, releasing and reallocating memory as it needs.  I trust the compiler until I see a reason not to (crashes, or run away memory).  I suggest you do the same, trust the compiler.  

 

You stated that memory is released properly when you close LabVIEW, and the memory is reused within the application when it needs to be so what is the problem?  Where is the bug?  If it doesn't release the memory then it must need it for a reason that I could not predict.  I do not see is as a bug, and I deal with very large amounts of data in the form of larger sized arrays of clusters of arrays all the time.

Share this post


Link to post
Share on other sites
I don't want to insult you, but you are generally a text based programmer aren't you?  I say that because at least from my perspective I don't care what LabVIEW does with memory, but I've seen other text based developers get hung up on trying to understand the low level of what is going on.  The compiler is better equipped to do as it pleases, releasing and reallocating memory as it needs.  I trust the compiler until I see a reason not to (crashes, or run away memory).  I suggest you do the same, trust the compiler.  

 

You stated that memory is released properly when you close LabVIEW, and the memory is reused within the application when it needs to be so what is the problem?  Where is the bug?  If it doesn't release the memory then it must need it for a reason that I could not predict.  I do not see is as a bug, and I deal with very large amounts of data in the form of larger sized arrays of clusters of arrays all the time.

 

I cannot agree: The VI posted here (memoryusage_reference.VI) is a reentrant VI that you can easily use to fill the memory in use by LabVIEW - even placing a Request Deallocation in its BD does not help if this VI produces an array of strings.  With a 32-bit LabVIEW, you quite quickly get the famous "Not enough memory to complete this operation" message. Sole reason:  Request Deallocation rejected to properly clean up.  Although the documentation promises to do so.

To my opinon, this is a bug in LabVIEW and as I take from this thread, it is still living in LabVIEW 2013.

I feel encouraged to move code like this to a C/C++ DLL ...

Share this post


Link to post
Share on other sites

Coming from someone who has had to tackle memory barriers a few times in LabVIEW before, I agree the memory manager can be a little aggravating at times, but I don't see a bug here.

 

As beefly presented the VIs when executing the main I can do a single string create operation and watch the memory allocation soar. When I follow up with a release operation memory decreases by a size that seems to correlate to the number of elements (approximately N*4 bytes), but leaves a larger portion of memory allocated to LabVIEW which appears to be the actual string data, consistent with Darin's observations. This is a good example, kudos.

 

However as Hooovahh pointed out, I don't believe the memory is leaked. It is not lost track of. I can indefinitely do create/release pairs with a stable peak memory allocation. The peak will only increase if I hang onto multiple references at a time which is required to support the multiple independent data spaces.

 

This is definitely not ideal. It would be nice if LabVIEW released the memory to the host operating system so, for example a LabVIEW process which has a tendency to balloon from time to time can restore itself to a small memory footprint when idle. While I don't like the behavior, I think it's still managed in a defined way so don't see a bug here.

Share this post


Link to post
Share on other sites

Glad that smb (guentermueller) agreed with me.

But there is a note: it was verified that reentrant VIs use the same memory. Can run SubVI five times, then close the references of subVIs, then run another 5 times and you will see that more memory will not allocate...
The problem is uncontrol of the process to free memory. In case of multi-tasking operating system the deallocation of unused memory is very important in any case. It's not just about how to live with it and how to get around this, but why there is such a difference in behavior in the case of one-dimensional and multi-dimensional arrays. I remind that memory will be never released. I understand that the lack of memory is easier to solve with it increases. But deallocation occurs in different ways, which should not be, I think. Deallocation should follow the same scenario.
Edited by beefly

Share this post


Link to post
Share on other sites

Is it really a 1D vs 2D problem? I didn't try with a 2D numeric array. I was under the impression it was the indirection in the array of strings that was causing issue- that is how the array is really just an array of handles to other data.

Share this post


Link to post
Share on other sites

After what I've read in this thread, I think there may be a bug here.  I've filed CAR 440207 to investigate this behavior.  I'll update this thread when I hear more information.  

 

While I understand and appreciate hoovah's sentiment that you should just trust the compiler, I think that blindly trusting it is not always the best idea.  There's a lot that can go wrong.  It almost always does the right thing.  And it reflects well on LabVIEW that you trust it so much.  But sometimes it doesn't do the right thing.  It's important to look at LabVIEW with a critical eye sometimes.

  • Like 2

Share this post


Link to post
Share on other sites
While I understand and appreciate hoovah's sentiment that you should just trust the compiler, I think that blindly trusting it is not always the best idea.  

 

No blind trust.  I trust it until I find an issue which to be honest isn't all that often.

 

I trust the compiler until I see a reason not to (crashes, or run away memory).

Share this post


Link to post
Share on other sites
But there is a note: it was verified that reentrant VIs use the same memory. Can run SubVI five times, then close the references of subVIs, then run another 5 times and you will see that more memory will not allocate...

 

Running the VI 5 times (I am always referring to the VI in the mode when it produces an array of strings) just uses a good amount of memory that you will not get back - but just can reuse by further calls to these reentrant instances.  So you cannot run another 5 instances of this VI in LabVIEW 32-bit - or do any other very memory consuming task that would have fit into memory before executing these reentrant VIs. Oops!

I am glad to hear that Mr Mike has announced a check of this behaviour and provided a CAR.

Share this post


Link to post
Share on other sites

It's good news that NI paid attention to this problem. Waiting for the result.

 

Running the VI 5 times (I am always referring to the VI in the mode when it produces an array of strings) just uses a good amount of memory that you will not get back - but just can reuse by further calls to these reentrant instances.  So you cannot run another 5 instances of this VI in LabVIEW 32-bit - or do any other very memory consuming task that would have fit into memory before executing these reentrant VIs. Oops!

 

I'm completely agree with you and it's obvious, that if you open hundred different copies of reentrant VI, the allocated memory will be one hundred times greater than for a single call, and physical memory can easily finish. However, in a previous post I just wanted to say that one copy of reentrant VI also reuses its memory after closing. Sorry for my english...

Share this post


Link to post
Share on other sites

Hey everyone,

 

the last response here was almost a year ago, so I wonder if there was a solution for the problem? I am sill working with LV2012 und currently running into problems with memory, that I can't release with the "Request Deallocation" function.

 

Thank you very much.

Share this post


Link to post
Share on other sites

As a work around, what if you open with Option set to x100?  it seems to release memory then.  I don't know if its bad form or opens another can of worms to open with "Prepare to call and collect"  but never collect. The documentation seems to indicate so.  

 

Quote:

If you use this option flag (x100), you must include one Wait On Asynchronous Call node for every call that you begin with a Start Asynchronous Call node to ensure that LabVIEW does not retain any started calls in memory indefinitely.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.