Memory deallocation bug

[beefly]

Hello!

It was found that the function Request Deallocation does not free memory after using an array of strings (or any other multi-dimensional array).

Description of the experiment:

There is a VI from which subVI is called by command. In the subVI only one-dimensional array (numbers or strings) generated. If the array element type is number (I32), then allocated memory is released (using the function Request Deallocation). If the array element type is string (or any other multi-dimensional array), the memory will not be released. Anyone faced this problem? If yes, what should we do?

Excuse for bad english.

main.vi

memoryusage.vi

[hooovahh]

Why are you using that function?  The reason I ask is because no program that I have ever written has needed it.  Every time I see it used it didn't need to be.  I'm not saying it doesn't serve a purpose I am just trying to say you don't need to use it. 

 

That function as stated will release memory not being used.  How do you know which memory is not being used?  The compiler will decide what memory it needs and doesn't and will best optimize itself.  So you running that function may do nothing and that would be expected.  Or it may do something but what the something is, will likely be unpredictable.

 

My advice is to just trust the compiler and not use that function to release memory, unless someone at NI has said to for some fix to an issue.

 

EDIT: Okay after re-reading the function description I think it could be used and have some benefit in some programs that perform some work seldomly, and you want to discard the data before exiting the application.  One reason your code may not work is there is a race condition, where you are requesting a deallocation in parallel with creating your array.  Perform one then the other using some data flow or sequence structure. 

[Neil Pate]

I do recall a colleague having quite a bit of memory problems with large 2D string arrays. The memory would never de-allocate even after the VI had finished running. This problem was unique to strings, it did not occur when the data type was changed to something else.

 

This was LV2012, so perhaps it has been fixed.

[Darin]

I also have described similar situations (I think on the NI forums).  The main takeaway was that the function is in fact named Request Deallocation, so I guess "Request denied!" is a valid response.  Actually I looked at it in some detail at the time and I seem to recall it worked like this:  string arrays are stored as an array of handles to the individual strings.  RD dutifully returns the memory used by the handle array immediately but the memory used by the individual strings is not freed.

 

I could troll AQ and discuss the LV garbage collector, but there is not one so that memory is not going to be freed in the normal sense.  I seem to recall that the memory is reusable by LV so it is not completely lost, but LV can be the RAM roach motel, bytes check in but they don't check out.

 

The quick and dirty solution I have used the few times this has been an issue is to put the offending code into an executable which I can then run dynamically in its own context.   Either as a one-shot process or as a daemon.  Reclaiming memory was as simple as shutting down the process and restarting.

[ShaunR]

I could troll AQ and discuss the LV garbage collector, 

Prepare to be shouted at

[jzoller]

...but LV can be the RAM roach motel, bytes check in but they don't check out.

 

Request deallocation can also be a huge time hit.  I've tried it several times on 2D arrays of strings in different situations, and it's... well, not ideal.  I ended up writing my string array to a file, and storing an array of mapped integer file offsets.

 

The deallocation problem first showed up with some changes in LV6.1... apparently, it's either not soluble or not a priority.

[beefly]

Hi everyone, thanks for the replies.

 

It is worth to note that the problem is not incorrectly using the Request Deallocation, and generally is not in this function. Even the subVI is run through the reference (see attachment), and then stoped and the reference is being closed in the main program, the memory is not released for any, any multidimensional array (it may be a 1d array of strings or 2d array of numbers)! It should be said that LabVIEW reuses this memory, but the  resources are not returned to the system. By the way in LabVIEW 2013 this problem remained. If this "feature" are known from the earliest versions LabVIEW, then why it is not in the official list of bugs? I understand, to leave this problem out, it is necessary either to avoide using any complex arrays (above 1st order) or work with multidimensional arrays in a separate executable.

main_reference.vi

memoryusage_reference.vi

Edited December 6, 2013 by beefly

[hooovahh]

I don't want to insult you, but you are generally a text based programmer aren't you?  I say that because at least from my perspective I don't care what LabVIEW does with memory, but I've seen other text based developers get hung up on trying to understand the low level of what is going on.  The compiler is better equipped to do as it pleases, releasing and reallocating memory as it needs.  I trust the compiler until I see a reason not to (crashes, or run away memory).  I suggest you do the same, trust the compiler.  

 

You stated that memory is released properly when you close LabVIEW, and the memory is reused within the application when it needs to be so what is the problem?  Where is the bug?  If it doesn't release the memory then it must need it for a reason that I could not predict.  I do not see is as a bug, and I deal with very large amounts of data in the form of larger sized arrays of clusters of arrays all the time.

[guentermueller]

I don't want to insult you, but you are generally a text based programmer aren't you?  I say that because at least from my perspective I don't care what LabVIEW does with memory, but I've seen other text based developers get hung up on trying to understand the low level of what is going on.  The compiler is better equipped to do as it pleases, releasing and reallocating memory as it needs.  I trust the compiler until I see a reason not to (crashes, or run away memory).  I suggest you do the same, trust the compiler.  

 

You stated that memory is released properly when you close LabVIEW, and the memory is reused within the application when it needs to be so what is the problem?  Where is the bug?  If it doesn't release the memory then it must need it for a reason that I could not predict.  I do not see is as a bug, and I deal with very large amounts of data in the form of larger sized arrays of clusters of arrays all the time.

 

I cannot agree: The VI posted here (memoryusage_reference.VI) is a reentrant VI that you can easily use to fill the memory in use by LabVIEW - even placing a Request Deallocation in its BD does not help if this VI produces an array of strings.  With a 32-bit LabVIEW, you quite quickly get the famous "Not enough memory to complete this operation" message. Sole reason:  Request Deallocation rejected to properly clean up.  Although the documentation promises to do so.

To my opinon, this is a bug in LabVIEW and as I take from this thread, it is still living in LabVIEW 2013.

I feel encouraged to move code like this to a C/C++ DLL ...

[mje]

Coming from someone who has had to tackle memory barriers a few times in LabVIEW before, I agree the memory manager can be a little aggravating at times, but I don't see a bug here.

 

As beefly presented the VIs when executing the main I can do a single string create operation and watch the memory allocation soar. When I follow up with a release operation memory decreases by a size that seems to correlate to the number of elements (approximately N*4 bytes), but leaves a larger portion of memory allocated to LabVIEW which appears to be the actual string data, consistent with Darin's observations. This is a good example, kudos.

 

However as Hooovahh pointed out, I don't believe the memory is leaked. It is not lost track of. I can indefinitely do create/release pairs with a stable peak memory allocation. The peak will only increase if I hang onto multiple references at a time which is required to support the multiple independent data spaces.

 

This is definitely not ideal. It would be nice if LabVIEW released the memory to the host operating system so, for example a LabVIEW process which has a tendency to balloon from time to time can restore itself to a small memory footprint when idle. While I don't like the behavior, I think it's still managed in a defined way so don't see a bug here.

[beefly]

Glad that smb (guentermueller) agreed with me.

But there is a note: it was verified that reentrant VIs use the same memory. Can run SubVI five times, then close the references of subVIs, then run another 5 times and you will see that more memory will not allocate...

The problem is uncontrol of the process to free memory. In case of multi-tasking operating system the deallocation of unused memory is very important in any case. It's not just about how to live with it and how to get around this, but why there is such a difference in behavior in the case of one-dimensional and multi-dimensional arrays. I remind that memory will be never released. I understand that the lack of memory is easier to solve with it increases. But deallocation occurs in different ways, which should not be, I think. Deallocation should follow the same scenario.

Edited December 9, 2013 by beefly

[mje]

Is it really a 1D vs 2D problem? I didn't try with a 2D numeric array. I was under the impression it was the indirection in the array of strings that was causing issue- that is how the array is really just an array of handles to other data.

[Mr Mike]

After what I've read in this thread, I think there may be a bug here.  I've filed CAR 440207 to investigate this behavior.  I'll update this thread when I hear more information.  

 

While I understand and appreciate hoovah's sentiment that you should just trust the compiler, I think that blindly trusting it is not always the best idea.  There's a lot that can go wrong.  It almost always does the right thing.  And it reflects well on LabVIEW that you trust it so much.  But sometimes it doesn't do the right thing.  It's important to look at LabVIEW with a critical eye sometimes.

[hooovahh]

While I understand and appreciate hoovah's sentiment that you should just trust the compiler, I think that blindly trusting it is not always the best idea.  

 

No blind trust.  I trust it until I find an issue which to be honest isn't all that often.

 

I trust the compiler until I see a reason not to (crashes, or run away memory).

[guentermueller]

But there is a note: it was verified that reentrant VIs use the same memory. Can run SubVI five times, then close the references of subVIs, then run another 5 times and you will see that more memory will not allocate...

 

Running the VI 5 times (I am always referring to the VI in the mode when it produces an array of strings) just uses a good amount of memory that you will not get back - but just can reuse by further calls to these reentrant instances.  So you cannot run another 5 instances of this VI in LabVIEW 32-bit - or do any other very memory consuming task that would have fit into memory before executing these reentrant VIs. Oops!

I am glad to hear that Mr Mike has announced a check of this behaviour and provided a CAR.

[beefly]

It's good news that NI paid attention to this problem. Waiting for the result.

 

Running the VI 5 times (I am always referring to the VI in the mode when it produces an array of strings) just uses a good amount of memory that you will not get back - but just can reuse by further calls to these reentrant instances.  So you cannot run another 5 instances of this VI in LabVIEW 32-bit - or do any other very memory consuming task that would have fit into memory before executing these reentrant VIs. Oops!

 

I'm completely agree with you and it's obvious, that if you open hundred different copies of reentrant VI, the allocated memory will be one hundred times greater than for a single call, and physical memory can easily finish. However, in a previous post I just wanted to say that one copy of reentrant VI also reuses its memory after closing. Sorry for my english...

[rahe]

Hey everyone,

 

the last response here was almost a year ago, so I wonder if there was a solution for the problem? I am sill working with LV2012 und currently running into problems with memory, that I can't release with the "Request Deallocation" function.

 

Thank you very much.

[bbean]

As a work around, what if you open with Option set to x100?  it seems to release memory then.  I don't know if its bad form or opens another can of worms to open with "Prepare to call and collect"  but never collect. The documentation seems to indicate so.  

 

Quote:

If you use this option flag (x100), you must include one Wait On Asynchronous Call node for every call that you begin with a Start Asynchronous Call node to ensure that LabVIEW does not retain any started calls in memory indefinitely.

[Rolf Kalbermatter]

Hey everyone,

 

the last response here was almost a year ago, so I wonder if there was a solution for the problem? I am sill working with LV2012 und currently running into problems with memory, that I can't release with the "Request Deallocation" function.

 

Thank you very much.

 

Generally, "Request Deallocation" is not the right solution for problems like this, even if it works more agressive than it did in previous versions, which I'm not aware of that it does.

 

You should first think about the algorithm and consider changing the data format or storage to a better suited way. LabVIEW is not C! In C you not only can control memory to the last byte allocated, you actually have to and if you miss one location you create memory leaks or buffer overflows. In LabVIEW you generally don't control memory explicitly and can't even do so to the byte level at all. It's similar with other high level languages where memory allocation and deallocation is mostly handled by the compiler system rather than the code programmer.

 

Writing a program in C that is guaranteed to not leak any memory or do buffer overwrites ever is a major exercise that only few programmers manage to do after lots and lots of debugging. So having this pushed into the compiler which can be formally designed and tested to do the right things is a major step into better programs. It takes away some control and tends to cost memory due to things like lazy dealloaction (mostly for performance reasons but also sometimes as defensive means since especially in multithreading environments there is not always a 100% guarantee that deallocating a memory area would be a safe operation at that point).

 

Request Deallocation basically has to stop any and all LabVIEW threads to make sure that no race condition can occur when a block marked as currently unused is freed while another thread attempts to reuse that block again. As such it is a total performance killer, not only because of causing many extra memory allocations and deallocations but also because it has to stop just about anything in LabVIEW for the duration of doing its work.

[EvgenKo423]

Unfortunately, it's still not fixed in LV 2018 SP1. Didn't check it specifically in 2020, but I guess it's the same.
I wonder what happened to this CAR?

[dadreamer]

1 hour ago, EvgenKo423 said:

Didn't check it specifically in 2020, but I guess it's the same.

Yeah! LabVIEW 2020 64-bit - RD doesn't work for strings as it should do. Even when the VI is unloaded, LabVIEW still holds some memory allocated and never releases.

1 hour ago, EvgenKo423 said:

I wonder what happened to this CAR?

No tracks of it on NI forums. Anyone with internal access to the list?

[Mr Mike]

I'm no longer with NI (as of 2014!), but I just got notification on this thread.  I'll forward it to someone who's still there and ask them to respond.

[rob.dye]

TL;DR: This is NOT a bug. It is all explainable by the normal behavior of the memory management mechanisms used by LabVIEW, including a memory allocator layer provided by SmartHeap (from MicroQuill).

Details:
Actually the original bug report in Dec 2013 by Mr Mike (bonjour, Mike!) was pretty accurately analyzed and documented by Ryan P in 2014 and the bug was closed then. Mike's post from today did manage to gain the attention of someone else at NI, who asked me to take a look. I reviewed the VIs from this page and decided I could explain all the behavior with actual numbers. See the enclosed picture of Process Explorer's trace of LabVIEW 2019 (64-bit) memory usage during a session looking at these VIs.

The labels I've placed in the picture attempt to explain it all, but I'll summarize by saying that all the "lost" memory (around 422MB) can be explained by the 10M master pointers managed by LV, plus 10M _freed_ small blocks sitting in pages managed by SmartHeap. 

These _freed_ blocks are not "lost". SmartHeap knows where they are and will let us use them again, although because they are small SH keeps them in special low-overhead pools that are used _only_ for small allocations, and who knows when LV will need 10M small blocks again. 

These _freed_ blocks were formerly the 10M strings with one "space" character in them, each of which actually take up probably 40 or 48 bytes. Each string block has a LV-managed 32 byte header, plus a 4 byte length, plus 1 byte for the "space" char. LV asks Smartheap for this 37 byte block, and they probably give us a 40 (rounded up to multiple of 8?) or 48 (rounded up to multiple of 16?) byte smallblock to contain our request. Small blocks in SH are low overhead because they only require a single bit to represent their inUse state and require no header of their own.

The numbers don't all add up exactly, but they are sufficiently in the ballpark that any slop is explainable by the the fact that a lot of other stuff is going on in the memory management arena in LV. It's complicated. There could be significant amounts of fragmented small free blocks already available. Process Explorer could be reporting in mebibytes vs. megabytes. etc.

Hope this helps.
Rob

[Rolf Kalbermatter]

7 hours ago, rob.dye said:

The labels I've placed in the picture attempt to explain it all, but I'll summarize by saying that all the "lost" memory (around 422MB) can be explained by the 10M master pointers managed by LV, plus 10M _freed_ small blocks sitting in pages managed by SmartHeap. 

Wasn't aware that LabVIEW still uses SmartHeap. I knew it did in LabVIEW 2.5 up to around 6.0 or so but thought you guys had dropped that when the memory managers of the underlaying OS got smart enough themselves to not make a complete mess of things (that specifically applied to the Win3.1 memory management and to a lesser extend to the MacOS Classic one).

Edited July 16, 2020 by Rolf Kalbermatter

[EvgenKo423]

Thanks for such a detailed explanation, Rob!

 

I just played with string array a bit more and wanna share my findings with the others.

15 hours ago, rob.dye said:

These _freed_ blocks are not "lost". SmartHeap knows where they are and will let us use them again, although because they are small SH keeps them in special low-overhead pools that are used _only_ for small allocations, and who knows when LV will need 10M small blocks again. 

These _freed_ blocks were formerly the 10M strings with one "space" character in them, each of which actually take up probably 40 or 48 bytes. Each string block has a LV-managed 32 byte header, plus a 4 byte length, plus 1 byte for the "space" char. LV asks Smartheap for this 37 byte block, and they probably give us a 40 (rounded up to multiple of 8?) or 48 (rounded up to multiple of 16?) byte smallblock to contain our request. Small blocks in SH are low overhead because they only require a single bit to represent their inUse state and require no header of their own.

According to SmartHeap specs, small should be considered blocks under 256 bytes, but LabVIEW starts to release memory as expected only for blocks which are 1048356 + 36 = 1048392 (1 MiB - 184) bytes in size and above. Is it still an expected behavior for LabVIEW?
Anyway, even 256 bytes is probably quite big length for a typical string to get properly garbage-collected.

TestStringArray.vi

Edited July 16, 2020 by EvgenKo423