Jump to content
beefly

Memory deallocation bug

Recommended Posts

@EvgenKo423: I don't know all the ins and outs of the SH implementation, but I do know that above a certain size threshold (I thought it was 64KiB) SH will just alloc straight from the Windows API (VirtualAlloc, I think). SH doesn't do any other management of these blocks, so when LV asks SH to free them, SH immediately calls VirtualFree. This reduction in mem footprint should be visible in Process Explorer (or Task Manager). Another complication, though, is that a performance analysis done by our NI Bangalore colleagues found that VirtualFree was quite an expensive operation, so I believe LV's memory manager layer currently keeps a small cache of freed large blocks in order to avoid the VirtualFree penalty when possible. As I said, these things are complicated. This cache was found to be effective in RFmx applications where large waveforms are being constantly allocated, resized, and freed. 

Share this post


Link to post
Share on other sites

Thanks, Rob! Very well done research with a lot of technical details, as we all here like. 🙂 After reading and re-reading your post and SH related documents and playing with the samples I still have one question. Can we control SH behaviour in any ways or is it up to LabVIEW Memory Manager completely? Say, could I make SH to empty its pools and free all the data cached, thus reclaiming the space occupied? Or it never gives it back to me entirely? Could I disable SH utilization somehow or is it hardcoded to be always on?

I found few private properties to control Memory Manager settings, e.g. Application.Selective Deallocation, Application.Selective Deallocation.EnableForAllVIs, Application.Selective Deallocation.LargeBufferThreshold and Application.NeverShrinkBuffers, but playing around these doesn't help much. I would say, it even worsens the situation in some cases. Currently I see no way to return the occupied memory, thereby LabVIEW can (and will) eat as much memory as it needs for its purposes. So, we have to live with it, don't we?..

upd: I think I found something. In [LabVIEW]\resource folder there are four variants of Memory Manager library:

  • mgcore_20_0.dll - no SH, no AT (Allocation Tracker)
  • mgcore_AT_20_0.dll - with AT
  • mgcore_AT_SH_20_0.dll - with both SH and AT
  • mgcore_SH_20_0.dll - with SH

LabVIEW uses SH version by default. If we switch to "no SH, no AT" version by backupping mgcore_SH_20_0.dll and renaming mgcore_20_0.dll to mgcore_SH_20_0.dll, then the memory consumption is somewhat reduced and we get more memory back after RD was called. On default mgcore_SH_20_0.dll I'm getting these values:

  • LabVIEW is opened and the example is running - 199 056 KB;
  • After the string array was once created (RD is on) - 779 256 KB (the peak value is at ~800 000 KB);
  • After the VI is stopped and closed - 491 096 KB.

On mgcore_20_0.dll I'm getting these values:

  • LabVIEW is opened and the example is running - 181 980 KB;
  • After the string array was once created (RD is on) - 329 764 KB (the peak value is at ~600 000 KB);
  • After the VI is stopped and closed - 380 200 KB.

Of course, it all needs more extensive testing. I see however, that "no SH, no AT" version uses less memory for the operations and so it could be prefferable, when the system is fairly RAM limited.

Edited by dadreamer
new info
  • Thanks 2

Share this post


Link to post
Share on other sites

@dadreamer, nice find!

Apparently these private properties should do exactly what you want: control SH behavior, but they allow to only control the size of large blocks (which shouldn't be the root of this problem) and don't seem to make any difference in my test anyway.
Library replacement, on the other hand, eliminates this problem completely, even for blocks <256 bytes.

Edit: Selective Deallocation is also available as an INI key along with a key "overanxiousMemoryDeallocation" (which doesn't help either).

Edited by EvgenKo423
  • Like 1

Share this post


Link to post
Share on other sites
On 7/20/2020 at 2:56 PM, EvgenKo423 said:

Apparently these private properties should do exactly what you want

Well, they're obviously not enough to have an absolute control over SH, including memory pools management as per SH API. Unfortunately I don't see any other functions or private nodes exposed, except maybe FreeSmartHeapPool function of mgcore, but that one crashes LV for some reason.

I'm afraid, my find about mgcore's switching is almost useless, because compiled app (i.e. EXE) uses lvrt.dll, which already has mgcore stuff integrated into it, so no way to disable SH in lvrt, as it would require its recompile from the sources. And I never saw any different versions of LVRT except a classic one and a Full Featured one. Honestly, I don't know, why LabVIEW is shipped with 4 variants of mgcore, even if it's using only one of them.

On 7/20/2020 at 2:56 PM, EvgenKo423 said:

key "overanxiousMemoryDeallocation" (which doesn't help either).

Yeah, it doesn't help much, because it's like you have inserted RD block in the end of every VI. In LabVIEW before 7.x there was "Deallocate memory as soon as possible" checkbox in the settings.

1213320735_21-07-20201-59-59.jpg.22ddb5c4eec9485648412cec46eb3907.jpg

Quote

Deallocate memory as soon as possible - Deallocates the memory of every VI after it completes execution. Doing this can improve memory usage in some applications because subVIs deallocate their memory immediately after executing. However, it slows performance because G must allocate and deallocate memory more frequently and in some instances it might lead to excessive memory fragmentation.

This setting was stored in INI as anxiousMemoryDeallocation token. In 7.x they removed the checkbox and likely renamed anxiousMemoryDeallocation token to overanxiousMemoryDeallocation. LabVIEW still tries to read overanxiousMemoryDeallocation on the start, thus it could be used if needed. Not much sense for that though.

By the way, this wiki page should be updated as well.

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Similar Content

    • By WILSTi
      Hello,

      I'm developing an application to play "Battleship" with another opponent through serial communication. I currently have the following difficulty:
       I have not been able to fill the Array with my opponent's ships, I can modify a position, but by changing the position, the result of the previous position returns to its original value.
      VI operation:

      For my turn, using the radio buttons, I select the coordinate and press the SEND button. My opponent answers me with a 1 if I hit or with a 0 if I failed, this data is what I couldn't keep in the array.
      Best regards,
      3Q.vi
    • By Ram Prakash
      Can anyone please tell what a DVR [ Data value reference ] is ? I want to know at what situation it will be used and what are the advantages we get by using DVR. I am really confused in this topic . If someone has any code in which they have worked with DVRs. kindly share it to me.
       
      Thank you.
    • By Chen Li
      The program obtains the number of messages in the hardware buffer through ZCAN_GetReceiveNum function, and then calls ZCAN Receive to complete the received data. When the amount of data reaches 1000 frames / s, the program will crash.
      UINT ZCAN_Receive(CHANNEL_HANDLE channel_handle,  ZCAN_Receive_Data* pReceive,  UINT len,  INT wait_time = -1);

      I think there is something wrong with creating the parameter array ZCAN Receive Data* pReceive. Has anyone ever encountered a similar problem?
       
      Receive.zip
    • By hooovahh
      View File Hooovahh Array VIMs
      Here is the Hooovahh Array VIMs.  This initial release contains 14 VIMs for manipulating array data, which are intended to replace OpenG functionality, but with the added benefit of data type propagation, and increased performance using newer array manipulation techniques.  In later versions other Array manipulation functions were added moving all the OpenG stuff to their own palette.
      Most of the OpenG functions are unchanged, but a few use the newer conditional and concatenating tunnels.  And a few functions have added performance based on other inputs.  For instance the Delete Array Elements can operate in a more efficient way if the input indexes are already sorted.  The Filter 1D array can also be more efficient if the input is known to not contain any duplicates.
      Because these packages contain VIMs, they require LabVIEW 2017 or newer.  Having these functions be VIMs mean all functions work with various array data types.  Included functions are:
      Conditional Auto-Indexing Tunnel Delete Elements from (1D or 2D) Array Filter 1D Array Index (1D or 2D) Array Remove Duplicates from 1D Array Reorder (1D or 2D) Array Reverse 1D Array Slice 1D Array Sort (1D or 2D) Array Convert 1D to 2D Convert 2D to 1D Find Subarray Force Array Min/Max Size Foreign Key Sort Submitter hooovahh Submitted 10/11/2017 Category *Uncertified* LabVIEW Version  
    • By Tim_S
      Thought I'd pass this along and see if anyone can reproduce with different versions of LabVIEW. Appreciate it if anyone has seen this and has a fix.
      I'm using shared variables to communicate between applications (1:N). I'd been seeing some memory creep that was inconsistent and somewhat bizarre. Eventually managed to track it down to that I'm programmatically opening a connection to a shared variable in one loop, then reading the value in a different loop (the different loops have to do with reconnecting on connection loss and startup). There is a functional global used to pass the variable to the second loop. The Read Variable primitive deallocates all but 4 bytes of memory for the previous loop handle and then allocates memory for a new handle on each iteration of the while loop, hence creating a leak. This behavior does not occur if there is only one loop where there is an open, while loop with a read, and a close.
      Main.vi demonstrates the issue. Main 2.vi is more like the NI example.
      I've got service request #7728859 with NI going, but I think I got the guy's first day.
      LabVIEW 2015 SP1 32-bit on Win7 64-bit. Shared Variables memory leak.zip
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.