[Bug?] X-Control Memory Leak with Large Data Array

[jgcode]

[LV2009]

[Cross-posted to ni.com]

I have found that if I pass a large data array (~4MB in this example) into an X-Control, it causes massive memory allocations (1 GB+).

Is this a known issue?

The X-Control in the video was created, then the Data.ctl was changed to 2D Array - it has not been edited in any other way.

I also compare the allocations to that of a native 2D Array (which is only ~4MB).

Note: I jiggled the Windows Task Manager about so that JING would update correctly, its a bit slow, but it essentially just keeps rolling up and doesn't stop.

Demo code attached.

Related to this thread by separated as posted to NI for support.

Cheers

-JG

<object id="scPlayer" class="embeddedObject" width="921" height="858" type="application/x-shockwave-flash" data="http://content.screencast.com/users/jgcode/folders/LabVIEW/media/1bfce9bd-6cea-4622-ad26-8da61f63adc7/jingswfplayer.swf"> <param name="movie" value="http://content.screencast.com/users/jgcode/folders/LabVIEW/media/1bfce9bd-6cea-4622-ad26-8da61f63adc7/jingswfplayer.swf"> <param name="quality" value="high"> <param name="bgcolor" value="#FFFFFF"> <param name="flashVars" value="thumb=http://content.screencast.com/users/jgcode/folders/LabVIEW/media/1bfce9bd-6cea-4622-ad26-8da61f63adc7/FirstFrame.jpg&containerwidth=921&containerheight=858&content=http://content.screencast.com/users/jgcode/folders/LabVIEW/media/1bfce9bd-6cea-4622-ad26-8da61f63adc7/X-Control%20Memory%20Leak%20Demo.swf&blurover=false"> <param name="allowFullScreen" value="true"> <param name="scale" value="showall"> <param name="allowScriptAccess" value="always"> <param name="base" value="http://content.screencast.com/users/jgcode/folders/LabVIEW/media/1bfce9bd-6cea-4622-ad26-8da61f63adc7/"> Unable to display content. Adobe Flash is required. </object>

X Control Bug LV2009.zip

[Olivier Jourdan]

Hello JG,

This memory leak is really frightening! Using LV2010 gives the same behavior.I've came across memory leaks on XControl. I've pointed issue in the undo/redo mechanism. I've tried to apply workaround used to fixed my issue on your example. It was unsuccessful.I've played few minutes with your example. I've noticed the following (maybe you've already note):

1. Replace "millisecond multiple" constant by a control
2. Execute your code
3. Change the value from 1 to 100

On my PC, when update speed is less than 50, memory increase. When it's more than 50, memory decrease and stay to the same amount.

There is something strange, if it was a "simple" memory leak the memory shouldn't decrease.

I think we need the lights of NI engineers.

[Jeffrey Habets]

On my PC, when update speed is less than 50, memory increase. When it's more than 50, memory decrease and stay to the same amount.

There is something strange, if it was a "simple" memory leak the memory shouldn't decrease.

I think we need the lights of NI engineers.

Seems something is being Queued up there..

[Antoine Chalons]

Seems something is being Queued up there..

It really does!

Last week I was helping a customer to solve speed issue with an X-Control, activating the "synchronous display" option solved the issue.

[jgcode]

Seems something is being Queued up there..

Hey Guys!

Thanks heaps for the testing and feedback, much appreciated.

Here are some more of my observations from further testing that tie in with yours - so I think this is starting to make sense!

• Using a smaller array (1 element instead of 100) as the element of the buffer, the issue went away
• I also made some changes that sped up the X-Control (e.g. update graph by explicit reference in subVI to implicit reference on facade) which also helped with the above
• If I smash the X-Control (fast, as per the demo) and hit the stop button, the VI stops but the X-Control continues updating for what can be a while.

So yes, I am inclined to believe the data is being queued somewhere -> because it can't keep up, and hence this is the cause of the allocations
The concerning thing is: my application (whilst doing other things in parallel) was only updating the X-Control every 1000ms and was still causing a 1-2+MB/s leak!
This X-Control could, instead of queuing if it ran late, just ignore data and catch up (given I am not buffering anything inside it) - but that is out of my control.
For the design I initially wanted to define the interface as a '2D Array of DBL's' (the buffer). So to allow if I want to user the buffer for something else.
Maybe I will have to look at something else. Ideas:

• I could buffer the data inside the X-Control so the behavior is like a Chart not a Graph - so the interface just becomes a '1D Array of DBL's'?
• Or I could reference an external buffer (i.e. using a DVR)? - but didn't want to do that.

Thoughts?

Cheers

-JG

<edit>Tried toggling Synchronous Display but it did not help the allocations, thanks for suggestion tho</edit>

[jgcode]

Damn - got stung by this one again, each time data (class) was passed to the X-Control's Property Node interface, a 2MB allocation occurred.

I pulled the code out of the X-Control (it was classes, so it wasn't that hard to do), ran the code on a VI with standard LabVIEW controls (XY graph, ring etc...) - memory leak fixed!

[SuperS_5]

Hi,

I stumbled on this and experimented a little bit. The exact execution of the loop that causes the queing seams to be base upon CPU saturation. If I do something that uses up a lot of cycles compounds the queing issue. It seams that the xnode is getting a lower execution priority, not matter what I selected in the VI properties. Even the smallest amount shortage of CPU cycles in the user interface thread causes the issue. In the referring post you mentioned that you have a CAR for this issue. Any response from NI?

[jgcode]

In the referring post you mentioned that you have a CAR for this issue. Any response from NI?

Hi SuperS_5

The issue is with R&D however, I followed up with NI AE to get the actual CAR Number - #257325.

I have been chasing it up periodically over the last 6 weeks, but there is nothing new to post.

(I have no clue really) but I don't imagine there would be a simple fix?

Also, I am hoping that an X-Control polish will be part of the 2011 stability release.

That would be sweet.

[SuperS_5]

Stability is always nice. I guess for now we have to be careful in the way we deal with x-nodes.

[SuperS_5]

Hi again. I was looking at a some other code, and had a light bulb moment. If you update a stack instead of the x-node, and call the x-node, which reads from that stack, allows more control over the queuing. Of course this means a lot more coding, but at least no memory leak if done correctly. Although I am only guessing, I think the x-node is called by the UI thread. This method, allows for the stack to run in a separate thread, with its own priority. When used in your example, I had no memory leak, while using a free running loop to update the x-node and stack.

Edit: I forgot to add that you would call the x-node using some other data. I used an empty array, but, theoretically any data could be used. The x-node data-type could even be something totally different.

Edited December 19, 2010 by SuperS_5

[jgcode]

Or I could reference an external buffer (i.e. using a DVR)? - but didn't want to do that.

I was looking at a some other code, and had a light bulb moment. If you update a stack instead of the x-node,

Are you talking about something similar to my suggestion above? Are you able to go into more detail?

[SuperS_5]

I guess I missed that post. I didn't get the "DVR" part. (The first meaning that I think of is Digital Video Recorder, but I know that I am just confused Past job experience.) I suppose its the same, but I was thinking a simple vi with an unregistered shift register. I would put all of the "real time" data processing code there instead of the x-control. That way the X-control would just pull it out of the "buffer." I guess it wasn't such a light bulb moment after all.

PS. I just realized that I used x-node above, where I meant x-control.

[jgcode]

I guess I missed that post. I didn't get the "DVR" part. (The first meaning that I think of is Digital Video Recorder, but I know that I am just confused Past job experience.) I suppose its the same, but I was thinking a simple vi with an unregistered shift register. I would put all of the "real time" data processing code there instead of the x-control. That way the X-control would just pull it out of the "buffer." I guess it wasn't such a light bulb moment after all.

PS. I just realized that I used x-node above, where I meant x-control.

DVR = Data Value Reference

[SuperS_5]

DVR = Data Value Reference

I had to look that up. My method would be a memory hog, but, still without a memory leak. The DVR method would be more efficient, and essentially the same external functionality, but with more programming.

PS. That link did not work, but I did find other NI pages on it. Thanks for the translation.:thumbup1:lol

[jgcode]

PS. That link did not work, but I did find other NI pages on it. Thanks for the translation.:thumbup1:lol

Judging by other thread, NI might have been down? - URL works for me now tho.

[Olivier Jourdan]

Or I could reference an external buffer (i.e. using a DVR)? - but didn't want to do that.

Hello JG

Reading again this thread, I notice that this workaround have a lack. You'll lose the recording mechanism integrated in the facade.vi. if you never write "Data".

This can be really annoying if you want to distribute your XControl. It won't behave as the user expect to.

We really need a bug fix from NI

Regards

[jgcode]

Hello JG

Reading again this thread, I notice that this workaround have a lack. You'll lose the recording mechanism integrated in the facade.vi. if you never write "Data".

This can be really annoying if you want to distribute your XControl. It won't behave as the user expect to.

We really need a bug fix from NI

Regards

I like your persistence!

I just prodded the support engineer again just then.

[SuperS_5]

I totally agree that a work-around is not a fix, but temporary solution. Thanks for prodding.

[jgcode]

Hey Guys

LabVIEW R&D has responded in cross-post, jump across to the darkside if you want to chime in.

Cheers

-JG