Buffer Allocation

[jgcode]

I had a memory leak that I had to plug and just want to share and discuss the results:

Background

Nothing new, but I have a Buffer Interface Class that is composed of an array LabVIEW Objects. I does all the required transactions for a buffer on the array.

I inherit from this Parent and create wrappers for specific datatypes to create a much nicer API for my application code.

After discovering the memory leak for a specific datatype (1D DBL + timestamp) I was able to squash it.

I have a 100 pt array of DBLs + timestamp (6528 bits) in a 5000 element buffer = (6528 * 5000)/(8*10^6) = 4.08 MB buffer

What really surprised me was the ferocity of the leak.

Check out the stats!

I initially used Build Array primitive originally, thinking that an allocation has to occur here anyways. As you can see from the Test VI on the left, these means that I am using Build Array in a loop which is LabVIEW 101 of what not to do.

That is fine, but what this resulted in was a 250+ MB with 50+ MB swings of memory allocations!

It was running so slow after 3000+ inserts I gave up waiting.

Switching to Insert Array primitive, the allocation was only 12MB which makes sense: the buffer, the FP of the test VI and most likely a subVI interface. A gradual allocation occurs because as the buffer fills up as the Buffer Interface returns a subset of the full buffer. Once the buffer is full tho, the full buffer is returned and no further allocations occur (yay!).

I am really surprised at the difference between the two primitives, I thought from using the build array I would get some of it back, instead it went crazy.

But it sure was a fun testing it all.

[Grampa_of_Oliva_n_Eden]

I had a memory leak that I had to plug and just want to share and discuss the results:

Background

Nothing new, but I have a Buffer Interface Class that is composed of an array LabVIEW Objects. I does all the required transactions for a buffer on the array.

I inherit from this Parent and create wrappers for specific datatypes to create a much nicer API for my application code.

After discovering the memory leak for a specific datatype (1D DBL + timestamp) I was able to squash it.

I have a 100 pt array of DBLs + timestamp (6528 bits) in a 5000 element buffer = (6528 * 5000)/(8*10^6) = 4.08 MB buffer

What really surprised me was the ferocity of the leak.

Check out the stats!

I initially used Build Array primitive originally, thinking that an allocation has to occur here anyways. As you can see from the Test VI on the left, these means that I am using Build Array in a loop which is LabVIEW 101 of what not to do.

That is fine, but what this resulted in was a 250+ MB with 50+ MB swings of memory allocations!

It was running so slow after 3000+ inserts I gave up waiting.

Switching to Insert Array primitive, the allocation was only 12MB which makes sense: the buffer, the FP of the test VI and most likely a subVI interface. A gradual allocation occurs because as the buffer fills up as the Buffer Interface returns a subset of the full buffer. Once the buffer is full tho, the full buffer is returned and no further allocations occur (yay!).

I am really surprised at the difference between the two primitives, I thought from using the build array I would get some of it back, instead it went crazy.

But it sure was a fun testing it all.

A quick loo at that first one reminds me of the condition we learned about in the "Clear as Mud" thread on the dark-side. WIring that buffer through the VI may let LV re-sue the buffer. I say "may" because I'd need to watch some buffer dots to be convinced.

Ben

[jgcode]

A quick loo at that first one reminds me of the condition we learned about in the "Clear as Mud" thread on the dark-side. WIring that buffer through the VI may let LV re-sue the buffer. I say "may" because I'd need to watch some buffer dots to be convinced.

Ben

Hi Ben!

Can you go into more detail or post a link?

...Bad news is that I squashed this leak only to find out my X-Control that displays the buffer is causing the same massive allocations!

Time to get my hands dirty again....

[Grampa_of_Oliva_n_Eden]

Hi Ben!

Can you go into more detail or post a link?

...Bad news is that I squashed this leak only to find out my X-Control that displays the buffer is causing the same massive allocations!

Time to get my hands dirty again....

About four years ago, the topic of discusion turned to one of the test in the VI Analyzer that did not make sense and even Darren could not explian why it was doing that check and could only tell us that "Greg McKaskle said to put that in there." (paraphrasing of course).

The check in question was enusirng the connector on the VI icon connector where located on the root of the diagram. I posted that I shut that ceck off since it did not make sense blah blah blah.

Well it turns out that if we constrcut our sub-VIs correctly LV can "look into" the sub-VI call and determine if it can pass the data as a ref or by value.

The thread in qwuestion can be found here

http://forums.ni.com/t5/LabVIEW/case-structure-parameter-efficiency/m-p/383019#M191864

I included benchmark code so you can try it for yourself.

Ben

[jgcode]

The thread in qwuestion can be found here

http://forums.ni.com.../383019#M191864

I included benchmark code so you can try it for yourself.

Thx Ben

I will check it out.

[asbo]

I am really surprised at the difference between the two primitives, I thought from using the build array I would get some of it back, instead it went crazy.

But it sure was a fun testing it all.

Sounds like you had a thrilling weekend Benchmarks like these always interest me, it's neat to see how we can tweak implementations that seem trivial but have staggering effects.

[jgcode]

Sounds like you had a thrilling weekend Benchmarks like these always interest me, it's neat to see how we can tweak implementations that seem trivial but have staggering effects.

Yer, it still blows my mind at how crazy it went with just a 4MB array!

And don't worry, I still have another installment (X-Control) waiting for me tomorrow!

At first glance there is no Build Array's etc... but there are options to transpose the buffer, then th buffer gets reformatted to an XY Graph.

That should be fun, can't wait!

[ShaunR]

Yer, it still blows my mind at how crazy it went with just a 4MB array!

And don't worry, I still have another installment (X-Control) waiting for me tomorrow!

At first glance there is no Build Array's etc... but there are options to transpose the buffer, then th buffer gets reformatted to an XY Graph.

That should be fun, can't wait!

Spooky

I was playing with Xconrols this weekend too (not fr buffer stuff though).

I shelved it in the end since I was really annoyed that a XControl doesn't inherit all the properties and methods of the base data type meaning you have to re-implement all the built in stuff.

[jgcode]

Spooky

I was playing with Xconrols this weekend too (not fr buffer stuff though).

I shelved it in the end since I was really annoyed that a XControl doesn't inherit all the properties and methods of the base data type meaning you have to re-implement all the built in stuff.

I have been playing with them more and more lately and getting a good feel for the style thats required to use them.

I have a few gripes too that make me stay away from them, e.g. I can't include it's reference as a member of a Class (works in src not in build) etc...

But some really powerful stuff there - they just need a little make-over IMHO.

[ShaunR]

I have been playing with them more and more lately and getting a good feel for the style thats required to use them.

I have a few gripes too that make me stay away from them, e.g. I can't include it's reference as a member of a Class (works in src not in build) etc...

But some really powerful stuff there - they just need a little make-over IMHO.

Indeed. My gripes are a bit more fundamental than that though.

All I wanted to do was add a "Clear" method,alternating row colours and make it accept arrays of strings instead of using the "Item Strings" property to a multicolumn list box. But the amount of effort in re-implementing all the standard stuff just doesn't make it worth it. Only took me 15 minutes to create the control and make the changes I wanted, then it looked like another 2 days to re-implement the standard stuff.

[Yair]

I shelved it in the end since I was really annoyed that a XControl doesn't inherit all the properties and methods of the base data type meaning you have to re-implement all the built in stuff.

You may wish to look at this -

I haven't used it myself, so I can't comment on it.

[ShaunR]

You may wish to look at this - http://lavag.org/fil...ol-inheritance/

I haven't used it myself, so I can't comment on it.

Thanks Yair. Unfortunately uses OGlibs so cannot try it. It talks about inheriting "daughters'" properties which might be the reverse of what I was seeking.But should we really use a tool to do this? I did briefly look at scripting to generate the VIs for the properties and methods, but couldn't get a list of them from the original control. But still. That's a lot of VIs to do what the control already does

[MikaelH]

Just one thing I saw, the codes will not give the same result since when you use the Insert Into Array, you have wired the Zero contant to the column input.

Do you get the same behavior when wiring it to the Row input?

Cheers,

Mikael

[jgcode]

Just one thing I saw, the codes will not give the same result since when you use the Insert Into Array, you have wired the Zero contant to the column input.

Do you get the same behavior when wiring it to the Row input?

Hi Mikael

To confirm, yes you get the same normal behaviour.

Cheers

-JG

[jgcode]

Ok, the X-Control seems to be a bug? that I can replicate with a default X-Control.

I have posted my findings here and will also follow up with NI support.

Please check it out if you have the time. I have posted video and the code to replicate it.

Cheers

-JG