OpenG filter array revised

[GregSands]

Greg: Yes, making the subVI inline does make it possible for the output to avoid calculation entirely. Doing it on any subVI call is something that the compiler optimization team has on its list of possible optimizations, but not currently implemented.

So, in the case described here, does that include not allocating the "Indices of removed elements" array before the For loop? If so, I'm pretty impressed.

[Mads]

I've now optimized the last of the array functions that can be noticeably improved; the search 1D array function.

The optimization I've done is to replace the repeated result array building with a dynamic array (preallocated and then resized in growing chunks only when needed). This will gain more speed the bigger the result array is, and should normally save some memory as well.

Test results:

0 (sought value covers 0% of the input array) to 6x (100% sought value) increase in speed.

OpenG Array functions improved Rev3.zip

[gregstoll]

So, in the case described here, does that include not allocating the "Indices of removed elements" array before the For loop? If so, I'm pretty impressed.

Yes - for unwired outputs in an inlined VI, we will dead code eliminate anything that is only used for that output (since it looks like the output of a node that no one's using). If you have particular VIs you want me to verify I can do that, but I'm pretty sure it should work. Inlining is neat!

[crossrulz]

Mads,

I'm curious about the reason for a FOR loop instead of a while loop. It looks to me like you are trying to catch that case where the item found is the last element in the array so an extra search isn't performed. Wouldn't it be better to use a while loop and compare the index to the highest index (i >= N-1, N-1 is calculated before the loop), OR this result with the equal to -1 to get the stop condition? I'll have to see if I can find some time to play around with it.

[crossrulz]

I was able to tweek out a little more performance by doing what I said above. In all, this VI is almost 9x faster (on my machine) than the current OpenG Search 1D Array.

It might be a good idea so see if we can make a good subVI for dynamically increasing the size of an array similar to what Mads made in his Search 1D Array. Maybe make it a replace element VI that will grow the array if needed? I'll have to think about that one.

Search 1D Array (DBL)__ogtk_improved.vi

[Mads]

Crossrulz,

The use of a for-loop was initially conincidental /the difference is less than it used to be now that the for loop can be aborted..), but I ran a comparison of it with a while loop and did not see a change in performance so I kept the for-loop for its simplicity. Perhaps there are test scenarios where the while loop has an edge though.

You have tidied up the code very nicely. I guess we could avoid an unnecessary last search by checking if the found index is at the end of the array too - but doing that check repeatedly might cost more than we gain.

I've started applying the new algorithms to all of the polymorph instances...(and I have to have a look at some 2D versions of those 1D ones) but if there are bugs to be found or further improvements to be done then it would be nice to hold that off until everything is checked out for the DBL versions. Or perhaps someone has already written scripts that change the data types automatically and generate the plymorph sets? That would definitely cut a lot of work:-)

Mads,

I'm curious about the reason for a FOR loop instead of a while loop. It looks to me like you are trying to catch that case where the item found is the last element in the array so an extra search isn't performed. Wouldn't it be better to use a while loop and compare the index to the highest index (i >= N-1, N-1 is calculated before the loop), OR this result with the equal to -1 to get the stop condition? I'll have to see if I can find some time to play around with it.

Edited August 1, 2012 by Mads

[crossrulz]

I guess we could avoid an unnecessary last search by checking if the found index is at the end of the array too - but doing that check repeatedly might cost more than we gain.

I found very little difference in my attempts. I'm kind of leaning towards taking out that check for simplicity reasons.

[Mads]

So here it is; a new and complete OpenG array library with 8 of its functions optimized for speed and memory usage.

All the polymorph instances have been revised. I have also backsaved it from 2011 to 2009 (which is as far back as the Remove Duplicates function will go now that it uses the In-Place structure...).

I guess this is as far as I can contribute. I really hope that we can see this (or even further improved versions) in an official release not too far into the future(?).

Mads

OpenG Array Revised R4 LabVIEW 2009.zip

OpenG Array Revised R4 LV2011.zip

[crossrulz]

Mads isn't going to like me here.

I was looking through the code and I looked at the Delete Elements from 1D Array VI. I found that I could tweek some more performance out by using the Empty Array? primitive instead of getting the array size and wiring the numeric into the case structure. Wasn't it mentioned earlier how boolean case structures are faster than numeric case structures?

The other thing I was looking at is the fact that we already have an Index 1D Array Elements VI. I put the VI where the for loop is to get the deleted elements and gave the performance gained with the Empty Array? right back. I guess overhead from calling a subVI? We could argue the reuse maintainability vs speed. Anyways, I took it back out.

Also did some slight cleaning up.

The other thing I just noticed (while typing this up), is that the current version is faster than our "improved" versions when the number of elements to delete is small (<25). I think Mads has already mentioned this, but I thought I would throw it out there still.

Delete Elements from 1D Array (DBL)__improved.vi

[Mads]

Hah, I do not have any problem with further improvements (although this one is relatively minor I would say) - as long as I'm not the one to update all those polymorph instances again ;-)

Edited August 2, 2012 by Mads

[GregSands]

Hah, I do not have any problem with further improvements (although this one is relatively minor I would say) - as long as I'm not the one to update all those polymorph instances again ;-)

Not just updating them all, but choosing what instances are provided - I use arrays of Images, but those are not useful for most people.

If there's ever a reason to have "generic" terminals, then this sort of polymorphic-heavy function gives a great example. Writing them as XNodes (thanks Gavin) means there's no need to code all the polymorphic instances, but there's a lot of extra coding still to do around the function itself. So yes, I know that generic terminals don't officially exist, and won't exist at all soon, and XNodes are somewhat hidden, but I hope there's a third alternative in the wings. Or that there's a slightly easier way to create XNodes.

Edited August 2, 2012 by GregSands

[Mads]

I just posted an idea on the idea exchange related to the polymorphic VI creation.

[Wouter]

I just posted an idea on the idea exchange related to the polymorphic VI creation.

Rather prefer C++ templates for creating VI's like this ;-) but that has been suggested already a numerous times in the idea exchange.

Edited August 3, 2012 by Wouter

[Mads]

Rather prefer C++ templates for creating VI's like this ;-) but that has been suggested already a numerous times in the idea exchange.

Any type of solution would be good. The "Polymorphism Wizard"-idea is possible to implement without any changes to the rest of LabVIEW / G though, so there is less reason for it not to happen, perhaps.

[hooovahh]

I just posted an idea on the idea exchange related to the polymorphic VI creation.

I'm actually quite surprised no one has made a scripting VI (or no one has shared it) that can do this. I agree it should be build into the IDE (which is why I gave a Kudo) but this doesn't seem that hard. If I get some time I may give it a try.

[crossrulz]

I'm actually quite surprised no one has made a scripting VI (or no one has shared it) that can do this. I agree it should be build into the IDE (which is why I gave a Kudo) but this doesn't seem that hard. If I get some time I may give it a try.

I started working on one about a month or so ago. Haven't touched it in awhile. It currently only works with numerics and it doesn't actually make the polymorphic VI, just the instances. I have it setup to take the dbl instance as a template and go from there. Build Polymorphic is the top level VI. Feel free to edit and share upgrades.

Build Polymorphics.zip

[Wouter]

Any type of solution would be good. The "Polymorphism Wizard"-idea is possible to implement without any changes to the rest of LabVIEW / G though, so there is less reason for it not to happen, perhaps.

No I disagree. First your code size will increase which is totally unnecessary. Second to make the functions work for more complex datatypes you will also need to create a VI for that. When you then want to change something to your algorithm you have to update all VI's, this can indeed be done with a polymorphism wizard or some kind, but again what about the more complex datatypes I think it will make your code base unmanageable.

Further I want to have a solution to the problem not a workaround which will for sure cost more time.

Edited August 4, 2012 by Wouter

[gb119]

If there's ever a reason to have "generic" terminals, then this sort of polymorphic-heavy function gives a great example. Writing them as XNodes (thanks Gavin) means there's no need to code all the polymorphic instances, but there's a lot of extra coding still to do around the function itself. So yes, I know that generic terminals don't officially exist, and won't exist at all soon, and XNodes are somewhat hidden, but I hope there's a third alternative in the wings. Or that there's a slightly easier way to create XNodes.

I have been following the thread with a view to incorporating the favoutrite algorithm into the XNode version of the OpenG Array tools when a good one was agreed on (and I had time between getting samples ready for beamtime experiments, writing next year's courses, installing LabVIEW 2012 and trying to explain to my IT department why installing 64bit everything on the computers that run our experiments isn't quite as simple as rolling out a new image onto a standard desktop...).

[Ton Plomp]

There is a (set of) VIs in the OpenG repository...

After the 2012 launch we'll have to discuss how to incorporate new features into the OpenG array tools.

Ton

[Mads]

No I disagree. First your code size will increase which is totally unnecessary. Second to make the functions work for more complex datatypes you will also need to create a VI for that. When you then want to change something to your algorithm you have to update all VI's, this can indeed be done with a polymorphism wizard or some kind, but again what about the more complex datatypes I think it will make your code base unmanageable.

Further I want to have a solution to the problem not a workaround which will for sure cost more time.

In an ideal world we would always jump to straight to the optimal solution, but if that's not an option it is still better to get something rather than nothing - unless that something is just a fraction better - but good enough for NI not to bother coming up with anything better after that...

[Mads]

There is a (set of) VIs in the OpenG repository...

After the 2012 launch we'll have to discuss how to incorporate new features into the OpenG array tools.

Ton

Are there any new features we can utilize for the array toolkit in 2012? The conditional tunnel VIs for example will still be nice to have in 2012. As the upgrade notice says:

"Note The Conditional tunnel option performs memory allocations as often as the Build

Array implementation. Therefore, just like with the Build Array function, National Instruments

recommends you consider alternatives to the conditional tunnel in portions of your application

where performance is critical."

Edited August 6, 2012 by Mads

[crossrulz]

Are there any new features we can utilize for the array toolkit in 2012? The conditional tunnel VIs for example will still be nice to have in 2012. As the upgrade notice says:

Since we are developing OpenG in 2009, we can't expect to use new features. When it is decided to upgrade the base version, then I agree that we should be looking at using the new features (inlining for an example).

[gb119]

I have been following the thread with a view to incorporating the favoutrite algorithm into the XNode version of the OpenG Array tools when a good one was agreed on (and I had time between getting samples ready for beamtime experiments, writing next year's courses, installing LabVIEW 2012 and trying to explain to my IT department why installing 64bit everything on the computers that run our experiments isn't quite as simple as rolling out a new image onto a standard desktop...).

New version of OpenG Array XNodes uploaded with Crelf's R4 version as the templates.

[crossrulz]

New version of OpenG Array XNodes uploaded with Crelf's R4 version as the templates.

Did you forget to attach? Where can we find your code?

[gb119]

Did you forget to attach? Where can we find your code?

Here