Jump to content

LabVIEW 2019 Map data type


Recommended Posts

So I got very excited when I saw that LabVIEW 2019 has a new native map type (aka hash array, associative array, dictionary) and so decided to have a play and see how it compared to my home-rolled LVOOP has array that uses variant attributes and I must admit that I'm slightly underwhelmed....

I've now benchmarked the 2019 native map class and a simple variant attribute by creating maps of 1 million elements of randomly generated 8 byte keys and then reading back 10,000 randomly selected elements and fairly consistently the native map is about an extra 50% slower than the variant attribute for both read and write operations. I'll admit my benchmark code is quite naive, but it's still a little disappointing that there is quite this performance hit....

Can any of the NI folks here comment on the performance - is it just that in fact variant attributes are blazingly fast?  I know I shouldn't be churlish and it's really great that 2019 has so many new features and a native map and set type have been long overdue...

 

Edit: and yes I've spotted that the variant type conversion was wired wrong and I should have been generating an array of 10000 I32 not error clusters - but no it doesn't make a significant difference....

map snippet.png

Variant Snippet.png

Edited by gb119
  • Like 2
Link to comment

You don't need to type cast to string when using maps. Change the keys in your Map code to be U64s instead of strings and let's see the benchmarks. Type casting is expensive. One of the benefits of using the map data type is that you no longer are forced to use strings as your keys.

Link to comment

So for my usual use cases I do want the keys to be strings - the type cast is just a means to generate a random string. Its also the same in both versions of the benchmark so cannot on its own explain why the map is slower. A better benchmark would probably be to pre-calculate the keys and then use the same keys for both map and variant attribute.

Link to comment

I would always recommend wiring up the output of the read nodes to an array indicator (outside the sequence so that it has no effect on timing).  Compiler optimisations can do weird things when you're not actually wiring up certain outputs.  For example, you're not using the data output of the variant but you are of the map. Not saying it explains the differences, but I've seen things like that wildly affect performance in the past.

I would look at the code, but I'm currently trying to get 2019 installed in a VM.

  • Like 1
Link to comment

Side note: I tried to look at the code, but I couldn't manage to get the snippets dragged into a LV 2019 diagram. Is there something special I have to do to download the .pngs so that they have the LV meta data necessary to make the snippet be droppable code?

Link to comment
53 minutes ago, Darren said:

Side note: I tried to look at the code, but I couldn't manage to get the snippets dragged into a LV 2019 diagram. Is there something special I have to do to download the .pngs so that they have the LV meta data necessary to make the snippet be droppable code?

Sorery dunno what has happened there - I created the snippets in LV 2019 64bit, saved as aPNG files and dragged it ontot he post in Chrome - perhaps somewhere between Chrome, and LAVAG it stripped the embedded LabVIEW code?

From your comments, I think the answer is simply that variant attributes are in fact quite fast and at least for string keys is the way to go for now.

Link to comment

LAVA has been known to strip out that meta data in the past.  I remember one instance where it was seen when the image wouldn't fully fit on the post and it would do some scaling on the server side.  I'll ping Michael but until then I'd recommend attaching the VI.

Link to comment

I had a go at recreating the benchmark (but with preallocated string keys), and maps did perform slightly worse than variants, but still within about 5% of each other. Flipping the map key type over from string to U64 increased the map performance by ~10%.

You may be seeing the sorting overhead of maps (and sets). AFAIK variant attributes are unsorted, whereas key/value pairs in maps are sorted on insert/delete. This exchange on Twitter has some more info:

aq_sets_rationale.PNG.04911af33ed3f1859d5e3643914a1d61.PNG

Link to comment

The ~5% slower Map vs Variant Attributes is consistent with what @altenbach reported in his NI Week presentation, for very large datasets.

Since one set is ordered, the other is not, it might be interesting to benchmark the "delete" operation and see if it is symmetrical. My intuition here would be that finding the key and deleting it would be ~5% faster in favor of Maps (for large datasets).

Since I rarely deal with large sets, I'm egotistically happy with NI's choice to make those sets and maps ordered.

Link to comment
4 hours ago, Antoine Chalons said:

Anyone knows if there is a way to do a RegExp search on a maps?

There doesn't appear to be a native way to do this, but if you write a VIM for this then you should be able to reuse that on any map type.

  • Like 1
Link to comment
  • 8 months later...

Figured someone over here might be interested... over the weekend, I built a new right-click menu item plugin for editing set controls and constants. The UI is bad, but it works; if anyone wants to work on the UI, you're welcome to do so. Building a map editor would follow the same pattern as the set editor.

https://forums.ni.com/t5/LabVIEW-Shortcut-Menu-Plug-Ins/Edit-A-Set-Value-llb/ta-p/4020190

  • Thanks 1
Link to comment
On 3/2/2020 at 1:23 PM, Aristos Queue said:

Figured someone over here might be interested... over the weekend, I built a new right-click menu item plugin for editing set controls and constants. The UI is bad, but it works; if anyone wants to work on the UI, you're welcome to do so. Building a map editor would follow the same pattern as the set editor.

https://forums.ni.com/t5/LabVIEW-Shortcut-Menu-Plug-Ins/Edit-A-Set-Value-llb/ta-p/4020190

What I would like to see is a way to minimize the map or set constant by double-clicking on it. Like you can do with cluster constants. Hand editing Maps is such a small use-case but useful for some I guess. You usually work with these things programmatically. Thanks for the tool.

Link to comment
  • 1 month later...
9 hours ago, Neil Pate said:

I am a bit late to the Map party. I love them though, thanks NI. 🤩

For those that have not tried them, take a quick look. I have only used the Map a few times (so cannot comment on Sets) but the API is nice and simple. Goodbye Variant Attributes 🙂

ditto.

Jump into sets, they're great as well and they can be combined with maps!

I rarely use 1D arrays anymore!

Edited by Antoine Chalons
Link to comment

I'm parenting today, so wait a bit for screenshot, but let's say you have 2 1d arrays of string and you want to compute the intersection or the diff, how do you do?
Also open the vim "remove duplicate from 1d array" in the array palette, it uses sets.

 

And check this https://forums.ni.com/t5/LabVIEW-APIs-Discussions/Tree-Map/td-p/3972244?profile.language=en

 

Edited by Antoine Chalons
  • Like 1
Link to comment
32 minutes ago, Antoine Chalons said:

I'm parenting today, so wait a bit for screenshot, but let's say you have 2 1d arrays of string and you want to compute the intersection or the diff, how do you do?
Also open the vim "remove duplicate from 1d array" in the array palette, it uses sets.

 

And check this https://forums.ni.com/t5/LabVIEW-APIs-Discussions/Tree-Map/td-p/3972244?profile.language=en

 

OK I see. Interesting I never really thought about doing anything like that. I will keep that technique in my pocket for next time I am doing 1D array stuff. Thanks!

Link to comment
On 6/4/2019 at 8:10 PM, Dataflow_G said:

You may be seeing the sorting overhead of maps (and sets). AFAIK variant attributes are unsorted,

Nope. Variant attributes and maps use the same — identical — underlying data structure. For reasons I don’t grasp, the C++ compiler adds a couple extra instructions in the maps case only when the keys are strings that aren’t in the variant attributes. Still, the conversion time to/from variant for the value tends to dominate for any real application. 

Link to comment
9 hours ago, Aristos Queue said:

Variant attributes and maps use the same — identical — underlying data structure.

Does that mean a LabVIEW map converts the data to/from variants behind the scenes, even though the datatype is fixed at edit-time?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.