Send data as Variant or Flattened String?

[jfazekas]

There is an article on the NI site entitled "Differences Between Flatten to String.vi and Variant to Flattened String.vi" http://digital.ni.com/public.nsf/allkb/36C...625729E0007AE75

Variants have a functional advantage over flattened strings due to the attribute feature. Personally I've never used attributes, but I am very interested to know if the variants flatten and unflatten data faster than the 'flatten to string' counterpart.

Here are two VI's I made to look at the speed difference between the two data abstraction methods. In one test case I only flattend and unflatten some data. In another case I flatten, send to Q, receive, unflatten.

It is interesting to me that the unflatten activity for both Variant and String methods are much faster than the flatten activity.

On my system the Variant method is always significantly faster (more than 50%).

Perhaps someone who is interested will test how the use of Variant attributes affects the flatten/unflatten speed.

[Aristos Queue]

Your post left me completely confused because of this line:

QUOTE

but I am very interested to know if the variants flatten and unflatten data faster than the 'flatten to string' counterpart.

I thought you were talking about comparing "Flatten To String" against "Variant To Flattened String" in order to do real transmission from one application instance to another application instance (say, over a TCP/IP link or somesuch).

You're just talking about handing data from one VI to another VI on the same machine.

So OF COURSE handling as variant is substantially faster. Why? VARIANTS AREN'T FLAT. The data is picked up off the wire whole, with all the hair hanging off of it (arrays of clusters of arrays of clusters of arrays of... etc) and put in the variant along with the type of the wire. Then when we convert it back to data, there's just a check that the type descriptors are compatible and we're done. When flattening to a string, there's the whole traversal of the type descriptor to create the flattened string and then unflattening requires not just traversal of the type descriptor but also parsing of the string and memory allocation.

You got a 50% speed difference in your test. That's with a simple array. The more complex the type the greater the difference between these will be.

But the original article that you linked to is talking about something entirely different.

[jfazekas]

QUOTE(Aristos Queue @ Feb 13 2008, 05:28 PM)

Your post left me completely confused because of this line:

I thought you were talking about comparing "Flatten To String" against "Variant To Flattened String" in order to do real transmission from one application instance to another application instance (say, over a TCP/IP link or somesuch).

You're just talking about handing data from one VI to another VI on the same machine.

So OF COURSE handling as variant is substantially faster. Why? VARIANTS AREN'T FLAT. The data is picked up off the wire whole, with all the hair hanging off of it (arrays of clusters of arrays of clusters of arrays of... etc) and put in the variant along with the type of the wire. Then when we convert it back to data, there's just a check that the type descriptors are compatible and we're done. When flattening to a string, there's the whole traversal of the type descriptor to create the flattened string and then unflattening requires not just traversal of the type descriptor but also parsing of the string and memory allocation.

You got a 50% speed difference in your test. That's with a simple array. The more complex the type the greater the difference between these will be.

But the original article that you linked to is talking about something entirely different.

I didn't mean to misconstrew the NI article. It was my starting point for considering the attribute ability of the variant.

I'm not 'just' talking about handling data from one VI to another on the same machine. Sometimes it is a necessary evil to send more than one data type over a single wire. There are a couple of example VIs I see on LAVA where different data types (flattened to a string) are passed between loops using Queues. (Publish-subscribe topic for example).

I was interested in discovering any performance differences between passing data types by variants or strings, that is all. You pointed out that Variants are not flat, which is good for people to know. It wasn't obvious to me.

[Aristos Queue]

QUOTE(jfazekas @ Feb 14 2008, 11:03 AM)

I'm not 'just' talking about handling data from one VI to another on the same machine. Sometimes it is a necessary evil to send more than one data type over a single wire. There are a couple of example VIs I see on LAVA where different data types (flattened to a string) are passed between loops using Queues. (Publish-subscribe topic for example).

Usually these examples are intended for intra-machine communication of some sort. I don't know the particular one to which you refer, but it would be my guess that it was written for one machine to publish and the other machine to subscribe. Over a TCP/IP or Datasocket link (or your favorite protocol) the data sent is just strings which the other side interprets.

[i2dx]

QUOTE(Aristos Queue @ Feb 13 2008, 11:28 PM)

[...]

So OF COURSE handling as variant is substantially faster. Why? VARIANTS AREN'T FLAT. The data is picked up off the wire whole, with all the hair hanging off of it (arrays of clusters of arrays of clusters of arrays of... etc) and put in the variant along with the type of the wire. Then when we convert it back to data, there's just a check that the type descriptors are compatible and we're done. When flattening to a string, there's the whole traversal of the type descriptor to create the flattened string and then unflattening requires not just traversal of the type descriptor but also parsing of the string and memory allocation.

You got a 50% speed difference in your test. That's with a simple array. The more complex the type the greater the difference between these will be.

QUOTE(Aristos Queue @ Feb 14 2008, 06:29 PM)

Usually these examples are intended for intra-machine communication of some sort. I don't know the particular one to which you refer, but it would be my guess that it was written for one machine to publish and the other machine to subscribe. Over a TCP/IP or Datasocket link (or your favorite protocol) the data sent is just strings which the other side interprets.

Thanks a LOT for pointing that out. Why? ... that statment saves me a lot of time for benchmarking my data transfer mechanisms in my distributed systems