Flattened LVOOP Class

[Eugen Graf]

Hello LAVAs!

I want know if it's possible to flatten a LVOOP Class to a binary string from e.g. C++, so I can unflatten them in LV to a LVOOP Class ?

Thanks, Eugen

[ragglefrock]

QUOTE (Eugen Graf @ Jun 24 2008, 04:52 PM)

Hello LAVAs!

I want know if it's possible to flatten a LVOOP Class to a binary string from e.g. C++, so I can unflatten them in LV to a LVOOP Class ?

Thanks, Eugen

The question's a little confusing. How are you using your LVOOP class in C++? Are you calling a LabVIEW-built DLL that creates and uses LVOOP classes?

To flatten the LVOOP class appropriately, you will need to use the LabVIEW Flatten to String function. You can create a wrapper VI that calls this function with a class input and create an exported DLL function to call this from C++.

[Eugen Graf]

QUOTE (ragglefrock @ Jun 25 2008, 12:03 AM)

The question's a little confusing. How are you using your LVOOP class in C++? Are you calling a LabVIEW-built DLL that creates and uses LVOOP classes?

To flatten the LVOOP class appropriately, you will need to use the LabVIEW Flatten to String function. You can create a wrapper VI that calls this function with a class input and create an exported DLL function to call this from C++.

I think to create a structure (=cluster) in C++ with all elements what LVOOP Class Contol Cluster has, than cast them to binary string (or byte array) and send over TCP/IP.

[Aristos Queue]

QUOTE (Eugen Graf @ Jun 24 2008, 05:08 PM)

I think to create a structure (=cluster) in C++ with all elements what LVOOP Class Contol Cluster has, than cast them to binary string (or byte array) and send over TCP/IP.

You can create a flattened cluster string in C++ and then write a custom unflatten routine for your class that can parse the string and populate the fields of your class. But if you want to use the Unflatten From String primitive, then you'll have to format the data like a LabVIEW class gets flattened.

What does an LVClass look like when we flatten it?

The string has four parts:

1. "NumLevels" The first 4 bytes represent a uInt32. This tells us how many levels of the hierarchy are recorded herein.
   SPECIAL CASE:
   If NumLevels is zero, absolutely nothing else follows. Zero indicates that this is an instance of LabVIEW Object, the ancestor of all classes which has no fields, no methods and inherits ex nihilo.
   
   
2. "ClassName". The next several bytes represent the fully qualified name of the class. First there is a single byte that tells the length of the qualified name of the class. This is followed by a sequence of Pascal strings , each one representing one segment of the qualified name. The last Pascal string is a length byte of zero. This section of the flattened string includes enough zero pad bytes to increase the class name section to a multiple of 4 bytes.
   
   
3. "VersionList". The next series of uInt16s that represent a list of version numbers. Each version number is 4 uInt16s, usually written in the format W.X.Y.Z, where W, X, Y and Z are each a uInt16. There are NumLevels of version numbers. The first version number is the version of ClassName class, followed by the version number for ClassName's parent, and so on to the version of the oldest ancestor (not including LabVIEW Object).
   SPECIAL CASE:
   If NumLevels == 1 and the version == 0.0.0.0, then this is the default data of the class. Skip section 4 -- no further data follows in this special case. If version is zero for any other value of NumLevels, the string is considered corrupt and the unflatten functions will return an error.
   
   
4. "ClusterData" -- This is a series of flattened clusters, one for each level of the hierarchy. The first cluster is the oldest ancestor class. Pay attention: The version numbers start at the descendant class and go up the hierarchy. The cluster data starts at the oldest ancestor and goes down the hierarchy. Each block of data starts with 4 bytes interpreted as an int32. This int32 represents the number of bytes in the data that follows. If this number is zero, then we use the default default data for this level of the hierarchy. If this number is non-zero then this is the standard flat data representation for the cluster at this level of the inheritance hierarchy. After each cluster there are enough pad bytes to get the length of the string back to a multiple of four.
   
   

Here, for example, is a flattened class object (I put the string into slash notation to make this "easier" to parse):

00021210Bacteria.lvclass000506070b0100000t0000000f0004abcd0000

The first 4 bytes are NumLevels. NumLevels in this case is 2. So we have a class that has one ancestor between it and LabVIEW Object.

Next is the PStr that is the name of the class: Bacteria.lvclass. After that are 2 NULL pad bytes to get the length back to a multiple of four.

Next is the first version number: 5.6.7.8. That's the version number for Bacteria.lvclass.

Next is the second version number: 1.0.0.11. That's the version number for Bacteria's parent class. We'd have to have Bacteria in memory to ask it "Hey, who was your parent class back when you were version 5.6.7.8?"

Next come the data blocks.

The first data block has data size 0. That means that we'll just assume that the parent cluster was the default default value.

The second data block has data size 15. So the next 15 bytes are a flattened cluster of the cluster of Bacteria when it was version 5.6.7.8. Remember that the Bacteria.lvclass currently in memory might be a much later version of the class -- LabVIEW will automatically handle the conversion for you (which you can read all about

here

). In this case, it looks like those 15 bytes of data contain a string ("abcd") and a couple other cluster elements.

Just write code in C++ to generate this string and you can have LV unflatten it. Contrariwise, you can have LV flatten the string and you write the C++ function to parse it.

IMPORTANT NOTE: The flattened form of LV data is readable on *any* endian platform. We always flatten int32s, uInt32s and other data types into little endian order. If you are on a big endian machine, make sure that your custom write function takes care of reversing the bytes.

[Jim Kring]

QUOTE (Aristos Queue @ Jun 24 2008, 06:36 PM)

What does an LVClass look like when we flatten it? The string has four parts:

Great info. I've added it to the http://wiki.lavag.org/LVClass' Data Storage Format' rel='nofollow' title='LabVIEW Wiki article on LVClass Data Storage Format' alt='Wiki article on LVClass Data Storage Format' style="border-bottom: 1px dotted #3366BB; color: #3366BB; cursor:pointer; text-decoration:none;" class="wiki">LVClass Data Storage Format wiki page. :thumbup:

[Yair]

Great info.

QUOTE (Aristos Queue @ Jun 25 2008, 04:36 AM)

Here, for example, is a flattened class object (I put the string into slash notation to make this "easier" to parse):

\00\00\00\02\12\10Bacteria.lvclass\00\00\00\05\00\06\00\07\00\b\00\01\00\00\00\00\00\t\00\00\00\00\00\00\00\f\00\00\00\04abcd\00\00\00\00

Next is the PStr that is the name of the class: Bacteria.lvclass. After that are 2 NULL pad bytes to get the length back to a multiple of four.

Which of the two bytes is the PStr length, 0x12 or 0x10? Bacteria.lvclass is 16 (0x10) characters . If 0x12 is the length and 0x10 is the beginning of the string, then the string is 0x11 bytes long and needs three null characters for padding, not two. If 0x10 is the length, there shouldn't be need for padding. Or am I missing something?

QUOTE

Next is the first version number: 5.6.7.8. That's the version number for Bacteria.lvclass.

Next is the second version number: 1.0.0.11.

Are you sure you didn't switch the 8 and 11 here? In the code, \b comes in the first part and \t comes in the second.

[Aristos Queue]

QUOTE (Yair @ Jun 27 2008, 05:31 AM)

Which of the two bytes is the PStr length, 0x12 or 0x10? Bacteria.lvclass is 16 (0x10) characters . If 0x12 is the length and 0x10 is the beginning of the string, then the string is 0x11 bytes long and needs three null characters for padding, not two. If 0x10 is the length, there shouldn't be need for padding. Or am I missing something?

What you're missing is that I didn't fully explain how the PStr is encoded.

a) The PStr is a length byte plus the text, so that is 17 bytes, not 16, and thus padding is required.

b) The first byte, 0x12, is the total length including padding: 18

c) The second byte, 0x10, is the total length not including padding: 16

d) Thus 18 + the two length bytes = 20

QUOTE

Are you sure you didn't switch the 8 and 11 here? In the code, \b comes in the first part and \t comes in the second.

\t is the tab character, which I thought was character 11, but I checked my ascii table. Turns out it is 9. I corrected my original post. I also added the hex view for clarity. \b is not hex 11 as you thought. It's the slashcode for character 8.

[Yair]

QUOTE (Aristos Queue @ Jun 27 2008, 02:18 PM)

What you're missing is that I didn't fully explain how the PStr is encoded.

a) The PStr is a length byte plus the text, so that is 17 bytes, not 16, and thus padding is required.

b) The first byte, 0x12, is the total length including padding: 18

c) The second byte, 0x10, is the total length not including padding: 16

d) Thus 18 + the two length bytes = 20

Well, I still don't really understand - shouldn't the total length, including padding (step b), be 20?

Or did you mean total length including the length bytes?

What's the actual use case for having two length bytes? If you have just one length pad, you should easily be able to calculate the needed padding, shouldn't you?

QUOTE

\t is the tab character, which I thought was character 11, but I checked my ascii table. Turns out it is 9. I corrected my original post. I also added the hex view for clarity. \b is not hex 11 as you thought. It's the slashcode for character 8.

I actually had a double mix-up here. I didn't check either and thought tab was ASCII 8 and backspace is ASCII 9 (when it's the other way around). That would explain why I thought that \b is ASCII 11.

I also fixed the wiki article.

[shoneill]

QUOTE (Yair @ Jun 27 2008, 01:33 PM)

Well, I still don't really understand - shouldn't the total length, including padding (step b), be 20?

Or did you mean total length including the length bytes?

What's the actual use case for having two length bytes? If you have just one length pad, you should easily be able to calculate the needed padding, shouldn't you?

Seeing as the descriptors can be very long and complicated, it's very useful sometimes to know how to easily skip to the next item in the descriptor. Hence the first value.

Of course one could deduce the next offset based on the data available, but I'd rather take the 0.25 Bytes (on average) cost of the second length value for faster response time in LabVIEW.

Isn't this all derived from the general and pretty much global method of defining type descriptors? Such type descriptors for Controls can get pretty involved.

Shane.

Ps I see your point regarding the value of 18 instead of 20. I think this might also be a result of a global method of handling type descriptors IIRC. I think the descriptor for a PStr lists the total length as the total length of the DATA and thus excludes the two bytes for the size information. It will thus always be 2 Bytes short of a multiple of 4.

[Aristos Queue]

QUOTE (Yair @ Jun 27 2008, 06:33 AM)

What's the actual use case for having two length bytes? If you have just one length pad, you should easily be able to calculate the needed padding, shouldn't you?

I have no idea. That's how PStrs are encoded whenever they're flattened in LabVIEW data.

[LAVA 1.0 Content]

QUOTE (Aristos Queue @ Jun 25 2008, 02:36 AM)

1. 
   
2. ... Note that there are enough pad bytes after the PStr to get the length of the string to be a multiple of four.
   
   
3. 
   
4. After each cluster there are enough pad bytes to get the length of the string back to a multiple of four.
   
   

Forgive my ignorance, but why should this be padded data?

How does this go for LabVIEW 64 bit? is the padding than 8 bytes?

Ton

[Aristos Queue]

QUOTE (Ton @ Mar 17 2009, 04:42 PM)

Forgive my ignorance, but why should this be padded data?

I forgive you. But I can't really explain it. I don't remember why it was done.

QUOTE

How does this go for LabVIEW 64 bit? is the padding than 8 bytes?

No change. It would be a real problem if it was changed. The format for flat information has to be the same no matter where it was flattened so anyone can unflatten it.

[Aristos Queue]

QUOTE (Yair @ Jun 27 2008, 06:33 AM)

What's the actual use case for having two length bytes? If you have just one length pad, you should easily be able to calculate the needed padding, shouldn't you? QUOTE (Aristos Queue @ Jun 27 2008, 08:23 PM)

I have no idea. That's how PStrs are encoded whenever they're flattened in LabVIEW data.

I realized the correct answer to this: The name of the class can be a whole string of PStrs connected together when the class is itself inside an owning library. So if the name of the class is "MyLib.lvlib:InnerLib.lvlib:MyClass.lvClass" then the string would be

WXMyLib.lvlibYInnerLib.lvlibZMyClass.lvclass

where W = length of the whole string

X, Y and Z = length of their respective substrings

[Jim Kring]

What does an LVClass look like when we flatten it? The string has four parts...

I'm now finding myself in need of a tool to parse the type descriptor of an LVClass. I can't use the flattened string data, since I'm actually parsing a reference (e.g. a Queue or Notifier reference).

I'd appreciate any info on the type descriptor format, if anybody has it.

Thanks,

[MikaelH]

I'm now finding myself in need of a tool to parse the type descriptor of an LVClass. I can't use the flattened string data, since I'm actually parsing a reference (e.g. a Queue or Notifier reference).

Tomi has made one parser that you can find in the OpenG root class template.

I've a version of that code in one on my VIs, it looks like this:

GetClassName.vi

//Mikael

[Jim Kring]

Tomi has made one parser that you can find in the OpenG root class template.

I've a version of that code in one on my VIs, it looks like this:

//Mikael

Hi Michael,

Tomi's parser did the trick. I needed to parse the TD, not the data string, in this case, so I wasn't able to use your example.

Thanks,

[Rolf Kalbermatter]

Here, for example, is a flattened class object (I put the string into slash notation to make this "easier" to parse):

00021210Bacteria.lvclass000506070b0100000t0000000f0004abcd0000

The first 4 bytes are NumLevels. NumLevels in this case is 2. So we have a class that has one ancestor between it and LabVIEW Object.

Next is the PStr that is the name of the class: Bacteria.lvclass. After that are 2 NULL pad bytes to get the length back to a multiple of four.

IMPORTANT NOTE: The flattened form of LV data is readable on *any* endian platform. We always flatten int32s, uInt32s and other data types into little endian order. If you are on a big endian machine, make sure that your custom write function takes care of reversing the bytes.

Hmm looking at the data I wonder if you mixed up Little Endian and Big Endian here. Especially since all other LabVIEW flattened formats use Big Endian.

The first four bytes making up Num Levels do look to me in MSB (most significant byte first) format which traditionally has always been referred to as Big Endian. Of course adding a new datatype you guys would be free to decide to flatten its data differently but the example would point to the fact that you stayed with the predominant Big Endian flatten format as used in all other flattened data in LabVIEW.

And therefore byte reversing would by now be necessary on any platform except the PowerPC based cRIOs.

As to the padding of Pascal Strings, this has been traditionally padded to a 16 byte boundary in the past but I guess that was in fact a remainder of the LabVIEW 68000 platform roots, where 16 byte padding was quite common to avoid access of non byte integers on uneven byte boundaries.

Rolf Kalbermatter

[PaulL]

Tomi has made one parser that you can find in the OpenG root class template.

I'm confused. Where would I find this? (I have the OpenG libraries, but I'm not sure what I'm seeking here. I'm hoping you will help me out.) Thanks!

[Jim Kring]

I'm confused. Where would I find this? (I have the OpenG libraries, but I'm not sure what I'm seeking here. I'm hoping you will help me out.) Thanks!

Mikael is referring to Tomi's OpenG Class Templates.

[PaulL]

Ah! OK! Thanks, Jim!

[mje]

Resurrecting this...

I needed to evaluate the version number of my objects at run-time so I've been parsing the flattened data and came across a slight inconsistency in the discussion.

a) The PStr is a length byte plus the text, so that is 17 bytes, not 16, and thus padding is required.

b) The first byte, 0x12, is the total length including padding: 18

c) The second byte, 0x10, is the total length not including padding: 16

d) Thus 18 + the two length bytes = 20

Take for example the Bacteria.lvclass we have been discussing, which yields the following flattened string (in hex words):

0000 0001 1210 4261 6374 6572 6961 2E6C

7663 6C61 7373 0000 0000 0000 0000 0000

The name section is highlighted in blue. The first byte of the name section (0x12) is the length of the entire name section including the size byte, but excluding the padding. The following byte is the length of the first PString, etc as described before, however I'll say that when dealing with qualified names, the colon character is not encoded (rather the qualified name appears as two distinct PStrings, with the colon appearing in neither). There is an additional word of padding added after the name segment to align with a four-byte boundary, then the four version words are present.

For example, a class called MyClass.lvclass produces the following bytes:

0000 0001 110F 4D79 436C 6173 732E 6C76

636C 6173 7300 0000 0000 0000 0000 0000

Note the name section is shorter (0x11 bytes), and the padding has increased to three bytes to align with the boundary. In both cases, the version words start at 0x18 bytes (red).

Both of these examples are generated by just dropping a class constant on the diagram, hence the 0.0.0.0 version number (and the first four bytes == 1).

Now what struck me while playing with this is that there seem to be two levels of default value. For example, if we now take the wire and operate on it explicitly, but make sure the class value is still default, we get a different data string (using MyClass.lvclass again):

0000 0001 110F 4D79 436C 6173 732E 6C76

636C 6173 7300 0000 0001 0002 0003 0004

0000 0000

Now we've recorded a version number of 1.2.3.4, and we also have a data segment (in green). The data segment starts with a U32 stating the length, which in this case is zero, meaning no data, or a default value (for this specific version?).

Now if we actually operate and store non-default values, we get what I'd expect according to the documentation and discussion in this thread:

0000 0001 110F 4D79 436C 6173 732E 6C76

636C 6173 7300 0000 0001 0002 0003 0004

0000 000C 4009 21FB 5444 2D18 1234 5678

The data string stores the length (0xC) followed by twelve data bytes.

Figured I'd clarify this. A default value does not always produce a version of 0.0.0.0. Example code is attached, LV2010.

-m

TestBinaryObjects.zip

[Aristos Queue]

Figured I'd clarify this. A default value does not always produce a version of 0.0.0.0. Example code is attached, LV2010.

True. It was inefficient to do an Equals test on the actual data whenever it was being flattened/unflattened. The only way the default value gets written out is if the value is still the shared copy of the default value.

For the record, something I hadn't realized before but recently realized was a side-effect of other decisions: if you use Flatten To String on a real-time target, the non-default flattened string is always what you'll get. On the RT target, we don't share a single copy of the default value (everything has to be preallocated to avoid jitter, so doing a copy-on-write under the hood is unacceptable at runtime). That means that we never use the shortcut representation of the class data.