Super Sized .lvclass files

[Daklu]

I was cleaning out some dead code from a project I've been working on lately and discovered two of my .lvclass files each are nearly 25 mb in size. I'm not one to spend extra effort trying to minimize the size of my source code, but 25 mb seems excessive. It's not like there's a huge amount of data on the class ctl. Maybe 15 double precision floats (all default values) and a few comments.

Even after removing all the private data, all the class members, and renaming the class to kill the mutation code it's still sitting there at ~25 mb. Granted this is with LV2009, but still...

Anyone ever run across this before?

Bloat.zip

[mje]

Odd:

<Property Name="NI.Lib.Version" Type="Str">1.0.0.0</Property>

I'd expect with a version number of 1.0.0.0, for all mutation history to have been removed. Something though is being stored in the <Val> element. Unfortunately since the bulk of lvclass files are encoded binary data, hard to say what.

But to answer your question, I've never seen anything like that before. I have a few classes in the hundreds of kilobytes area, but they're also loaded with mutation histories. I understand why mutation history is in LV classes, but I still don't like it. I'd much rather have a defined interface which gives me programmatic control over how LabVIEW creates/copies/mutates my data. But I digress, I can't even say that's the cause of your problem.

[Daklu]

I'd expect with a version number of 1.0.0.0, for all mutation history to have been removed. Something though is being stored in the <Val> element. Unfortunately since the bulk of lvclass files are encoded binary data, hard to say what.

I tried opening the file using Notepad++ but it choked. What did you use?

I was very surprised to see a 25mb text file compress into 200 kb. Makes me wonder if there's a huge amount of white space embedded in there somewhere.

I understand why mutation history is in LV classes, but I still don't like it. I'd much rather have a defined interface which gives me programmatic control over how LabVIEW creates/copies/mutates my data.

I don't mind that LV tracks it automatically, but I do wish there were a way to hook into the process and override the default behavior.

[mje]

Definitely not whitespace:

XMLSpy works very well, but forces artificial line feeds every 4096 columns. Hence a lot of the "lines" are not real. The <Val> element, for example is a 24248022 character "line".

[Aristos Queue]

There are VIs in vi.lib\utility\editlvlibs\lvclass to allow you to inspect the mutation history information.

[Daklu]

There are VIs in vi.lib\utility\editlvlibs\lvclass to allow you to inspect the mutation history information.

Thanks for the tip. I discovered renaming the class doesn't clear the mutation history in 2009. There was still 46 or so entries in the list. However, I was able to use the Set Mutation History vi to clear the list and reduce the file size back down to where it should be.

[Aristos Queue]

Thanks for the tip. I discovered renaming the class doesn't clear the mutation history in 2009. There was still 46 or so entries in the list. However, I was able to use the Set Mutation History vi to clear the list and reduce the file size back down to where it should be.

Right... renaming still preserves a lot of the history because any child class that tries to unflatten its old version may need the mutation records of the parent as it was.

[Daklu]

Right... renaming still preserves a lot of the history because any child class that tries to unflatten its old version may need the mutation records of the parent as it was.

That makes sense. I've noticed the mutation history size increases much faster when a class contains other classes as private members. The other problem class has 3 member classes and grows ~1.5 mb with each save. Does a class extract and store the mutation history of all member classes as well?

This also explains a lot of the edit-time performance issues I've had in LV 2009. Any operations on the two 25 mb classes that required saving or applying changes took several minutes to complete. Once I removed the mutation history the problem went away. (Note to others: The vis in the folder AQ mentioned operate on the class, not on the object. It alters the .lvclass data in a way that affects all instances of the class from that point forward.)

Since I refactor a lot and rarely persist objects to disk using built-in functions, I'd like to script a tool that automatically clears the geneology. My thought is to delete all but the most recent item in the array. Other than not being able to load persisted objects from disk, are there any gotchas you know of that I should watch out for?

[Aristos Queue]

Other than not being able to load persisted objects from disk, are there any gotchas you know of that I should watch out for?

Nope, that's the only one.

[MikaelH]

<A name=OLE_LINK1>Regarding large lvclass files, I have a colleague that used the NI Themed Custom Control Suite for all his controls in the class private data. These controls contains massive images so don’t use them in TypeDefs or Class Private Data is my recommendation.

In my collegues case, this made the lvclass so big so LabVIEW crashed and run out of memory on his computer.

Cheers,

Mike

[Götz Becker]

Nope, that's the only one.

Is something like this:

allowed if I want to reset the version number together with the history? The direct write access to the library version using VI server is probably not allowed on purpose?!?

[Aristos Queue]

Is something like this:

Just write an empty array to clear the mutation history. But the version number is never allowed to be set backwards because even with the version number wiped out, there could still be data out there of the earlier version number, and that needs to return a "this data is too old to be inflated" error. Only "Save As >> Rename" sets the version number back to 1.0.0.0.

[Daklu]

what can cause Error 42 when trying to clear the Mutation history?

I don't remember the error code, but I know I received an error when trying to write an empty array. My solution was to delete everything in the existing array except for the last one.

[Aristos Queue]

Hitting "apply" after I have made any typedef/class changes terrifies me at the moment as it seems so easy to nuke everything.

Yes, but now you and Daklu are educated users... exactly the sort of folks that I've been hoping for a long time would take my under-the-hood VIs and write some sort of nice editor for manipulating the mutation history. :-)

[Daklu]

Yes, but now you and Daklu are educated users... exactly the sort of folks that I've been hoping for a long time would take my under-the-hood VIs and write some sort of nice editor for manipulating the mutation history. :-)

Wow, talk about pressure.

I've actually thought about how handy that would be. Deleting selected entries to reduce .lvclass file is trivial. Creating a tool that allows users to build a history from scratch is trickier. I think core code is managable, but I've been stuck on the UI. (I say "stuck" like I've actually written code. I haven't. All I've done is think about the problem.) I can imagine a user interface that would make it fairly intuitive to build a history. Unfortunately I have absolutely no idea how to implement it.

--A little later--

Looking through this again I'm not sure your vi exposes all the information I'd need to build a mutation history from scratch. In particular, I see the Old Name Index is incremented every time the class is renamed, but there's no information about what those names actually are. Isn't the actual former name necessary information in the mutation history?

[Götz Becker]

For my current usecase, preparing a release version of a OOP-based framework and medium sized sample application, I tried my inplace reset snippet on all .lvclasses in the project.

After restarting LV and reloading the project everything seemed OK, I could open the .lvclasses and see the version was reset to 1.0.0.0.

Opening the main VI looked good too, running it resulted in a LV crash (no errors, LV just gone). Reopening and masscompile using the project, same result. Reopening and forced recompile (ctrl+shift+run) & save all, did fix the crash.

Is this a expected behavior since I changed the XML file directly? Or did I hit some other problem?

[Daklu]

Is this a expected behavior since I changed the XML file directly? Or did I hit some other problem?

This is pure speculation, but my guess is manually forcing the version to 1.0.0.0 has created a conflict. If there are relatively few classes, I'd try doing a Save As on all your classes so LV sets them back to 1.0.0.0 itself.

[Aristos Queue]

Isn't the actual former name necessary information in the mutation history?

Nope.

Contained class and container class --- when one class has another class as a member of its private data control, the one in the cluster is the contained class and the other one is the container class.

Child classes and container classes include in their own flattened data the data of another class -- the parent class and/or the contained class. When a class is renamed, its version number resets to 1.0.0.0. It could then be edited, moving its version number forward. If another class has written down a version number of 1.0.0.1, that version number might be the way the class looked before the rename or after the rename. The child and container classes have a name index recorded in their mutation history that lets them figure out which mutation record of the renamed class is the right one to use.

When you rename a class (say rename "Old" to "New"), the only other classes that can take advantage of the mutation are classes that are in memory at the time that the class renames. Child classes and container classes have the renamed class as part of their flattened data. When the renamed class changes name, those classes update their inheritance records or their cluster records to note that they are aware of the rename. They don't actually care about the name at all. All they need to know is that the version number that they wrote down in their flattened data is not a *current* version number, but is instead a version number from before the last reset of the renamed class.

For the record, you can always open the .lvclass file in a text editor, find this tag:

<Property Name="NI.LVClass.Geneology" Type="Xml">

and delete that tag and everything through the matching

</Property>

tag. That removes the mutation history entirely and is always safe to do.

This is pure speculation, but my guess is manually forcing the version to 1.0.0.0 has created a conflict. If there are relatively few classes, I'd try doing a Save As on all your classes so LV sets them back to 1.0.0.0 itself.

It's good speculation. When you reset the version number manually, you need to delete the geneology records entirely because there's no way you can update them by hand to account for the rename (which means updating the name index, and modifying all the existing records to note that they are for the previous name).

[Götz Becker]

This is pure speculation, but my guess is manually forcing the version to 1.0.0.0 has created a conflict. If there are relatively few classes, I'd try doing a Save As on all your classes so LV sets them back to 1.0.0.0 itself.

Currently there are about 90 classes involved. "Save as" would reset the version but then I would have to fix all child classes to their new base class (+SCC and documentation overhead). That's why I am looking for a way of doing the reset in place.

I'll try the XML-only approach, removing NI.LVClass.Geneology and reseting the version string directly.

[Daklu]

Nope.

Ahh... I see. The name index and version number define the unique key to look up the correct mutation record.

So, I know it's possible to unflatten an object when the original class has been destroyed, but it requires the developer to have intimate knowledge of the flattened object's class properties. What about this scenario...

A developer creates simple class C (non-container, non-child) and persists object 1 (C version 1.0.0.1), object 2 (C version 1.0.0.2), and object 3 (C version 1.0.0.3) to disk. The developer's cat then walks across the keyboard and deletes C.lvclass such that it is unrecoverable. The developer knows he has these flattened objects on disk, but he can't remember what data types each of them contains or their class version number.

What I'd like to do is create a tool that helps the developer recreate class C so the persisted data is not lost forever. (If I had that tool I'd feel much more comfortable persisting objects directly to disk.) If I recall correctly, unflattening an object goes through several checks before it returns successfully:

1. The object's class name must match the class name wired to the unflatten type terminal.

2. The object's name index and version number must match with one of the mutation records.

3. The object's data cluster must match the data cluster in the mutation record found in step 2.

If any of these steps fail, the unflatten fails. Since the data required to unflatten an object is embedded in the flattened data, it should be possible to read that information from the three flattened objects and create a mutation history for a new version of class C. I'll have to scour old threads to see if you've mentioned how that information is encoded when flattened.

[Edit - I just remembered (or at least think I remember) data types aren't recorded as part of the flattened data. If that's true, recovering a flattened object would be a guess-and-check process.]

Child classes and container classes include in their own flattened data the data of another class -- the parent class and/or the contained class.

Let me rephrase a bit to make sure I understand...

Flattened child and container objects include the flattened data of the parent and/or contained object, right? And from what I understand a flattened object only contains the class name, index/version key, and data values. A flattened object does not include mutation history or even an entire mutation record; that information is looked up in the lvclass file. If the flattened object is a child or container object, I'd need to recurse* through the data doing the same process for each object I encounter before I can successfully unflatten it. That's going to be an extremely difficult process, perhaps impossible.

Given the difficulties and uncertainty involved, is there any pratical value in having a tool that can edit the mutation history versus one that can simply delete unneeded records?

(*Damn. And damn again. Recurse.)

For the record, you can always open the .lvclass file in a text editor...

Sure, as long as your lvclass file doesn't grow to 25 mb. (Thanks, that's a good tip to know.)

[Aristos Queue]

What I'd like to do is create a tool that helps the developer recreate class C so the persisted data is not lost forever.

Three words: Source Code Control.

[jgcode]

Three words: Source Code Control.

Three words: Kill the cat

[Daklu]

Three words: Source Code Control.

Yeah, I guess that's the only solution currently available. It'd be nice if there were a better way to deal with the issue, but it is what it is.

So that leaves my other question unanswered. Other than deleting records to reduce the class file size, are there any use cases where editing the mutation history would be beneficial?

[ShaunR]

Three words: Kill the cat

Ooooh. You like living dangerously eh? .

[Aristos Queue]

So that leaves my other question unanswered. Other than deleting records to reduce the class file size, are there any use cases where editing the mutation history would be beneficial?

I theorized that it would be beneficial for backing out edits. I make an edit to the class that removes a field. I save the class. I realize, oops, I want that back. Add it back in. But to the mutation code, this looks like "delete the data and add a new field with default values." Editing the mutation history would let you address changes like that so that the middle step you record as having no real changes (assuming you don't actually have any flattened data saved at that middle step).