OpenG and Object compatibility

[pawhan11]

Also worth to mention:
https://forums.ni.com/t5/Actor-Framework-Documents/AQ-Character-Lineator-working-draft-version-of-serialization/ta-p/3512840?profile.language=en

[bjustice]

2 minutes ago, pawhan11 said:

Also worth to mention:
https://forums.ni.com/t5/Actor-Framework-Documents/AQ-Character-Lineator-working-draft-version-of-serialization/ta-p/3512840?profile.language=en

Yes!  Did a deep dive through that code as well.  We're taking a similar approach

[pawhan11]

Did You came up with something better? In this lineator I don't like the idea that it must be a base class for all other classes. Wondered if it could be made using interfaces in 2020, but never made anything worth using...

[bjustice]

We adopted the notion that classes must inherit from a base "serializable.lvclass".  This activates the Project Provider plugin (project tree scripting methods).  We didn't see a convenient way around this, and we didn't see this as too much of an inconvenience.

Where we noticeably diverged from AQ Char Lineator is that we (through scripting) create a "SerializableData" cluster typedef that gets dropped into the class private data.  Only data within this cluster is consumed by the serialization framework.  all other data is ignored.  This simplified things in that we only need a single future-proof data read/write accessor for this cluster.  A user can update serializable data by only editing this typedef... they don't have to touch any VI source code.

Pros and cons to this solution.

[pawhan11]

Nice, Did You include scenarios where members of this cluster can be other classes, collections of classes, variant atributes of classes etc. that inherit or not from base serializable class?

[bjustice]

Our code supports nested and arrays of classes that inherit from the base "Serializable.lvclass" gracefully.

Classes are represented as a nested cluster in the serializable string (JSON, XML, TOML)

Objects that don't inherit from the base "Serializable.lvclass" are treated in the same manner as JSONtext handles objects... flatten/unflatten from string.  If error on unflatten, then return default object

Edited January 20, 2021 by bjustice

[pawhan11]

Neat! 

[drjdpowell]

On 1/19/2021 at 9:13 PM, bjustice said:

I now understand why James made this SubVI for his JSONtext library.
I hope that he built this thing with scripting.  Brute force, but seems to work well.

Brute force.  I suck at scripting.  I add 20-30 more cases every so often.  Maximum possible with this method is 255.  

This VI is part of the "JDP Science Common Utilities" vipm package, BTW, so one doesn't need to install JSONtext to use it.   Originated in the conversations on this idea: Convert an Array of Variants into a Cluster.  I wish NI would just implement that idea so this stopgap would be unnecessary, so please kudo that idea.  

I used to use OpenG Variant Tools a lot, but replaced them with the newer NI functions once they became faster (about LabVIEW 2012, I recall) and I had this workaround.   The only missing functionality from OpenG is the ability to set the name of data in a cluster (the above SubVI create a Cluster with generic element names; I decided to just live without that.

[bjustice]

Thanks JDP for the background information.  I just kudo'd your idea.

[LogMAN]

21 hours ago, bjustice said:

JSONtext branch with object support:
https://bitbucket.org/logmanoriginal/jsontext/src/lvobjectserialization/

This was meant as a proof of concept to see if it can be done and if it's something worth investigating. I should probably mention that this branch has a few bugs that I haven't fixed yet.

22 hours ago, bjustice said:

Internally, we made the early decision that we didn't want to directly access class private data through string flattening or by inspecting the class' *.ctl file.

Certainly not something I would use in production right now but still, I believe there is some value in this - especially for general-purpose libraries like JSONtext. Anyway, I'll back save and upload when I have access to LV. By the way, the details are explained on the Wiki: LabVIEW Object - LabVIEW Wiki

I haven't found a better way to do this without adding (or scripting) methods to every class. The only function that currently breaks encapsulation natively is Flatten To XML, which has its own limitations.

[bjustice]

2 minutes ago, LogMAN said:

This was meant as a proof of concept to see if it can be done and if it's something worth investigating. I should probably mention that this branch has a few bugs that I haven't fixed yet.

Certainly not something I would use in production right now but still, I believe there is some value in this - especially for general-purpose libraries like JSONtext. Anyway, I'll back save and upload when I have access to LV. By the way, the details are explained on the Wiki: LabVIEW Object - LabVIEW Wiki

I haven't found a better way to do this without adding (or scripting) methods to every class. The only function that currently breaks encapsulation natively is Flatten To XML, which has its own limitations.

Ha!  I now feel silly for responding to your first post with a link back to your own JSONtext branch.

Did you encapsulate your object (de)composition into a reuse library of sorts?  Or is it pretty entangled in your JSONtext branch?
I read through your code and did indeed find that wiki page.  Very cool stuff.
There are certainly large benefits to doing it the way that you did things in your JSONtext branch.  You don't have to inherit from a base class, so all classes become inherently serializable.  And there is no need for scripting or creation of data member accessors to allow for the library to do its job.

Yeah, we noticed some pretty big limitations to the "Flatten to XML" primitive as well.  This tends to leave out fields in the output XML if they are default value in the object.

[LogMAN]

It's a separate library. Object composition was actually much more difficult to figure out than the other way around. I have attached the library for LV2017 (without test suites and package configuration). I'll also put this on GitHub in the near future.

Here is an example that overwrites elements in the private data cluster (the outer IPE addresses the class hierarchy).

Here is an example that uses JSONtext to extract data from a private data cluster. I was looking into this particular case as a way to transition from clusters to/from objects

Both examples are included in the package.

Object Decomposition LV2017.zip

[bjustice]

Kudos, this is very cool

[LogMAN]

The library is now available on GitHub (including test cases) https://github.com/LogMANOriginal/LabVIEW-Composition

I also discovered this project, which provides some useful methods to work with variant data.

Edited January 23, 2021 by LogMAN
Why so sad?

[Francois Normandin]

On 1/21/2021 at 4:04 AM, drjdpowell said:

Brute force.  I suck at scripting.  I add 20-30 more cases every so often.  Maximum possible with this method is 255.  

This VI is part of the "JDP Science Common Utilities" vipm package, BTW, so one doesn't need to install JSONtext to use it.   Originated in the conversations on this idea: Convert an Array of Variants into a Cluster.  I wish NI would just implement that idea so this stopgap would be unnecessary, so please kudo that idea.  

 

Here's a snippet you can use if you want an unbound cluster size that preserves element names.
(Note that the "Create Cluster" method is found in vi.lib/Utility/GetType.llb)

 

It is much slower than your direct cast, but in the cases where you absolutely want to keep the element names, it might be worth the hit. 

Here are some quick comparison benchmarks on my machine:

- 256 named elements => 7ms (compared to ~microseconds for the fixed-sized method)
- 1000 named elements => 23ms
- 10000 named elements => 135ms

 

PS. This is a method found in DataManipulation package on VIPM. Since it is licensed under 0-BSD, feel free to extract at will (no attribution needed).

 

Edited February 11, 2021 by Francois Normandin
fixed type, added link

[drjdpowell]

49 minutes ago, Francois Normandin said:

It is much slower than your direct cast, but in the cases where you absolutely want to keep the element names, it might be worth the hit. 

Thanks.  I have often considered providing a flattening-based implementation for arbitrarily large clusters, but have always paused because of the big step change in performance.

[Aristos Queue]

On 1/20/2021 at 2:07 PM, pawhan11 said:

Did You came up with something better? In this lineator I don't like the idea that it must be a base class for all other classes. Wondered if it could be made using interfaces in 2020, but never made anything worth using...

Using interfaces would be a BUG. You cannot add serialization to a class if all ancestors do not support it or you end up with an insane class. That's why the Lineator uses class inheritance. 

[Aristos Queue]

I really didn't expect serialization to be weaponized like this when I wrote it, and over the last few years, I've become aware that it can be abused like this. I am going to have to prioritize making flatten/unflatten an option to toggle on classes, one that defaults to "off" for new classes. This is the sort of "cast to void" thing that completely sinks C++ projects. 

[LogMAN]

In my opinion NI should finally make up their mind to whether objects are inherently serializable or not. The current situation is dissatisfying.

There are native functions that clearly break encapsulation:

• Flatten To XML
• Variant To Flattened String
• Flatten To String

Then there is one function that doesn't, although many users expect it to (not to mention all the other types it doesn't support):

• Flatten To JSON

Of course users will eventually utilize one to "fix" the other. Whether or not it is a good design choice is a different question.

[drjdpowell]

On 3/1/2021 at 5:29 PM, Aristos Queue said:

Using interfaces would be a BUG. You cannot add serialization to a class if all ancestors do not support it or you end up with an insane class.

You can add them to classes whose ancestors have no data (such as LVObject).   You might have a "Flatten to JSON" interface and a "Flatten to XML" interface and a "Store in My Special Format" interface, and can decide which formats to implement.  Inheritance only works once.

[Aristos Queue]

7 hours ago, drjdpowell said:

You can add them to classes whose ancestors have no data (such as LVObject).  

That would still be a bug because a child doesn't know whether its parent will remain without private data permanently. 

Quote

Inheritance only works once.

That's why the inheritance only supplies the ability to serialize. It says nothing about the format/structure/etc. 

[drjdpowell]

On 3/4/2021 at 12:45 AM, Aristos Queue said:

That would still be a bug because a child doesn't know whether its parent will remain without private data permanently. 

Just top-level parent classes that inherit from LVObject, then?  If LVObject gets private data, even an inheritance-based serializer will fail.

On 3/4/2021 at 12:45 AM, Aristos Queue said:

That's why the inheritance only supplies the ability to serialize. It says nothing about the format/structure/etc. 

I haven't looked at your "lineator" in a long time, but I feel there must have been some architectural choices made that other developers may have reasons for making other choices.  Thus, there is a need to be able to support more than one type of serializer in the same class hierarchy.  Thus interfaces, maybe?

[drjdpowell]

For example, my Interface in JSONtext (if JSONtext were based in 2020 rather than 2017) would just implement "To JSON" and "From JSON" methods, whose default implementations would just use the standard flattening of the class in a JSON string.   I've have been holding off implementing this as a parent class because I was waiting for interfaces.   Should I just go ahead and make this a Class?  Note that a User could not use your Lineator to actually do the conversion to JSON, as they cannot inherit off your Lineator if they are already inherited off my Class.  If Interfaces were used, they could use your Lineator to produce JSON inside my JSONtext subVIs.

Edited March 5, 2021 by drjdpowell