[CR] JSON LabVIEW

[ShaunR]

That is an interesting idea... I would imagine that being able to work in place could definitely offer better performance for large / nested JSON objects.

 

 

From the poking around I did, the sorting that takes place is an implicit sort when adding the JSON elements as variant attributes to a containing element. They end up getting stored alphabetically (in the variant attribute tree) so when "Get all attributes" is done in the "flattening to JSON string" VI, the objects come out in alphabetical order. The workaround I threw together kept a list of names so that instead of "get all attributes" a "foreach name, get attribute" was done (which preserved original order).

 

So can we not prepend an index to the name in the variant attribute list to enforce a sort order and strip it off when we retrieve?

(I really should download it and take a proper look, but having to find and install all the OpenG stuff first is putting me off...lol)

Edited October 8, 2014 by ShaunR

[JamesMc86]

Hi Guys,

 

A couple of issues to look at. I'm happy to look at myself if you want them in.

 

1. Can we rename GET (polymorphic) and SET (polymorphic) to JSON GET (polymorphic) and JSON SET (polymorphic), it would make them easier in quick drop.

2. One I came across today is that there isn't currently support for escaping strings. I remember there is something in the VI.lib extras library for it.

EDIT: Just seen reference to this in the commit history for 1.3, will check this first.

 

Cheers,

James

Edited October 10, 2014 by JamesMc86

[drjdpowell]

I believe Quick Drop uses the VI Title, rather than Name, so we can easily change the titles without causing any other issues.

 

String values are escaped; however I realized recently that the name strings in JSON Objects aren’t escaped.  That’s on my list of changes to make when I have the time.  

 

—James

 

PS> since we are listing issues that need to be addressed, the parsing fails on empty subObjects, such a {subObj:{},â€OtherItemâ€:1}

[drjdpowell]

The latest JSON package, where I have fixes some issues with corner cases, mostly by changing the string escaping (failed on strings like “C:\\“).  Also extended escaping to the name strings of JSON Objects (which we had neglected to do).   I will make this the latest CR version in a week or so if there is no objection.

lava_lib_json_api-1.3.1.25.vip

[JamesMc86]

Hi,

 

Thanks for the update. The escaping problem turns out to be mine!

 

I've been trying to compare output with that of the JSON library in Node.js. It actually only escapes the bare minimum which does not include / despite there being an escape character, so both are valid JSON, just not comparable.

[drjdpowell]

 It actually only escapes the bare minimum which does not include / despite there being an escape character, so both are valid JSON, just not comparable.

I had never noticed that JSON allows, but does not require, / to be escaped as \/.   Googling suggests this is due to some use of JSON embedded in other formats such as HTML that do use / as a control character.  I believe our library as it stands should accept both versions.   But what should it do on output?

[JamesMc86]

The argument I have seen for not escaping them is because escaping increases the size of the data, in this case necessarily.

 

From my point of view either could work, it just meant I had to change my code and do it properly instead of cheating!

[Stobber]

I had never noticed that JSON allows, but does not require, / to be escaped as \/.   Googling suggests this is due to some use of JSON embedded in other formats such as HTML that do use / as a control character.  I believe our library as it stands should accept both versions.   But what should it do on output?

Accepting both and outputting the spec-conformant version seems like the right thing to do, especially if that version is smaller and can be modified by the application to add escaping for that character as needed.

[drjdpowell]

I’ve modified the code to only escape ‘/' if it follows ‘<‘ (so  â€˜</‘ becomes ‘<\/‘), as it’s use in HTML <script>..</script> seems to be the reason that escaping of ‘/‘ is allowed in JSON.

[ShaunR]

Hmmm. Marked as broken?

 

If i hadn't stumbled into the code repository area by mistake, I wouldn't have known. I wonder when that was reported?

 

P.S

Can we change the spelling of my name in the VI package description? (just noticed)

Edited March 23, 2015 by ShaunR

[hooovahh]

It says it was reported on Jan 26 2015.  I am looking into a way to get that notice removed.  I feel it is the kind of complaint that should be discussed before considering a submission broken.  Until then here is what the user complained about.

 

 

 

This file has been reported as broken because: I have started using this JSON tookit, It works really good when input JSON data size is less.

Having issues when i give a JSON data size >60K to "from JSON vi".

On Windows Environment#
After it runs, LabVIEW becomes unresponsive for quite some.

On NI LabVIEW RT(PharLap)#
Actually i used to call the attached vi as one of the subvi from Main VI. Once the application terminate in RT, CPU utilization in RT(PXI) goes upto ~90% communication betwen RT and host becomes unresponsive quite some time and comes back to normal.

I feel there seems to be an issue in memory free after the big data is parsed, please let me know if what needs to be done on this...

I dont know how to attach my code in this report, I would like to share the input data to "from JSON.vi"

Thanks & Regards,
Venkatesh.C.S

[drjdpowell]

I got a PM from the user, Venkatesh.C.S,  and we found that the issue was only on LabVIEW 2014.   No problem on 2013.  I'm not working with 2014 yet so I haven’t followed up. Venkatesh consulted NI tech support, I think, but they hadn’t yet responded.

[ShaunR]

I got a PM from the user, Venkatesh.C.S,  and we found that the issue was only on LabVIEW 2014.   No problem on 2013.  I'm not working with 2014 yet so I haven’t followed up. Venkatesh consulted NI tech support, I think, but they hadn’t yet responded.

I don't have a Pharlap target, so I won't be able to investigate. However. If I had to guess at a starting point; I would be looking at the variant  type library since it uses UI (orange) calls to LabVIEW.

[Stobber]

Any chance of adding support for Inf and NaN to this API? I want to move off of the native LV prims because of their rigidity with data type and missing items in a JSON stream, but I can't let go of their support for these DBL values. Maybe you can add a property node to the JSON Value class that sets whether or not to allow these values in numeric fields?

 

Edit: Let me clarify, since playing with the API further shows that there is some support for those values. When I try this code on a stream that doesn't have the "updated period " field in its clusters, I don't get NaN out. I get the default value for the data type, instead.

 

Edited December 2, 2015 by Stobber

[ShaunR]

Any chance of adding support for Inf and NaN to this API? I want to move off of the native LV prims because of their rigidity with data type and missing items in a JSON stream, but I can't let go of their support for these DBL values. Maybe you can add a property node to the JSON Value class that sets whether or not to allow these values in numeric fields?

 

Edit: Let me clarify, since playing with the API further shows that there is some support for those values. When I try this code on a stream that doesn't have the "updated period " field in its clusters, I don't get NaN out. I get the default value for the data type, instead.

 

 

It was discussed in the past. The problem is that the JSON spec forbids it specifically.

[Stobber]

And that's that, huh? No interest in making it interoperable with the LV API by "supporting LV's extensions to JSON"?

[ShaunR]

And that's that, huh? No interest in making it interoperable with the LV API by "supporting LV's extensions to JSON"?

 

Not at all. Just saying it's been discussed before and there was no consensus. People were more worried about NULL

Edited December 2, 2015 by ShaunR

[Stobber]

Is consensus among the original authors needed to act, or is there a public repo I can fork to add the feature?

[ShaunR]

Is consensus among the original authors needed to act, or is there a public repo I can fork to add the feature?

 

Got for it

 

Hmm.

The version in the CR is later than in the repository. Maybe drjdpowell is building off of his own branch?

Edited December 2, 2015 by ShaunR

[drjdpowell]

Any chance of adding support for Inf and NaN to this API? I want to move off of the native LV prims because of their rigidity with data type and missing items in a JSON stream, but I can't let go of their support for these DBL values. Maybe you can add a property node to the JSON Value class that sets whether or not to allow these values in numeric fields?

It looks easy enough to add a Boolean to enable use of NaN, Infinity, -Infinity.   It will have to be default False, though, as the default should be to meet the JSON standard.  I would like to see this myself, as I use JSON mostly LabVIEW-to-LabVIEW.   Maybe this should be an enum instead of a boolean, in case we want an alternate extension in the future?

Edit: Let me clarify, since playing with the API further shows that there is some support for those values. When I try this code on a stream that doesn't have the "updated period " field in its clusters, I don't get NaN out. I get the default value for the data type, instead.

 

 

Not sure we can get default values of clusters in arrays (also, which array element should be used?  First? Same Index?).   As a work around, you can just index over the JSON Array elements in a FOR loop and convert each to a Cluster individually.  Then you can either provide the same cluster as default, or have an array of different default clusters.

[Stobber]

It looks easy enough to add a Boolean to enable use of NaN, Infinity, -Infinity.   It will have to be default False, though, as the default should be to meet the JSON standard.  I would like to see this myself, as I use JSON mostly LabVIEW-to-LabVIEW.   Maybe this should be an enum instead of a boolean, in case we want an alternate extension in the future?

FWIW, I'd stick to the Boolean. There's no reason to assume any other extensions would need to be mutually exclusive from the LV numeric extensions. An enum would enforce that exclusion.

 

 

Not sure we can get default values of clusters in arrays (also, which array element should be used?  First? Same Index?).   As a work around, you can just index over the JSON Array elements in a FOR loop and convert each to a Cluster individually.  Then you can either provide the same cluster as default, or have an array of different default clusters.

Same index, of course. I'm specifying the default value of the aggregated data structure. If I want all elements to have the same default values, then I'll either need an interface that lets me specify the element type of the array (instead of the array), or I'll have to use the "Initialize Array" prim.

[drjdpowell]

 There's no reason to assume any other extensions would need to be mutually exclusive from the LV numeric extensions. An enum would enforce that exclusion.

I mean alternate treatments of Inf, -Inf, and NaN.  I can think of alternatives that are arguably better than the choice made in the LabVIEW primitive.  Personally, I would choose NaN==Null, Inf/-Inf==“Infinityâ€/“-Infinity†(strings), as this would be valid JSON.  The LAVA JSON package should already be able to read this (I think).   Regardless, other JSON packages that we may want to interact with may have made other choices.

[drjdpowell]

Here is a Beta version with a new “Treatment of Infinity/NaN†terminal on the JSON write functions.

Three options:

-- Default JSON: write all as nulls.
-- Extended: use JSON strings for Infinities ("Infinity", "-Infinity"); NaN still null.
-- LabVIEW extentions (compatable with the LabVIEW JSON primatives): Infinity, -Infinity, NaN (not in quotes)

Note that the LabVIEW extentions are not valid JSON.  The Extended option IS valid JSON, but not all JSON parsers will process strings as numeric values.

 

lava_lib_json_api-1.4.0.28.vip

[ShaunR]

Here is a Beta version with a new “Treatment of Infinity/NaN†terminal on the JSON write functions.

Three options:

-- Default JSON: write all as nulls.

-- Extended: use JSON strings for Infinities ("Infinity", "-Infinity"); NaN still null.

-- LabVIEW extentions (compatable with the LabVIEW JSON primatives): Infinity, -Infinity, NaN (not in quotes)

Note that the LabVIEW extentions are not valid JSON.  The Extended option IS valid JSON, but not all JSON parsers will process strings as numeric values.

 

Is this no longer on Bitbucket? Where is the repository now?

[drjdpowell]

Is this no longer on Bitbucket? Where is the repository now?

Sorry, forgot to push the changes.  It’s there now.