Jump to content

Recommended Posts

@ShaunR "Didn't that do JSON?"
Yes. Initially I rolled my own JSON parser, but it was rewritten to call JSONText as subVIs. The Character Lineator is a better framework (my opinion) for importing and exporting objects into/out of JSON than trying to hook the serializer directly. Raw JSON doesn't provide any schema version management nor any data massaging capacities. It's just fields without context. The JSONText library is important for the raw string handling. You need more infrastructure sitting above it to actually use JSON. CL is one possible choice for that infrastructure. The serializer hook that was added to JSONText in its last revision allows users to roll their own infrastructure.

Link to comment

Alternative (6): all attributes are returned as JSON values.  Rather than get the attribute as the actual type, the User would get the attribute as a string, then use From JSON to convert to type. 

One could make VIMs that combine these two steps, and these VIMs might possibly be made to work with non-JSON attributes as well.

  • Like 1
Link to comment
9 hours ago, ShaunR said:

A paean to the utility of JSONText. :)

Kind of. The JSONText VIs are more efficient overall than my originals that I slapped together in a weekend. But the current JSONText VIs do have some inefficiencies that my originals didn’t have (most notably, see the earlier post about accessing all fields of an object) that crop up when translating schemas. And the single  serializer hook is insufficient to do some operations — support for sets/maps or custom translations of complex data structures, for example. These are things that the CL architecture accounts for that I wish JSONText was refactored to support. Having said that, I’m happy enough for my own projects with the existing toolkit and unwilling to donate the refactoring time. 
 

It is definitely a good toolkit. But it still has a long way to go in my opinion. 

Link to comment
10 hours ago, Aristos Queue said:

But the current JSONText VIs do have some inefficiencies that my originals didn’t have (most notably, see the earlier post about accessing all fields of an object)

The Object thing is not where I would spend limited effort in performance improvements, as I would expect the User to actually use the item Values and thus require a copy at some point.   I would rather improve the JSONpath search algorithm.

Link to comment
3 minutes ago, drjdpowell said:

I hope Users can do whatever they want using the lower-level functions plus subJSON with the <JSON> tag.

Indeed. I don't know what AQ is referring to with CL (Command Line? Common Lisp?) but I have a feeling he has an eye on the library supporting JSON Schema-another IETF brain-fart which made HTML a nightmare until they ditched it and brought us XML. They have a habit of taking nice simple solutions and "formalising" them them into bloated, complicated solutions that nobody uses.

10 hours ago, Aristos Queue said:

But it still has a long way to go in my opinion.

I think the library is pretty much feature complete at this point-maybe some data types missing but can't think of any off the top of my head - so this sounds peculiar...unless JSON schema is your target.

Remember. Many of us only use toolkits like these because NI refused to make the native JSON primitives' fit-for-purpose.

Link to comment
12 hours ago, ShaunR said:

Remember. Many of us only use toolkits like these because NI refused to make the native JSON primitives' fit-for-purpose.

Dont misunderstand me: I think it’s a great toolkit, and I think it should be a G toolkit not a language primitive. I’m just finding it’s edge cases. 🙂

Link to comment
9 hours ago, Aristos Queue said:

I think it’s a great toolkit, and I think it should be a G toolkit not a language primitive.

Possibly in this case as it has searching. But we wouldn't have to write parsers if the current primitives were useful 

I don't know what the current status of the G toolkit is, since the last person that took over maintenance and release is no longer active on this forum.

9 hours ago, Aristos Queue said:

 I’m just finding it’s edge cases. 🙂

That's fair enough and I'm sure appreciated. However JSON Schema - if that's your target - isn't an edge case. Your Lineator would be the place to implement that, so I'm confused by the comment "long way to go" for a library that I think is pretty much feature complete.

Link to comment
  • 2 weeks later...

Found two bugs in "Variant to JSON Text.vi".

The fixed VI is attached, saved in LV 2015. Included in this is a whole lot of manual diagram cleanup to help me follow some cases I was trying to understand. I noticed that there are several calls to the To JSON primitive that do not have the "enable LV extensions" boolean wired up. I did NOT change any of those, but they may warrant a review. There are also a few places where the error cluster is dropped that I don't think it should be.

FIRST Here's the test case for the major bug:

image.png.ced291d98ca54ee4060606e348efd357.png

Expected JSON:

{"a":null,"b":null}

Actual JSON:

[null, null]

SECOND A simple miswire: the LV extensions bool was wired to the pretty print terminal in the Waveform case.

image.png.11db748849956c643bb3a978ea231098.png

Variant to JSON Text.vi

Link to comment
Posted (edited)
11 hours ago, Aristos Queue said:

FIRST Here's the test case for the major bug:

image.png.ced291d98ca54ee4060606e348efd357.png

Expected JSON:

{"a":null,"b":null}

Actual JSON:

[null, null]

 

That is arguably not a bug, as Variants involves two names: the name of the Variant, and the name of the data contained in the Variant.  In your case your empty Variants contain unnamed void data, and JSONtext flattens clusters of unnamed elements as JSON Arrays. 

It is not obvious if it is the best choice to use the data name rather than the Variant name.  However, I am sort of tied to the behaviour of the Variant-to-Data node that I use internally to convert Clusters to Arrays of Variants.  See this example:

1072512207_2021-08-0509_03_58-Untitled2BlockDiagramonJSONtext.lvproj_MyComputer_.png.128081aeae178f2777d34c3495253877.png

Note that Typed clusters and Clusters of Variants containing Typed Data both convert identically, while Variant elements of a cluster never retain the names of the Variants themselves, just the name on the Data in the Variant).  The produced JSON matches these choices, made by the programmers of the Variant-to-Data node.

Edited by drjdpowell
Link to comment

I've had multiple requests to "support Variants", but there is a mismatch between Variants (Data plus full type description including name) and JSON (Data with only weak type info: string, number, etc.).   To support unflattening JSON to Variant would require a form something like this:

{
  "VarA":{"Variant type":"long","name":"TypedA","Value":123},
  "VarB":{"Variant type":"long","name":"TypedB","Value":456}
}

But this is a rabbit hole I'm not going down.  I'd rather ask "Why are you using Variants here instead of subJSON?"

This gives you the behaviour you expect:

664794586_2021-08-0511_15_17-Untitled2BlockDiagramonJSONtext.lvproj_MyComputer_.png.088dac376dafd4ffc0a0a45f335870f3.png

Link to comment

People who wish to use Variants and JSON generally want to do this:

  1. Put Static-Typed Data in a Variant in a larger data structure
  2. Convert larger structure to JSON
  3. Send the JSON somewhere
  4. Convert JSON back to larger structure (containing Variant)
  5. Convert Variant to Static-Typed Data

But Step (4) is a big problem, as at that point we don't have access to the Type-description information to rebuild the Variant.

But I say do this:

  1. Convert Static-Typed Data to JSON
  2. Assemble larger JSON from this subJSON
  3. Send the JSON somewhere
  4. Extract the subJSON
  5. Convert subJSON to Static-Typed Data

This avoids the problem, as we only need the Type at the first and last steps,where we statically know the type.

Link to comment

Our use  case is nullable strings and nullable integers. We want to be able to differentiate between "this is the set value of the parameter (including possibly empty string)" and "this parameter has never been set, so defer to the default". So a cluster of variant correctly should record "null" for an empty variant and not-null for a non-empty variant.

Quote

That is arguably not a bug

Recording an array when passed a cluster is definitely a bug, regardless of the behavior of the variants.

Link to comment
4 hours ago, drjdpowell said:

To support unflattening JSON to Variant would require a form something like this:

For integers you can just reconstitute as i64 regardless. That covers most of them. A note telling the dev that this is so, means they can do things after for the edge cases. Most of the time the native coercion will cover the edge cases when the dev wires them up.

Same goes for Singles and Doubles as Extended by detecting the decimal point, lack of quotes and the E in the JSON string to differentiate them from integers.

So you end up with only 2 numeric types. I64 Integer or extended depending on the JSON format of the number.

Link to comment
1 hour ago, Aristos Queue said:

Recording an array when passed a cluster is definitely a bug, regardless of the behavior of the variants.

LabVIEW Clusters and Arrays do not map directly onto JSON Objects and Arrays.   

  • LabVIEW Clusters are fixed-size ordered sets of optionally-named values
  • JSON Objects are variable-size, unordered sets of must-be-named values
  • LabVIEW Arrays are variable sized ordered sets of values all the same type
  • JSON Arrays are variable sized ordered sets of values that can all be different type

So a LabVIEW Cluster with unnamed elements cannot map onto a JSON Object, but can map onto a JSON Array.  Similarly, mixed-type JSON Arrays cannot be converted to a LabVIEW array, but can be converted to a LabVIEW cluster (assuming you know the number of elements and types).  JSONtext thus supports two mappings for LabVIEW Clusters, as explained in Help>>JDP Science>>JSONtext...

 

1888267577_2021-08-0516_51_02-ClusterandArrayconversiontoJSON_html.png.c6bfc9fec3c954cdf12dc7360e6a24f2.png

Link to comment
Quote

Recording an array when passed a cluster is definitely a bug, regardless of the behavior of the variants.

Allow me to rephrase then.

Recording an array when passed a NAMED cluster is definitely a bug, regardless of the behavior of the variants.

Link to comment
1 hour ago, Aristos Queue said:

Our use  case is nullable strings and nullable integers. We want to be able to differentiate between "this is the set value of the parameter (including possibly empty string)" and "this parameter has never been set, so defer to the default". So a cluster of variant correctly should record "null" for an empty variant and not-null for a non-empty variant.

Consider just not including those parameters.  Rather than {"A":123,"B":null,"C":789} just have {"A":123,"C":789}; then "B" will be default.

Alternately you could put named-Variant values inside your variants (which teh Variant-to-Data node will pass through:

1090371450_2021-08-0517_26_42-Untitled2BlockDiagramonJSONtext.lvproj_MyComputer_.png.63907e2224240eac829f7dadb8bfa9d8.png

Variants are quite tricky, as they can serve both as a temporary container for a value, and a value itself.

Link to comment
Posted (edited)
15 minutes ago, Aristos Queue said:

Recording an array when passed a NAMED cluster is definitely a bug, regardless of the behavior of the variants.

Again, Variants are tricky, and have two names and serve two purposes.  Viewed as a container of named data, your Variants are just placeholders, not actual data themselves (which is why I suggested in the last post to put a named Variant as Data inside the placeholder).

Edited by drjdpowell
Link to comment

I am trying to use this to convert a map with an enum as its key. It seems to convert the map to something which looks sensible but when I try and convert the JSON back to a map I am getting an error saying that the it is trying to create a map with identical keys.

I haven't had time to look into what is causing this properly, I was hoping someone had some thoughts.Enum Map.vi

Link to comment
7 hours ago, Niatross said:

I am trying to use this to convert a map with an enum as its key. It seems to convert the map to something which looks sensible but when I try and convert the JSON back to a map I am getting an error saying that the it is trying to create a map with identical keys.

I haven't had time to look into what is causing this properly, I was hoping someone had some thoughts.Enum Map.vi

Here is what happens:

  1. Scalar JSON text to Variant.vi uses the index output of the Search 1D Array function (I32) for the Enum value (U16). JSON to Map.vi then uses Variant To Flattened String (special).vi to extract the data. Enum U16 has 2 Bytes but I32 has 4 Bytes of data, so the Map data gets offset by 2 Bytes for each Key.
  2. Scalar JSON text to Variant.vi uses the output of Get U32.vi for all unsigned integers. JSON to Map.vi then uses Variant To Flattened String (special).vi to extract the data. U16 has 2 Bytes but U32 has 4 Bytes of data, so the Map data gets offset by 2 Bytes for each value.

The solution is to cast all values according to their respective types. Here are the two offending sections for your particular case:

image.png.7e6757601af6e37aad0af6cb246f657a.png

image.png.bd00e7e18fd0808e720ff6bb9c068817.png

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By LogMAN
      I recently stumbled upon this issue while debugging an application that didn't handle JSON payload as expected.
      As it turns out, Unflatten From JSON breaks on NUL characters, even though Flatten To JSON properly encodes them as ("\u0000").


      I have confirmed the behavior in LabVIEW 2017, 2019, and 2020 (all with latest service packs and patches), currently waiting for confirmation by NI.
      Workaround: Use JSONtext
    • By drjdpowell
      Package for working with JSON.  Uses high-speed text parsing, rather than building an intermediate representation as with prior LabVIEW JSON libraries (this is much faster).  Allows easy working with "subitems" in JSON format, or one can convert to/from LabVIEW types.  Uses the new "malleable VIs" of LabVIEW 2017 to convert to any LabVIEW type directly.
      JSON text makes use of a form a JSON Path notation, allowing easy and rapid access to the subitems of interest.
      Requires LabVIEW 2017 and install by VIPM 2017 or later.
      Original conversation about JSONtext.
      On the LabVIEW Tools Network.
      JDP Science Tools group on NI.com.
      Copyright 2017 JDP Science Limited
    • By drjdpowell
      View File JSON LabVIEW
      JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. This is yet another one motivated by this hijacked conversation originally about a different project to convert JSON into LabVIEW Variants.

      This project uses a set of LVOOP classes to match the recursive structure of JSON, rather than variants. It allows conversation to and from JSON. All functionality is available through two polymorphic VIs: Set and Get. In addition to Get and Set VIs for common data types, one can also convert directly to or from complex clusters via variant-JSON tools.

      Copyright 2012-2016 James David Powell, Shaun Rumbell, Ton Plomp and James McNally.
      [Note: if you are using LabVIEW 2017, please also see the JSONtext library as a an alternative.]
      Submitter drjdpowell Submitted 10/04/2012 Category General LabVIEW Version  
    • By ThomasGutzler
      Hi,
      I'm receiving a JSON string from a web API, which I'm trying to convert into a cluster (of clusters) but I've run into some problems:
      1) Sometimes the order of the elements in the JSON string changes, which causes my conversion to fail.
      2) Sometimes the "object" returned via JSON is null, which causes my conversion to fail if I use clusters within clusters. It works with variants in clusters but then I need to convert the all the variants manually
      Is there any way to improve my code to fix those problems? The attached snippet is a simplified version. In my project, the first JSON string to data is done in a library. The second conversion from "result" variant to data is done in my application. This makes a direct conversion from JSON string more difficult.

    • By drjdpowell
      JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. This is yet another one motivated by this hijacked conversation originally about a different project to convert JSON into LabVIEW Variants.

      This project uses a set of LVOOP classes to match the recursive structure of JSON, rather than variants. It allows conversation to and from JSON. All functionality is available through two polymorphic VIs: Set and Get. In addition to Get and Set VIs for common data types, one can also convert directly to or from complex clusters via variant-JSON tools.

      Copyright 2012-2016 James David Powell, Shaun Rumbell, Ton Plomp and James McNally.
      [Note: if you are using LabVIEW 2017, please also see the JSONtext library as a an alternative.]
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.