Using JSONText to parse a poorly formatted JSON string

[jed]

Hello!  I have a JSON string that looks something like this...

 

[
   {
      "id":12345,
      "name":"E1",
      "functions":[
         "randoNameX",
         "randoNameY",
         "randoNameZ"
      ],
      "variables":{
         "randoName1":"int",
         "randoName2":"string",
         "randoName3":"dbl"
      }
   },
   {
      "id":54321,
      "name":"E3",
      "functions":["randoName"]
   },
   {
      "id":31645,
      "name":"E5"
   }
]

 

Things to note:
1) The functions and variables elements may or may not be present in each element.  This is no problem parsing individual elements, but for some reason it fails when processing them as an array.

2) "functions" is a simple array of strings, so that parses easily into a [STR] element

3) variables is more complicated, it's a LIST of random key/value pairs.  That is, I don't know what the keys will be, and they are arranged into a JSON element, not an array.

So looking at another post, I tried using JSONtext and accessing those "variable" elements through the path.  But while I can use a wildcard to access the values and ignore the keys, I can't seem to figure out how to use a path to find the key names.  I can't use a numeric to access their position in a list (as if it was an array), that errors out.

any ideas?

[jed]

Figured it out...

I was using the path $.[X] to try and access the top level array, but that's actually $[X] without the period.

What I ended up doing was to create a cluster with a variable array that looked like this [{name, type}];  but I named it "variables.and.types" so that the parser would ignore it.  Then, after doing a full parse of the entire cluster, I used the getMultipleItems (non typed) with the path $[i].variables.* and looped over the now known number of elements (substituting i) in the top level array.  This returned the names and types in two separate string arrays, which I then recombine into the array of clusters.

[drjdpowell]

You've solved your problem, but this is the solution I just did:

[drjdpowell]

Just a comment, but I have noticed that people very often deal with JSON by looking for the "monster cluster" that completely converts the JSON into a monolithic LabVIEW structure.  I suggest people think a bit more modularly in terms of "subJSON".  The first question I would ask you if I were working with you on your actual project is why do you need (at this code level, at least) to convert your variables from their perfectly reasonable JSON format to an array of clusters?

Edited May 18, 2021 by drjdpowell

[jed]

Awesome, thank you for that idea.

To answer your question...  I tend to only work with JSON as a monolithic static element.  That is, my applications seldom require me to adjust a portion of the JSON.  If you are working in another language, it makes sense to use the JSON as the actual data cluster that is modified as it moves through one's code.  But the thought process for me has always been  Retrieve JSON (as a file or HTTP request) -> Convert to LVCluister -> Process.   Now, combine that with the other part of this... the JSON to me is just a tool to get data in and out.  I am usually tasked with some other important process, not learning how JSON works.  And the tools provided in LV are pretty monolithic.  So all together, it's never occurred to me to treat the JSON element as you did above... I just want to turn it into a cluster and get to work!

(For this specific application, I am processing the response of a call to get a list of web-connected devices and their properties (what I posted is a small subset of many fields).  This JSON object would not be used by my code, so there's no reason to maintain it as JSON; converting to a LVCluster upon return is both convenient and simplifies my code.  Of course, this assumes it's a relatively simple matter to convert to LVClusters.  If I have to really crunch this thing I will probably only convert the data I anticipate using)

Of course, now that I am working with more web APIs, the need to get into the fundamentals is more important.  And WOW, your tool is one of the best things I have seen out there.  So thank you.

I must say, dealing with some of these APIs is frustrating.  It's clear to me that these functions were coded by multiple people who didn't coordinate.  In my example above, why would one not place those variable objects in an array?  I know in other languages enumeration is simple, but it's also even simpler to have a logical structure (and I won't even mention how this particular API reports errors; no consistency whatsoever).

Also... I looked at several JSONtext pages but could not find this answer... is there any documentation that would tell me how to use that <JSON>variables element?  I am still uncertain whether this just populates the string automatically with JSON or if the <JSON> tag causes this.

(A guide for newbies on using your tool would be great; I would contribute to it.  Please let me know if you would want me to put some examples together, including how to process non-straightforward examples like this.  It would also help me get familiar with your tool).

[drjdpowell]

2 hours ago, jed said:

I looked at several JSONtext pages but could not find this answer... is there any documentation that would tell me how to use that <JSON>variables element?  I am still uncertain whether this just populates the string automatically with JSON or if the <JSON> tag causes this.

Did you look at the detailed help on teh JSONtext VIs?  They should all contain links to further documentation pages (or you can go to Help>>JDP Science>>JSONtext...). The "<JSON> tags" page explains:

Quote

<JSON> tags in JSONtext control labels

<JSON> tags at the front of labels of many string outputs of JSONtext functions serve to mark those strings as being JSON text, rather than standard LabVIEW text.  Functions that input LabVIEW types (including clusters and arrays of those types) look for these tags and treat such strings as already being in JSON format.  

For example, the string control "MyString" with value '123' will be converted by "To JSON Text.vim" to the JSON String "123", but the string control "<JSON>MyString" with the same value will be converted to the JSON number 123.

For support please see [CR] JSONtext.

 

[drjdpowell]

There is a page on Path notation too:

Quote

JSON Path notation in JSONtext

Paths specify an item in a tree of nested Elements, and start with $. Each element is either in the form .name, for Object items, or [n] for Array index ([*] for all elements). Multiple Array index can be specified ([n1,n2,n3,...]), and multiple Object items can be specified by the list form .[abc,def,...].   All elements/items can be specified with a * ([*] for Array elements, or .* for object items).

If the Path input does not start with a $, then it is interpreted as a single Object Item name, rather than a path.

Path Elements:

$	Root JSON Value
.name	Object item "name"
.[name1,name2,..]	Object items
.*	All Object items
[n]	Array index n (negative n counts back from end; -1 == last element)
[n1,n2,n3,...]	Multiple Array indexes
[*]	All Array elements
[#]	For appending to the Array, indicates element past the end

Example JSON:

{"A":[{"C":123,"D":4},{"C":567,"D":8}],"B":{"F":9},"E":true}

• At path $.A[*].C are 123 and 567
• At path $.A[1].C is 567
• At path $.B (or just B) is {"F":9}
• At path $.B.F is 9
• At path $.[B,E] are {"F":9} and true
• At path $.* are [{"C":123,"D":4},{"C":567,"D":8}], {"F":9}, and true

Object items can also be specified as their proper JSON string.   This allows the use of characters in names that would otherwise not parse correctly.  For example:

$."My Name that contains \"quotes\", []brackets, and dots." 

Note that this must be proper JSON (thus the escaped quotes in the example).

Lastly, please note that JSON Path is not standardized, and will differ from other path implimentations.  

For support please see [CR] JSONtext.

 

[jed]

Thank you again for those last comments.  I went to the detailed pages for each VI, but did not see the link to the larger help (which may have just been me not paying close enough attention).  I don't even bother looking for things in the Help... menu since only a handful of tools utilize it.  I'll try and remember to check it going forward, since you took the time to write all that.

Thanks again!

[drjdpowell]

All the Tools Network libraries are supposed to have help documentation of some kind.  At some point, they insisted I do it.