Jump to content

Recommended Posts

I'm trying to parse this JSON data:
 

{
  "id":"fb66aaec-f713-424c-ba04-386d0986fb12",
  "planId":"4c7cbe9c-ba67-4d8b-b731-32d3de0de700",
  "name":"AJ3",
  "redrillOfHoleId":null,
  "design":{
    "angle":0.0,
    "bearing":4.376555,
    "comment":null,
    "decks":[
      {
        "number":1,
        "horizon":25.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":2,
        "horizon":20.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786469,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":3,
        "horizon":15.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":4,
        "horizon":10.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429169,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":5,
        "horizon":5.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":true,
        "isBackfill":false
      },
      {
        "number":6,
        "horizon":0.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786469,
        "isVariable":false,
        "isBackfill":false
      }
    ],
    "diameter":300.0,
    "length":30.0,
    "x":-194.97681394419092,
    "y":67.241148799992772,
    "z":0.0,
    "whenCreated":"2020-01-28T01:51:43.4065706Z",
    "whenUpdated":"2020-01-28T01:51:43.4065706Z"
  },
  "adjustedDesign":{
    "angle":0.0,
    "bearing":4.376555,
    "comment":null,
    "decks":[
      {
        "number":1,
        "horizon":25.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":2,
        "horizon":20.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":3,
        "horizon":15.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":4,
        "horizon":10.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":5,
        "horizon":5.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":6,
        "horizon":0.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786,
        "isVariable":false,
        "isBackfill":false
      }
    ],
    "diameter":300.0,
    "length":30.0,
    "x":-194.97681394419092,
    "y":67.241148799992772,
    "z":0.0,
    "whenCreated":"2020-01-28T02:04:07.6143372Z"
  },
  "actual":{
    "angle":null,
    "bearing":null,
    "calculatedDecks":[],
    "comment":null,
    "depthToWater":null,
    "diameter":null,
    "expectedDeckCount":null,
    "length":null,
    "accessories":[],
    "measuredDecks":[],
    "notDrilled":false,
    "userName":null,
    "x":null,
    "y":null,
    "z":null
  },
  "drillAdjustedDesign":null,
  "drillActual":null,
  "holeType":"Hole type 1",
  "materialType":null,
  "isDeleted":false,
  "isNonDesignedHole":false,
  "allowTruncatedDesign":false,
  "disableChangeDesignRules":false,
  "scaleDepthOfBurial":null,
  "strataIntercepts":{},
  "tags":["drilling_designed","loading_designed"],
  "whenCreated":"2020-01-28T01:51:43.4065706Z",
  "whenUpdated":"2020-01-28T01:51:43.4065706Z"
}

But am currently getting this error:
 

Quote

JSONtext.lvlib:JSON text to Variant.vi:4410001<ERR>
State: Parser Error
Near: ':nu'
in '"2020-01-28T02:04:07.6143372Z"},"actual":{"angle":null,"bearing":null,"calculatedDecks":[],"comment":null,"depthToWater":null,"diameter":null,"expectedDeckCount":null,"length":null,"accessories":[],"me'


<b>Complete call chain:</b>
     JSONtext.lvlib:JSON text to Variant.vi:4410001
     JSONtext.lvlib:JSON text to Variant.vi:4410003
     Parse Hole JSON.vi

 

Link to comment

@drjdpowell Also if you go through and set the comment fields to empty strings, the next parsing error hit is this:
 

Quote

Error 1 occurred at JSONtext.lvlib:JSON text to Variant.vi:4410003

Possible reason(s):

Cannot Get as integer: null


Complete call chain:
     JSONtext.lvlib:JSON text to Variant.vi:4410003
     JSONtext.lvlib:JSON text to Variant.vi:4410002
     JSON Parsing Error.vi

This is replicated using the same VI shared above by first setting the comment keys to actual strings

Link to comment

That one is trickier.   What is null supposed to mean, interpreted as an integer?   Unlike for Floats, there is no Not-an-Integer value, nor is there an obvious "null" value like an empty string.   Should one just use zero as null?   Or should one consider the null to be that same as if the item was not present at all in the JSON, those using your default value in the supplied cluster.  Why is whatever generated this JSON providing null values at all?  Especially in place of strings and integers?

Link to comment
13 hours ago, camow7 said:

Until a fix is done I could precheck specific fields that may error and set them to empty strings before passing into your conversion vi.

A possibly better option is to replace your string/integer values with JSON strings, which is LabVIEW strings with <JSON> at the start of the name.  So "<JSON>Comment" rather than "Comment".   JSONtext will happily parse out the value into your cluster as JSON, then when you actually need your value you can convert it to string/integer, with appropriate handling if it is null or throws an error.

This method, by the way, is also how to handle JSON that has variable structure.

  • Like 1
Link to comment
2 hours ago, drjdpowell said:

Why is whatever generated this JSON providing null values at all?  Especially in place of strings and integers?

I imagine it is C , C++ or Javascript. NULL is a specific type of invalid pointer in C/C++ languages and "not an object" in Javascript (as opposed to "undefined"). In LabVIEW we don't really have either concepts.

In JSON, it is a valid type so it depends on how you want to translate it back into a LabVIEW type. Historically I have converted in the following manner: string->empty, numeric->0, boolean->false etc. Unless, of course, it is in quotes. In that case it is the string "NULL", in whatever case, as it's implicity typed.

Edited by ShaunR
Link to comment

Is there a plan to implement conditional filters in the JSON paths?  For instance the following should return a single book title, but I get a parsing error due to the question mark.

Or is this already implemented in a different manner?

--tested on jsonpath.com--

$.store.book[?(@.category=='reference')].title

 

{
    "store":
    {
        "book":
        [
            { "category": "reference",
              "author": "Nigel Rees",
              "title": "Sayings of the Century",
              "price": 8.95
            },
            { "category": "fiction",
              "author": "Evelyn Waugh",
              "title": "Sword of Honour",
              "price": 12.99
            },
            { "category": "fiction",
              "author": "Herman Melville",
              "title": "Moby Dick",
              "isbn": "0-553-21311-3",
              "price": 8.99
            },
            { "category": "fiction",
              "author": "J. R. R. Tolkien",
              "title": "The Lord of the Rings",
              "isbn": "0-395-19395-8",
              "price": 22.99
            }
        ],
        "bicycle":
        {
            "color": "red",
            "price": 19.95
        }
    }
}

 

Link to comment
On 1/31/2020 at 11:25 PM, drjdpowell said:

A possibly better option is to replace your string/integer values with JSON strings, which is LabVIEW strings with <JSON> at the start of the name.  So "<JSON>Comment" rather than "Comment".   JSONtext will happily parse out the value into your cluster as JSON, then when you actually need your value you can convert it to string/integer, with appropriate handling if it is null or throws an error.

This method, by the way, is also how to handle JSON that has variable structure.

Tested this and it worked well. Had no idea you could add the <JSON> tag to parse complex data types like dictionaries. Super handy. This looks likes it's solved my problems. Thanks!

Link to comment
On 2/3/2020 at 5:01 AM, camow7 said:

Any plans to include a "Set Null to Default" feature?

Unfortunately this contrasts with the current behaviour that null-->NaN for a floating-point number, rather than being the default number input.  In standard JSON, the float values NaN, Infinity and -Infinity have to become null, and to convert them back to a default value doesn't make sense.  We could add an option to "ignore null items" which would treat nulls as equivalent to that item not existing. 

  • Like 1
Link to comment
On 1/31/2020 at 5:59 PM, pjr1121 said:

Is there a plan to implement conditional filters in the JSON paths?  For instance the following should return a single book title, but I get a parsing error due to the question mark.

Or is this already implemented in a different manner?

--tested on jsonpath.com--

$.store.book[?(@.category=='reference')].title

Plan?  Yes, but not a priority.   Note, though, that JSONtext returns substrings, meaning you can implement filtering in LabVIEW without making data copies, so the following code implements your example:

1925139157_JSONtextfilteringwithLabVIEW.png.9c67cd15e4d89c0d78552941e184b98b.png

More effort, of course, but it should be fast, and using LabVIEW is very flexible, and more debug-able.  

Link to comment
  • 1 month later...

Help

Is there a way to parse this JSON string into this cluster in the following way:
image.png.4ca6fc2304370ea4d98288a6434e8ad6.png

- The order of elements inside the "Parameters" cluster does not matter. No error
- I can use enums
- The name of the elements matter. "TWOoo" shouldn't end up populating "two" (even if the order of elements matches the cluster). I want an error here instead of "two" being the default value
- Don't care if JSON text has additional elements that aren't in the cluster. No error

In short, features: Strict name checking, order independence, error if element missing

I've tried so many things but they all fail in one way or the other. See attached snippet

snip.png

  • Like 1
Link to comment

@ThomasGutzler here is a version that does most of what you describe.

  • The order of elements inside Parameters doesn't matter.
  • You can use enums.
  • It doesn't matter if the JSON string has additional elements (no error)

However, you don't get an error if an element is missing ("two" in your example). That would require something like a "strict" mode, which I don't think is supported by JSONtext.

526902039_ExtractParameters.png.f887cb8dbf8b55807ce18965b533cc42.png

Unfortunately code is removed from VI snippets on LAVA, so here is the VI. Extract Parameters.vi

Link to comment
4 minutes ago, LogMAN said:

However, you don't get an error if an element is missing ("two" in your example). That would require something like a "strict" mode, which I don't think is supported by JSONtext.

Actually, if you turn your cluster back into JSON you can compare it to the original JSON string to see if there are any differences. The string is empty if every element is defined in the source string.

Try this: Extract Parameters.vi

Notice the "Differences" string, which lists all differences (missing elements) in your source JSON string.

675891111_ExtractParametersV2.png.4a6efda39861851394cc5ff03f2aed93.png

  • Thanks 1
Link to comment

Standard JSON specification for JSON objects is that they are a collection of named elements with order unimportant, so any JSON library should ignore order and match on names.

Missing elements is a design choice, though, and I've gone with "use supplied default" rather than "throw error", for a number of reasons.  But as LogMAN shows, there are ways to do checking yourself.

Link to comment
On 2/4/2020 at 4:28 AM, drjdpowell said:

Unfortunately this contrasts with the current behaviour that null-->NaN for a floating-point number, rather than being the default number input.  In standard JSON, the float values NaN, Infinity and -Infinity have to become null, and to convert them back to a default value doesn't make sense.  We could add an option to "ignore null items" which would treat nulls as equivalent to that item not existing. 

The native LabVIEW Unflatten From JSON Function might be of use.

I used a Salesforce REST API that returned JSON. There were numerous string and numeric fields that returned as NULL.

If you use the native LV function and change 'default null elements?(F)' to true and set 'enable LabVIEW extensions? (T)' to false, you can get the default values assigned in your input type & defaults cluster.

https://zone.ni.com/reference/en-XX/help/371361R-01/glang/unflatten_from_json/

 

Link to comment
  • 3 weeks later...
On 7/26/2019 at 3:35 PM, andy_r said:

Just small recommendation to change the library.

Concerning timestamp to string conversion:

The current format string "%Y-%m-%dT%H:%M:%S%3uZ" in "Variant to Json Text.vi" is dependent on localization. On my german windows this leads a to timestamp according to Iso 8601 of "2019-07-26T14:17:20,053Z". Notice the comma for the fractional seconds. According to the Iso both, comma and point, is valid (Apparently, the Iso even recommends the comma https://en.wikipedia.org/wiki/ISO_8601#cite_note-25). However, it seems that the point is used more regularly in practice, for example in .net (see Json.net package). So I would suggest to change the format like this: "%Y-%m-%dT%H:%M:%S%.;%3uZ". This ensures using the point instead of comma., hence the format is independent of your local setting.

Sorry, I had missed this comment at the time.   That is a good idea.

https://bitbucket.org/drjdpowell/jsontext/issues/53/force-use-of-decimal-point-in-timestamps

Link to comment
  • 2 months later...

Bug? JSONText doesn't apply input default values to missing cluster items of array elements.

image.png.cfac8c824a0c9d7cb52ca3a937cbf166.pngimage.png.8ae1b813f1c246373cf99f02b3bc8356.png

I was hoping to see "Default", 1 and "2", NaN but instead get "",1 and "2",0

I think that it is due to the behavior of the Get Array Information VI.

The workaround is to convert the array elements individually as clusters:

image.png.30316820d66dc9e23dc0c3ae7f3760c8.pngimage.png.4022f34cc2d9604696e4e09bf7fe0789.png

Link to comment
4 hours ago, Porter said:

JSONText doesn't apply input default values to missing cluster items of array elements.

Thanks.  Issue 62: https://bitbucket.org/drjdpowell/jsontext/issues/62/jsontext-doesnt-apply-input-default-values

Although, I'm not sure this is doable, as your default values are the default values of your on-diagram array constant, not part of the data.  As that array is empty, I don't think I can access those default values inside a subVI.  If you had passed a non-empty array with a default value as first element, then I could read that and use it as default.

Edited by drjdpowell
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.