Jump to content

Recommended Posts

I'm trying to parse this JSON data:
 

{
  "id":"fb66aaec-f713-424c-ba04-386d0986fb12",
  "planId":"4c7cbe9c-ba67-4d8b-b731-32d3de0de700",
  "name":"AJ3",
  "redrillOfHoleId":null,
  "design":{
    "angle":0.0,
    "bearing":4.376555,
    "comment":null,
    "decks":[
      {
        "number":1,
        "horizon":25.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":2,
        "horizon":20.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786469,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":3,
        "horizon":15.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":4,
        "horizon":10.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429169,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":5,
        "horizon":5.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":true,
        "isBackfill":false
      },
      {
        "number":6,
        "horizon":0.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786469,
        "isVariable":false,
        "isBackfill":false
      }
    ],
    "diameter":300.0,
    "length":30.0,
    "x":-194.97681394419092,
    "y":67.241148799992772,
    "z":0.0,
    "whenCreated":"2020-01-28T01:51:43.4065706Z",
    "whenUpdated":"2020-01-28T01:51:43.4065706Z"
  },
  "adjustedDesign":{
    "angle":0.0,
    "bearing":4.376555,
    "comment":null,
    "decks":[
      {
        "number":1,
        "horizon":25.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":2,
        "horizon":20.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":3,
        "horizon":15.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":4,
        "horizon":10.0,
        "length":5.0,
        "productId":"f3fb75ac-1065-4d9c-bd6a-fdf01c34837b",
        "weight":0.0,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":5,
        "horizon":5.0,
        "length":5.0,
        "productId":"0fa2dd6d-6b0c-4bfd-998b-e0b50ee64481",
        "weight":353.429,
        "isVariable":false,
        "isBackfill":false
      },
      {
        "number":6,
        "horizon":0.0,
        "length":5.0,
        "productId":"5800171c-e16e-48e7-9431-ac9b88f4852f",
        "weight":441.786,
        "isVariable":false,
        "isBackfill":false
      }
    ],
    "diameter":300.0,
    "length":30.0,
    "x":-194.97681394419092,
    "y":67.241148799992772,
    "z":0.0,
    "whenCreated":"2020-01-28T02:04:07.6143372Z"
  },
  "actual":{
    "angle":null,
    "bearing":null,
    "calculatedDecks":[],
    "comment":null,
    "depthToWater":null,
    "diameter":null,
    "expectedDeckCount":null,
    "length":null,
    "accessories":[],
    "measuredDecks":[],
    "notDrilled":false,
    "userName":null,
    "x":null,
    "y":null,
    "z":null
  },
  "drillAdjustedDesign":null,
  "drillActual":null,
  "holeType":"Hole type 1",
  "materialType":null,
  "isDeleted":false,
  "isNonDesignedHole":false,
  "allowTruncatedDesign":false,
  "disableChangeDesignRules":false,
  "scaleDepthOfBurial":null,
  "strataIntercepts":{},
  "tags":["drilling_designed","loading_designed"],
  "whenCreated":"2020-01-28T01:51:43.4065706Z",
  "whenUpdated":"2020-01-28T01:51:43.4065706Z"
}

But am currently getting this error:
 

Quote

JSONtext.lvlib:JSON text to Variant.vi:4410001<ERR>
State: Parser Error
Near: ':nu'
in '"2020-01-28T02:04:07.6143372Z"},"actual":{"angle":null,"bearing":null,"calculatedDecks":[],"comment":null,"depthToWater":null,"diameter":null,"expectedDeckCount":null,"length":null,"accessories":[],"me'


<b>Complete call chain:</b>
     JSONtext.lvlib:JSON text to Variant.vi:4410001
     JSONtext.lvlib:JSON text to Variant.vi:4410003
     Parse Hole JSON.vi

 

Link to post
Share on other sites
  • Replies 110
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

View File JSONtext Package for working with JSON.  Uses high-speed text parsing, rather than building an intermediate representation as with prior LabVIEW JSON librari

working on it

Definitely on my list of JSONtext additions.  Ideally, I would want to implement RFC 7396: JSON Merge Patch, with the ability to generate a "patch" of the differences between two JSON texts, then appl

Posted Images

The error is coming from where you attempt to convert null (as in "comment":null) to a string.   Intuitively, one would think that a null should be equivalent to an empty string (just as when converted to a number, null becomes NaN and does not throw an error).  Thank you for the report.

Link to post
Share on other sites

No problem. Thanks for following it up so quickly. Until a fix is done I could precheck specific fields that may error and set them to empty strings before passing into your conversion vi.

Link to post
Share on other sites

@drjdpowell Also if you go through and set the comment fields to empty strings, the next parsing error hit is this:
 

Quote

Error 1 occurred at JSONtext.lvlib:JSON text to Variant.vi:4410003

Possible reason(s):

Cannot Get as integer: null


Complete call chain:
     JSONtext.lvlib:JSON text to Variant.vi:4410003
     JSONtext.lvlib:JSON text to Variant.vi:4410002
     JSON Parsing Error.vi

This is replicated using the same VI shared above by first setting the comment keys to actual strings

Link to post
Share on other sites

That one is trickier.   What is null supposed to mean, interpreted as an integer?   Unlike for Floats, there is no Not-an-Integer value, nor is there an obvious "null" value like an empty string.   Should one just use zero as null?   Or should one consider the null to be that same as if the item was not present at all in the JSON, those using your default value in the supplied cluster.  Why is whatever generated this JSON providing null values at all?  Especially in place of strings and integers?

Link to post
Share on other sites
13 hours ago, camow7 said:

Until a fix is done I could precheck specific fields that may error and set them to empty strings before passing into your conversion vi.

A possibly better option is to replace your string/integer values with JSON strings, which is LabVIEW strings with <JSON> at the start of the name.  So "<JSON>Comment" rather than "Comment".   JSONtext will happily parse out the value into your cluster as JSON, then when you actually need your value you can convert it to string/integer, with appropriate handling if it is null or throws an error.

This method, by the way, is also how to handle JSON that has variable structure.

  • Like 1
Link to post
Share on other sites
2 hours ago, drjdpowell said:

Why is whatever generated this JSON providing null values at all?  Especially in place of strings and integers?

I imagine it is C , C++ or Javascript. NULL is a specific type of invalid pointer in C/C++ languages and "not an object" in Javascript (as opposed to "undefined"). In LabVIEW we don't really have either concepts.

In JSON, it is a valid type so it depends on how you want to translate it back into a LabVIEW type. Historically I have converted in the following manner: string->empty, numeric->0, boolean->false etc. Unless, of course, it is in quotes. In that case it is the string "NULL", in whatever case, as it's implicity typed.

Edited by ShaunR
Link to post
Share on other sites

Is there a plan to implement conditional filters in the JSON paths?  For instance the following should return a single book title, but I get a parsing error due to the question mark.

Or is this already implemented in a different manner?

--tested on jsonpath.com--

$.store.book[?(@.category=='reference')].title

 

{
    "store":
    {
        "book":
        [
            { "category": "reference",
              "author": "Nigel Rees",
              "title": "Sayings of the Century",
              "price": 8.95
            },
            { "category": "fiction",
              "author": "Evelyn Waugh",
              "title": "Sword of Honour",
              "price": 12.99
            },
            { "category": "fiction",
              "author": "Herman Melville",
              "title": "Moby Dick",
              "isbn": "0-553-21311-3",
              "price": 8.99
            },
            { "category": "fiction",
              "author": "J. R. R. Tolkien",
              "title": "The Lord of the Rings",
              "isbn": "0-395-19395-8",
              "price": 22.99
            }
        ],
        "bicycle":
        {
            "color": "red",
            "price": 19.95
        }
    }
}

 

Link to post
Share on other sites
On 1/31/2020 at 11:25 PM, drjdpowell said:

A possibly better option is to replace your string/integer values with JSON strings, which is LabVIEW strings with <JSON> at the start of the name.  So "<JSON>Comment" rather than "Comment".   JSONtext will happily parse out the value into your cluster as JSON, then when you actually need your value you can convert it to string/integer, with appropriate handling if it is null or throws an error.

This method, by the way, is also how to handle JSON that has variable structure.

Tested this and it worked well. Had no idea you could add the <JSON> tag to parse complex data types like dictionaries. Super handy. This looks likes it's solved my problems. Thanks!

Link to post
Share on other sites
On 2/3/2020 at 5:01 AM, camow7 said:

Any plans to include a "Set Null to Default" feature?

Unfortunately this contrasts with the current behaviour that null-->NaN for a floating-point number, rather than being the default number input.  In standard JSON, the float values NaN, Infinity and -Infinity have to become null, and to convert them back to a default value doesn't make sense.  We could add an option to "ignore null items" which would treat nulls as equivalent to that item not existing. 

  • Like 1
Link to post
Share on other sites
On 1/31/2020 at 5:59 PM, pjr1121 said:

Is there a plan to implement conditional filters in the JSON paths?  For instance the following should return a single book title, but I get a parsing error due to the question mark.

Or is this already implemented in a different manner?

--tested on jsonpath.com--

$.store.book[?(@.category=='reference')].title

Plan?  Yes, but not a priority.   Note, though, that JSONtext returns substrings, meaning you can implement filtering in LabVIEW without making data copies, so the following code implements your example:

1925139157_JSONtextfilteringwithLabVIEW.png.9c67cd15e4d89c0d78552941e184b98b.png

More effort, of course, but it should be fast, and using LabVIEW is very flexible, and more debug-able.  

Link to post
Share on other sites
5 hours ago, direheart said:

is it possible to make it compatible with Labview 2014?

No, unfortunately, as it uses VIMs that are only availablee in 2017+

Link to post
Share on other sites
  • 1 month later...

Help

Is there a way to parse this JSON string into this cluster in the following way:
image.png.4ca6fc2304370ea4d98288a6434e8ad6.png

- The order of elements inside the "Parameters" cluster does not matter. No error
- I can use enums
- The name of the elements matter. "TWOoo" shouldn't end up populating "two" (even if the order of elements matches the cluster). I want an error here instead of "two" being the default value
- Don't care if JSON text has additional elements that aren't in the cluster. No error

In short, features: Strict name checking, order independence, error if element missing

I've tried so many things but they all fail in one way or the other. See attached snippet

snip.png

  • Like 1
Link to post
Share on other sites

@ThomasGutzler here is a version that does most of what you describe.

  • The order of elements inside Parameters doesn't matter.
  • You can use enums.
  • It doesn't matter if the JSON string has additional elements (no error)

However, you don't get an error if an element is missing ("two" in your example). That would require something like a "strict" mode, which I don't think is supported by JSONtext.

526902039_ExtractParameters.png.f887cb8dbf8b55807ce18965b533cc42.png

Unfortunately code is removed from VI snippets on LAVA, so here is the VI. Extract Parameters.vi

Link to post
Share on other sites
4 minutes ago, LogMAN said:

However, you don't get an error if an element is missing ("two" in your example). That would require something like a "strict" mode, which I don't think is supported by JSONtext.

Actually, if you turn your cluster back into JSON you can compare it to the original JSON string to see if there are any differences. The string is empty if every element is defined in the source string.

Try this: Extract Parameters.vi

Notice the "Differences" string, which lists all differences (missing elements) in your source JSON string.

675891111_ExtractParametersV2.png.4a6efda39861851394cc5ff03f2aed93.png

  • Thanks 1
Link to post
Share on other sites

Standard JSON specification for JSON objects is that they are a collection of named elements with order unimportant, so any JSON library should ignore order and match on names.

Missing elements is a design choice, though, and I've gone with "use supplied default" rather than "throw error", for a number of reasons.  But as LogMAN shows, there are ways to do checking yourself.

Link to post
Share on other sites
On 2/4/2020 at 4:28 AM, drjdpowell said:

Unfortunately this contrasts with the current behaviour that null-->NaN for a floating-point number, rather than being the default number input.  In standard JSON, the float values NaN, Infinity and -Infinity have to become null, and to convert them back to a default value doesn't make sense.  We could add an option to "ignore null items" which would treat nulls as equivalent to that item not existing. 

The native LabVIEW Unflatten From JSON Function might be of use.

I used a Salesforce REST API that returned JSON. There were numerous string and numeric fields that returned as NULL.

If you use the native LV function and change 'default null elements?(F)' to true and set 'enable LabVIEW extensions? (T)' to false, you can get the default values assigned in your input type & defaults cluster.

https://zone.ni.com/reference/en-XX/help/371361R-01/glang/unflatten_from_json/

 

Link to post
Share on other sites
  • 3 weeks later...
On 7/26/2019 at 3:35 PM, andy_r said:

Just small recommendation to change the library.

Concerning timestamp to string conversion:

The current format string "%Y-%m-%dT%H:%M:%S%3uZ" in "Variant to Json Text.vi" is dependent on localization. On my german windows this leads a to timestamp according to Iso 8601 of "2019-07-26T14:17:20,053Z". Notice the comma for the fractional seconds. According to the Iso both, comma and point, is valid (Apparently, the Iso even recommends the comma https://en.wikipedia.org/wiki/ISO_8601#cite_note-25). However, it seems that the point is used more regularly in practice, for example in .net (see Json.net package). So I would suggest to change the format like this: "%Y-%m-%dT%H:%M:%S%.;%3uZ". This ensures using the point instead of comma., hence the format is independent of your local setting.

Sorry, I had missed this comment at the time.   That is a good idea.

https://bitbucket.org/drjdpowell/jsontext/issues/53/force-use-of-decimal-point-in-timestamps

Link to post
Share on other sites
  • 2 months later...

Bug? JSONText doesn't apply input default values to missing cluster items of array elements.

image.png.cfac8c824a0c9d7cb52ca3a937cbf166.pngimage.png.8ae1b813f1c246373cf99f02b3bc8356.png

I was hoping to see "Default", 1 and "2", NaN but instead get "",1 and "2",0

I think that it is due to the behavior of the Get Array Information VI.

The workaround is to convert the array elements individually as clusters:

image.png.30316820d66dc9e23dc0c3ae7f3760c8.pngimage.png.4022f34cc2d9604696e4e09bf7fe0789.png

Link to post
Share on other sites
Posted (edited)
4 hours ago, Porter said:

JSONText doesn't apply input default values to missing cluster items of array elements.

Thanks.  Issue 62: https://bitbucket.org/drjdpowell/jsontext/issues/62/jsontext-doesnt-apply-input-default-values

Although, I'm not sure this is doable, as your default values are the default values of your on-diagram array constant, not part of the data.  As that array is empty, I don't think I can access those default values inside a subVI.  If you had passed a non-empty array with a default value as first element, then I could read that and use it as default.

Edited by drjdpowell
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By drjdpowell
      Package for working with JSON.  Uses high-speed text parsing, rather than building an intermediate representation as with prior LabVIEW JSON libraries (this is much faster).  Allows easy working with "subitems" in JSON format, or one can convert to/from LabVIEW types.  Uses the new "malleable VIs" of LabVIEW 2017 to convert to any LabVIEW type directly.
      JSON text makes use of a form a JSON Path notation, allowing easy and rapid access to the subitems of interest.
      Requires LabVIEW 2017 and install by VIPM 2017 or later.
      Original conversation about JSONtext.
      On the LabVIEW Tools Network.
      JDP Science Tools group on NI.com.
      Copyright 2017 JDP Science Limited
    • By drjdpowell
      View File JSON LabVIEW
      JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. This is yet another one motivated by this hijacked conversation originally about a different project to convert JSON into LabVIEW Variants.

      This project uses a set of LVOOP classes to match the recursive structure of JSON, rather than variants. It allows conversation to and from JSON. All functionality is available through two polymorphic VIs: Set and Get. In addition to Get and Set VIs for common data types, one can also convert directly to or from complex clusters via variant-JSON tools.

      Copyright 2012-2016 James David Powell, Shaun Rumbell, Ton Plomp and James McNally.
      [Note: if you are using LabVIEW 2017, please also see the JSONtext library as a an alternative.]
      Submitter drjdpowell Submitted 10/04/2012 Category General LabVIEW Version  
    • By ThomasGutzler
      Hi,
      I'm receiving a JSON string from a web API, which I'm trying to convert into a cluster (of clusters) but I've run into some problems:
      1) Sometimes the order of the elements in the JSON string changes, which causes my conversion to fail.
      2) Sometimes the "object" returned via JSON is null, which causes my conversion to fail if I use clusters within clusters. It works with variants in clusters but then I need to convert the all the variants manually
      Is there any way to improve my code to fix those problems? The attached snippet is a simplified version. In my project, the first JSON string to data is done in a library. The second conversion from "result" variant to data is done in my application. This makes a direct conversion from JSON string more difficult.

    • By drjdpowell
      JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. This is yet another one motivated by this hijacked conversation originally about a different project to convert JSON into LabVIEW Variants.

      This project uses a set of LVOOP classes to match the recursive structure of JSON, rather than variants. It allows conversation to and from JSON. All functionality is available through two polymorphic VIs: Set and Get. In addition to Get and Set VIs for common data types, one can also convert directly to or from complex clusters via variant-JSON tools.

      Copyright 2012-2016 James David Powell, Shaun Rumbell, Ton Plomp and James McNally.
      [Note: if you are using LabVIEW 2017, please also see the JSONtext library as a an alternative.]
    • By Pavanb
      Hi There 
      I want to go from TestStand's output i.e. database to HTML5 SPC charts. Sorta like http://moderndata.plot.ly/using-plotly-from-labview-via-python/ Has anyone done this or used Bokeh or Enthough's data tool before?  What about these tools with TDMS or HDF5?
      Thanks. I appreciate it.
      Best,
      Pavan 
      By the way, https://www.enthought.com/services/consulting/case-studies/TAIPAN/ (like TestStand without TestStand)
       



×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.