Jump to content

[CR] JSON LabVIEW


Recommended Posts

  • Replies 240
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

The easiest way by far is to use the [un]Flatten to XML primitives, which can take objects and variants as arguments, and convert the XML to/from JSON. with simple regex/search/replace. Why they didn'

View File JSON LabVIEW JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. T

I tried to use this tool. And it looks great, however the function 'Variant to JSON' fails with a variant inside an object. I made the following changes: -Added support for Variant. By using 'Unwrap

Posted Images

Shaun,

I am looking at the VIPM package that Ton and I have put together. Shall we divide into two packages, one not dependent on OpenG and one that contains all the Variant stuff? So people who can’t use OpenG can still use the core functions or your extended API? I can do that just be replacing one OpenG function, Trim Whitespace, in the core (can we use your “Fast Trim” instead).

Ton,

You currently have the package installing under LAVA; don’t we need a LAVA tools approved package before doing that?

Link to post

Shaun,

I am looking at the VIPM package that Ton and I have put together. Shall we divide into two packages, one not dependent on OpenG and one that contains all the Variant stuff? So people who can’t use OpenG can still use the core functions or your extended API? I can do that just be replacing one OpenG function, Trim Whitespace, in the core (can we use your “Fast Trim” instead).

Ton,

You currently have the package installing under LAVA; don’t we need a LAVA tools approved package before doing that?

If you can wait until next weekend, I'll replace all the openG stuff so it's a self contained lib. Just a bit busy at the moment finalising the websocket demo, but that will be finished this week.

Link to post
If you can wait until next weekend, I'll replace all the openG stuff so it's a self contained lib. Just a bit busy at the moment finalising the websocket demo, but that will be finished this week.

OK, but to me it seems silly for anyone to avoid OpenG yet use this (far, far less tested through experience) third-party open-source software. I’d rather offer the whole package to OpenG.

Now, do it with higher performance and you’ll get me interested. :)

Link to post

OK, but to me it seems silly for anyone to avoid OpenG yet use this (far, far less tested through experience) third-party open-source software. I’d rather offer the whole package to OpenG.

I'm not opposed to that either but I thought Ton was talking about it going into the Lavag Tools Network..

Now, do it with higher performance and you’ll get me interested. :)

Yeah. Well its got LVPOOP in it so that's not an option. :P

Link to post
Yeah. Well its got LVPOOP in it so that's not an option. :P

Theoretically, the not-on-the-pallet VIs in vi.lib\Utility\VariantDataType should blow the OpenG stuff out of the water, as OpenG has to flatten the data to access it (expensive), but my only experience with the VariantDataType Vis is that they are glacially slow.

Link to post

Theoretically, the not-on-the-pallet VIs in vi.lib\Utility\VariantDataType should blow the OpenG stuff out of the water, as OpenG has to flatten the data to access it (expensive), but my only experience with the VariantDataType Vis is that they are glacially slow.

In theory. Practice and theory are the same. In practice, they are not :P

Historically the openG stuff is generally more performant since a lot of them perform the same functions as are available in LV, just optimised. Of course with the NI grown ones they can have C functions in the exe to do the heavy lifting, but usually they use nodes that run the the UI thread which mitigates any performance gained

Saying that. I use the vi.lib getTypeInfo,GetCluster/numeric etc but I haven't bench-marked them since they are "Hobsons Choice" for me.

Link to post
Saying that. I use the vi.lib getTypeInfo,GetCluster/numeric etc but I haven't bench-marked them since they are "Hobsons Choice" for me.

I’m pretty sure you’ve got some non-UI thread substitutes lying around. I think you posted one once. :shifty:

Link to post
  • 1 month later...

I have a 2D array in a cluster in a 1D array in a cluster. after flattening, in the JSON string, the 2d array looks like 1d. if I try to unflatten this string to my cluster, the variant to data primitive throws me an incompatible datatype error. it seems, 2d arrays are generally unsupported?

Link to post

I had not considered multi-dimentional arrays. JSON doesn’t have a multi-dimentional array type, but we could just have arrays of arrays. I will look into supporting it. Thanks.

N dim arrays don't have a "type" but they are represented

e.g. Array:[[1,2,3,4,5],[1,2,3,4,5]]

This causes us a problem in the way we convert to type from a variant (recursion works against us due to the fact that it is not a recursive function. it is iterative). If we were just dealing with strings, then it would be fairly straight forward.

For example. Using the "Set From Variant" in a for loop works fine for 2D arrays. But for 3D arrays it will give incorrect results (try with the "Example Creation of JSON string.vi"-bug).

One way forward is to detect the number of dims and have a different "Set JSON Array.vi" for each (max 4 dims?). But this is ugly (POOP to the rescue?).

Link to post

It looks like it can be done with a little OpenG gymnastics (though not trivial). But this is making more work for you if you want to avoid an OpenG dependancy. How do you want to proceed with that?

Well. Decoding an N dim array is not that hard (a 1d array of values with offsets). But I'm not sure how you envisage representing it internally .

Edited by ShaunR
Link to post
Well. Decoding an N dim array is not that hard (a 1d array of values with offsets). But I'm not sure how you envisage representing it internally .

If you can give me a 1d array, and the dimensions to calculate the offsets, it will be easy to recursively convert to nested JSON arrays inside of JSON arrays (and it will work for any number of dimensions).

— James

BTW> One other issue that occurs to me is “ragged” arrays, arrays of other arrays of varying size. In LabVIEW, I would make a ragged array from an array of clusters containing arrays; in JSON, it would be just an array of arrays of varying length. I would probably try and add support for conversion between these types, too, if we’re going to support multi-D arrays.

Link to post

If you can give me a 1d array, and the dimensions to calculate the offsets, it will be easy to recursively convert to nested JSON arrays inside of JSON arrays (and it will work for any number of dimensions).

Done.

BTW> One other issue that occurs to me is “ragged” arrays, arrays of other arrays of varying size. In LabVIEW, I would make a ragged array from an array of clusters containing arrays; in JSON, it would be just an array of arrays of varying length. I would probably try and add support for conversion between these types, too, if we’re going to support multi-D arrays.

Agreed.. All LV arrays are "square" arrays, however this is inefficient for large data-sets. We could actually handle non-square arrays by using arrays of DVRs internally (again, the premise being that we would only convert to "square" when necessary on extraction).......just a thought!

Link to post
Done. Agreed.. All LV arrays are "square" arrays, however this is inefficient for large data-sets. We could actually handle non-square arrays by using arrays of DVRs internally (again, the premise being that we would only convert to "square" when necessary on extraction).......just a thought!

Square arrays? Equal dimension size? You must mean rectangular arrays or?

  • Like 1
Link to post

Square arrays? Equal dimension size? You must mean rectangular arrays or?

Quite right. Poor terminology on my part.

Done. Agreed.. All LV arrays are "rectangular" arrays, however this is inefficient for large data-sets. We could actually handle non-rectangular arrays by using arrays of DVRs internally (again, the premise being that we would only convert to "rectangular" when necessary on extraction).......just a thought!

Link to post
  • 2 weeks later...
I have a 2D array in a cluster in a 1D array in a cluster. after flattening, in the JSON string, the 2d array looks like 1d. if I try to unflatten this string to my cluster, the variant to data primitive throws me an incompatible datatype error. it seems, 2d arrays are generally unsupported?

I am having a problem uploading a new version to the Code Repository, but you can download it from bitbucket.  It is extended to work with multi-D (rectangular) arrays.

Link to post
  • 1 month later...
  • 2 months later...

Hey Guys,

 

Had an issue with JSON to Variant where types mismatch because of no enum support so variant to data threw an error.


Though it easiest to keep seperate so there is a patch attached. I see it already encodes the enums as string anway so I used the OpenG Variant tools to use this and set the variant value as a string.


EDIT: Won't let me upload a patch, shall I push it into the bitbucket repo or fork first?

 

Cheers,

James

Link to post
Had an issue with JSON to Variant where types mismatch because of no enum support so variant to data threw an error.

It’s supposed to work with enums, and my test case seems to work.  "Variant to JSON" delegates enums to OpenG’s “Scan from String” which calls OpenG “Set Enum String Value”, so it should work.  What error did you get and what enum string did it have?

Link to post
  • 2 months later...

Hi,


Thought I had already posted back, sorry!

 

The root cause is that the OpenG Scan From String puts the enum string through a scan from string with %s which removes anything after a space so this bug specifically effects enum strings with spaces.

 

A customer of mine has also had issues with strings in clusters. In JSON Scalar.lvclass:Variant Decode.vi it assumes all array elements are the same size. In the case of an array of strings or an array of clusters with strings this is not the case. I guess this comes back to your discussions about ragged 2d arrays as well. You can see the issue in the attached VI. He has managed to write a fix for this that we can share as well in whatever format is easiest.

Link to post

Hi James,

 

Could you fork the repo at bitbucket, add the fix to your fork, and request a Pull request. It woul be great if it can be supplied in LV2009 format.

 

Regards,

 

Ton

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By drjdpowell
      View File JSONtext
      Package for working with JSON.  Uses high-speed text parsing, rather than building an intermediate representation as with prior LabVIEW JSON libraries (this is much faster).  Allows easy working with "subitems" in JSON format, or one can convert to/from LabVIEW types.  Uses the new "malleable VIs" of LabVIEW 2017 to convert to any LabVIEW type directly.
      JSON text makes use of a form a JSON Path notation, allowing easy and rapid access to the subitems of interest.
      Requires LabVIEW 2017 and install by VIPM 2017 or later.
      Original conversation about JSONtext.
      On the LabVIEW Tools Network.
      JDP Science Tools group on NI.com.
      Copyright 2017 JDP Science Limited
      Submitter drjdpowell Submitted 08/02/2017 Category General LabVIEW Version 2017 License Type BSD (Most common)  
    • By drjdpowell
      Package for working with JSON.  Uses high-speed text parsing, rather than building an intermediate representation as with prior LabVIEW JSON libraries (this is much faster).  Allows easy working with "subitems" in JSON format, or one can convert to/from LabVIEW types.  Uses the new "malleable VIs" of LabVIEW 2017 to convert to any LabVIEW type directly.
      JSON text makes use of a form a JSON Path notation, allowing easy and rapid access to the subitems of interest.
      Requires LabVIEW 2017 and install by VIPM 2017 or later.
      Original conversation about JSONtext.
      On the LabVIEW Tools Network.
      JDP Science Tools group on NI.com.
      Copyright 2017 JDP Science Limited
    • By ThomasGutzler
      Hi,
      I'm receiving a JSON string from a web API, which I'm trying to convert into a cluster (of clusters) but I've run into some problems:
      1) Sometimes the order of the elements in the JSON string changes, which causes my conversion to fail.
      2) Sometimes the "object" returned via JSON is null, which causes my conversion to fail if I use clusters within clusters. It works with variants in clusters but then I need to convert the all the variants manually
      Is there any way to improve my code to fix those problems? The attached snippet is a simplified version. In my project, the first JSON string to data is done in a library. The second conversion from "result" variant to data is done in my application. This makes a direct conversion from JSON string more difficult.

    • By drjdpowell
      JSON is a data interchange format (sometimes compared to XML, but simpler). There are multiple projects to create a JSON package for LabVIEW. This is yet another one motivated by this hijacked conversation originally about a different project to convert JSON into LabVIEW Variants.

      This project uses a set of LVOOP classes to match the recursive structure of JSON, rather than variants. It allows conversation to and from JSON. All functionality is available through two polymorphic VIs: Set and Get. In addition to Get and Set VIs for common data types, one can also convert directly to or from complex clusters via variant-JSON tools.

      Copyright 2012-2016 James David Powell, Shaun Rumbell, Ton Plomp and James McNally.
      [Note: if you are using LabVIEW 2017, please also see the JSONtext library as a an alternative.]
    • By Pavanb
      Hi There 
      I want to go from TestStand's output i.e. database to HTML5 SPC charts. Sorta like http://moderndata.plot.ly/using-plotly-from-labview-via-python/ Has anyone done this or used Bokeh or Enthough's data tool before?  What about these tools with TDMS or HDF5?
      Thanks. I appreciate it.
      Best,
      Pavan 
      By the way, https://www.enthought.com/services/consulting/case-studies/TAIPAN/ (like TestStand without TestStand)
       



×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.