Jump to content

Un-flatten from JSON to Array of cluster with contained variant loses data


Recommended Posts

Hi!

I have to convert a dynamically generated array into a JSON string and back. Unfortunately I found that the un-flatten method loses the variant data. See the screenshot of FP and BD and the comments inside.

JSON_Text_test.vi

 

grafik.png.f5b5e50177a88e8546ef88edefe1de62.png

 

Is this a bug in JSON Text or is my data-construction not supported as expected? In case of the letter I have modify huge parts of my code. So I hope that it is a bug 😉

 

The 2nd thing I recognized is that the name "Value" of the cluster is not used during flatten. Instead the name of the connected constant / control / line is used. I found the green VI ("Set Data Name__ogtk.vi") at OpenG Toolkit that allows me to programmatically set the variant data name. As you can imagine I would prefer not to need the OpenG VI.

 

Thanks in advance for your kind help 🙂

 

Edited by AndyS
Link to comment
27 minutes ago, AndyS said:

Is this a bug in JSON Text or is my data-construction not supported as expected?

27 minutes ago, AndyS said:

The 2nd thing I recognized is that the name "Value" of the cluster is not used during flatten. Instead the name of the connected constant / control / line is used.

Here is a similar post from the CR thread. The reason for this behavior are explained in the post after that.

JSONtext essentially transforms the data included in your variant, not the variant itself. So when you transform your array into JSON, the variant contains name and type information. But this information doesn't exist when you go the other way around (you could argue that the name exists, but there is no type information). The variant provided to the From JSON Text function contains essentially a nameless element of type Void. JSONtext has no way to transform the text into anything useful. To my knowledge there is no solution to this.

The only idea I have is to read the values as strings, convert them into their respective types and cast them to variant manually.

Edited by LogMAN
Link to comment

I would encourage you to try the "subJSON" described in the conversation LogMAN linked to.  Basically, replace your "Value" variant with a string labelled "<JSON>Value", and convert your values to JSON (at all the places where you now have "Set Data Name__ogtk.vi").  Then, wherever you are converting your variants to strict type, instead convert the subJSON to that type.

The problem with using Variants and JSONtext is that the JSON doesn't contain the specific type information needed to recover the data in the Variant.

 

Link to comment
On 12/8/2021 at 4:02 PM, LogMAN said:

JSONtext essentially transforms the data included in your variant, not the variant itself. So when you transform your array into JSON, the variant contains name and type information. But this information doesn't exist when you go the other way around (you could argue that the name exists, but there is no type information). The variant provided to the From JSON Text function contains essentially a nameless element of type Void. JSONtext has no way to transform the text into anything useful. To my knowledge there is no solution to this.

Ok, it took a while but yes, you're right. How can LabVIEW know the data type of a variant if the only you offer is the value information. It can't.

But: JKI JSON can ... I tested it shortly and somehow they are able to reconstruct the variant. I didn't had the time to went deeper into the code but I assume that they take a closer look to the data. E.g. everything that comes in " " must be a string. Everything without must be a number, enum, bool, ... every number with a . included must be a floating point. That's my assumption without seeing the code. I also assume that the input variant and the output variant might be not exactly the same if there are unusual data types used.
The problem on JKI JSON is the low speed in comparison to JSON text - so I would highly prefer to go on using JSON text. I'm not sure if there is a way to combine the best of both worlds but it would be really cool.

Edited by AndyS
Link to comment

I believe JKI JSON converts to Variants containing the basic JSON-matching types: String, Boolean, Number (DBL, I'm guessing).  Then it also provides, if I remember, a special VI to convert that Variant into your cluster of actual specific data types.

One of the reasons I don't like that is that it ties things to a monolithic LabVIEW type, the Cluster, that is not as flexible as JSON Object/Arrays.  You show the same "LabVIEW tunnel vision" in your design of an array of "Group", "Name", "Value" clusters.  That's a common way to handle the limits of LabVIEW.   

In JSON, the natural structure is this:

{
"Basic": 
   {
     "config file":"c:\\test\\config.ini",
     "Operator:"Muller"
   },
"Extended": 
   {
     "Room Temp":21.4,
     "Test No":0,
     "ping":null
   }
}

That isn't really doable with your LabVIEW Arrays and Clusters, even if I give you some extra Variant support.  But you have tools to do stuff like this in JSONtext.  

Link to comment
On 12/8/2021 at 5:33 PM, drjdpowell said:

I would encourage you to try the "subJSON" described in the conversation LogMAN linked to.  Basically, replace your "Value" variant with a string labelled "<JSON>Value", and convert your values to JSON (at all the places where you now have "Set Data Name__ogtk.vi").  Then, wherever you are converting your variants to strict type, instead convert the subJSON to that type.

The problem with using Variants and JSONtext is that the JSON doesn't contain the specific type information needed to recover the data in the Variant.

 

Ok, I implemented it like suggested and it's working fine. Nevertheless I'm semi-happy with this solution because I have now a dependency to JSON text in a toolkit where I havn't had it before.

One final question for my understanding (see the following picture). In opposite to the screenshot of my first post, I connect now the original array with all variant data information to "From JSON Text.vim". Nevertheless the value can still not be reconstructed. Why?

grafik.png.3e28e50be79da3904107b1fa6a22c633.png

Link to comment

You still have the problem of the embedded variant in your datatype. JSON is inherently string format with some indications to make a distinction between numbers and strings but nothing more. A variant is an AnyType.

What you try to do is: Convert this string (which from the JSON syntax only can be inferred to be a numeric, object, list or string, into Anything! Anything what? The JSON library could elect to convert strings into a string, numerics into a double and anything else into nothing, but that will break down as soon as you want to use that variant as a specific type as you pushed it into the incoming variant in the begin. The JSON string simply lacks every information about the actually wanted type in the variant and therefore you can not convert it into a variant without loosing significant compatibility and functionality. This is a fundamental problem that can not be solved with throwing more code at it, but only by rethinking the way you want to save your configuration data.

Edited by Rolf Kalbermatter
Link to comment
46 minutes ago, Rolf Kalbermatter said:

This is a fundamental problem that can not be solved with throwing more code at it, but only by rethinking the way you want to save your configuration data

The way I usually get around this is either by not using variants (preferred) or saving as a string and specifying the type as another entry. e.g.

{

"config_file:"c:\temp\config.ini",

"config_file~type":"str",

"value":1.0E6,

"Value~type":"ext"

}

This can be transparent to the user but not applicable if you have a strict field list for a file. Thank [insert deity] we can switch on strings in a case structure, eh?

Link to comment
On 12/15/2021 at 7:48 AM, AndyS said:

Nevertheless the value can still not be reconstructed. Why?

The Type input just sees "array of clusters".  Normally one just inputs an empty array; there is no code to look at individual elements of the array.  Theoretically, this could be done, but that seems a very low-value use case so chances of me doing that are zero.

Actually, it was a mistake of mine to implement the first part of your code; doing it again I would convert Variants in Clusters to 'null' and throw a "Variants not supported" error.  This would keep my options open to introduce, without breaking changes, a better Variant support that includes encoding teh data type (along the lines of what Shaun describes).  There is a number of possibilities, for example:

{
  "config file:["c:\temp\config.ini",{"LVtype":"Path"}],
  "Room Temp":["21.4",{"LVtype":"DBL"}]
}

I am held back from that, partly because it would now be a breaking change, and partly because I have yet seen an example of someone using Variants and JSON that would not be far better implemented in just JSON.  

Link to comment
11 hours ago, drjdpowell said:
{
  "config file:["c:\temp\config.ini",{"LVtype":"Path"}],
  "Room Temp":["21.4",{"LVtype":"DBL"}]
}

That is far more difficult to implement.

consider this-an array in a variant and a normal array:

{
"temps":[21.4, 23.2, 33.6],
"press":[1006];
}

LabVIEW arrays are single types so this is straight forward.

{
"temps":[21.4, 23.2, 33.6],
"temps~lvtype":"dbl",
"press":[1006]
}

You don't need to pars the array string ("temps") any more than you do now.

Including the type in the array, you now have to inspect the elements of the array (at least one, if the format is the first or last element) parse it then delete it to return the array. If there are only two elements in the array, you have to see if there is a lvtype to decide if it is really an array or it's a scaler with a type. 

Edited by ShaunR
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By Guru Prasath
      Hello Everyone;
      How I plot the timestamp in the graph of the x-axis and numerical slider the timestamp given in 2d array.

      The array in 2d array of string...

      I will try this but it not plotting full timestamp can you help me to short of this problem.

      It will be a plot, but a few timestamps will plotting 
      How I plot full timestamp and date...
       
      I have attached my problems like the jpg file and .vi file for your reference.

      Does anyone have solution comments below...
      5_6285044541540009090.vi
      Thanks and Best regards...


    • By torekp
      I was trying a python http communication tutorial - https://aiohttp-demos.readthedocs.io/en/latest/tutorial.html#views - when I had to disable the NI Application Web Server to proceed.  And then I thought, what the **** am I doing?  Maybe I should take the free (well, prepaid) gift of a working web server.
      Here's my task.  A central HQ computer will have a GUI that monitors five machine stations, each of which has its own computer.  Every approx 10 ms (negotiable), each station gives a report consisting of two arrays, the larger being 2048 data points, the other much smaller.  Whenever HQ feels like it, HQ can tell a station to start or stop (its computer stays on).  A local IP connection is used, with a router at each end.  There is also a Raspberry Pi with its own IP address at each station's router, that can send camera frames to HQ.  The station-computers use Python and C++ to do their work, not counting whatever needs to be added to communicate with HQ.
      Your advice please?  Should I use Labview?  On both ends or just the HQ?  And which if any of these helpful add-ons suggested by Hooovahh should I use?
       
    • By kartik.azista
      HAs anyone tried creating a sub vi programmatically by selecting the set of blocks through scripting?????
    • By TDF
      TDF team is proud to propose for free download the scikit-learn library adapted for LabVIEW in open source.
      LabVIEW developer can now use our library for free as simple and efficient tools for predictive data analysis, accessible to everybody, and reusable in various contexts.
      It features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy from the famous scikit-learn Python library. 
       
      Coming soon, our team is working on the « HAIBAL Project », deep learning library written in native LabVIEW, full compatible CUDA and NI FPGA.
      But why deprive ourselves of the power of ALL the FPGA boards ? No reason, that's why we are working on our own compilator to make HAIBAL full compatible with all Xilinx and Intel Altera FPGA boards.
      HAIBAL will propose more than 100 different layers, 22 initialisators, 15 activation type, 7 optimizors, 17 looses.
       
      As we like AI Facebook and Google products, we will of course make HAIBAL natively full compatible with PyTorch and Keras.
       
      Sources are available now on our GitHub for free : https://www.technologies-france.com/?page_id=487
    • By mhsjx
      Hi,
      I'm a beginner in labview, and now test cRIO about two weeks. I still can not solve the problem. I attach my test project for explanation.
      I want to realize that , for example, with time sequence t1, t2, t3, t4,  DO outputs T, F, T, F, AO1 outputs A1, A2, A3, A4, AO2 outputs B1, B2, B3, B4, and the delay of AO1 and AO2 should as small as possible(AO1 and AO2 may comes from difference modules).
      I search in Google, NI forum, and decide to use for loop and loop timer in FPGA.
      The reason as follow:
      1. To realize the specific time interval, I can use Wait and Loop timer. But in "FPGA 0--Test DO.vi", it can't not realize specific time interval by several us's error(maybe large). And to complete once of while loop, it needs 134us. I can't explain that it can realize time interval below 134us, even I acturally realize a delay of 10us, but the input is not acturally 10us, so it's not accurate. 
      And by NI example, I use the Loop timer.
      2. In "FPGA 1--Test DO and AO.vi", I find that the loop timer helps me to realize accurate time interval, however, it ignore the first time interval. Such as, t1, t2, t3, t4, with disired output A1, A2, A3, A4. It goes A1(t2), A2(t3), A3(t4), A4(t1). And in "FPGA 2--Test DO and AO.vi", it has same problem. DO0 and AO1 goes A1(t2), A2(t3), A3(t4), A4(t1). And AO0 is always ahead of DO of t1. 
       
      The people of NI forum advice that I should put AO0 and AO1 into one FPGA/IO node and use SCTL. But up to now, I don't find any example about it(Google or NI forum, maybe it's primary).  Mainly that AO0 and AO1 must go with different timeline, the dimension of input array is different.  Can anyone offer advice for me?
      Thanks
      Test.7z
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.