drjdpowell

I note that you haven't given any performance numbers.  If I were looking at slow code with four steps, I would have learned how long each step roughly took within a few minutes**.  It is not uncommon in such cases for one step to be the bottleneck, taking significantly longer than all the other cases put together.  In that case, it does not help to try and do steps in parallel, and you should instead try and improve the performance of that one step.

I will withhold any other suggestions, as I think you are possibly just digging a hole for yourself by layering on "performance improvements" without measuring performance.

**Using the "Quick Timer" from "Cyclic Table Probes".

It's the inconsistent state that is the problem, where the by-value bit is inconsistent after the by-ref bit is changed in parallel.   That doesn't always happen; sometimes either the value or the reference bit never changes.   But one needs to be aware of this issue.

49 minutes ago, Francois Normandin said:

It is much slower than your direct cast, but in the cases where you absolutely want to keep the element names, it might be worth the hit. 

Thanks.  I have often considered providing a flattening-based implementation for arbitrarily large clusters, but have always paused because of the big step change in performance.

On 11/20/2020 at 8:47 AM, drjdpowell said:

I will just do a simple Messenger-Library-only service discover implementation.

In the LAVA-CR is a 1.14.5 version that includes a service-discovery implementation using UDP messages.  See the example "Example Reconnecting TCP Client with UDP Service Discovery.vi".

Sets and Maps are 2019, so I'm not sure they can be supported till JSONtext is in 2019.

2 hours ago, TiborK said:

Hi, is there a chance to release build version for LabVIEW 2015?
In VIPM is allowed just >=2017.

I'm afraid it uses Malleable VIs that were introduced in 2017, meaning it can't, as a whole at least, be back ported.

1 hour ago, Jordan Kuehn said:

I have a request from non-LV programmers who are using the JSON output I am providing them to provide the schema so that they can ensure they are accurately parsing the data structure. I have used a tool similar to this: https://www.liquid-technologies.com/online-json-to-schema-converter to generate it, but it is imprecise. I’d prefer to just give them the typedef file, but they have no idea what to do with that. 

Ah, you mean this then: http://json-schema.org/understanding-json-schema/, and you want a function that converts a LabVIEW Cluster/Array to a basic JSON Scheme.  

That does not exist in JSONtext but could at some point.  

6 hours ago, Jordan Kuehn said:

Also, is there a way to generate a JSON schema based on a LabVIEW data structure?

Not sure I understand the question.  Can you rephrase it?

6 hours ago, Jordan Kuehn said:

Is there a way to limit the number of decimal places stored in a double numeric?

No.  Only by custom formatting the numbers into JSON yourself.  I often use single numerics, as 7 decimal places is more reasonable looking.  

Googling suggests PQ should be threadsafe: https://www.postgresql.org/docs/9.1/libpq-threading.html

1 hour ago, Antoine Chalons said:

I'm confused... what is the connection between SQLite and PostgreSQL?

My mistake.  There is an SQLite on Linux RT conversation also going on right now and I mixed the two up.

On 1/11/2021 at 2:18 PM, JKSH said:

I'm guessing that the library's functions are not thread-safe. Without forcing the CLFN to use the UI thread, it could use different threads to call the library functions... thus causing a crash.

SQLite is thread safe unless it is changed from the default "Serialized" mode: https://www.sqlite.org/threadsafe.html

It is difficult for me to give good advice without really understanding the project.  Some random thoughts:

Where are the performance bottlenecks?  The first thing I would do is prototype and benchmark the steps and get a feel for what are time-limiting, high-CPU steps.   Be wary of designing a complex custom communication pattern "for performance" before you understand what and where the performance issues are.  How long does it take to convert your IMAQ image into a DBL array, for example?  You might find that effort on a custom queue system is better spent on some other area.

Remember that you IMAQdx task is already an asynchronous actor-like thing, with a buffer of images.  I often see people placing importance on the loop "taking data" piping that data to an async analysis loop, when really the IMAQdx or DAQmx task is really the one taking data.

Also note the design pattern used by IMAQdx Events, where the "Frame Done" event doesn't contain the Image itself, but tells one that an image can be read from a separate queue (ring buffer).  If you find you need a custom queue for data, you could consider using this pattern.  I'd also try and experiment to see if I could have more than one loop pull different Frames off the IMAQ ring buffer at the same time, as this gets parallelism working as early as possible.

If you are working with reference data, either IMAQ images or DVRs, then 75 messages per second is not high, and I personally would just use the standard Messenger Library Actor Event-Notification Messages for passing references to the data, at least at first.  Design for simplicity and add complexity only later if your performance benchmarks show value in it.

On 1/19/2021 at 9:13 PM, bjustice said:

I now understand why James made this SubVI for his JSONtext library.
I hope that he built this thing with scripting.  Brute force, but seems to work well.

Brute force.  I suck at scripting.  I add 20-30 more cases every so often.  Maximum possible with this method is 255.  

This VI is part of the "JDP Science Common Utilities" vipm package, BTW, so one doesn't need to install JSONtext to use it.   Originated in the conversations on this idea: Convert an Array of Variants into a Cluster.  I wish NI would just implement that idea so this stopgap would be unnecessary, so please kudo that idea.  

I used to use OpenG Variant Tools a lot, but replaced them with the newer NI functions once they became faster (about LabVIEW 2012, I recall) and I had this workaround.   The only missing functionality from OpenG is the ability to set the name of data in a cluster (the above SubVI create a Cluster with generic element names; I decided to just live without that.

On 1/19/2021 at 9:43 AM, Conner P. said:

for really high data rates

What kind of data and data rate?

My own UIs, when I polish them, tend towards mostly white.  I could give NI some suggestions on how to improve that UI, but I wouldn't change the basic whiteness.

Although my UIs are graph heavy, which I find look better as all white.  In a UI whiteout graphs the reasoning is less strong.

Theoretically, there are no roadblocks to developing it to work on RT.  See the SQLite Library package, which has conditional disables to point to the .so on Linux RT, as an example.

I note that even though I created a framework, Messenger Library, I did not re-architect any existing project using my framework, but instead just used it to add new features to those projects.  Then I used Messenger Library for all new projects, armed with actual experience.  Trying to rework the entire architecture of a large project sounds very expensive and risky.  

You could also use a DVR, or similar mechanism, to manage access to a com port, instead of using a non-reentrant subVI.  

Try just taking out the generated column.  Those were added recently, in SQLite 3.31.0.  If the NI Linux package for SQLite is an earlier version than that then it will throw an error at the Create Table statement.

What error was given?   The error should have given useful debug info.

18 hours ago, Matt_AM said:

My biggest issue I can see is having child classes return different data types without it getting overly complicated (any suggestions on designs on this front are more than welcome).

You need see past your specific motor types to think what your generic motor needs to do.  Is it returning configuration info?  That can be JSON, rather than a typed cluster.  Or a Variant, or an object of a "config" subclass.

My advice is to NOT learn OOP with a Motor Parent class unless you have enough experience with different motor types so that you can form a solid view of what a generic motor does.  

On 11/17/2020 at 12:34 PM, ShaunR said:

The major use case for UDP I would have is for COAPS

I'll have a read, but that looks a bit more complex than I have the time for.  I will just do a simple Messenger-Library-only service discover implementation.

On 11/17/2020 at 12:54 PM, Thoric said:

Also, our network admins have strict policies which could interfere with UDP. Although it's allowed, it can't cross certain hardware boundaries between networks. So although we can get TCP Messenger to cross networks, UDP would likely fail.

Another possible service-discovery design is a central broker; an actor with TCP Server at a known IP/Port that all services connect to to register their location, and to which clients connect to lookup services.

A beta version is now posted: https://forums.ni.com/t5/JDP-Science-Tools/UDP-Messengers/td-p/4099792

This is of the UDP comms itself; haven't done a service-discovery implementation yet.  

3 hours ago, RedBeard said:

i found an issue converting a array of clusters from text to data. 

https://bitbucket.org/drjdpowell/jsontext/issues/68/report-issue-converting-a-array-of