drjdpowell

Speculative, to establish that the work is legitimate R&D of my company.  I want to be able to show a tax inspector a company-branded product on the LabVIEW Tools Network.

BTW, I’ve found I can separate the new stuff into a separate, non-conflicting project, so LAVA JSON can be left as is (I’ll bring out a new version with some modest performance improvements).

1 hour ago, ShaunR said:

What is the criteria for the Tax exemption that requires this move? What makes this toolkit a good choice over, say, you messaging library which has little or no collaborative contributions  so far?

I’m recording my work on Messenger Library and all other reuse libraries, and hope to claim at least some of them also.   JSON just happens to be what I am working on right now.  

JSON LabVIEW is BSD licensed.  It’s not “Copy Left”, so one doesn’t have to open-source any derivative work.  

BTW, what I’m working on is a major change, that gets rid of the intermediate representation as LVOOP objects entirely.   The only representation is text.  So I may be able make it an independent package that can be used in parallel with the LAVA JSON library.

Because I forget to submit to NI.   Updating LAVA-CR is quick and instant, and I intend to send to NI later when any bugs are discovered, but then it drops of my radar.  

1 hour ago, ShaunR said:

I've no idea what you are talking about (R&D Tax credits?)

https://www.gov.uk/guidance/corporation-tax-research-and-development-tax-relief-for-small-and-medium-sized-enterprises

1 hour ago, ShaunR said:

I would prefer you create a new product derived from the current.to mark a distinct change rather than an attempt a seamless absorption. In time, people will probably move to your new product in preference  but I think a distinction should be made and the current project control handed over to LavaG admins.

I was thinking of that as an option.   The LAVA community one would be version 1.4.1.  I’m afraid I’ve checked in some things starting with a switch to 2013; you should ignore those and branch from the last 2011 checkin.   I’ll set up a new repo and a new project name.  

I’m doing a lot of work at the moment on a “JSON 2.0” version, particularly intended to improve performance in common use cases that I’ve encountered. For example, I have a new VI that flattens a Variant directly to JSON text (skipping the Objects entirely) that works on large arrays of numerics about 30 times faster.   

I’d like to claim R&D tax credits on the work I do on this (which is otherwise unpaid), and to do that the result of the work needs to be owned by my limited company, so I’d like to release the new version under “JDP Science Limited”, rather than “LAVA”.   I would put a BSD license on the new version, and properly comply with the BSD license of the current LAVA version.  I hope none of the contributors of "JSON LabVIEW” object?   The R&D credit would work out to about 30% of charged rate (if I do the tax math right, which is far from obvious).  30% is a lot better than zero.

11 hours ago, MarkCG said:

A very good way to take an N-order derivative of a noisy signal is the Savitski-Golay filter, which is in LabVIEW. Read up on it, you will see it is much better than the naive method of taking the derivative (the simple forward, backward, and central approximations)

I can also recommend Savitski-Golay, which is basically just fitting to a Nth-order polynomial, and estimates multiple derivatives at once.  Apply some “straight-line” criteria such as all derivatives beyond the first being “small”.

You have to outsmart the compiler when performance testing.  In your code your constant array is copied at the to-variant step because it is needed at the next iteration.  Make a new array for each iteration, like this:

tests (Modified to avoid Constant Folding).vi

Yes, I try not to shoot myself in the foot.

1 hour ago, ShaunR said:

There are no rules (except self imposed ones) that say the data has to be an integral part of a message. 

Actually, that is an Actor Model rule ("messages should be immutable").  It’s a rule I sometimes break, but not without pause for thought.  Where I have broken the rule I have sometimes had race-condition bugs.  The value of rules is not in blindly obeying them, but in understanding why one should be reluctant to break them.

17 hours ago, pawhan11 said:

I think sometimes globals or DVRs will be more suitable  than messages. For example when we have a large buffer data points that some process is acquiring and stores in memory, others will use that data. By using globals/dvrs it is just basically set and get. Using messages involves flattening data to variant/string in order pass that data by message implementation. With large data this delay might be significant.

Variants don’t “flatten”.  Putting something in a variant doesn’t involve altering or copying the data.  They have overhead but I don’t think the size of the data matters.

Never use Globals for big data.  Globals always copy when you read them.  So does any “Get”.   Avoid a copy by extracting only the required data inside the structure you are using.  So, inside the IPE with a DVR, inside the Action Engine, or inside the message sending code.  

BTW on Action Engines:  are they really that simple to pick up for a novice?  “Sending a message” seems to me to be a much easier concept than “non-reentrant subVIs with uninitialized shift registers”.

9 hours ago, JKSH said:

I still use AEs for very simple event logging. My AE consists of 2 actions:

1. "Init" where I wire in the log directory and an optional refnum for a string indicator (which acts as the "session log" output console)
2. "Log" (the default action) where I wire in the message to log.

Then, all I have to do is plop down the VI and wire in the log message from anywhere in my application. The AE takes care of timestamping, log file rotation, and scolling the string indicator (if specified)

Having more than 2 actions or 3 inputs makes AE not that nice to use -- I find myself having to pause and remember which inputs/outputs are used with which actions. I've seen a scripting tool on LAVA that generates named VIs to wrap the AE, but that means it's no longer a simple AE.

Make it that simple and perhaps I do use Action Engines.  Turn the enum into an “(Re)Initialize” boolean, or have the code initialize itself on first call, and you’ll find such things in my code.  But none of them will be usable to communicate between code modules like AEs are often used.  Logging is a good example of where multiple things can share something without affecting each other.  

On September 10, 2016 at 8:35 AM, ShaunR said:

Well. there is no "Total bytes downloaded" emitted by the downloaders ...

I must not be understanding the problem you’re presenting.   Perhaps you could explain a AE/DVR solution, and illustrate how it cannot be done by messages.  

On September 10, 2016 at 8:35 AM, ShaunR said:

I'm not sure why Dispatcher is thrown in here. That isn't a messaging framework or a memory accessor or really anything close to what we are discussing. Dispatcher is some turbocharged TCPIP primitives for a specific purpose (pub/sub). It has more in common with network streams than a messaging framework. It is meant to be a component in your system not the system itself. I suppose when an actor is just code that does something and messaging is just information flow then even a simple API becomes a "Messaging Framework". I don't subscribe to the philosophy of tortured vocabulary, though.

If you are looking for my "Messaging Framework" then you need to look at the VIM Demo where it is one VI @ ~700KB. The merits of each messaging approach is irrelevant in his discussion though. We should instead concentrate on the OPs question.

You mention Daklu, who only has LapDog as a reuse library.  LapDog is less a messaging framework that your Dispatcher.   Dispatcher is an implementation of a “Message Broker” messaging pattern, I believe, so one is making more of a design choice by using it than one would be by using LapDog.  My “Messenger Library” is meant as a flexible message passing library, where you don’t need to use the “actor” stuff at all if you’d rather not (most of the examples installed with Messenger Library are simple message passing).  Its central messaging pattern is “Request-Reply”.  The Actor Framework, on the other hand, is very much intended to be a framework to enforce a certain style of program architecture.  One should (I hope) find Messenger Library very useful even if one doesn’t follow similar design principles to me, but it is (deliberately) hard to use the AF in a way not intended by AQ. 

Perhaps, though, you actually meant “principles” rather than “framework”, as Daklu has written a lot about the “actor-oriented” design principles he follows.

1 hour ago, ShaunR said:

This is solvable with some sort of protected global storage (a DB, an AE, a DVR, a Go Pro) but not by pure messaging alone.

Your kidding, right?  Just register on “Total bytes downloaded” and look at the increase (which stops when they finish).  Trivial.

Edit>> or register a Queue for all the “Throughput” messages and average once every 10 secs.   Also easy.  And are you really worried about sub-millisecond timing uncertainties on a 10-second average?  If so, I think you have those with an AE also, just due to OS jitter.

1 hour ago, ShaunR said:

But it is not learning "messaging". It is learning someone's particular flavour of framework Your's, AQs, Michaels, Daklus.

Don’t forget your “Dispatcher".

23 minutes ago, ShaunR said:

An action engine to do the averaging over N channels is about 2 minutes to produce one VI (~2k), that can be explained to a novice in 30 seconds regardless of the architecture you use. 

A class with a DVR is about 5 minutes, 3 or 4 Vis (about 30K) and you have to promise there will be only one (but that's probably OK for Niel) and could be explained to a novice in 30 secs IF  he was OK with classes to begin with.

What's your overhead and how long until a novice would know it "very well"?

Well, the simplest way to do that would be to Register Notifiers (well, NotifierMessengers) for the info one wants and then For-loop over the Notifiers to get your snapshot.  I’m not that fast at coding but it’s no faster for me to make an AE.  And don’t you have to modify all your data sources to add calls to this new AE?  I don’t have to modify any of the data-producing actors, my actors aren’t code-coupled via this subVI and can be reused as is, and some of my actors can be on remote systems (try that last one with your action engine!).  

How long would it take for a novice?   I’m not as positive on the ability of novices to understand action engines (they seem to love Local variables, from my experience) but learning messaging patterns is more of a learning curve.  But learning is an investment.  Learning messaging is well worth it.

1 hour ago, Neil Pate said:

I used to do things like this, where the data was in a message itself. However I found maintenance of the user events became too much of a pain. Now I just have simple "data changed" events, and then the thing listening polls the data it care about from the global data store.

My actors themselves are all 100% ByVal, I just use the data store as the mechanism for access the data across processes

I have one User Event per “actor”, that basically carry Text-Variant messages.  No maintenance at all.  The main weakness of polling on “data changed” is that you can miss updates.

53 minutes ago, ShaunR said:

Because of aggregation.

The goal is to average a number of values all being acquired asynchronously. There needs to be a way to snapshot the multiple values at an instance in time. If you use messaging then by the time you messaged each subsystem and got a response the values are not at the same instance in time. If you use events, then the event structure only operates sequentially so you have a similar problem. The AE makes this trivial. Other solution require all sorts of hoops.

My first thought is “consider doing views of past history for real and use SQLite”, as I’m sure you agree.  My second thought is “that sounds trivial to do with the Register-Notify messaging pattern”.  Why in the world would you message subsystems to get data in response?   This is what I mean by focussing on a few complimentary techniques and learning them very well.  SQLite is very complimentary, but if one mixes limited-capability messaging with some similarly simplistic “tag” implementation then you end up with an overly complex result that is weaker than just picking one technique and really understanding it.

5 minutes ago, ShaunR said:

There are "Tags" too which are global. IIRC I suppose they could be leveraged for this but I've never really looked at them

Oh yeah, there are quite a few “tag” libraries on NI.com.  Using one of those might be better than rolling your own.  I don’t use them because they wouldn’t give me anything I can’t do with messages, and they increase the amount of stuff I need to know (gotta know messaging, tags, and how tags interact with message timing).  

1 hour ago, Neil Pate said:

The way the architecture is designed there will only ever be a single writer of each data item, but these scattered across multiple actors (different hardware components and software interfaces).

The data in my application "runs slowly". Most new data comes in around about once per second so I do not really need to worry about extracting every last clock tick of performance out of my PC. Computers are crazy fast anyway these days, the bottleneck is certainly my skills.

Single writer is good.  Why “global” then?   I generally have data flowing around by messages and locally stored by anyone who needs to store it.

The other issue is constantly polling a DVR for changes, versus receiving notification messages only when things change, but for “slow” apps that doesn’t matter much (but it affects your ability to reuse your solution in “fast” apps).  

11 minutes ago, Neil Pate said:

I am experimenting with a DVR based variant repository for my "global" data storage needs. Each actor in my system (everything is an actor) is able to reference this DVR and thus get at *any* data. Perhaps a bit dangerous, but as I said, this is an experiment. 

What are you “global storage needs”?  How many writers are there of each data item?  

On September 7, 2016 at 11:56 AM, ShaunR said:

Actors and AEs aren't equivalent.

Hey, let’s infinite-loop the conversation!  

On September 7, 2016 at 0:20 PM, drjdpowell said:

Actors, AEs and by-ref classes can all be used as wrappers around shared resources that serialize parallel access to them and protect critical sections of code.  They aren’t equivalent, but they have considerable overlap.   

I should clarify.  Message-handling loops, Action Engines, and by-ref classes can all be used to organize a complex application into subcomponents and control interactions between the subcomponents.  The means of interacting may be different, async messages versus sync subVI/method calls, but the purpose of these structures is the same.  Because they are different, but used for the same underlying purpose, I would recommend that a developer focus on one.  It reduces complexity if you standardize, and allows you to really get competent at the techniques you have chosen.  If asked, I would personally recommend using messages over any by-ref techniques, and also not using globally-accessibly things like standard AEs, but it is more important that one pick a technique and get good at it.

2 hours ago, ShaunR said:

Generalising to nebulous "actors" (which is just a hand-wave and means "insert your meaning here")

Well, even at its most nebulous, “actors” interact by passing asynchronous messages, and are actively handling those messages.  “Message-handling loops”, basically.  That is significantly different than AE/DVRs, which passively wait to be synchronously called.  More specifically, “actors” should follow the Actor Model, and I should probably have used the more generic "Message-Handling Loop” as a thing to compare with Action Engines.

1 hour ago, ShaunR said:

To conflate them with processes, subsystems and services where state rules supreme is a fallacy.

Eh? Where was that in the discussion?  I was comparing AE's, DVR-based by-ref objects, and "actors".  One can do complex or very simple things with any of them.  

21 minutes ago, ShaunR said:

Standard terms; not peculiar to LabVIEW.

Link title is "Passing Arguments by Value and by Reference"

Ah, so you’re meaning pass by reference, as applied to arguments to function calls (terminals in LabVIEW), and some of the things we are talking about are not passed as arguments.  

I’m meaning “act on by reference”, or “having a reference to something”, meaning I have the means of acting on something that others also can act on.  I don’t care if the reference is passed as an argument or is statically compiled in (like a Global or a subVI call).