Poll on Architecture and Frameworks

[drjdpowell]

I was interested in the relative numbers of LAVA developers using different "frameworks" or types of architecture.  Apologies, my list of poll options shows my biases towards a message-passing, actor-ish, style of architecture.

Sorry, I seem to have forgotten to allow more than one option, so you'll have to pick the the best answer and explain more in a post. 

I pick "Messenger Library", though as I developed that, my honest answer is that I use my own internal framework.

Edited September 25, 2019 by drjdpowell

[crossrulz]

When not doing TestStand, I tend to use something similar to a Queued Message Handler.  Possible commands to a module/actor are the public interfaces in a library while the queue/event/etc are stored in a private Action Engine.

[jcarmody]

We use the JKI State Machine, but have begun adding DQMHs.  My last project launches up to twelve of them; my current project launches one or two.  Both depend upon the number of pieces being tested (and  were written by my CLA-colleague).  I wanted to use JKI SMOs, but was given DQMH (and am happy about that, too).

[hooovahh]

Since 2011 I've been refining what I call Glued (Global User Event Driven) architecture.  It is User Event driven using a publisher and subscriber model, with variants and variant attributes as the data containing things like the Request Data, event name, who requested the action, who is to perform the action, and GUID among other things.

These parallel actors default to non-reentrant so that probing and debugging is easy (and possible on RT), and don't get spun up or down but instead have a known number of them at test start.  They all start at once, and they all stop at once.  If an actor needs to restart it can go through the queued message handler states that perform initialization but the only way to actually stop the actor is to stop the application with a global event, which is an event that all actors subscribe to.  The actors are part of a Library but not Class.  The architecture also supports events that are request only, or ones that will wait on a reply, and can do this by leveraging Futures or Promises.  There is scripting tools for generating actors, template projects, or adding methods to existing actors.  Manual panels, error actor, configuration actor, and probe actor are included with a template project since the majority of projects I have use them.  Actors can be ran as the top level VI and events can be sent to itself.  This way individual actors can be developed and debugged without the whole application, and without any other actors needed.  Each actor can publish to a Sensor which is a scalar double or a string and is name based, or publish to it's own global which is a variant and can contain any data as a variant attribute and read with VIMs.  These globals are read only to actors other than itself.

Over the years this has been improved to support remote actors running on other targets such as RT with Network Streams being the back end for this.  Requests for action can be sent to these remote actors, and reading the actor's global, or Sensors can also be done locally or remotely.

It sounds like a lot of extra bells and whistles and there are times I'll start a new project just doing my own thing from scratch.  And every time I've done this I ended up regretting it because some feature was built into Glued that I would have to re-write or hack in my custom project.  Over the years I've just convinced myself that even small projects should start with some actor based design.

[Rolf Kalbermatter]

The comment about being able to choose more than one selection is not true since it is a radio button list that resets any previous selection when you select something new.

For me I selected my own (company framework) which can sometimes vary since there are customers who use their own framework too. But I also have used DCAF and similar systems which had a CVT backend for most of the data handling. 

[ShaunR]

I use service oriented architectures. Modular, self contained services that can be transplated from one project to another which interact via string messaging APIs (kind of like SCPI). An execution engine (also usually a service) orders and drives the messages which is usually custom for a particular project. If anyone remembers the HAL demo for Vims, it used this architecture with an Event Driven State Machine as the execution engine.

Edited September 25, 2019 by ShaunR

[LogMAN]

We use a variety of frameworks/templates/patterns for our architecture:

• The Actor Framework is used for asynchronous UI operations and long-running data processing tasks.
• Queued Message Handlers (not using DQMH or QMH template) handle simple asynchronous tasks.
• Action Engines encapsulate privately shared data for our translation and general I/O libraries.
• (Queued) State Machines ensure that everything runs in order.
• The proxy pattern is used to interface most customer libraries.
• We have our own frameworks for the test execution engine and test libraries (message based).

I find the publicly available frameworks and templates (DQMH, Messanger Library, NI templates, etc...) very valuable for learning and to get things started quickly.
More advanced projects, however, require a deeper understanding of the underlying patterns in order to develop your own architecture (which may or may not utilize these frameworks/templates).

4 hours ago, drjdpowell said:

I pick "Messenger Library", though as I developed that, my honest answer is that I use my own internal framework.

In your case these are synonymous 😋

3 hours ago, hooovahh said:

It sounds like a lot of extra bells and whistles...

"Any sufficiently advanced technology is indistinguishable from magic." (Arthur C. Clarke)

For some reason that just popped into my mind...

3 hours ago, hooovahh said:

...there are times I'll start a new project just doing my own thing from scratch.  And every time I've done this I ended up regretting it because some feature was built into Glued that I would have to re-write or hack in my custom project.  Over the years I've just convinced myself that even small projects should start with some actor based design.

If your entire architecture and thought process is fundamentally based on actors, any small project will of course have to depend on it as well. That is, unless you are willing to rethink (and probably reimplement) the fundamental architecture.

Then again, why reinvent the wheel?

[G-CODE]

SMO. I'm guessing Francois will say the same thing since he wrote a large part of it.

Several months ago I was comparing three common community frameworks and I had Francois chime in. I attached a screenshot of what we put together with our rankings obscured. If he's ok with me showing it, I'd love to reveal our rough rankings for the categories shown.

My superficial summary of each?

• Actor Framework - you better love having LOTS of classes in your project and opening DO methods to figure out how the code works.
• JKI SMO - Use this framework and Steven Mercer will castigate you because no human alive can write code with reference class data that isn't riddled with race conditions and bugs. Honestly though, if you're that worried you can impose some rules when developing with this framework to make it more actor-like.
• DQMH - The fact that it was developed for customers who don't understand object oriented programming suggests that if an experienced developer wants to combine it with classes that it might be unnecessarily complicated. Sure enough, I watched the presentation on how to do hardware abstraction with classes and DQMH and it confirmed my suspicions. It makes one think about what a framework would look like if your initial design goals were NOT to hide classes from less experienced developers. Turns out that framework might look something more like SMO.

In the same project that is using SMO, I'm currently using DCAF at the lower level to manage I/O because there are hundreds of I/O tags. Now that I've spent a fair amount of time developing a variety of DCAF dynamic modules for this project I think I have a good idea of DCAF's strengths and shortcomings. It is open source so if I want some of the bugs and nuisances fixed, I'm going to have to fix it myself. Whoever wrote it got us 80-90% of the way there, but that final polish is going to take a fair amount of work. A lot of the code under the hood just ain't that pretty and some people don't really care about that. I do so I find myself getting annoyed every time I dig into it.

[MarkCG]

I use DCAF for most of the things running on compactRIO. I like it, very easy to write static modules, much harder to write dynamic ones where it works correctly with the configuraton editor, though I have done it. Nice to be able to manage tags with CSV files.  My biggest gripe about it is that there is no built in events / messages/ triggers. So if you have a module that needs to perform some sort of sequence that you want to initiate on user command, you have to do something hacky like increment an integer tag, or change a boolean tag's state. I guess you could also do that with RTFIFOs or queues, sending this kind of command data from outside or even inside the framework but I have not invested any time in that, and seems like it would be hard to debug.

Edited September 26, 2019 by MarkCG

[Antoine Chalons]

When I had to pick one, the options I considered were :

• DQMH
• AF
• JKI SMO
• G#
• ALOHA
• my own that would be much better than all of the above combined

I picked DQMH because I love the API tester the scripting tools, also I feel there is a somewhat large community adoption and support.

[drjdpowell]

14 hours ago, G-CODE said:

SMO. I'm guessing Francois will say the same thing since he wrote a large part of it.

Several months ago I was comparing three common community frameworks and I had Francois chime in. I attached a screenshot of what we put together with our rankings obscured. If he's ok with me showing it, I'd love to reveal our rough rankings for the categories shown.

Apologies to JKI for not explicitly including State-Machine-Objects in the poll.  Though I think what you compared was two common community frameworks, plus the framework you use, and largely developed, regardless of how common it is.  Difficult to make such comparisons, as the stuff-you-know-well usually trumps stuff-you've-read-about.  Even getting unbiased comparison categories is hard; I'd probably have "messaging pattern support" and "network communication", two things Messenger Library does well, yet not "Teststand Integration", which I've never tried.

[drjdpowell]

39 minutes ago, Antoine Chalons said:

G#

Didn't think to include G#, GOOP and similar, as I've never used them.

[G-CODE]

2 hours ago, drjdpowell said:

... the stuff-you-know-well usually trumps stuff-you've-read-about.

That doesn't necessarily have to be true if you're willing to dig in to specifics of existing frameworks even if you haven't spent a lot of time developing with them. For example, Antoine points out that DQMH has good scripting tools. I agree with him, even though I haven't developed a large project with it. DQMH seems like it makes it very easy to create new messages with custom data types.

3 hours ago, drjdpowell said:

I'd probably have "messaging pattern support" and "network communication", two things Messenger Library does well

Those are good suggestions to add to the comparison chart! This raises the question of what a framework should be: process startup and management, dependency management, data management, interprocess communication, etc... The answer is probably different for everyone. It depends on what's important to the project and the developers working on the project.

Now you've got me interested in digging into Messenger Library which I've never taken the time to do. SMO and DQMH assume you are going to be using events to broadcast and receive messages.

[G-CODE]

15 hours ago, MarkCG said:

much harder to write dynamic ones where it works correctly with the configuraton editor

Tell me about it! I think I've written seven of them so far and every time I create a new one I have to reference my previous modules to help me get the new one working. It's not trivial.

15 hours ago, MarkCG said:

So if you have a module that needs to perform some sort of sequence that you want to initiate on user command, you have to do something hacky like increment an integer tag, or change a boolean tag's state

For that reason I moved complex logic outside of DCAF to a higher level. DCAF manages I/O like scan engine, serial/modbus instruments, calculated tags. I wrapped DCAF in a subsystem that all other subsystems must use to access I/O. Since DCAF exposes I/O data through NI's CVT, I make sure not to use the CVT elsewhere throughout my application as that would bypass the protections I put in place for writing to tags.

[drjdpowell]

5 hours ago, G-CODE said:

That doesn't necessarily have to be true if you're willing to dig in to specifics of existing frameworks even if you haven't spent a lot of time developing with them. For example, Antoine points out that DQMH has good scripting tools. I agree with him, even though I haven't developed a large project with it. DQMH seems like it makes it very easy to create new messages with custom data types.

Ahh, your illustrating my point.  DQMH is similar to SMO in having custom-type "plumbing", with new messages requiring changes in multiple places.  AF also has its own busywork of multiple things to do for each message.  Now, all this work is highly repetitive and thus scriptable.  So you can value the strength of DQMH's scripting tools, because similar tools would be a help for SMO.

But from the point of view of Messenger Library, which uses more generic messages, scripting is just a way of reducing one of the weak points of custom data-type messages: all the extra work to carry the new custom types through the communication code.  Scripting Tools aren't a strength, they're the minimization of a disadvantage that is shared by DQMH, SMO and AF.  

 

[G-CODE]

1 hour ago, drjdpowell said:

But from the point of view of Messenger Library, which uses more generic messages, scripting is just a way of reducing one of the weak points of custom data-type messages: all the extra work to carry the new custom types through the communication code.  Scripting Tools aren't a strength, they're the minimization of a disadvantage that is shared by DQMH, SMO and AF.

I have to be careful what I say here because I haven't explored Messenger Library, but my inclination is to think that I'm going to have to know the type at some point in time whether it be at edit time or at run time. So at what point in time would I rather know it?

I just downloaded the package and flagged your YouTube videos to watch.

[Francois Normandin]

Like Eric predicted... I use JKI SMOs the most.
Although I also actively maintain a couple of Actor Framework projects.

[drjdpowell]

8 hours ago, G-CODE said:

I'm going to have to know the type at some point in time whether it be at edit time or at run time. So at what point in time would I rather know it?

Your right, the question of strict versus weak typing is an important question and involves trade offs, and different frameworks make different choices.  I, in using Messenger Library, seem to spend almost no time bothered by problems caused run-time type mismatches, so I am not much impressed by the edit-time type safety if it comes with the negative trade offs of requiring large scripting tools to work productively.  Why that is might be a long discussion about other features of how I code, and what information is put in a message.  I've tried to point out before that code frameworks support mental frameworks, sets of guidelines that work together, and it can be difficult to compare code frameworks because it is hard to full grasp the full set of its mental guidelines.  You found that yourself when AQ's strong criticism of State-Machine-Objects didn't really move you, because you just have to "impose some rules when developing with this framework".

[ShaunR]

10 hours ago, drjdpowell said:

But from the point of view of Messenger Library, which uses more generic messages, scripting is just a way of reducing one of the weak points of custom data-type messages: all the extra work to carry the new custom types through the communication code.  Scripting Tools aren't a strength, they're the minimization of a disadvantage that is shared by DQMH, SMO and AF.  

I agree totally with this. The use of scripting is, more often than not, used as a copy&paste re-use tool which is the antithesis of re-use.

Edited September 27, 2019 by ShaunR

[drjdpowell]

BTW, I was partly prompted to do this poll after seeing there is yet-another-framework on the Tools Network: WORKERS by Scarfe Controls. 

[smithd]

On 9/25/2019 at 6:35 PM, G-CODE said:

A lot of the code under the hood just ain't that pretty and some people don't really care about that. I do so I find myself getting annoyed every time I dig into it.

Yes, I don't really care about that, and I will use block diagram cleanup until its pried from my cold dead hands 👻

Some of the key parts (like the engine) are relatively neat (by my standards) but due to the complexity its definitely still a hot mess. And of course a lot of the code dealing with all of the data types is scripted because aint nobody got time for that.

Your other comments are fair though, some definite mistakes were made in the design, but all told I think it does a decent job.

On 9/25/2019 at 8:31 PM, MarkCG said:

My biggest gripe about it is that there is no built in events / messages/ triggers.

Yeah, I made an aborted attempt (https://github.com/LabVIEW-DCAF/ModuleInterface/tree/StreamingAndEvents  and   https://github.com/LabVIEW-DCAF/ExecutionInterface/tree/StreamsAndEvents) but...then I left NI. The nature of labview merging is such that those branches are probably now useless

edit: theres probably a few implementation-side branches too like https://github.com/LabVIEW-DCAF/StandardEngine/tree/StreamsAndEvents

Edited September 27, 2019 by smithd

[MarkCG]

5 hours ago, smithd said:

Yeah, I made an aborted attempt (https://github.com/LabVIEW-DCAF/ModuleInterface/tree/StreamingAndEvents  and   https://github.com/LabVIEW-DCAF/ExecutionInterface/tree/StreamsAndEvents) but...then I left NI. The nature of labview merging is such that those branches are probably now useless

 

That's great, I had no idea! I'll take a look at it at least and see if I can maybe come up with something similar.

[MarkCG]

On 9/26/2019 at 9:24 AM, G-CODE said:

Tell me about it! I think I've written seven of them so far and every time I create a new one I have to reference my previous modules to help me get the new one working. It's not trivial.

For that reason I moved complex logic outside of DCAF to a higher level. DCAF manages I/O like scan engine, serial/modbus instruments, calculated tags. I wrapped DCAF in a subsystem that all other subsystems must use to access I/O. Since DCAF exposes I/O data through NI's CVT, I make sure not to use the CVT elsewhere throughout my application as that would bypass the protections I put in place for writing to tags.

Makes sense. To me it seems DCAF if very close is so close to doing everything I want it to, I'd hate to add yet another layer on top of it.

Honestly my feeling is that all these frameworks are great but they are all using a lot of complexity to transform and stretch LabVIEW into something completely different from what it was meant to do originally. There has to be a better way, were Actors , state machines, tags vs command vs streaming data, and easily handling thousands of I/Os are fundamental concepts of the language.

[ShaunR]

57 minutes ago, MarkCG said:

Honestly my feeling is that all these frameworks are great but they are all using a lot of complexity to transform and stretch LabVIEW into something completely different from what it was meant to do originally. There has to be a better way, were Actors , state machines, tags vs command vs streaming data, and easily handling thousands of I/Os are fundamental concepts of the language.

I think the main problem is that most people confuse frameworks with architectures. People tend to pick one framework then try to shoe-horn in incompatible features. The obvious one is the difficulty in sequencing actors in the actor framework. It's a concurrent paradigm so is very resistant to sequencing and all sorts of contortions have to be realised to make it useful in certain circumstances.

This is why I like SOA. A service can use any of the frameworks that fit the requirement for that service and the system state is external to the services. Many, if not all, of the frameworks embed the system state inside the framework and therfore you run into problems when, for example, you need to sequence concurrent operations (AF) or concurrently execute sequential operations (QMH). Because of that you end up with silly things like "Promises". 

Edited September 27, 2019 by ShaunR

[G-CODE]

On 9/27/2019 at 2:14 AM, drjdpowell said:

I, in using Messenger Library, seem to spend almost no time bothered by problems caused run-time type mismatches, so I am not much impressed by the edit-time type safety if it comes with the negative trade offs of requiring large scripting tools to work productively.

Now you've got me thinking. I'm so habituated to creating typedefs, events, and registrations for every message. Maybe there's a better way...