Simple Modular MVC

[abrink]

I've built (and re-built) a medium-sized application over the past months, learning more about LabVIEW in the process. The design is simplistic and reaching its limits by now, so I've been re-thinking the design lately. I'm looking for feedback on my design choices before I start implementing... all criticism and tips are welcome!

 

The goal: flexible software for an ever-changing university laboratory environment, to be used and extended by students/staff with limited LabVIEW skills.

 

Decisions:

1. Avoid LVOOP unless clearly called for
   OOP has it's benefits but it makes code more complicated to grasp for novices.
2. Avoid asynchronous commands unless clearly called for
   Many hardware operations are synchronous; these should be carried out synchronously (with time-outs) to minimize complexity. No message queues, definitely no Command architecture. Of course, queues are still OK for inter-thread communication, e.g. a data acquisition thread in a DAQ module.
3. Modular Model-View-Controller (MVC) paradigm
   This should facilitate our need for extendible and adaptable software
4. Every module needs to be able to run independently
   A central GUI will usually be running, but it should be possible to start/operate any module separately during run-time. This simplifies debugging and allows for flexibility during measurements ('let's quickly check the signal by hooking up a scope') 
5. Every module must be able to 'require' and 'unrequire' other modules
   If a measurement module is loaded, any required equipment should be initialized unless it's already running. 
6. A module should gracefully shut down when there are no modules left requiring it
   This is vital to prevent equipement from melting during the night!

 

What I've come up with (core ideas are underlined):

 

MVC implementation

1. Each module is a lvlib containing Typedefs, Public Functions, Private functions, and Panels.
2. The Model is a typedef'd cluster containing Settings, Data, Events, and Resources (queues, refs, etc.)
3. The Model cluster is stored as a Data Value Reference (DVR) in a Functional Global Variable (FGV). The "First Call?" functionality within the FGV is used to initialize the Model (read Settings from a .ini file, create User Events, spawn background threads, etc.) and create the DVR. I could be re-inventing the wheel here, but I'm pretty enthusiastic about this approach. Note that the Model could alternatively be stored in a Class instance holding the DVR. This is useful in applications where you expect multiple copies of the same module (maybe you have six DAQs attached) but it complicates application design as you need to specify which instance you'd like to access before each Controller function call.
4. The Controller is a set of synchronous functions operating on the Model. Each function obtains the Model DVR from the FGV, and uses an in-place element structure (IPES) with DVR Read/Write terminals to access and modify the data. Note that this provides some protection against race conditions, and reduces memory usage. Inside this IPES could be a second IPES, with Cluster Unbundle/Bundle terminals, to increase readability and again reduce memory usage.
5. All Controller functions that change the Model generate appropriate User Events when doing so.
6. Views are panels that execute public Controller functions to interact with the Model. The View registers for (a subset of) the User Events emitted by the Controller to reflect changes in the Model.

Modularity

1. Each module has public Require and Unrequire functions. These maintain a list of dependent modules, and call a (private) Stop function if the list is empty. Right now, I'm simply using the Call Chain string to identify the calling module, but there should be a more appropriate solution for this (in light of OOP, where the Call Chain is no longer unique).
2. A View Requires its own Controller, and Unrequires it on Panel Close.
3. A controller can Require other modules, which it must Unrequire when it no longer needs them.
4. Hardware interfaces, software components, and measurements are all modules.

Practical Example

1. User runs the View (panel) of a Measurement Module, which starts the related Controller.
2. The measurement controller Requires some hardware (say a DAQ board and a camera) and software (say an Error Logger) and loads certain Views (panels) to them into SubPanels (e.g. it display the Error Log).
3. The Required modules in turn Require other modules (the camera Requires an Image Processor) or the same modules (the camera also Requires the Error Logger) which are initialized if not already running.
4. Uses carries out his measurement, calling public functions on his required controllers. All changes are immediately reflected in the open Views through event handling. 
5. User stops his measurement and closes the view.
6. All required modules (the DAQ, Camera, and Error Logger) are Unrequired.
7. In turn, these models Unrequire their respective Required modules; the Camera Unrequires the Image Processor, etc.
8. All modules shut down gracefully when there are no modules left requiring them.

Concerns

1. Unexpected errors may break the require/unrequire chain, causing modules to run indefinitely. Similarly, if a users forgets to Unrequire a module, it will live on (possibly without a panel) indefinitely. A sort of application-wide emergency stop system (using events or notifiers) could be used to 'fix' this, but I'd like to refrain from using any central structures of possible.
2. Controller lifetime. Say two modules Require the same third module. The third module will be 'owned' by the first caller, causing it to abort when the first caller leaves memory. This leaves the second caller module with a problem. I think this can be prevented by having each controller maintain a reference to itself, which it only releases upon graceful shutdown. Will this work, and will it be reliable?
3. Stop code in a View Panel (notably Unrequiring its controller) might not run. Again, I think this can be circumvented by having the panel maintain a reference to itself, which is only released after calling graceful shutdown code in the Panel Close event case.

 

Wow, that sure turned into a short novel. If you're still with me, I'd love to hear your thoughts!

[ShaunR]

Well. You mention that you don't want to use LVOOP because it makes it difficult to grasp for novices but then advocate a Muddled Verbose Confuser (MVC) architecture which even experts on that design pattern can't agree on what should be in which parts when it comes to real code.

 

As it needs to be simple for novices, I also suggest you throw rotten tomatoes at anyone that mentions "The Actor Framework".

 

Since there maybe many people who build on the code and many will have limited experience, Have you thought about a service oriented architecture? With this approach you only need to define the interfaces to external code written by "the others". They can write their code anyway they like but it won't affect your core code if they stuff it up. You can then create a plugin architecture that integrates their "modules" that communicate with your core application via the interfaces.

 

The module writers don't need to know any complicated design patterns or architecture or even the details of your core code (however you choose to write it). They will only need to know the API and how to call the API functions..

Edited October 11, 2014 by ShaunR

[abrink]

Thanks for the quick reply. The point is that the end users will have to create components as well as measurements. I'll be gone in two years, and if my successors buy a new piece of equipment I want them to be able to integrate it without hassle. My approach should allow them to copy a template lvlib, replace some code (aided by documentation) and get measuring.

The necessity of simple interfacing is met by writing modular code; the question is, what should a module look like? Subsequently, how should modules interact? I've described a solution above, and I'm hoping for feedback from experienced LabVIEW developers.

[Tim_S]

I would not say that LVOOP should be avoided unless clearly called for, but (like any tool) used appropriately. In the case of the modules, a parent object could define the interface to the larger application and carry in to a child object common data/settings. (See plug-in archetecture.)

 

Since you intend for this to change, grow and mutate without you, as generic a way for modules to communicate would be best. Two ways that has been done before is using a cluster of an enum and variant, or a string. You may want to look at SCPI commands to get some ideas for using strings.

[abrink]

By now, I've created my implementation with little difficulty. The main challenge was data persistence, since data tends to leave memory when the original 'requiring' caller aborts. Hence, 'moving' a component from one owning module to the next releases all User Events and other allocated resources. The solution was to asynchronously launch an 'owning' vi on a component's first launch, which initializes the component and then just sits in memory. An added benefit of this approach is that you can implement an event handler for an 'emergency stop' event in this owning vi.

 

As I mentioned, I had a hunch that I was reinventing the wheel here. Indeed, it appears that I have basically recreated the Extensible Session Framework (https://decibel.ni.com/content/docs/DOC-12813). Oh well, I learned a lot 

[drjdpowell]

By now, I've created my implementation with little difficulty...

Could you explain more?  Your initial description gave the impression of being very complex, a lot more complex that the Extensible Session Framework.

[ShaunR]

Could you explain more?  Your initial description gave the impression of being very complex, a lot more complex that the Extensible Session Framework.

 

I think the OP was just having difficulties figuring out how to do #5 and #6. He'll be back again when he runs into #2 between modules.

[abrink]

Nope, everything is running smoothly. I will post a barebone example once I find the time.

It boils down to this: when calling Start on a module, a 'holder' vi is launched asynchronously. This initializes the module data and events and creates a DVR to that data, which it stores in a FGV. The holder then starts listening for a module stop event, which it also stores in a FGV. All functions which operate on the data get the DVR and modify it in an IPE structure, providing protection against race condition. Calling Stop on a module gets the stop event FGV and triggers it, causing the holder vi to run exit code and destroy the DVR.

The require/unrequire functionality is trivial, maintaining a list of depending modules and calling Start/Stop when the list is one element new / empty.

Edit: I should add that I ended up creating a Core module that is required by everything else. This allows me to shut down all modules (gracefully) at once by having each module listen for a Stop All event as well as their own Stop Moduld event. Furthermore, it will contain some settings like a debug mode boolean and a parsed config.ini

Hope this is a bit clear!

Edited October 15, 2014 by abrink

[abrink]

I'd like to leave a small note for anyone reading this topic: this turned out to be a highly unmaintainable solution to the challenge. I've bit the bullet and moved to a LVOOP approach, which is converging towards the ESF solution. The required LV skill for using classes is definitely outweighed by the benefits, mainly inheritance and property nodes on DVR wires of the class.

[odoylerules]

I'd like to leave a small note for anyone reading this topic: this turned out to be a highly unmaintainable solution to the challenge. I've bit the bullet and moved to a LVOOP approach, which is converging towards the ESF solution. The required LV skill for using classes is definitely outweighed by the benefits, mainly inheritance and property nodes on DVR wires of the class.

 

Remember you said this.... 

 

LVOOP comes with its own set of troubles that i'm sure you will stumble upon in time.  I'm not saying its not the better route, its just not the end all be all

Edited October 28, 2014 by odoylerules

[abrink]

Funny you should say that; I've now completed my LVOOP implementation and I hate it. I'm moving back to the solution I've described in this topic; it is more straightforward and looks way cleaner on the block diagram. The added value of inheritance turned out to be limited for our purposes.

[hooovahh]

I must say, reading this whole thread all at once for the first time causes a roller coaster of emotions.

[ShaunR]

Funny you should say that; I've now completed my LVOOP implementation and I hate it. I'm moving back to the solution I've described in this topic; it is more straightforward and looks way cleaner on the block diagram. The added value of inheritance turned out to be limited for our purposes.

 

Indeed. LVOOP is a bit like a religion - a dogmatic ideology that everybody seems to still accept, in spite of all the evidence.

 

Well. That's a bit unfair. It's not just LabVIEW. I've been laughing my gonads off about the implementation of "namespaces" that were introduced in PHP 5.3. Another example of taking a perfectly usable language and making it worse to fit the doctrine.

[MarkCG]

Indeed. LVOOP is a bit like a religion - a dogmatic ideology that everybody seems to still accept, in spite of all the evidence.

 

Well. That's a bit unfair. It's not just LabVIEW. I've been laughing my gonads off about the implementation of "namespaces" that were introduced in PHP 5.3. Another example of taking a perfectly usable language and making it worse to fit the doctrine.

 

It's got its uses, it's definitely helped me simplify certain problems. Just to not be vague, hardware abstraction classes for communicating to various random instruments and hardware, loading those dynamically according to a config file, is one clear slam dunk for LVOOP to me. There are others. 

 

I definitely get very annoyed by the "everything must be a class" philosophy, with vaguely implied benefits of "reusability" or "so we can change it later" (inheritance) being the justification.

 

Using OOP does not necessarily practically imply either benefit IME. If you can already see the whole picture of your hierarchy of classes before implementation, and see how it will benefit you, do it. But just using LVOOP that it will somehow be easier or "better" without a clear picture of what you will accomplish with it will just make things more complicated than needed. 

[ShaunR]

It's got its uses, it's definitely helped me simplify certain problems. Just to not be vague, hardware abstraction classes for communicating to various random instruments and hardware, loading those dynamically according to a config file, is one clear slam dunk for LVOOP to me. There are others. 

 

I definitely get very annoyed by the "everything must be a class" philosophy, with vaguely implied benefits of "reusability" or "so we can change it later" (inheritance) being the justification.

 

Using OOP does not necessarily practically imply either benefit IME. If you can already see the whole picture of your hierarchy of classes before implementation, and see how it will benefit you, do it. But just using LVOOP that it will somehow be easier or "better" without a clear picture of what you will accomplish with it will just make things more complicated than needed. 

 

The thread seems to have run its course, so I will respond here as we are now way off topic and it shouldn't hurt

 

Hardware abstraction (at least for devices) was solved years ago by hardware engineers. With the proliferation of boxed solutions, SCPI and robust comms converters, the purpose behind most peoples HALs was obsoleted. It reached critical mass about 6 years ago when to not be SCPI compliant became the exception rather than the rule, there was a large choice in comms-to-comms converters and all PCs came with high speed USB and TCPIP.

 

VI drivers were designed to normalise instrument command interfaces. SCPI changed all that and things have moved on but still people write HALs around these (what I consider legacy) drivers instead of directly interfacing the devices to their messaging systems. If you don't use VI drivers, SCPI devices slot straight in to string based messaging systems and open up scripting and performance benefits. Even NI DAQ, Profibus and Modbus devices can be given a thin SCPI translation wrapper in a pinch if you have no control over device selection and laptops become viable test systems.

 

Once you get that far, you suddenly realise that now the devices don't have to be in the same country, let alone hanging off the same computer, as hardware and device specifics have been completely removed from the software. The emphasis then becomes managing and routing the messages, security and scripting test harnesses - very little thought or indeed programming needs to go into how to talk to the hardware.

 

So. The benefit of OOP for hardware abstraction is moot IMO and wasn't solved by OOP or even software, anyway. There seems to be a lot of trumpeting of LVPOOP for solving complex software problems that either don't exist or where created by using LVPOOP in the first place-singletons are a good example of the latter.However, I know what you mean about "vaguely implied benefits" and the evidence is never proffered. There is however evidence that the benefits are just marketing hype.

 

 

Our results indicate that in a commercial environment there may not be a consistent statistically significant difference in the productivity of object-oriented and procedural software development, at least not for the first couple of generations of an object-oriented product. The reason may be low reuse level, but it could also be the underlying business model.

 

spetep- printable.pdf

Edited November 15, 2014 by ShaunR

[Tim_S]

VI drivers were designed to normalise instrument command interfaces. SCPI changed all that and things have moved on but still people write HALs around these (what I consider legacy) drivers instead of directly interfacing the devices to their messaging systems. If you don't use VI drivers, SCPI devices slot straight in to string based messaging systems and open up scripting and performance benefits. Even NI DAQ, Profibus and Modbus devices can be given a thin SCPI translation wrapper in a pinch if you have no control over device selection and laptops become viable test systems.

 

Normalizing the communication bus (Ethernet/IP, ModbusTCP, Profibus, CAN, Flexray, UDP, etc.) is where I have found LVOOP to be useful. Could I do this without LVOOP? Certainly. Is it a good use of the LVOOP tool? I believe so.

 

The hardware manufacturers I've been working with don't seem to have heard of SCPI or consider it outdated. The types of devices I've worked with are drives, particle counters, valve manifolds, remote I/O, and weather stations. Most of the hardware is from customers' approved components list.

[smithd]

Normalizing the communication bus (Ethernet/IP, ModbusTCP, Profibus, CAN, Flexray, UDP, etc.) is where I have found LVOOP to be useful. Could I do this without LVOOP? Certainly. Is it a good use of the LVOOP tool? I believe so.

This is what came to mind for me when I read Shaun's post, but I don't know anything at all about SCPI. From what the wiki tells me it defines a generic set of messages to be used kind of like j1939 for instruments. It seems like it has just replaced one type of HAL (instrument centric) with another (network abstraction). You still need to handle communication over ethernet, usb, serial, etc. (ie what VISA does). 

 

It also doesn't seem to help at all on the device side. If I'm making more than one device that talks SCPI, I would want to have some standard network interface which takes commands and passes them to an abstraction layer before returning the response. The abstraction layer would handle the differences between my devices when responding to each command.

 

Anyway, to me lvoop amounts to dynamic cluster with a function pointer and occasional linking issues. For when I need to have a dynamic cluster and function pointer, I use lvoop. When I don't need those things I don't use lvoop. 

[ShaunR]

This is what came to mind for me when I read Shaun's post, but I don't know anything at all about SCPI. From what the wiki tells me it defines a generic set of messages to be used kind of like j1939 for instruments. It seems like it has just replaced one type of HAL (instrument centric) with another (network abstraction). You still need to handle communication over ethernet, usb, serial, etc. (ie what VISA does). 

 

J1939. Wow. That's going back a bit. OBDII has been the standard for at least 10 years now (1998?).

 

In response to both of you though.......... Not really. As I think I mentioned. The hardware guys & gals mitigated hardware interfaces with cables (and hubs). There used to be a few teething issues with things like USB compatibility, but they have pretty much disappeared now. If a device doesn't use TCPIP (most offer it as an option, you just need to spec it) Just use one of the many hubs or cables (Ethernet<->USB/RS422/RS485/SERIAL). It s not unusual to use a 20 port USB or 485 hub dangling off of a Ethernet cable, for instance. You can even connect wirelessly if you like, Combine that with standardised string messages (SCPI) and the worse you need for the OCD programmer who cannot abide odd messages is a lookup table from a home grown SCPI hierarchy for the legacy device to whatever ASCII or byte representations the old instruments use. Just copy and paste from their manuals.

 

And testing? Well. That's a whole other area of ease of use. Just throw all the commands into files and squirt them at the devices in whatever order you want with whatever instruments you want. Just one simple VI that loads the file with commands and expected responses. Send them to one device/service/module to do a full factorial message test or to different devices/services/modules to do system tests. Simples.

 

The hardware guys really have solved it, even if small manufacturers are still doing what they did from 20 years ago. I wouldn't be surprised if over 95% of those on the Instrument Drivers Network were SCPI compliant for instance (haven't counted, just saying).

 

PS.

We don't care about the instrument side, that's the firmware developers problem

Edited November 18, 2014 by ShaunR

[smithd]

PS.

We don't care about the instrument side, that's the firmware developers problem

Lol well ok then, I guess I can't argue with that

 

 

J1939. Wow. That's going back a bit. OBDII has been the standard for at least 10 years now (1998?).

 

Perhaps in some areas, but I can say with unfortunate certainty that people still use that protocol. Very unfortunate certainty

[ShaunR]

Lol well ok then, I guess I can't argue with that

 

 

That was a little unfair, I know.The fact that firmware engineers did solve it for the rest of us doesn't acknowledge their difficulties in achieving it. Maybe they abstracted so we don't have to? Firmware engineers used to plan for serial and a MAX chip in front did the rest. That is why with so many devices the interface is an option. It depends on which daughter-board they install. So again; it was solved with hardware (did one myself, back in the day).

 

With ARM and modern PICs coming with in-built UARTs, USB  and TCPIP stacks, I don't really know but it may highlight the reason for your question. Once the comms interface has been pushed inside the System On A Chip boundary, how does the firmware switch interfaces? What if the device has an interface that isn't propagated to the outside world?. The solution will still be to get it into a character string and then define the SCPI command set that the firmware can work with but how you get the string may be a pain. Saying that.. The firmware engineers did do it and the major suppliers normalised it and for that, I am thankful because it makes my life so much easier.

Edited November 19, 2014 by ShaunR

[abrink]

Time for another on-topic update! I've been playing around with the concept of 'simple' modular labview code, focusing on flexibility and ease of development. After many revisions, the basic module design has come down to a somewhat surprising layout! Module data is now stored into Global Variables, which are restricted to the Private scope of the module lvlib. A semaphore should be used to protect write operations, if you're worried about race conditions. This approach results in clean block diagrams and quick coding, and I'm really liking it so far.

I'll share some code if there is any interest!

Edited December 25, 2014 by abrink