Jump to content

Over my head. Need help on designing a framework.


Mojito

Recommended Posts

Hi all,

 

I've been maintaining and improving a LabVIEW project which controls and automates a prototype microscope array for 2 years. I'm an engineering apprentice so I don't have that much experience with LabVIEW.

 

The current framework of the project is a simple "Producer/Consumer" which has served well but is no longer future proof and scalable. I want to revamp the program which is quite big and complex. And I need help since this will be the first time of actually starting a real project from the ground up.

 

The most modular and scalable framework I found was the QMH (Queued Message Handler). It's similar to the basic P/C loop and has the possibility to have multiple parallel consumer loops. But I have no experience on starting from 0.

 

If any you have docs or give advice on starting the project would be appreciated. Especially something on codding with a QMH structure would be helpful.

 

 

Cheers from France!

 

 

Link to comment

Many of us aren't fans of QMH. I'm somewhat ambivalent about them but a couple of points I will make from your description.

 

1. Future proofing is marketing speak.

Adding complexity for something that is not required and just a guess as to future requirements is unproductive, will bite you and adds nothing except complexity. Design your system for the requirements now and iteratively innovate when necessary to encompass new requirements. Experienced engineers will recognise patterns in solutions and code in a way that they know will be easy to modify in the future but the key point there is easy to modify, not seamlessly integrate.

 

2.If you are going to have multiple consumers of your messages/data then queue based message handling is probably the wrong solution.

You will get tied in knots re-transmitting messages/data since queues are a many to one topology and you are trying to create a one to many or many to many architecture. An event based architecture is much more suited to this but you will find very little in the way of formal examples or templates outside of user interfaces.

 

3. You can have more than one architecture within your program!

I see this all the time with even knowledgeable engineers. They choose a magic bullet framework, shoehorn all their code into it and it becomes a mess that only the developer can ever understand and is riddled with bugs. Modular code can have frameworks or architectures at the module or abstraction level and you can mix and match dependant on module requirements (QMH, State Machines, Producer Consumer etc). You can then choose one or more architectures to sew all the modules together to form a cohesive program with defined areas of responsibility that can be debugged in isolation. When new to designing software systems I would suggest the following axiom when using LabVIEW.

 

Design the system top down (from the UI usually) but create the modules/code bottom up.

A natural partitioning of code reuse arises from bottom up that is different from the partitioning of responsibilities that top down produces. It is reconciliation of these two partitioning views that yields robust and modular code.

 

So. Enough sermonising :D Here are some insightful write-ups you may find useful

NI QMH Template

QMH's Hidden Secret

Some Context

 

Link to comment

If I were looking at improving an old program I would first look for and refactor any natural ‘objects’, bits of data and actions on that data.  You, for example, might have a ‘Camera’ object for taking images, or a “Sample Positioner’ object.   Only after this would I ask if some of these objects should exist in separate loops so they can do things in parallel.  

 

Re the various QMH designs, I gave a talk recently about some of the issues (main point: don’t use the NI QMH template; use something like the JKI template).

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By cyro2015
      Hi,
      I tried to create a template based on OOP for QMH. During development I have been confronted with infinite crashes of LabVIEW so I decided to slow down with this project and open it to the community. I finished my working example and stopped for now.
      So if anyone is interested to play around with the code, see attached ZIP file (LV 2020).
      Cu,
      Peter
       
      MHT.zip
    • By GregSands
      The Parallel For Loop is perfect for parallel processing of an input array, and reassembling the results in the correct order, however this only works if the array is available before the loop starts.  There is no equivalent "Parallel While Loop" which might process a data stream - so what is the best architecture for doing this?
      In my case, I'm streaming image data from a camera via FPGA, acquiring 1MB every ~5ms - call this a "chunk" of data - and I know I will acquire N chunks (N could be 1000 or more).  I then want to process (compress) this data before writing to disk.  The compression time varies, but is longer than the acquisition time.  So I'd like to have a group of tasks which will each take chunks and return the results - however it's no longer guaranteed that the results are in the same order, so there's a bit of housekeeping to handle that.
      I have a workable architecture using channels, but I'd be interested in any better options.  Easiest to explain with a simplified code which mimics the real program:

      It requires the processing to use a Messenger channel (i.e. Queue) because a Stream channel cannot work in a Parallel For Loop, but this doesn't maintain order.  And the reordering is a little messy - perhaps could be tidied using Maps but I don't have 2019 at the moment.  The full image is too large to keep in memory (I'm restricted to 32-bit because the acquisition is from an FPGA card), so I need to process and write the data as it becomes available.  I've considered writing a separate file for each chunk, but writing millions of small files a day is not particularly efficient or sustainable.
      Is there a better approach?  Have I missed something?  I feel like this must be a solved problem, but I haven't come across an equivalent example. Could there be a Parallel Stream Channel which maintains ordering, or a Parallel While Loop which handles a defined number of tasks?
      Thanks.
      Greg
    • By Benoit
      Manufacturing a satellite or a simple pen require to test the quality of the product before delivery to the customer.
      LabVIEW is widely used for that purpose. Since 20 years of LabVIEW development I saw numerous test framework. I was wondering if people where interested to work in a collective and open source test framework.
      Per example the following feature can be included:
      HAL (hardware abstraction layer)
      Database to record test results with the data viewer (PostgreSQL)
      single/asynchronous/batch/debug mode
      multi-language support
      Image analysis (open CV) + bar code reader
      User access level
      Remote monitoring
      Jig identification to prevent user error (testing the wrong product with the wrong jig/test sequence)
      HTML/xml/txt report
      and so on....
      Benoit
    • By RedAndGreen
      I often have to create interfaces for power supplies. I often use the QMH for my overall architecture.
       
      I have created a QMH that controls multiple power supplies. It was designed in such a way that it takes a minute to add an additional power supply. Of course I am using Dynamic Dispatch to use multiple types of power supplies. It works quite well.
       
      I am trying to get better at LVOOP and plug-ins. I was thinking of creating an abstract class that contains a QMH for just one power supply. Of course I would create children for several types of power supplies. My plan with the single QMH for each power supply would be to use it as a plug-in or use a sub panel so that all the QMHs are on one UI.
       
      If anyone has any advice on the methodology please feel free to comment. Also I am new to LAVA so I hope I am not asking something that has already been solved.
       
       
    • By Fab
      Hi LAVA friends,
       
      We are pleased to announce that the Delacor Queued Message Handler (DQMH) is now available via the LabVIEW Tools Network:
       
      http://sine.ni.com/nips/cds/view/p/lang/en/nid/213286
       
      You can read about the thought process behind DQMH at our blog post: Ours is not better than yours (or YAF=Yet Another Framework) at WalkingTheWires.com
       
      The DQMH is based on the NI QMH Project Template. The DQMH expands on the NI QMH by providing safe, event-based message handling and scripting tools to encourage same style between different developers in the same project and improve efficiency. The DQMH is ideal for applications where multiple modules must run in parallel, possibly at different rates, while communicating with each other. The DQMH can also be used for applications that have a single module, where the developer would benefit from having a Tester with the capability of eavesdropping on the different DQMH events and messages. 
       
      The DQMH integrates with TestStand since all development and troubleshooting can take place under LabVIEW, while calling the public API VIs as individual steps in the TestStand sequence. The tester can eavesdrop during the execution of the TestStand sequence. This is specially useful when the LabVIEW code is written during the research and development or prototyping phase, because the same code can be called by TestStand in production or manufacturing sequences without any changes.
       
      The DQMH uses LVOOP for some internal functions, but developers need not be familiar with LVOOP to use or understand this architecture.
       
      Try it out and let's us know what you think.
       
      Fab and the Delacor team.
         
       
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.