Jump to content

Techniques for componentizing code


Recommended Posts

Thanks so much for the insightful, inspiring write-up! Plenty of food for thought, I'd say :)

I rather like the idea of wrapping messages with method VIs, to ensure data validity. It does seem a bit cumbersome, but I guess that it may well be a lifesaver in bigger projects. I'll have to consider it...

One small question: what's the data type of your queues?

Link to comment

Just curious.......

In terms of code metrics. How much is framework and how much actually does something in the physical world (e.g read/writes from an device, updates a display, plays Mozart on a xylophone etc)

Any program needs it's infrastructure (of course). I'm just wondering what the trade-off is (you don't want to write 100 VIs just to turn an LED on/off for example).

Link to comment

This answer was posted before I posted the question.

Really? You were just asking yourself, "I wonder what crazy idea Dave has cooked up recently?" :lol:

[Comment regarding the similarity to the Actor framework he recently published seems to have been edited out.]

Yep, this is similar to the Actor framework. It's the result of a conglomeration of ideas I've picked up from various places, and the many discussions I've had with you has been a primary source of those ideas. I don't think the slave loop concept is revolutionary, but for me it is an evolutionary step forward in my code.

I view the slave loop object as an intermediate step between a naked loop and an actor object. Since there's no dynamic instantiation the slave loop is easier to develop and debug than an actor object. It's also easier to understand than my prior actor object implementations. One thing I really like about this pattern is that if the time comes when I need to convert it to an actor, I can do that by adding a single DynamicLaunch method to the class and replacing the ExecutionLoop method with the DynamicLaunch method. None of the messaging needs to change.

One small question: what's the data type of your queues?

The queue data type is Message.lvclass; however, the "queue" wire you see is not a queue refnum. It's the class wire for MessageQueue.lvclass. (More details below.)

Daklu: Am I correct in guessing that "Output Queue" is a class that specifically only exposes a Dequeue method and "Input Queue" is a class that specifically exposes only an Enqueue method? You don't explicitly call that out in your text, but it seems like that's what you're doing.

Nope. I have a generalized object-based messaging library that is a foundational component of all my code. (Link to a copy that is very similiar to what is shown--right down to the icons.) [Edit - Ahh, I think I understand the potential confusion. InputQ and OutputQ aren't the names of the classes--they are labels signifying how each of the objects are used within the class. Both objects are instances of the MessageQueue class.] MessageLibrary contains a handful of core classes:

MessageQueue.lvclass - This is essentially a wrapper for the queue prims with a couple exceptions. First, you can't define the data type. The primitive queue data type is always Message.lvclass. Second, the DequeueMessage and PreviewMessage methods contain logic to output special messages if there's an error on the input terminal ("QueueErrorInMessage",) if the queue prim generates an error ("QueueErrorMessage",) and if the primitive times out ("QueueTimeoutMessage".) Both DequeueMessage and PreviewMessage output the message name (wired into the case structure) and the message object for retrieving the message data.

Message.lvclass - This is a base class from which all messages inherit. It is for sending messages that don't contain any data. It contains a single property ("Name" - string) and two methods: CreateMessage and Get MessageName. Here's how they look on the BD.

post-7603-0-49160900-1297443107_thumb.pn

Get MessageName is a static dispatch method so child classes cannot override it. It's an accessor with some extra logic. If the user defined a message name when the object was created, Get MessageName returns that string. If a name was not defined, Get MessageName returns the name of the class as the message name. Here's the BD.

post-7603-0-43855900-1297443469_thumb.pn

ErrorMessage.lvclass - This is a Message subclass that contains an error cluster and two methods: Create ErrorMessage (with a Name input) and Get ErrorCluster. It's part of the framework because it is the object type returned by the DequeueMessage and PreviewMessage methods when they deliver QueueErrorMessages and QueueErrorInMessages. I also use it extensively for transmitting error information around my apps.

Originally I subclassed Message.lvclass for every message I wanted to send and did not provide the ability to give the messages names (aside from Message.lvclass, which always needed a message name.) The strict typing and class-based message name meant I didn't have to worry about matching the message name string in the sending vi with the message handling case strings in the receiving loop. I just had to make sure the message handling case strings in the receiving loop matched the class names in the project window. It was much easier for me to verify the code was correct.

In practice I found that to be quite cumbersome during development and confusing, especially as messages propogate up and down through application layers. I've recently added some additional capabilities and add-on libraries that give up some type safety in exchange for simplifying the dev process. Unfortuately I haven't rolled them into the released package yet.

How much is framework and how much actually does something in the physical world.

What metric should I use? Number of vis? Bytes of hard drive space? Block diagram area? ;)

As for "doing something in the real world," I assume you mean "actually helps meet the functional requirements?" There's a lot of processing taking place that doesn't have a physical aspect but is still required to meet the functional requirements.

Link to comment

Really? You were just asking yourself, "I wonder what crazy idea Dave has cooked up recently?" :lol:

I was contemplating about producer/consumer design pattern and LVOOP.

Well, it's a real world app I'm coding in LV7.1, so I don't really have OOP. I use type def'ed clusters and accessor VIs, organization 'by-class' in folders and Tree.vi, marking Accessors as 'private' by the Icon. Pretty primitive, no inheritance and dynamic dispatching. But as this is all self-made rules to mimic some of the advantages of OOP, I need to do a lot of thinking about such design decisions.

Now the code is simple DAQ/presentation. A super-cluster/über-class (think of it as a composition of objects for all kind of settings made for the measurement) is coming into that VI. Then it's 'split' in a 'method' of that super-cluster/über-class. The super-cluster/class continues to the DAQ/producer loop with all the settings for the actual DAQ measurement and the signal processing (might be split into more loops later).

A kind of factory method is creating a new object that contains the display-settings and the pre-initialized data. This goes down into the consumer loop. At this stage, I also create the P/C queue and it goes into both clusters/classes.

To compare this to your implementation:

* I use a factory method to create the slave-classes. This is where I pass the queue (in this case I also create it) and other information (*), the queue is invisible on the top level.

* As this is a pure data P/C and not a message passing, I stop the consumer by destroying the queue.

* Because the consumer is writing to an indicator, the 'loop' is top-level, but only contains a single VI+the indicator.

(*) I'm unsure how this will evolve. Maybe all the settings needed in the consumer are just a pseudo-class on their own.

But queue and those settings would be a nested-classifier only shared between the über-class and the consumer class.

Now, just some thinking I had about this design:

The main concern is merging the data from producer and consumer. The measurement report will need the settings from the über-class and and the measurement data. So several options I have:

* Merge both loops clusters, kind of brute-force.

* Modify the 'Send data' method to both call an (private) accessor of the über-class and hold the measurement data in both wires. Then I just 'kill' the consumer/display object.

* Just put everything into the über-class. Then I call a parallelize method that creates the queue, branch the wire in both loops. Combine with above method. When the producer/consumer part is finished, actually both classes should be equal. Then I can reconsider everything when moving onto the next task.

So, I was really planning to post the question here.

But part of this projects challenges is, that I always get distracted by something more pressing. So it's hunting me since more than 5 years. And of course, each time I rewrite all from scratch because I know a much better way of doing it. :frusty:

Felix

Link to comment

* I use a factory method to create the slave-classes.

Does each of your slave classes have it's own factory method? If so, then I think we do the same thing but just give them different names. In my mind, if a method instantiates the class it's a member of, I call it a creator. It is (again in my mind--no idea if this is universally true) a "factory" if it instantiates and returns an instance of a different class. Here's a factory method I have implemented as part of a factory class. This particular factory method correlates data from a bunch of different sources and creates a GutStep object based on that data.

post-7603-0-22319800-1297461838_thumb.pn

The main concern is merging the data from producer and consumer. The measurement report will need the settings from the über-class and and the measurement data. So several options I have:

"Producer" and "consumer" don't have to be fixed attributes of a specific loop. They can be roles. Furthermore, the roles can change at various stages of your application. Nearly all my loops both produce messages for other loops and consume messages from other loops. What's preventing you from adding a 'receive' message queue to your producer loop and having your consumer loop send it the required information?

Any chance you could throw up a block diagram?

Link to comment

Does each of your slave classes have it's own factory method? If so, then I think we do the same thing but just give them different names. In my mind, if a method instantiates instances of the class it's a member of, I call it a creator. It is (again in my mind--no idea if this is universally true) a "factory" if it instantiates and returns an instance of a different class. Here's a factory method I have implemented as part of a factory class. This particular factory method correlates data from a bunch of different sources and creates a GutStep object based on that data.

Yes, that's how I do it. Creator is a better term than factory in this case.

The creator-object knows (directly or indirectly) all information that is needed to initalize the object-to-be-created. Using a create-method, there is no need to expose this information publicly.

"Producer" and "consumer" don't have to be fixed attributes of a specific loop. They can be roles. Furthermore, the roles can change at various stages of your application. Nearly all my loops both produce messages for other loops and consume messages from other loops. What's preventing you from adding a 'receive' message queue to your producer loop and having your consumer loop send it the required information?

Interesting point. I guess, to use 'roles', I'd need inheritance.

But the requirements are really simple for that section of code. Aquire->Analyze/Process->Present. So it could even be done with 3 VIs wired as a chain inside a single loop (for now, aquire and process are still together).

There is no need to call back to the aquisition loop.

But the code needs to scale well and be modular, as I have to implement more features (from single channel-measurement to multi-channel measurement, this will lead to n Process-loops; changes in the Aquisition code should not affect processing).

Any chance you could throw up a block diagram?

I can make a screenshot the next time I develope on the target machine. But it's just prototype-code that gives us a running setup where I and others (non-software people) can see the progress of our work/test whatever we do. Other modules are not clean-coded yet, but just a fast hack on top of the low-level drivers.

Felix

Link to comment

Daklu, any chance for you to attach your code instead of just images?

I'm planning on putting up SlaveLoopTemplate.lvlib (you know... the uninteresting part) soon, but unfortunately I can't post the code from the real-world apps. I did convert the Timer Loop shown above into a generic slave loop. I'll see if I can dig it up and post it next week if you'd like.

Interesting point. I guess, to use 'roles', I'd need inheritance.

Nope. You're overthinking it. The 'producer' is simply the loop that sends the information. The 'consumer' is the loop that receives the information. All it means is that (almost) every loop has both a receive queue and one or more send queues.

But the requirements are really simple for that section of code. Aquire->Analyze/Process->Present.

But the code needs to scale well and be modular, as I have to implement more features (from single channel-measurement to multi-channel measurement, this will lead to n Process-loops; changes in the Aquisition code should not affect processing).

Interesting problem. Why n process loops? Is each channel analyzed differently?

Link to comment

Interesting problem. Why n process loops? Is each channel analyzed differently?

The DUT is excited by n-channels. I only measure a single response signal.

Each of the excitation channels is modulated by a unique frequency.

Each of those n processing loops will filter out the response at one of these frequencies. Each of these channels will be also presented seperatly.

Ahh, things will even be more complicated, as I measure other channels as well, do the same kind of processing.

With all this data, now I'll do some physical calculations. Now I get some more data sets to present, both n-channel and 1-channel. :wacko:

But I have a clear directional data flow, as opposed to a messageing system where the objects talk in both directions. So certain stages of the processing could be done better by traditional procedural chain wireing of vi's with some for-loops.

But then again this is tricky when it comes to the filters, as they need to keep the history of the signal. So I'll need to keep their state seperately for the different channels. Well, nothing I really want think too much about sunday morning.

Felix

Link to comment

The DUT is excited by n-channels. I only measure a single response signal.

Each of the excitation channels is modulated by a unique frequency.

Each of those n processing loops will filter out the response at one of these frequencies. Each of these channels will be also presented seperatly.

Ahh, things will even be more complicated, as I measure other channels as well, do the same kind of processing.

With all this data, now I'll do some physical calculations. Now I get some more data sets to present, both n-channel and 1-channel. :wacko:

But I have a clear directional data flow, as opposed to a messageing system where the objects talk in both directions. So certain stages of the processing could be done better by traditional procedural chain wireing of vi's with some for-loops.

But then again this is tricky when it comes to the filters, as they need to keep the history of the signal. So I'll need to keep their state seperately for the different channels. Well, nothing I really want think too much about sunday morning.

Felix

Thats fairly painless with a database.

Log all the raw data to the DB. Make your filter re-entrant (notch/bandpass filter?).and just operate it on Selects from the database as many times as you like - in parallel with different settings. Any channel, any number of data points, any frequency - 1 re-entrant filter (simple case of course). Compare channels, cross correlate,,,you name it. You also don't need to keep state information or bung up your memory with a huge history.

Just a thought.

Link to comment

Thats fairly painless with a database.

Log all the raw data to the DB. Make your filter re-entrant (notch/bandpass filter?).and just operate it on Selects from the database as many times as you like - in parallel with different settings. Any channel, any number of data points, any frequency - 1 re-entrant filter (simple case of course). Compare channels, cross correlate,,,you name it. You also don't need to keep state information or bung up your memory with a huge history.

Just a thought.

Thanks for the input, but I'm unsure if this will work.

There is no written specs for this app, and I even plan to use in in a whole family of setups. That's why I dedicate a layer to the 'componentizing'.

One worst case estimate would be running the meaurement over the weekend at 500kS/s. With processing I get an estimate 1S/s (from literature, my processing is better, so I'll certainly hit other limitations; well they can be overcome as well and then ....).

But a hybrid solution could work, do the signal pre-processing with producer-consumer pattern and push the reduced data into the DB. Then do the physical calculations as a post-processing. The more I think about this, the more I like it to use a DB... I'll add it to the feature list.

Felix

Link to comment

Thanks for the input, but I'm unsure if this will work.

There is no written specs for this app, and I even plan to use in in a whole family of setups. That's why I dedicate a layer to the 'componentizing'.

One worst case estimate would be running the meaurement over the weekend at 500kS/s. With processing I get an estimate 1S/s (from literature, my processing is better, so I'll certainly hit other limitations; well they can be overcome as well and then ....).

But a hybrid solution could work, do the signal pre-processing with producer-consumer pattern and push the reduced data into the DB. Then do the physical calculations as a post-processing. The more I think about this, the more I like it to use a DB... I'll add it to the feature list.

Felix

I don't want to hijack Daklus thread.(too important).

I have to do acquisition stuff all the time. I used to do the classic acquire-process-display or producer-consumer. Now I just run acquisitions daemons direct to the DB and completely decouple the acquisition from the processing and reporting. Much more flexible and means you can swap reporting modules in and out (even while live acquisition is taking place) without having to manage the transitions or worrying about filling queues, memory or synchronising. However, direct streaming is only feasible up to about. 500 samples per second, reliably, without (as you rightly say) some intermediary processing and buffering. But that is part of the daemon so you don't lose any of the benefits.

I think your proposal would work extremely well since you can decide how much processing you want to put on either side of the divide (it could be staged and split).

Link to comment
  • 2 weeks later...

Just ran into some error handling issues and reviewed this document.

How do you handle errors? Specifically, when in the master loop an error occures, I would guess the ExitLoop-Message won't be send.

My own problem was that an error in my producer loop did make the Accessor VI for the queue return a 'not a refnum'-const so the Destroy Queue didn't stop the consumer.

Felix

Link to comment

How do you handle errors? Specifically, when in the master loop an error occures, I would guess the ExitLoop-Message won't be send.

Here's how I approach error handling... (hopefully I can explain in coherently.)

I imagine each loop as a separate entity in my app. Each of these loops has a specific responsibility: mediator, data producer, data processor, etc. Then, within the context of that loop's responsibilities, I figure out how I want that loop to respond to errors. I try to make each loop self-recover as much as possible. Sometimes that means the loop will "fix" the error on it's own. (Got a bad filename for saving data? Create a temporary file name instead.) Lots of times the loop simply reports the error and goes into a known, predefined state, after which the app user can retry the process that caused the error.

I know that's a pretty vague description. Here are some more specifics. They all tie together into a system that works well for me, but it's past the stupid hour and I can't figure out how to lay it out all nice and neat...

--- a. All my loops support bi-directional communication. A single queue for receiving messages (input queue) and at least one queue for sending messages to other loops (output queue(s)).

--- b. The preferred way to stop loops is by sending it an "ExitLoop" message.

--- c. A loop automatically exits if it's input queue is dead. (Can't receive any more messages. Fatal error... Time to stop.)

--- d. The loop automatically exits if it's output queue(s) is dead. (Can't send any more messages. Fatal error... Time to stop.)

--- e. The loop always sends a "LoopExited" message on its output queue(s) when it exits to let whoever is listening know. If the output queue is already dead the error is swallowed.

--- f. I don't connect a loop's error wire directly to the loop's stop terminal. I want the loop to stop when *I* tell it to stop, not because of some arbitrary unhandled error. (Cases c and d are safety nets.)

LapDog's messaging library (especially the DequeueMessage method) is an integral part of my error handling. In fact, having the error wrap around the shift register and go back into the DequeueMessage method's error in terminal is the main way errors get handled. DequeueMessage reads the error on the error in terminal, packages the error cluster in an ErrorMessage object, and instead of dequeuing the next message it ejects a message named "QueueErrorInMessage." In that message handling case I'll check the error and decide what action to take.

When an owning loop gets a message that a slave loop has exited, it decides whether it can continue operating correctly without the slave. If so, it will make adjustments and continue. If not, it shuts down its other loops and exits too.

My own problem was that an error in my producer loop did make the Accessor VI for the queue return a 'not a refnum'-const so the Destroy Queue didn't stop the consumer.

Hmm... do you mean to say the prior error on the queue accessor's error in terminal triggered the error case, which in turn returned 'not a refnum' and left the queue active?

Link to comment

First, I've ditched the rule of wrapping all sub-vi code in an error case structure. (Blasphemy! Burn me at the stake!) About 9 months ago AQ and I had a discussion about this...

Hey Daklu,

This thread has been great. I am starting to use AQ's Actor Framework with success on my systems. My derived classes add some features that probably break the proof of stability, but it is a risk that I accept. One of the latest things I did was to not propagate an error up into 'ActorCore.vi'. Once that happens, the Actor is dead.

That is not why I am posting however. Your last two posts on Error handling, I think, deserve a thread of its own. Those coming into the top of this thread for the first time may never get to this gem of insight. I can start it if you want.

-kugr

Link to comment

Kugr! Dropping by for your bi-yearly check in? ;)

Ha Ha! :D

Yup, I am pretty much a voyeur on this this site. I do enjoy your contributions though.

I am not full time on LabVIEW programming but I do want to get a common framework that I can use on systems here. AQ's ActorFramework fit that bill for me with some enhancements on the derived Actor classes. For instance, my Actors have a name and know their receive queue - sometimes an actor will send itself a message. Actors also retain the queues of Observers. It seems the intent of the ActorFramework (especially for stability) was that an Actor that created another Actor was the only one that could communicate with that new Actor. AQ can correct me here but that was my take. There was nothing inherent in the framework that restricted this, it was just my understanding. I did not like that restriction and I generally allow a full network of messaging, though I do have the concept of retaining what Actors another created.

The biggest change to AQ's framework was to make the Message:Do.vi re-entrant. Within an actor, the queue will keep processing to one message at a time. I was OK to have all actors processing messages in parallel though this does make debugging crazy at times. And I just added queue names to make my debugging a bit easier - now I can see the name of the actor who sent the message. Oh, yes, all of my messages allow the sender to add it's reply queue. Generally the Do.vi only gets the creator's queue, but sometimes I want to reply directly back. This is an added property in my derived message.

I have added an ActorFacade class that can be derived to hide the nuances of all of the messages. A message is essentially wrapped in a method of an ActorFacade. The ActorFacade is not an actor though and cannot receive messages. It is usually contained within an Observer. The ActorFacade actually becomes very useful in updating UI controls via CtlRefnums. So it is a bit more than just a Facade with respect to a design pattern, but I am not sure what to call it.

I also have ManagerActor (creates new actors via a message), a UIActor (an easy way to put a user interface on an actor), and a PollingActor (for those times when I need to watch for something to change that does not generate events).

From there the derivations continue to my specific requirements. I deal with moving fluids so there are generally valves, pressures and temperatures to deal with. They boil down to analog and digital IO.

Not sure this was very helpful but it is what I am up to.

I will set the thread up shortly for Error Handling.

-kugr

Edited by kugr
Link to comment
  • 7 months later...

I have a question. What is the best way to handle multiple instances of a slave? Specifically I want to write a slave loop that does communications with an RS232 device but I have at least two of these devices. What I am not clear on is how best to handle slave responses in my mediator loop. The slaves will send messages on the output queue but how do I differentiate between the slaves so I know who (which instance) sent what?

One thing I was thinking is to maybe give the slave loop a name property. The name would be set when the slave is created and become part of the response message name. Cases in my mediator would be something like:

  • SerialDevice:DeviceA:Response
  • SerialDevice:DeviceA:Error
  • SerialDevice:DeviceB:Response
  • SerialDevice:DeviceB:Error

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.