LVOOP messaging scalibility

[Yair]

Consider the following issue:

You want to pass messages of different types. Different messages may have parameters of different types associated with them.

There are any number of ways people do this. For example:

• Flatten the data to a string and unflatten it on the other side based on the actual message you got (kind of similar to what the JKI state machine does).
• Do the same, but use a variant.
• Have a dynamically registered event for each message type. Then, each message is handled in a different frame of the event structures and get the specific data type it needs.
• Have a class for every message type, build the message as an object and send that object to the message handler. When the message handler gets a message, it dynamically dispatches to the "process message" VI.

I like the last option, but I was wondering about its performance and usability. What happens if you want to have 100 message types? Do you want to have 100 classes in your project? Do you want to load the classes at run-time?

Stephen, I guess this question is mainly for you and also has to do with the general performance of having many classes. What's the largest number of classes (dynamic and non-dynamic loading) you know of?

[Aristos Queue]

I like the last option, but I was wondering about its performance and usability. What happens if you want to have 100 message types? Do you want to have 100 classes in your project? Do you want to load the classes at run-time?

I like the last option also, yes, even at 100 classes. In fact... I'll probably say something more about it in about two weeks.

I guess this question is mainly for you and also has to do with the general performance of having many classes. What's the largest number of classes (dynamic and non-dynamic loading) you know of?

Let me see...

• Most classes in one app that I've seen: 116. This is an NI product... I think about 25% are dynamically loaded. I have to work with this hierarchy on a near daily basis.
• Most classes in one app that I've seen from a user: 109 in the Endevo GOOP Toolkit (but come to think of it, you might not consider all the things installed by the installer to be all one app)
• Most classes in one app that I've seen from a user definitely all one app: 66
• App Builder: 34
• Getting Started Window: 19

[PaulL]

We do exactly what you suggest (essentially an implementation of the Command Pattern), even over a network (flattening the object to strings, which means both senders and receivers must share the same class definitions, of course, so version control is crucial), and it works quite well. I haven't counted the class definitions in our hierarchy but a quick survey shows there are about 100.

[Yair]

In fact... I'll probably say something more about it in about two weeks.

'K, I guess I'm gonna have to wait.

At the moment I don't have something I need which has a lot of messages, but I just wanted to check.

[Norm Kirchner]

'K, I guess I'm gonna have to wait.

At the moment I don't have something I need which has a lot of messages, but I just wanted to check.

I have used this technique before which requires a new class for each command, but the added benefit is the ability to make sub-class tree of commands which comes in handy when dealing with a ton of commands.

I've found that it's really helpful if you define naming and file organization decisions before or early into the process of creating commands since making changes en-masse is a pain.

But other than making sure both sender and receiver are working on the same version it's a sound tactic.

The max number of commands I worked with was 17 and no issues. I was sending them across VI server using a Call By Ref node so there wasn't even any flattening to string needed.

Good Luck

~,~

Norm

[Aristos Queue]

We do exactly what you suggest (essentially an implementation of the Command Pattern), even over a network (flattening the object to strings, which means both senders and receivers must share the same class definitions, of course, so version control is crucial), and it works quite well. I haven't counted the class definitions in our hierarchy but a quick survey shows there are about 100.

Just a note: The receiver can have a later version of the class than the sender... LV classes include in their flat representation the info for mutating the flat data into future formats. We don't have support for flattening to older versions, so the communication is one-way. For some applications, that's acceptable. But if you need two-way communication, Paul is correct.

[huotom]

I think that Queue may be useful.

[Michael Aivaliotis]

I think that Queue may be useful.

That's very insightful.

[Yair]

That's very insightful.

You're kinder than I am.

houtom, the issue is not passing the data, but decoding it when you have different types and I don't need a solution.

Just to clarify, my concern was not really with run-time performance. I was wondering more about edit-time behavior and whether having a large number of classes in the project would be fast and reasonably easy to use.

[Yair]

Here's an example of this sort of thing. I'm assuming the classes there are more complex than what I'm thinking about, but we can't know unless Daklu steps in. In any case, this is exactly the sort of thing I'm concerned about.

[Aristos Queue]

Here's an example of this sort of thing. I'm assuming the classes there are more complex than what I'm thinking about, but we can't know unless Daklu steps in. In any case, this is exactly the sort of thing I'm concerned about.

I just posted a reply in the ni.com thread.

[Aristos Queue]

When I said in my previous post that I'd comment more about it in a couple weeks, this is what I meant.

[Daklu]

Here's an example of this sort of thing. I'm assuming the classes there are more complex than what I'm thinking about, but we can't know unless Daklu steps in. In any case, this is exactly the sort of thing I'm concerned about.

I've been debugging the issue in that thread over the past couple days and I discovered there is a device driver on my computer that is starting interrupt storms. When they hit LV slows to a crawl and completely locks up during certain operations. (Save all, mass compile, right clicking the project window when lots of vis are selected, etc.) The only way to stop the storm is to put the computer to sleep and wake it up again. That gives me anywhere from 1 minute to 2 hours of unhindered programming.

If I load the project when my computer is storm free it takes ~3 minutes, including LV startup time. This is more in line with what I would expect. However, it is still longer than I like and I prefer using subprojects to manage the amount of code that's loaded up at any one time.

I just posted a reply in the ni.com thread.

AQ, in your response you said, "All library files listed in the project are full loaded into memory when the project loads. VIs that are listed by the project are not..."

If non-library vis in a project aren't loaded when the project is loaded, how does LV populate the Dependencies section with its non-project sub vis? I just ran the following experiment:

1. Start a new project. Create a main vi. Create a sub vi and put it on the main vi's bd.
2. Remove the sub vi from the project.
3. Save and close everything.
4. Check the .lvproj file with a text editor. The sub vi shows up in the Dependencies section. Close the text editor.
5. Without opening the project, open the main vi. Create another sub vi and add it to the main vi.
6. Save and close everything.
7. Open the project with Labview. Sub VI 2 appears in the Dependencies section.
8. Open the .lvproj file with a text editor. Sub VI 2 does not show up in the Dependenies section.

LV is discovering Sub VI 2 is a dependency at the time the project is loaded, so there *must* be more going on than just checking to see if the files listed in the .lvproj file are at the specified location in the file system.

[Ton Plomp]

LV is discovering Sub VI 2 is a dependency at the time the project is loaded, so there *must* be more going on than just checking to see if the files listed in the .lvproj file are at the specified location in the file system.

I think LabVIEW started automatically filling the dependencies in LabVIEW 8.5, in earlier versions (at least 8.0) you had to hit the 'Update Dependencies' button.

Ton

[Daklu]

I think LabVIEW started automatically filling the dependencies in LabVIEW 8.5, in earlier versions (at least 8.0) you had to hit the 'Update Dependencies' button.

Ton

But the question is how is LV discovering the dependencies. It's doing more than just reading the dependencies from the .lvproj file, even for those files that it's not supposed to be loading.

In any case, this is exactly the sort of thing I'm concerned about.

I just did some quick experiments with dynamically calling classes and if you're concerned about the number of classes in your project that's the way to go. The Factory Pattern example is an excellent starting point. The Orange class and Blue class can be removed from the project and they don't show up in dependencies, meaning they won't be loaded when the project is opened. Since the main app generally only uses vis from the parent class (Generic Plugin, in this case) you don't have to go hunting for vis from Orange and Blue.

On a large project with many classes I would have a main project that contains the top level vis and parent classes, but not the dynamically called child classes. To simplify child class development I would have a separate project file for each class hierarchy that contains the parent and all the child classes as part of the project. Using this kind of scheme would have greatly reduced our project load time.

[Yair]

If non-library vis in a project aren't loaded when the project is loaded, how does LV populate the Dependencies section with its non-project sub vis?

I'm assuming it uses the linker methods which read the info from the binary data of the VI without loading it into memory.

...if you're concerned about the number of classes in your project that's the way to go.

I was more concerned about the performance if you have a large number of small classes (since each would probably have only two methods) and about the ease of use of loading them dynamically. In any case, it's kind of theoretical, since I'm planning on doing anything like that at the moment.

[Daklu]

I'm assuming it uses the linker methods which read the info from the binary data of the VI without loading it into memory.

Ah ha, I don't know anything about the linker methods. I take it each vi has a table at a fixed offset containing all the sub vis it calls that allows the project to retrieve the information directly from disk?

[Aristos Queue]

If non-library vis in a project aren't loaded when the project is loaded, how does LV populate the Dependencies section with its non-project sub vis?

You may have heard of the scripting method "Read Linker Info"... people ask about it from time to time on the forums. The path information of a LV file can be extracted from the file without loading the VI itself into memory. The linker info for various VIs is cached between runs of LabVIEW to accelerate the population of the dependency information.