LVOOP & the JIT compiler

[Daklu]

One of the benefits of using interfaces (or abstract classes) in code packages with text languages is that as long as you don't mess with the interface you won't have to recompile the code that depends on that package. Labview's just-in-time compiler allows us to ignore recompiles; however, hiding them under the covers also prevents me from learning how much JIT compiling impacts performance and exactly what circumstances require a recompile. In particular, the scenario I'm imagining is one with a large, multi-package reusable code base where there are various dependencies (inheritance, composition, references, etc.) between packages and the packages are released/updated on different schedules. I've run a few very simple tests, but I can't think of any way to test this at a large scale without actually building a a bunch of packages.

Along these lines, I have a few questions:

1. What's the best way to minimize the time spent doing JIT recompiles when starting an application that uses those packages?
2. Do we know which types of dependencies are 'clean' and which are 'dirty?' For example, package A can depend on package B, but if the only interaction they have is through a string-based messages it's unlikely changing B will trigger a recompile in A. On the other hand, if package A inherits from a class in package B, changes in B are more likely to require a recompile in A. What if A contains B as data? What if A contains a reference to B as data? What if A member vis have B constants on the block diagram? What if A member vis use B member vis on their block diagram?
3. Does JIT compiling have a practical impact on load times or am I splitting hairs?

Dave

Edited August 6, 2009 by Daklu

[Adam Kemp]

JIT refers to compiling right before executing starting from a partially compiled binary. For instance, .Net code compiles to a bytecode format (not directly to machine code), but instead of a bytecode interpreter they compile the bytecode right before you run it into machine code optimized for your processor. LabVIEW doesn't actually do this kind of JIT compiling. The LabVIEW runtime engine doesn't do any compiling whatsoever, so if you have an interface parent class and a bunch of implementation child classes, and all of those are compiled and saved, then you don't have to worry about the runtime engine trying to recompile them.

The only thing you might need to worry about is what we call "inplaceness". This is an optimization our compiler uses to allow wires to share the same place in memory, even if they pass through a node or subVI. For instance, if you have an array wire that you connect to a VI that just adds 1 to every array element, then it may be possible (depending on how you wrote it) for that subVI to use the exact same array as its caller without any copy being made.

Dynamic dispatching (and call by reference) complicates this a bit because it could turn out that the specific implementation you end up calling at runtime has difference inplaceness requirements than the one you compiled with. We do handle this at runtime if we find a mismatch, so it can add some overhead. I think some people solve this by always using an inplace element structure (even for your empty parent class implementation) for your dynamic dispatch methods where you really want a certain input/output to always be inplace. This just prevents the mismatch from occurring.

[Daklu]

The LabVIEW runtime engine doesn't do any compiling whatsoever, so if you have an interface parent class and a bunch of implementation child classes, and all of those are compiled and saved, then you don't have to worry about the runtime engine trying to recompile them.

There's something going on in the IDE I don't understand. Suppose I have a Base class, a Child class, a composite of the Base class, and a composite of the Child class. If I make a change in the base class that requires a recompile, such as adding a private data member, Base class and Child class objects on block diagrams are dimmed. I've always thought this essentially means Labview can't execute this code until that file is saved (or has its changes applied.) If that's not the case, what is the significance of dimmed objects? Dimming the child class suggests a tighter coupling between the Base and the Child than between the Base and the Composite Base.

The reason for the question is I have packages that will expose abstract classes which other packages will use to derive their own classes and objects. I can save a lot of duplicate code if I include some functionality and private data in the abstract class, but I don't want get bogged down in package-dependent recompiles when I change its private functionality. (This applies to both the runtime engine and the IDE.) I was considering making the exposed base classes purely abstract just to make sure I wouldn't have to change it, but it sounds like there's nothing to gain by doing so. Correct?

[Adam Kemp]

The dimming is showing that the class's private data control has changes which have not been applied. As you are editing the control we don't want to constantly be recompiling and messing with other VIs because you could be doing multiple things at once, changing your mind, etc. Instead we let you make your changes and then when you save the class, close the private data control window, or choose File->Apply Changes in the private data control window then we will update all the other VIs that need to be updated. While the class is in this intermediate state it's considered broken because we don't yet know whether your changes will break other VIs, so it makes no sense to allow you to run them yet. Once you apply the changes then we can do a real check to see if anything broke, and if not then those constants stop being dimmed.

None of this is JIT, though. This is all compile-time stuff. The child class is only broken because the parent class is broken, and the parent is only broken because it is in this intermediate state. As soon as you apply the changes then the parent class becomes unbroken, and thus the child class becomes unbroken as well. As long as you end up with a good (non-broken) parent class then nothing you change in the parent's private data will cause a recompile of the child class VIs. They don't even have to be in memory when you make the change, and they won't notice if they come into memory after the fact.

The only thing you really need to worry about with editing the parent class is making sure that the child classes still meet all the requirements (i.e., dynamic dispatch VIs have the same connector pane). If you change the connector pane of a dynamic dispatch VI then you definitely have to modify your child classes.

[Daklu]

Thanks for the detailed responses Adam. Letting your answers percolate for a few weeks helped clarify thing for me, but of course also raised a few new questions. To put the questions in context, I'm currently in the early phases of designing internal frameworks that will be distributed to other Labview developers for creating applications to test our devices. I'm planning on deploying the components as multiple OpenG packages and these questions come up when I consider updating a package without having the application source code loaded into memory. Also, since these are under source control I'd prefer not to have to checkout and resave a bunch of vis when updating a package.

1. In the .lvclass file there is a property named "NI.LVClass.FlattenedPrivateDataCTL." What information is actually stored in Composite (Base).lvclass when a Base.lvclass object is included as private data? Given that Composite (Base).lvclass doesn't need to be resaved when Base.lvclass changes, it implies that Composite (Base).lvclass is in fact storing a link to Base.lvclass rather than a representation of the object (and data) itself. What happens if I use a Base object with non-default data as private data in Composite (Base).lvclass? Composite (Base).lvclass needs to maintain a representation of the data, right? Suppose I go back later and delete a private data member from Base.lvclass that is no longer needed. How does Composite (Base).lvclass handle this?

2. If my package exposes a typedeffed cluster for clients to use, what changes can I make to the typedef in the package without breaking (either broken run arrow or unexpected runtime behavior) client code? Obviously deleting items could break clients. What about renaming items? Reordering? Renaming and reordering?

3. How does the answer to Q2 change with respect to enums rather than clusters? (My gut is that renaming or adding would be okay but reordering would be bad.)

JIT refers to compiling right before executing starting from a partially compiled binary. For instance, .Net code compiles to a bytecode format (not directly to machine code), but instead of a bytecode interpreter they compile the bytecode right before you run it into machine code optimized for your processor.

I was using 'JIT' to refer to the background compiling that happens in the Labview dev environment. What do you call that process? Precompiling? A level 1 compiler? On large projects I've seen a several second delay between hitting the run button and having the application actually start, so I don't think it compiles directly to machine code.

[Aristos Queue]

I was using 'JIT' to refer to the background compiling that happens in the Labview dev environment. What do you call that process? Precompiling? A level 1 compiler? On large projects I've seen a several second delay between hitting the run button and having the application actually start, so I don't think it compiles directly to machine code.

That's just plain old compiling. And that's what we call it. :-) The fact that you as user don't have to explicitly invoke it is a service on our part, but the compilation happens before you run, which is exactly when it happens in every other compiled programming language.

[Daklu]

That's just plain old compiling. And that's what we call it. :-)

So the background compiler generates machine code on the fly in a single step and stores it in memory? Or does it compile down to function calls for the runtime engine to translate into os specific commands?

The fact that you as user don't have to explicitly invoke it is a service on our part

Ohhhhh, so that's why Labivew is so expensive. (I'll gladly use a compile button if it knocks a few thousand dollars off the price.)

[Aristos Queue]

So the background compiler generates machine code on the fly in a single step and stores it in memory? Or does it compile down to function calls for the runtime engine to translate into os specific commands?

When you write a C++ program, you write it in some editor (MSVC++, XCode, emacs, Notepad, Textedit, etc). Then you compile it. If you're tools are really disjoint, you use gcc to compile directly on the command line. If you have an Integrated Development Environment (XCode, MSVC++), you hit a key to explicitly compile. Now, in MSVC++, you can hit F7 to compile and then hit F5 to run. OR you can hit F5 to run, in which case MSVC++ will compile first and then, if the compile is successful, it will run. All of this is apparent to the programmer because the compilation takes a lot of time.

There's a bit of hand-waving in the following, but I've tried to be accurate... The compilation process can be broken down as

1. Parsing (analyzing the text of each .cpp file to create a tree of commands from the flat string)
2. Compiling (translation of each parse tree into assembly instructions and saving that as a .o file)
3. Linking (taking the assembly instructions from several individual .o files and combining them into a single .exe file, with jump instructions patched with addresses for the various subroutine calls)
4. Optimizing (looking over the entire .exe file and removing parts that were duplicate among the various .o files, among many many many more optimizations)

LabVIEW is a compiled language, but our programmers never sit and wait 30 minutes between fixing their last wire and seeing their code run. Why do you not see this time sink in LabVIEW?

1. Parsing time = 0. LabVIEW has no text to parse. The tree of graphics is our initial command tree. We keep this tree up to date whenever you modify any aspect of the block diagram. We have to... otherwise you wouldn't have an error list window that was continuously updated... like C++, you'd only get error feedback when you actually tried to run. C# and MSVC# does much the same "always parsed" work that LV does. But they still pay a big parse penalty at load time.
2. Compile time = same as C++, but this is a really fast step in any language, believe it or not. LabVIEW translates the initial command tree into a more optimized tree, iteratively, applying different transforms, until we arrive at assembly instructions.
3. Linking time = not sure how ours compares to C++.
4. Optimizing time = 0 in the development environment. We compile each VI to stand on its own, to be called by any caller VI. We don't optimize across the entire VI Hierarchy in the dev environment. Big optimizations are only done when you build an EXE, because that's when we know the finite set of conditions under which your VIs will be called.

[Adam Kemp]

Suppose I go back later and delete a private data member from Base.lvclass that is no longer needed. How does Composite (Base).lvclass handle this?

Magic. Actually, the .lvclass file also contains a history of edits you've made so that it can automatically mutate any old data on load. It's mostly transparent to the developer, and it makes mixing versions of classes much easier than in other languages. The only time you'll run into problems is if you have an older version of the class in memory and then later load some VIs that have data from a later version. I'm not sure exactly what happens in that case, but I think it would fail to load the VI.

2. If my package exposes a typedeffed cluster for clients to use, what changes can I make to the typedef in the package without breaking (either broken run arrow or unexpected runtime behavior) client code? Obviously deleting items could break clients. What about renaming items? Reordering? Renaming and reordering?

There are few if any changes you can make to a typedef that won't break your clients. With typedefs you pretty much have to have all the VIs that use that typedef in memory when you make the change, and then you have to save those VIs after making the change. Classes are far superior for this use case. In fact, that's the main advantage to using classes. It's called encapsulation: hiding the details of the inside of the class so that the clients won't notice when those details change.

3. How does the answer to Q2 change with respect to enums rather than clusters? (My gut is that renaming or adding would be okay but reordering would be bad.)

With enums I think you can add to the end of the enum, but removing or renaming an existing item will break the clients. If you really need to do something that would break a client then you could introduce a new type and a new VI which takes that type, and then deprecate the old type and the old VI. Rewrite the old VI to convert the old type into the new type and then forward the call to the new VI.

I was using 'JIT' to refer to the background compiling that happens in the Labview dev environment. What do you call that process? Precompiling? A level 1 compiler? On large projects I've seen a several second delay between hitting the run button and having the application actually start, so I don't think it compiles directly to machine code.

Aristos explained this decently. We do compile directly to machine code, but if you're making edits then we have to recompile next time you run. Once we've done that, though, there won't be any delay next time you run it (unless you make more edits). If you then save those VIs or build an executable then next time they're loaded or when the .exe runs there won't be any compiling going on. They'll just run.

So the background compiler generates machine code on the fly in a single step and stores it in memory? Or does it compile down to function calls for the runtime engine to translate into os specific commands?

We do compile and store in memory, but we also save the compiled code in your VI so that we don't have to recompile again next time you load. The one caveat to that is that sometimes changing a subVI causes the callers to need to recompile, so you might get a prompt to save a VI you never changed directly. That's because we recompiled that VI to adapt to the changes in its subVI(s), and we want to save that new code so you don't have to recompile again next time you load.

As I mentioned before, dynamic dispatch VIs (and call by reference) do extra work at runtime in case the VI you're calling changed inplaceness, so that's a case where you don't need to worry as much about breaking callers. You just have to keep the possible performance impact in mind.

Also, we compile directly to machine code and calls to the runtime engine. For simple functions we just compile machine code, but sometimes it's easier and more efficient to just compile machine code to call a function we wrote in C++. That function will be in the runtime engine. Almost all compilers do that, including MSVC and GCC. That's why they also need runtime libraries.

[Daklu]

It's been said many times but I'll repeat it again... it sure is nice to have NI insiders responding to questions and comments on public forums. It leads to a level of understanding that otherwise would take years to achieve. Thanks.

Magic. Actually, the .lvclass file also contains a history of edits you've made so that it can automatically mutate any old data on load. It's mostly transparent to the developer, and it makes mixing versions of classes much easier than in other languages. The only time you'll run into problems is if you have an older version of the class in memory and then later load some VIs that have data from a later version. I'm not sure exactly what happens in that case, but I think it would fail to load the VI.

I like magic. So much of the world can be explained with it.

I don't think I've run into this problem yet, but I can see how it might happen when deploying reusable code packages to an application subdirectory instead of a common shared directory.

There are few if any changes you can make to a typedef that won't break your clients.

Got it. Typedefs within a package = good; typedefs across package boundaries = bad. (Unless you are 100%-for-sure-guaranteeing-cross-my-heart-hope-to-die-swear-on-my-mother's-grave that it won't change later.)

With enums I think you can add to the end of the enum, but removing or renaming an existing item will break the clients. If you really need to do something that would break a client...

I've been more concerned with a developer accidentally breaking a client with a future change. The specific example I was thinking of with the question of enums was the factory pattern example with Orange and Blue classes. Using an enum as the class selector makes it easier for the developer using the factory, but it also seems like a prime spot for bugs to appear. LV developers are used to being able to change enums and having the changes propogate through all the code.

Good unit testing is certainly needed but I'd prefer to prevent the errors from making it into the code in the first place. I'm tempted to limit cross-package data to natural data types (numbers, strings, bools, etc.) and objects. Hopefully that would remove the temptation to make "easy" changes. I'm not convinced the added developer overhead in maintaining string processing is worth it in the long run though.

Aristos explained this decently. We do compile directly to machine code...

If you don't mind my asking, how do you manage to allow us to step through the code if it's primarily machine code? VB6 was interpreted so I can understand how they managed that. C#/VB.NET are compiled to IL so I can see how they could do it there too. There's a big gap in my knowledge between high level programming languages (LV, C#, VB) and my low level hardware knowledge (68HC11 & AVR microcontrollers) so if it's too much to explain don't worry about it.

The one caveat to that is that sometimes changing a subVI causes the callers to need to recompile, so you might get a prompt to save a VI you never changed directly. That's because we recompiled that VI to adapt to the changes in its subVI(s), and we want to save that new code so you don't have to recompile again next time you load.

And the changes that trigger that might be...?

[Adam Kemp]

I've been more concerned with a developer accidentally breaking a client with a future change. The specific example I was thinking of with the question of enums was the factory pattern example with Orange and Blue classes. Using an enum as the class selector makes it easier for the developer using the factory, but it also seems like a prime spot for bugs to appear. LV developers are used to being able to change enums and having the changes propogate through all the code.

If you're using typedefs for your enums then the rules are pretty much the same: they work well within a project (we can update the uses of that typedef automatically), but they can cause problems for VIs that aren't in memory when the change is made. I still think you're ok if you just add to the end, though.

This is tricky in any language, really. An enum is basically a symbolic representation of a numeric value. If you change the definition of that enum such that the symbols map to different values than they did before then you may break existing code. It's just one of the things you have to keep in mind when writing a reusable API.

If you don't mind my asking, how do you manage to allow us to step through the code if it's primarily machine code? VB6 was interpreted so I can understand how they managed that. C#/VB.NET are compiled to IL so I can see how they could do it there too. There's a big gap in my knowledge between high level programming languages (LV, C#, VB) and my low level hardware knowledge (68HC11 & AVR microcontrollers) so if it's too much to explain don't worry about it.

More magic. When debugging is enabled we emit a little extra code between nodes that handles things like probes and breakpoints. We could do what C debuggers do (use the processor's breakpoint mechanism), but that's difficult to do in a cross-platform way. Our current mechanism allows us to do the same thing on all platforms, and even allows us to do remote debugging and probes with the same mechanism. It's just more flexible.

And the changes that trigger (a recompile of a caller after changing a subVI) might be...?

We try to optimize the callers based on what we can get away with toavoid copies of data, but that means that when things change in thesubVI then it sometimes affects the callers. For instance, if you have an input and output terminal of the same type and they weren't "inplace" before (meaning the compiler couldn't use the same spot in memory for both) but they are now, then the caller may need to change. Or it could be the opposite (they were inplace, but now they're not). It could also be that an input was modified inside the subVI before but now it's not (or the other way around). If you use dynamic dispatch or call by reference then you're somewhat shielded from these concerns (we can adapt at runtime), but you lose some of those optimizations. You may end up with copies of data that aren't strictly necessary.