Script Engine - MView

[j_meier]

Hello there,

first post here, so perhaps I should start with introducing myself. I've used LabView for about 10 years. For work to hack together some quick automated test routines for instrument control and data acquisition. Outside work I use it like a kind of hobby program language, mainly just messing around with it to see how it works and what it can do. For many reasons (read the NI ads...) it has become one of my favourite program languages, next to Matlab.

I never released any code to the public. I never had formal training, and I thinks this shows in my code.

What I'm releasing here is something I worked out in the last few weeks as a hobby project, it's now at a point where I think it could actually become useful. I'm looking for comments and criticisms on the code and the general question if this is useful and should be completed.

What I tried to achieve is to add a script language to Labview. It should be a simple language, text-based, capable to interact with arbitrary front-panel objects, easy to add to an existing vi and working in the absence of the development environment. In difference to the MathScript or formula nodes it should be able to interact with anything on the front panel and not being limited to controls hardwired to the node. The script should be exchangeable at run-time, even when an executable has been built. Basically, it should be able to do everything a person can do with the front-panel.

My approach was to use the GOLD parsing system ( http://www.devincook.com/goldparser/ ) to create a language similar to Matlab, the Intel x86 Assembly Engine by Ege Madra ( http://www.devincook.com/goldparser/engine/assembly-x86/index.htm ) to parse the user created scripts, and the attached Labview code to make it work. Somehow. It's not finished by far.

What is working so far:

- basic math and logic operations

- comparisons

- variables (doubles, strings, 1D and 2D arrays)

- flow control (for loop, while loop, repeat until, if-elseif-else)

- reading and setting values on the front panel.

The attached zip file contains all vi's necessary to run what I have so far, inclusive dependencies (OpenG vi's and the Tree Control Toolkit by Norman J. Kirchner). A demo application is TestLoop.vi, there are three scripts in the script-folder that work with this application.

Flaws I'm aware of are that it's slow, verging on sluggish (I've used a recursive interpreter design pattern, and recursions are a weakness with Labview, eh?); that I changed it so often to get to this point that I would need to do a serious cleanup of the code; that there is no documentation and of course, IT'S NOT FINISHED.

Anyway, please take a look, and as said above, any sort of feedback is welcome.

Ciao,

Joachim

[jgcode]

The attached zip file contains all vi's necessary to run what I have so far

Hi Joachim

There are no attachments with your post.

Cheers

-JG

[j_meier]

Hi all,

Hi jgcode:

Been working on the attachment issue...

Had to disable all Add-ons in Firefox, now I see a "Attach This File" Button. Hope this works...

Will need th hunt down what this is in my browser.

Joachim

[j_meier]

Hi Neal,

Hi everyone:

thanks for your reply in general and for the positive response on top of it And sorry for the long time for me to come back to my own thread...summer vacation, and then all hell broke loose at work. I hope you DON'T know what I'm talking about.

..difficult to follow...

Yeah, after taking a long break working on it, I had the same problem.

...straight wires...

My big weakness: good planing. First there is so much space, and then it gets crammed, and then...curves, bends, this vi could still fit in that white space, now where to put this wires? etc. How do you fight this?

...icons...

Learned this now. I'm not good with graphics, so I'm just typing the vi-name in the icon. Helps a lot about readability.

..SVN...

Yes, I'm using TortoiseSVN. As I'm working alone on a hobby project, it's more of a backup system, but the most useful I know. I'm using the Labview Builder to export the project with it's dependencies, and it grabs the .svn folders when I'm embedding directories in the project. I'll ignore that for now

About the project: In the time I found to work on it some changes happened:

The tree-control did eat a lot of time. It's gone. It was nice to visualize the syntax tree, but...so much CPU time, for so little!

The recursion is gone. It was the second time-waster. Now the interpreter uses a stack. To implement this was fun, but the code is now totally unreadable. Better planning would have helped. To much trash is now left from the first version and then while I was figuring out how to do this some concept errors happened that are now not fixable. Without rewriting most of it again.

The grammar is almost fully implemented, the only important thing I put in the definition file for the parser that is not in the interpreter are the switch structure and the break/continue commands for flow control.

Ability to call subroutines (functions like in Matlab, and other scripts with a "call" command) works now. And there are predefined functions implemented in Labview in this version. See the file "internal functions.m" in the "Scripts" folder for a list.

I recompiled Ege Madras parser engine with a higher limit for the token stack. Modified source code and compiled dll is in the attachment.

What is still missing: Cleaning up the code. Bug-fixes. Manual. Yes. A manual.

Anyway, code is attached, if I find the time I'll clean it and hopefully bring it to certified standard. Might be a while, though...

HTH,

HAND,

Joachim

[LogMAN]

Anyway, code is attached, if I find the time I'll clean it and hopefully bring it to certified standard. Might be a while, though...

Hi, j_meier

I think the MView.lvproj file is missing in your new build.

However, your code is just amazing!!!

first post here... ...never released any code to the public

Very efficient...

...straight wires...

My big weakness: good planing. First there is so much space, and then it gets crammed, and then...curves, bends, this vi could still fit in that white space, now where to put this wires? etc. How do you fight this?

It's true, caring large VIs is important and necessary.

If you need space on your block diagram, hold the 'Ctrl' key and use the left mouse button to make space. Release the mouse button to commit.

Carefull:

marking an rectangle will cause all code to move in 2D.

To make space in one direction, move your mouse until there is a straight line (almost not visible).

Tip: Use a dummy- VI for testing.

...icons...

Learned this now. I'm not good with graphics, so I'm just typing the vi-name in the icon. Helps a lot about readability.

Yes, it's hard to read and the VIs need some Artwork, but

the functionalities are amazing!!!

I suppose your code will be used in many projects from now on (Of course after some enhancements).

Cheers, LogMAN

Edited October 28, 2010 by LogMAN

[asbo]

My big weakness: good planing. First there is so much space, and then it gets crammed, and then...curves, bends, this vi could still fit in that white space, now where to put this wires? etc. How do you fight this?

Personally, I rarely have a 'plan' for my layout when I start, but often will do a bit of reorganization on-the-fly as I place nodes. That way, the clean-up is progressive and I'm not left with a big wire-y mess when the functionality is finally done. As LogMAN mentioned, you can select a rectangular area with the mouse (using the pointer tool) but you can also use Shift+Click to add or remove individual objects to the selection. Another handy shortcut I use is to hold Control and then click and drag; this expands the diagram from the point you begin your selection outward in the direction(s) you drag. You might have to give that one a try to get what I'm saying

[j_meier]

Hi, j_meier

I think the MView.lvproj file is missing in your new build.

You are right. And a test-vi is broken. Here they are:

Put them in the MView directory and it should work.

I also had to learn that some of the included demo-scripts don't work since my last round of ..ahhm.. bug-fixes.

I'm preparing a new release with this issues resolved.

However, your code is just amazing!!!

Thanks.

It's true, caring large VIs is important and necessary.

If you need space on your block diagram, hold the 'Ctrl' key and use the left mouse button to make space. Release the mouse button to commit.

Carefull:

marking an rectangle will cause all code to move in 2D.

To make space in one direction, move your mouse until there is a straight line (almost not visible).

Tip: Use a dummy- VI for testing.

Thanks to you and asbo for the tips. I knew already how to force empty space into a diagram. I have two problems with it: What you said - the movement off all other code. It makes it sometimes look worse. And in a vi like this interpreter, with its endless sequence of case-structures, should I force space into one of them, I have a larger number of mainly empty cases. Most of the time no issue, but if the vi grows over my screen-size, I stop. The cleaner way would be to make a sub-vi. But one little change still fits in, right? To avoid the work of making a new icon, wiring up the front panel, giving good names to the FPOs, etc. And then another fix is need...lets squeeze everything a little closer together, right?

[asbo]

Thanks to you and asbo for the tips. I knew already how to force empty space into a diagram. I have two problems with it: What you said - the movement off all other code. It makes it sometimes look worse. And in a vi like this interpreter, with its endless sequence of case-structures, should I force space into one of them, I have a larger number of mainly empty cases. Most of the time no issue, but if the vi grows over my screen-size, I stop. The cleaner way would be to make a sub-vi. But one little change still fits in, right? To avoid the work of making a new icon, wiring up the front panel, giving good names to the FPOs, etc. And then another fix is need...lets squeeze everything a little closer together, right?

I agree, in the interpreter situation, you're going to have loads of case structures. You're definitely right, a subVI is the proper way to do things and that's really what I suggest, especially since interpreters can be a little difficult to read. If the code is already written, you shouldn't need more than 5 minutes to create the subVI. Having to labels the wires will give you a chance to review your logic as well. Squishing down code is only a good choice when you know it's perfect.

[LogMAN]

You are right. And a test-vi is broken. Here they are:

Put them in the MView directory and it should work.

Thanks, it worked just fine.

Thanks to you and asbo for the tips. I knew already how to force empty space into a diagram. I have two problems with it: What you said - the movement off all other code. It makes it sometimes look worse. And in a vi like this interpreter, with its endless sequence of case-structures, should I force space into one of them, I have a larger number of mainly empty cases. Most of the time no issue, but if the vi grows over my screen-size, I stop. The cleaner way would be to make a sub-vi. But one little change still fits in, right? To avoid the work of making a new icon, wiring up the front panel, giving good names to the FPOs, etc. And then another fix is need...lets squeeze everything a little closer together, right?

I know exactly what you mean.

Using LabVIEW as IDE makes programming easy like playing a game.

But design in LabVIEW is as important as description in conventional programming languages.

A good way to clean programming is a design concept.

You have to define your programming design and use it in all your programms.

(Believe me, after some time it goes easier and easier)

Here are some concept ideas I use normaly. They are easy and fast to implement

after some training:

1. Do never create wires behind structures, you might search errors for hours!!! (Move wires above or below the structure.)

2. Use one connector pattern in all of your VIs (except of special VIs), with error in and error out connected to opposite connectors at the same place every time. (This will help you creating a "VI line" and using the "flow" concept of LabVIEW. (Even change the pattern of automatically created Sub-VIs.)

3. Do not move wires around like snakes. (I know it's hard at the beginning, so right clicking the wire and selecting "cleanup wire" helps )

4. Programming VIs, larger than your desktop should only expand in one direction (horizontal, vertical).

5. Use labels on your block diagram for documentation. (I normaly use a special color for documentation labels)

6. If any function might be usefull somwhere else, create a sub-VI. (VI icons should at least have text within)

7. Are you using parallel loops, larger than your desktop? Create seperate VIs for each of them.

8. Move inputs on the left side, outputs on the right side of your block diagram.

9. move labels of inputs on the left border, labels of outputs on the right border. (Implemented as option in LabVIEW 2010)

10. Do never lecture your concept ideas... aaah...

I know these concepts does not work always, but it's much easier to maintain your VIs, if you have a clean concept.

The "Right Click Framework" is also a good way to clean VIs (Thanks to all).

There are two snippets attached as sample for the concept. (First is original, second is changed)

Wow, my hands are burning... I'll need something to cool my head.

Cheers, LogMAN

[j_meier]

<snip Design-Guidelines>

Good tips. Now I only need to learn to do something like that CONSISTENT.

I'm weak, I'm weak, ... I dream of free-hand wires...

Just kidding.

The "Right Click Framework" is also a good way to clean VIs (Thanks to all).

Have to check this out.

There are two snippets attached as sample for the concept. (First is original, second is changed)

It's obviously better to read. Gosh, how do you sustain this during debugging/trying out when your not sure what will work best?

A little addendum to the project:

Included here is a modified version of the demo vi and reworked example scripts. The demo assumes that there is a loop handling the user interface, a loop doing some hardware stuff (I'm imitating a temperature controller), and one want's to include scripting.

I hope anyone who has experience with a text-based language can understand how MView works after reading through the example scripts.

And a LabView question:

The memory management of this code is...well...lets say not great. I wanted something quick and easy to implement, so I ended up with arrays in clusters in arrays, and I see problems with speed and expect problems with memory allocation because of the requirement to have arrays in continuous memory. For now I won't change it, but just a thought: Labview has an interface to it's memory manager for external code, with ways to dynamically allocate memory when needed. Is there a way to use this from within LabView without writing an extensive wrapper?

Ciao,

Joachim

[LogMAN]

It's obviously better to read. Gosh, how do you sustain this during debugging/trying out when your not sure what will work best?

It's not like my programs are perfectly designed at the beginning, but sometimes there are some functions I know I will use more often,

so I take the time to make it more readable!

The only thing I do always is correct wiring, because it makes it easier reading code after a day or so.

You know:

You can do 80% of work in 20% time, but you'll need 80% time getting the last 20% work done!

(I forgot who said this sentence first )

So, after finishing a program in LabVIEW, I go through the code writing it more readable and placing documentation all over my

VIs. It's hard work, but it worth the time.

However - Your code is much more readable as most of the code I have to read all the day.

Code like this: Caution: Bad code! is very common!

One thing that's really great with your project: The way you organize your VIs in your *lvproj.

So, don't worry and keep on programing, writing code more readable one step after another.

(Do one thing better each project - )

I hope anyone who has experience with a text-based language can understand how MView works after reading through the example scripts.

Your MView programming language is well designed, as far as I concern.

Even if I don't use your project as is, It's a great base concept for me to write a similar distribution!

I was going write a similar project like yours, but I'm to lazy for taking the time of a concept like your's.

So, thanks!

And a LabView question:

The memory management of this code is...well...lets say not great. I wanted something quick and easy to implement, so I ended up with arrays in clusters in arrays, and I see problems with speed and expect problems with memory allocation because of the requirement to have arrays in continuous memory. For now I won't change it, but just a thought: Labview has an interface to it's memory manager for external code, with ways to dynamically allocate memory when needed. Is there a way to use this from within LabView without writing an extensive wrapper?

I don't know much about memory management in LabVIEW, so it might worth the work of writing a new post (Interesting question).

You could write an external DLL in a conventional language, using it in LabVIEW as "Memory" (Not Recommanded)

You might try some of the 'In place element structure' functions (LabVIEW 2009 and 2010 are getting better with that step by step.)

One thing about your memory management just now:

It's an interesting way of thinking and writing something like heap and stack (pseudo, but however) in your project!

Nice to read and something to use somewhere else...

Thanks for that kind of idea too.

Greetings, LogMAN

[ShaunR]

Code like this: Caution: Bad code! is very common!

That's fantastic

Just goes to show.

"There are no bad programs, only bad programmers".

I think I'll set that up as my wallpaper....move over Grace Park

[Tim_S]

You know:

You can do 80% of work in 20% time, but you'll need 80% time getting the last 20% work done!

(I forgot who said this sentence first )

I prefer the 99 rule of project scheduling...

The first 90% of the project will take 90% of the time.

The last 10% of the project will take 90% of the time.

[asbo]

I prefer the 99 rule of project scheduling...

The first 90% of the project will take 90% of the time.

The last 10% of the project will take 90% of the time.

Sounds like an over-optimistic 80/20 to me

[LogMAN]

That's fantastic

Just goes to show.

"There are no bad programs, only bad programmers".

I think I'll set that up as my wallpaper....move over Grace Park

Great idea. I know some people who think if they don't see the canvas, the program is using the memory more efficient.

(I havn't yet proved that)

I should upload a wallpaper collection for you (slide show).

[sam]

I Tried looking at your code but there are some missing functions that I can not run it. Mainly You have used LabVIEW add-on toolkit that everyone may not be using.:

somewhere you have used Mathscripts: Looking at the scripts : can you use PureG or even "Formula Node".

There are some links to office toolkit.

Lastly, have you looked at Python, and more importantly LabPython? what is it that MView offers over-beyond LabPython?

I ask because I'm interested..

[j_meier]

Sam:

First, thanks for the feedback. Perhaps I should use this to give a feedback what happened to this projekt, considering that I posted first on June 17, 2010 and it's been now two years:

The big thing I intended to do did not happen - cleaning up the code. I did change some parts of it, but mainly to fix bugs and add some functionality, but it gets nowhere to the standard for a submission in the LAVA code repository. And extrapolating from the last two years this will not happen...

I can say that the tips some of the residents gave me two years ago have been helpful for my work. This is in general true for a lot of threads on the LavaG forum and the reason I follow the RSS feed. (That's why I saw your post!) Kudos to all the regulars!

I've attached the latest version I have, I would be happy if someone can use it. It is now for Labview 2011, and I had to delete the old atachments to stay below 10MByte - so the 8.6 version is gone.

MView.zip

This file does not contain the OpenG vi's I'm using!

Considering the trouble I had the last time to upload something useful, please let me know if you run in trouble opening the project.

About your questions:

I'm using the Report Generation Toolkit, so yes, if you don't have it the old version won't work. Sorry, not been thinking. In the version I uploaded I commented it out. I believe this will allow you to open and run everything. If you run in problems, the offending vi is a subvi in

MViewFunctions.vi

just delete the one disabled vi in the Default case.

The Mathscript node is totally removed - I hope. It's been a long time. IIRC, I only used it for the colon operator; if you want to know why I did this in the first version look in Colon.vi and at

http://www.mathworks...lution=1-4FLI96

Using the Mathscript node was an order of magnitude easier to do what Matlab does. After I got Labview 2011 I switched everything to formula nodes and changed the colon.vi to do what the m-file on the Mathworks website does.

About LabPython and similar scripting solutions like LuaView:

Yes, I know them. But my original objective was not to redo their work, but to learn something. I'm programming since I was 12 - 28 years ago - and in all this time I never learned how a compiler or interpreter works. One day I wanted to know, and reading books alone gets boring after some time, so some programming was needed. Next thing I did was cheating and using the GOLD Parser builder as lexer/parser. But the interpreter is mine. And it's Labview all the way.

When you come from the application side:

Cons:

- My code is slow compared to everything else.

- Handling different datatypes is pretty limited - it only knows bool, double and string arrays up to dimension 2 (i.e. scalars, row/column vectors and 2D arrays)

- Don't try to work with huge amounts of data, the internal memory management is bad - everything in a huge array of clusters of arrays... The ability to dynamically allocate memory would help here, but this is Labview. Love it or... (Hey, I just had an idea: Dynamically calling a vi that acts as buffer - similar to a functional global - but is reentrant, use the reference to the clones like a pointer in c...could mimic malloc... I have to think about this...)

- There is no such thing as a manual...just a few demoscripts. If you know any textual language, I'm sure you figure out what is going on.

- There is one lone developer who did this for fun, and has now found other fun things to do, and LabPython and Python, well, I heard some rumours that there is more than one.

Pros:

- You can change the script at runtime. The Labview Formula Node can't do that. LabPython can as long as you don't use the script node but the vi's. I'm not sure about LuaView.

- It can get and set the data from any front panel object that can be mapped in the above listed datatypes. AFAIK, LabPython needs to be feed with data with the Set/Get Data vi's, and those need to be hard-wired. I don't know how LuaVie does this. Anyway, everything MView needs is that the user knows the name of the FPO.

- If you understand the interpreter code, you can call native Labview vi's (Look into MViewFunctions.vi and it's sub-vi's - The absence of comments make it probably unreadable, but that's my fault.). Then it gets actually fast!

- I'm able to train someone who has a basic knowledge in textual programming within one hour to use this to script everything (well, almost ) an user sitting in front of a Labview application can do.

- Compared with LuaView it's cheap.

I hope this helps you, if you have more questions - if I can I'll answer.

[Rolf Kalbermatter]

Let me comment on some of these things

Full disclosure: I'm currently maintaining LuaVIEW and I'm the lone LabPython programmer, who did this in the first place to find out how the script node could be used by someone outside of NI. And once I had that, I realized that wrapping those functions into VIs would allow real dynamic access to the Python engine. At about the same time my collegue started to develop LuaVIEW for a rather large customer project. We had quite some fun arguing over if Lua or Python was the better language. While that view is a personal taste it is clear that Lua is a very much self contained and extremely compact scripting environment that is much easier to embed in other systems like LabVIEW. In fact Python, at that time at least, had no real intentions to actively support embedding of its engine into other environments. The API was there and it could be done, but the Python developer community was in general unresponsive to any suggestions of improvements in that part.

- You can change the script at runtime. The Labview Formula Node can't do that. LabPython can as long as you don't use the script node but the vi's. I'm not sure about LuaView.

Unlike LabPython LuaVIEW does NOT have a script node interface but only a VI interface that not only allows but in fact requires to pass a script at runtime.

- It can get and set the data from any front panel object that can be mapped in the above listed datatypes. AFAIK, LabPython needs to be feed with data with the Set/Get Data vi's, and those need to be hard-wired. I don't know how LuaVie does this. Anyway, everything MView needs is that the user knows the name of the FPO.

While LuaVIEW doesn't do that out of the box currently it would be not a to complicated project to develop that. But I'm not convinced about the need for that.

- Compared with LuaView it's cheap.

Aside that LuaVIEW is free for non commercial use, the initial purchase costs are usually the smallest parts of a projects cost. Any decent software programmer will incur the license costs of a commercial LuaVIEW license in two days of programming an alternative solution. Two days is very little time for such a thing as a scripting engine.

[j_meier]

Let me comment on some of these things

Full disclosure: I'm currently maintaining LuaVIEW and I'm the lone LabPython programmer, who did this in the first place to find out how the script node could be used by someone outside of NI.

....

Hi Rolf:

Thanks for taking the time to reply. Always good to hear the history of other software packages.

Just make one thing clear: I don't see MView as a competition to LuaView. I would advise anyone in a commercial environment to not use MView. I can not give support beyond posting here, and the moment you have to dig through the source code to make a small change or fix a bug you have lost any savings on the initial investment. On top of that I'm building on the work of two other "lone" projects (parser builder and the lexer/parser engine) which may become unusable in an other environment (think new version of the OpSys or Labview, 32/64 bit etc...)

While LuaVIEW doesn't do that out of the box currently it would be not a to complicated project to develop that. But I'm not convinced about the need for that.

The reason I like it is that it decouples the script from the Labview application - you do get full access to all features of the non-scripted application without work for the Labview developer. On the other hand, this can be a desaster when someone writes in FPOs nobody has a business to write (MView does no check if a FPO is an indicator or a control or if the control is disabled, it just tries to set the value via property node).

A second advantage is that the script can actually react to a changing FPO while it runs. Like watching for a FPO value to settle down and then continue. This changes the form of use from "Do something and give me the result" to "interact with the FPO and/or the hardware the FPO represents".

The two combined were the reasons I set it up that way. Something a LabView developer could just drop on the diagram and give his user the ability to use scripts within the final application. I never thought about something that resides inside the LabView application and is inaccessible to the end user.

Apart of that I have no clue how to make a script node. The more I think about that the more curious I get - do you have a description how you did this for LabPython somewhere?

[Rolf Kalbermatter]

Apart of that I have no clue how to make a script node. The more I think about that the more curious I get - do you have a description how you did this for LabPython somewhere?

All the documentation for that is in the C source code to the LabPython DLL in the sourceforge repository. But note that you can't get away without writing a very specific DLL. And LabPython does very tricky dynamic loading to allow separation of the actual LabPython core functionality from the scriptnode plugin. Without that you get into trouble since the LabVIEW VIs wouldn't search in the script node plugin for the DLL.

[gb119]

I too have played around with writing my own scripting language to drive my LabVIEW applications (a couple of times over even..). Generally I've cheated out of trying to write my own experssion parser and just content my selft with a call to another scripting language - my current version uses LabPython as a parser, but I used to use Perl. What I like about my homebrew versions is that it is easy(ier) to extend the scripting language - every command is a LabVIEW class, so it's relatively simple to make an existing application script aware by wrapping code to interact with it in new script commands. Having the language implemented in LabVIEW also means that the variable name space can be accessed from LabVIEW code too. Obviously it's slower than a 'native' environment, but in most cases I'm limited by waiting on instruments to do their stuff.

I think LuaVIEW can do the call backs to LabVIEW code, but if it was able to work with something like a variant with attributes and was able to suspend and resume after each line of script, then that would be really very nice...

[Rolf Kalbermatter]

I think LuaVIEW can do the call backs to LabVIEW code, but if it was able to work with something like a variant with attributes and was able to suspend and resume after each line of script, then that would be really very nice...

You are right that callback from Lua code to LabVIEW and vice versa is an integral part of LuaVIEW. However there is a limitation: You can't do that recursively at this time. This is due to the fact that call stack management across those borders is limited and can't be easily chained in a recursive matter, both because of very different parameter interfaces between LabVIEW and Lua, with LabVIEW not even having a classical call stack at all, but also because of complications in error unwinding when backing out of recursive call sequences. I have been looking into relaxing that limitations with a fully recursive call stack management, but the necessary effort is rather high in comparison to the benefits, and that feature also offers a very easy way to create overly complicated architectures that are bound to let a user shoot in its own feet.

Variant integration would be great but unfortunately the C side of the LabVIEW variants are not only undocumented but have also varied between different LabVIEW versions.