LVSpeak 2.0 - Release Discussion

[Norm Kirchner]

I'm having the same experience that John had above. "remove and rewire" doesn't work, but everything else does. I'm not sure that Execute QEC VI is running. I opened it and turned on Retain Wire Values. When I speak "remove and rewire", I see the command appear in the LVSpeak applet, but nothing happens. The wires in the Execute QEC VI are all empty. (Granted I'm a beginner with LVOOP, so I'm not sure that dynamic dispatch isn't running another memory instance of the VI.)

Can we tap into other QD scripting shortcuts (e.g. Darin.K's add-on for "change to array")?

Can we tap into JKI RCF actions (e.g. "wire nodes by corners")?

-Jason

Jason, I'm really excited that you've taken to LVSpeak the way I hoped many would.

To address the questions you have, I've noticed the way remove and re-wire behaves, but hadn't taken the time to fully debug myself (not often used by myself). But the way you would debug is to invoke the command "Show Quick Edit" or right click the notification tray and use that same right click option. once open you should be able to put a breakpoint right before the dynamic dispatch call 'Execute QEC Command'. Once the program breaks at that point, 'step into' the QEC command and it should dispatch you to the BD of remove and rewire. You have to take this convoluted route because Quick Edit operates in the NI.LV.Editor context and the only way to debug in these contexts is to hack into a VI in non-traditional ways. Ideally, I should put some debugging code in there that properly reports any errors of a variety of styles.

WRT other editor assistants like QD shortcuts and JKI RCF plugins, you most definitely can tap into their IP. For QD Shortcuts, the process is as simple as making a new plugin that calls the QD shortcut VI like for remove and rewire and assigning it a speech command. Ideally, I would create a plugin that just scans that directory and assigns names based on one of their properties

For RCF, the process is a bit more involved, because there is not 1 predefined VI that the action code exists in. So you need to open the plugins, find the needed code in their framwork, and then plop that into the appropriate case for the Execute QEC command.

I'm not sure if all of that made sense to you, but please ping me back w/ other concerns or findings while you debug.

-Norm

[Bean]

Jason, I'm really excited that you've taken to LVSpeak the way I hoped many would.

To address the questions you have, I've noticed the way remove and re-wire behaves, but hadn't taken the time to fully debug myself (not often used by myself). But the way you would debug is to invoke the command "Show Quick Edit" or right click the notification tray and use that same right click option. once open you should be able to put a breakpoint right before the dynamic dispatch call 'Execute QEC Command'. Once the program breaks at that point, 'step into' the QEC command and it should dispatch you to the BD of remove and rewire. You have to take this convoluted route because Quick Edit operates in the NI.LV.Editor context and the only way to debug in these contexts is to hack into a VI in non-traditional ways. Ideally, I should put some debugging code in there that properly reports any errors of a variety of styles.

WRT other editor assistants like QD shortcuts and JKI RCF plugins, you most definitely can tap into their IP. For QD Shortcuts, the process is as simple as making a new plugin that calls the QD shortcut VI like for remove and rewire and assigning it a speech command. Ideally, I would create a plugin that just scans that directory and assigns names based on one of their properties

For RCF, the process is a bit more involved, because there is not 1 predefined VI that the action code exists in. So you need to open the plugins, find the needed code in their framwork, and then plop that into the appropriate case for the Execute QEC command.

I'm not sure if all of that made sense to you, but please ping me back w/ other concerns or findings while you debug.

-Norm

Norm-

Thanks for the help. Regarding "remove and rewire", I followed your instructions and probed around but don't see any errors going into or leaving R.vi. See below for a screen shot with error probe and probe for the VI refnum input. Is that VI refnum what R.vi is expecting?

-Jason

Edited April 14, 2010 by Bean

[Norm Kirchner]

Norm-

Thanks for the help. Regarding "remove and rewire", I followed your instructions and probed around but don't see any errors going into or leaving R.vi. See below for a screen shot with error probe and probe for the VI refnum input. Is that VI refnum what R.vi is expecting?

-Jason

Your probing the right place, but the reference is invalid,

Because of the nature of working within a tool that tries to figure out what is currently selected/active/hovered over, you must not interact w/ the VI your debugging in the process.

So set the probe in the same manner, and most likely a breakpoint as well, go back to the VI of interest that you're attempting to operate on and then perform the operation.

The Execute QEC should breakpoint and the probe be correct. Take a picture of that, and if it still is not functioning properly then we have identified the source of the bug

*****

EDIT

*****

Ok, the source of the bug is known and I thought I already dealt with it actually. Guess I inadvertently undid a fix which is funny due to the nature of the bug. What is happening is that I already setup an 'UNDO' session in the first VI in the QEC command, and there is another 'UNDO' session being set within the remove and rewire VI.

If you haven't figured it out by now, that is a no-no.

The fix, that I'll roll into this, is to artificially end the undo session before calling remove and rewire using a transaction.end undo method before you call 'remove and rewire'

I just tested this out on my PC at home and that fixed the issue.

What will need to happen is that either I make a generic plug-in for all QuickDrop shortcuts that addresses this first. or make a sub-class of QEC Command that is specific for QD shortcuts, that all QD shortcut LVS implementations use.

Thoughts?

Did you try the fix?

[Bean]

What will need to happen is that either I make a generic plug-in for all QuickDrop shortcuts that addresses this first. or make a sub-class of QEC Command that is specific for QD shortcuts, that all QD shortcut LVS implementations use.

Thoughts?

Did you try the fix?

Hey Norm-

Resolved.

The following maybe what you meant above...

Regarding the implementation, the thought that comes to mind would be to have a "QEC - Quick Drop" sub-class of the "QuickEdit Command" class. Then, inside "QuickEdit Command.lvclass:Execute QEC" VI, have a subVI that runs as a no-op unless dispatched by the "QEC - Quick Drop" sub-class. When dispatched by "QEC - Quick Drop", you could run the "Transaction.End Undo" method. In the future, this allows for expansion, where you might have a "QEC - Right Click Framework" sub-class (of the "QuickEdit Command" class) that requires invoking other methods/functions before dispatching the "QEC - Right Click Framework.lvclass:Execute QEC" VI. Essentially, when accessing other scripting APIs, you would have the ability to dispatch custom routines based on their class if you group them accordingly.

(I guess I'm making the assumption that the "Transaction.End Undo" method is necessary for both FP and BD. If not, then you would need it in every BD case within the "QEC - Quick Drop.lvclass:Execute QEC" VI. And, therefore, dispatching another subVI from the "QuickEdit Command.lvclass:Execute QEC" VI, at least for Quick Drop, would be unnecessary.)

At first, placing the Quick Drop commands in their respective Plugins categories [e.g. "remove and rewire" in the Replace (QEC - Replace.lvclass) category] seems organizationally appealing, but I wonder if segregating them doesn't make the code more logical. This would also allow us to quickly tell which Quick Drop shortcuts, Right Click Framework add-ons, etc. are present.

Aside: How do I place down a "Transaction.End Undo" method without copying it from some other diagram? I may not have installed VI Scripting after re-rolling my PC, so that probably explains it.

*thanks*

-Jason

Edited April 16, 2010 by Bean

[Norm Kirchner]

Jason

You are indeed correct and apparently in phase and frequency locked with the way my brain is spinning. Well done.

Yes, I'm thinking that making unique sub-classes for QD and RCF plugins is in order to address these exact kind of issues.

WRT the 'end undo' this is just a byproduct of how Darren set guidelines for ppl to create QD shortcuts (explicitly handle the undo action in the scripting VI rather than the framework doing it for you). This is a totally valid stance and at the end of the day, the cat is still bald. So regardless of if it's FP or BD, if the person who created that plugin, took into consideration Darrens guidelines, we can presume that all QD shortcuts will need to prematurely end the QEC undo operation so the QD transaction can begin.

You would find the 'end undo' within the parent call of Execute QEC.vi

<a href="http://content.screencast.com/users/NJKirchner/folders/Jing/media/96c8d298-b581-40a4-9eba-017a8429d87d/2010-04-16_1027.png"><img'>http://content.screencast.com/users/NJKirchner/folders/Jing/media/96c8d298-b581-40a4-9eba-017a8429d87d/2010-04-16_1027.png"><img class="embeddedObject" src="http://content.screencast.com/users/NJKirchner/folders/Jing/media/96c8d298-b581-40a4-9eba-017a8429d87d/2010-04-16_1027.png" width="371" height="290" border="0" /></a>

If you didn't notice, I actually call the parent class twice, once at the beginning and another at the end. The nice thing about this design, is that now when it comes time to make a QDS QEC class, we can structure these parent level calls to indeed call {QEC::Execute -> QDS::Execute (which will end undo) -> QDS_SHORTCUT::Execute (Specific Shortcut via enum case within )

Making one for QD will be pretty darn easy. RCF I imagine will be much more difficult because of determining if/how we can circumvent the framework and just execute the needed code. At this point, it's a time and resource issue for me. But it needs to be done at some point so we can converge our methodologies.

[Bean]

Thanks to your example, I've seamlessly integrated some of my favorite QD shortcuts in the "QEC - Replace" class. (Nice work auto-populating the command list).

You are indeed correct and apparently in phase and frequency locked with the way my brain is spinning. Well done.

Glad to hear we're sync'ed up. I mentioned before that I'm a beginner with LVOOP, so thinking through this *small* aspect LVSpeak has been a timely challenge for me. It's good to see a real-world example demonstrating the benefits of dynamic dispatch.

Making one for QD will be pretty darn easy. RCF I imagine will be much more difficult because of determining if/how we can circumvent the framework and just execute the needed code. At this point, it's a time and resource issue for me. But it needs to be done at some point so we can converge our methodologies.

I could be dreaming here, but because I REALLY value efficiency (ref. interest in LVS, reason I studied ISE) and in the spirit of code reuse...

[***EDIT*** « SHOULD BE A LIGHT BULB EMOTICON HERE]

maybe if "we" (Norm, Darren, Jim) put our our minds together, their could be a common interface for LVS, QDS, RCF, etc. to call worker VIs. Worker VIs = VIs that do the placing, deleting, organizing... (e.g Align left edges.vi, Move labels to side and justify text.vi, Go to pizzahut.com.vi, Wire nodes by corners.vi, etc.)

This way, users could access the same scripting features they enjoy using the method they chose [voice (LVS), keyboard (QD), mouse (RCF)] all the while making scripting developers more productive (ref. adage about 1 small rock and 3 fowl).

Edited April 16, 2010 by Bean

[Bean]

maybe if "we" (Norm, Darren, Jim) put our our minds together, their could be a common interface for LVS, QDS, RCF, etc. to call worker VIs. Worker VIs = VIs that do the placing, deleting, organizing... (e.g Align left edges.vi, Move labels to side and justify text.vi, Go to pizzahut.com.vi, Wire nodes by corners.vi, etc.)

So, with a CSI (Common Scripting Interface) / USI (Universal Scripting Interface), if folks preferred (after R&D implements the following), they could just as easily use their LVSpeak, QD, RCF add-ons with Darren's Plug-in VIs for Right-Click Menu Options. There are probably already discussions going on about having a CSI elsewhere, but I just came across that Plug-in VI idea by Darren again, and wondered: Do we really need to have 4 different implementations for each scripting tool?

[Norm Kirchner]

<br />So, with a CSI (Common Scripting Interface) / USI (Universal Scripting Interface), if folks preferred (after R&D implements the following), they could just as easily use their LVSpeak, QD, RCF add-ons with Darren's <a href='http://labviewartisan.blogspot.com/2008/12/plug-in-vis-for-right-click-menu.html' class='bbc_url' title='External link' rel='nofollow external'>Plug-in VIs for Right-Click Menu Options</a>. There are probably already discussions going on about having a CSI elsewhere, but I just came across that Plug-in VI idea by Darren again, and wondered: Do we really need to have 4 different implementations for each scripting tool?<br />

<br /><br /><br />

Once again, you're spot on.

I've spoken with both Jim and Darren and we're all in agreement (almost)

Each of our frameworks implement the same flow (get input from user -> execute a specific VI), but the specific differences make things a little more complicated (not impossible though) and just need more thought put into.

A great example is that QD has a default user field that people can type into. How does that flow work into QD or LVS?

another example is that RCF has an 'Options' ability and cascaded menus. How does that flow work into QD or LVS?

But before you start to try to answer those questions (already have some thoughts on it) realize this;

LVSpeak is not 'just' a speech recognition engine... well technically it is, but the LVS package also includes Quick Edit.

IMHO: Quick Edit is intended to be this central unifying engine that enables users to invoke a VI through whatever mean they so choose (speech, QuickDrop typing, right clicking). Because of it's plug-in and object oriented approach each of these enabling technologies can take advantage of it in the same manner that LVS has.

This does not mean that it has all capabilities ready to rock and roll to ensure no loss of functionality for the other methods, but at the heart of the issue QD shortcuts are a hack on the QD window. And RCF, although very feature rich, has the meat of the code that does the work 'very' tightly integrated within the framework.

So what is needed (and I feel QD as is, is a good step in this direction) is an engine that JoeLVUser can tap into to execute VIs to help his development, Scripting, ordering pizza, or just dropping structure.

Would you agree with this from what you've seen?

[Bean]

<br /><br /><br />

Once again, you're spot on.

I've spoken with both Jim and Darren and we're all in agreement (almost)

Each of our frameworks implement the same flow (get input from user -> execute a specific VI), but the specific differences make things a little more complicated (not impossible though) and just need more thought put into.

A great example is that QD has a default user field that people can type into. How does that flow work into QD or LVS?

another example is that RCF has an 'Options' ability and cascaded menus. How does that flow work into QD or LVS?

But before you start to try to answer those questions (already have some thoughts on it) realize this;

LVSpeak is not 'just' a speech recognition engine... well technically it is, but the LVS package also includes Quick Edit.

IMHO: Quick Edit is intended to be this central unifying engine that enables users to invoke a VI through whatever mean they so choose (speech, QuickDrop typing, right clicking). Because of it's plug-in and object oriented approach each of these enabling technologies can take advantage of it in the same manner that LVS has.

This does not mean that it has all capabilities ready to rock and roll to ensure no loss of functionality for the other methods, but at the heart of the issue QD shortcuts are a hack on the QD window. And RCF, although very feature rich, has the meat of the code that does the work 'very' tightly integrated within the framework.

So what is needed (and I feel QD as is, is a good step in this direction) is an engine that JoeLVUser can tap into to execute VIs to help his development, Scripting, ordering pizza, or just dropping structure.

Would you agree with this from what you've seen?

Here's the hierarchy of actions that I'm visualizing:

User Input [Ctrl-Space (QD), Hold Ctrl key (LVS), Ctrl-Alt (as I've configured RCF)]

>>Framework [convert user input into worker VI inputs and call respective worker VI]

>>>>Worker VI* [one worker VI per action]

>>Framework [cleanup]

User Input [if needed, e.g. for LVS, release Ctrl key]

*Worker VIs = VIs that do the placing, deleting, organizing... (e.g Align left edges.vi, Move labels to side and justify text.vi, Go to pizzahut.com.vi, Wire nodes by corners.vi, etc.)

So, the framework (QD, LVS, RCF) would be responsible for specifying the user interface (media, GUI, etc.), processing / differentiating user input to pass as inputs to the worker VIs, and cleaning up after the worker VI runs.

This could require some decoupling / re-tooling from the various frameworks in order to standardize "what" the worker should and should not be responsible for (e.g. Transaction.End Undo). I don't know that it's necessary to standardize on a common framework or even subsection of the framework (e.g. Quick Edit).

Certain worker VIs are going to have inputs that are unsupported by some frameworks. For instance, in QD we can change VI server class based on what a user types. So, QD allows passing an action with data. I don't know that it's imperative that LVS, RCF, etc. support action + data worker VI calls. And, that's okay because QD fills that void. So that class / type of worker VI would not be supported by all frameworks BUT could still be a sub-class of some workerVI.lvclass.

Q: Can we assume that folks writing scripting worker VIs are familiar with LVOOP? If not, "we" (DN, NK, JK...) could specify worker VI templates without LVOOP (e.g. "action", "action + data", etc.).

I know that was abstract, but maybe there's something in there that confirms / challenges what you were already thinking or leads to some completely new idea...

Edited April 20, 2010 by Bean

[Bean]

Norm-

I added "scalar to array" and "array to scalar" as we've discussed earlier via the QD interface (Ctrl-A created by Darin.K). It works when I highlight a terminal and speak "scalar to array" on the diagram, but not on the front panel. I added the code to the "BD" and "FP" cases in the QEC - Replace.lvclass:Execute QEC VI. See below for a screen shot of QuickEdit Command.lvclass:Execute QEC VI, which is reporting FP.IsFrontmost = False. I've highlighted the FP control before speaking "scalar to array", so the FP of my VI should be frontmost; however, the reference that I've probed (going into the FP.IsFrontmost method) points to the QuickEdit Command.lvclass:Execute QEC VI (not my VI).

Any ideas as to why this would work on the diagram but not the panel? (QD works fine either place.)

Thanks...

-Jason

[Norm Kirchner]

Norm-

I added "scalar to array" and "array to scalar" as we've discussed earlier via the QD interface (Ctrl-A created by Darin.K). It works when I highlight a terminal and speak "scalar to array" on the diagram, but not on the front panel. I added the code to the "BD" and "FP" cases in the QEC - Replace.lvclass:Execute QEC VI. See below for a screen shot of QuickEdit Command.lvclass:Execute QEC VI, which is reporting FP.IsFrontmost = False. I've highlighted the FP control before speaking "scalar to array", so the FP of my VI should be frontmost; however, the reference that I've probed (going into the FP.IsFrontmost method) points to the QuickEdit Command.lvclass:Execute QEC VI (not my VI).

Any ideas as to why this would work on the diagram but not the panel? (QD works fine either place.)

Thanks...

-Jason

I'll look into it, but it happens to be a command I already built before Darren's shortcut so I've never seen it not work. (I call mine pluralize, but I suppose his is more descriptive).>

>

I just looked at your image though, and you've caught the wrong reference. Unfortunately when debugging you have to be extra careful what is active when you invoke the command. As you have shown is the QEC execute as the active VI, which I assume to not be correct.

[Bean]

I'll look into it, but it happens to be a command I already built before Darren's shortcut so I've never seen it not work.

It's Darin.K's shortcut (not Darren's) [what's the chance of the 2 guys producing QD shortcuts having the same "name"?]

I call mine pluralize, but I suppose his is more descriptive

Well, his is just called "A.vi", but the nice thing about your implementation is that I get to name things according to the way that my mind works (and my my ability to enunciate).

I just looked at your image though, and you've caught the wrong reference. Unfortunately when debugging you have to be extra careful what is active when you invoke the command. As you have shown is the QEC execute as the active VI, which I assume to not be correct.

Yep. If I step into VIs until they're all open and then try the command again (without stepping), then I get the correct reference and no errors. Not sure why it's won't work for the panel, but other QD plugins also work on the diagram but not the panel (e.g past excel numeric, paste excel string, ...).

[rex]

I finally had some time to spend getting LVSpeak set up, after you blew my socks off at NI week with that little video they played one morning and I have to say this is simply the coolest community submission I have ever even heard of. I am a LVSpeak junkie already, this is so sweet and so fast.

I have been taking notes while using LVSpeak and I am compiling quite a wish list of commands I would love to see implemented. I might be able to implement a few myself if I knew how.

Is there any way you guys could post a video (like the other ones) on the basics of implementing a command, even if its just a basic overview I could start from? It was briefly mentioned in one of the videos but I would really like to make some custom command packages to post for everyone if I can figure out how. Just looking at the code I must say that you are far beyond my LV abilities, but that just means there is room for me to improve right? lol

Thank you so much for the massive amount of time and effort you put into this. I can't thank you enough and I look forward to updates and command packages posted by the community.

rex

[Bean]

Norm-

If I hold down the Ctrl key (when selecting items, when scrolling through cases, when I'm thinking, etc.), LVS will drop random functions. Is there a way to remap the keyboard "activate LVS" shortcut?

Thanks,

Jason

[Norm Kirchner]

Norm-

If I hold down the Ctrl key (when selecting items, when scrolling through cases, when I'm thinking, etc.), LVS will drop random functions. Is there a way to remap the keyboard "activate LVS" shortcut?

I've run into the same thing before and know what you mean.

To answer your question, definitely.

Unfortunately it's a hard code in the software at the moment, but it is indeed configurable.

Open the VI

C:\Program Files\National Instruments\LabVIEW 2009\project\_LVSpeak\ENGINE - LVSpeak.vi

Within that VI you will see a callback registration and some user parameters.

This is where you'll change your values

There will eventually be a re-architecture of this that will happen that will make this configuration easier to change, but for now this is the way it needs to be done.

[Bean]

It worked. For reference, I deleted the first item in the array, leaving the following image.

Now, when I press (Ctrl), nothing happens; however, when I press (') and (1) together, I can use LVSpeak.

Thanks Norm!

Edited May 13, 2010 by Bean

[Bean]

[Aside: Not sure why my previous image blew up after I posted it. If you click the image, you should see the correct size. This hasn't happened with previous posts.]

[asbo]

[Aside: Not sure why my previous image blew up after I posted it. If you click the image, you should see the correct size. This hasn't happened with previous posts.]

It's an on-going issue.

[Bean]

Thanks asbo...

[rex]

I really find LVSpeak useful. I use it every day and it rocks. I have been taking notes while I use it and I have come up with a wish list of features and functionality that I wish LVSpeak had. With respect, I submit my wishlist...

Voice Command Set Wish List

Block Diagram

1. Block diagram cleanup (implement both selective or clean-up all)
2. Show error list
3. Run VI, Save VI, New VI, make subvi (from selection or if nothing selected, from nothing)

UI Front Panel Commands

1. Align front panel items (all the align functionality of the block diagram!) like “vertical gap” and all the distribution commands
2. Convert all controls and indicators to system controls at once (“system style”)
3. Change colors of hovered target (common colors and customizable ones) transparent at least
4. Activate controls at runtime
5. Initialize to default value (all selected)

Improvements to LabVIEW Speak

Speak Interface

1. The option to select a different hotkey of my choice, right from the settings. I would like to pick one of the extra buttons on my mouse, for instance. I use the control key for a lot of other things in labview and windows. I don’t want it performing commands based on interpreted noise during that time.
2. Ability to search available commands (preferably intelligently) in the command list window. I would like to be able to click into a text box in the command list and type text to search. It should begin the search as I type, like the quick drop menu or a google search.
3. Labview Speak Panel – Slimmer and dockable to windows so I can dock it to the bottom of the VI I am working on. It doesn’t need to be a big box like that, I just need to see a small LED and the interpreted command on one line. Maybe you could select different window styles in settings?
   If labview speak was able to write to that status text on the bottom border of the VI, that would be the sweetest! Maybe change an icon to green or red when speak activated (like the current panel LED functionality).
4. The option to only press down the hotkey while speaking, rather than all the way until the requested action has been taken, would be very nice.

Speak Engine

I would like to not need to say the entire command for it to be able to place it. Rather be able to say the first few words of it and if those words didn’t match any other command at all to go ahead and inform me of the guess so I could quit speaking. If there were multiple vi’s with that command it could choose the first on its list or something. I am sure it would be wise to have this only apply to commands of 2 words or more, including the freaking fast drop stuff. This would work sort of like typing your search in google and it already having an idea of what you are trying to search for. I know that the Microsoft voice detection engine only analyzes your statement once there is a second pause so it can’t start guessing with each new word. What I am saying is sometimes I don’t remember the full name of a command I and I would like to be able to only say what I remember and it would get a little closer to getting the command right. For instance it would be nice to be able to say “open create” and it know that I meant “open/create/replace file.vi”. There seems to be some fill in the blank type functionality but usually it guesses the completely wrong function or nothing at all. Perhaps this ‘guessing’ could be a feature that you could enable or disable based on your preference.

[Mike Tan]

hi, everyone, i have installed the lvspeak2.0.5.1 to my labVIEW2010, however it is not working as expected.

I have downloaded and install the speechsdk51

How can i correct this prob?

FYI, I am doing an application in labview using speech to toggle switchs. Is there any sample labview codes related that i can reuse?

Edited January 10, 2011 by Mike Tan

[Norm Kirchner]

Mike,

Which version of windows are you running?

I'll have to validate it on my install of 2010.

Give me a few days, and be sure to reply w/ the windows version.

Rex,

I'm sorry I've neglected your post.

Most of the commands I've thought of, and you could implement easily as well.

Have you seen the option under the tools>>quick edit>>create new plugin...

I personally don't mind still using keys for anything that my left hand can do by itself. Most of my commands for shortcuts were motivated by needing to remove my hand from my mouse. (Including freaking fast drop)

I'll check back in soon w/ some updates (I hope)

~,~