LVSpeak - tell LV what to do

[TobyD]

I have to say that I like the idea ever since I first saw Norm's demo of it last year, but I haven't used it all due to not wanting to install the speech recog utility or speaking all the time in the office.

I have found that I can speak very quietly and still get picked up. None of my co-workers have started calling me crazy yet, so I guess it's not bothering them.

My favorite part of this tool is the ability to quickly align code and labels in any number of ways. It saves so much time - especially when dealing with other people's code who might not be as fastidious as I am in keeping things pretty.

[Norm Kirchner]

I have found that I can speak very quietly and still get picked up. None of my co-workers have started calling me crazy yet, so I guess it's not bothering them.

My favorite part of this tool is the ability to quickly align code and labels in any number of ways. It saves so much time - especially when dealing with other people's code who might not be as fastidious as I am in keeping things pretty.

You're using it?!!!! Really!! I've got my first official user!!!! HELL YEAH! I'm going to give you a reputation point just for that .

Yup, the alignment is about the most used one, and then probably the quick drop integration.

I have to say that I like the idea ever since I first saw Norm's demo of it last year, but I haven't used it all due to not wanting to install the speech recog utility or speaking all the time in the office.

Another thing which I think is a major hinderance is the list of options. I think that what QE needs to really take off is a smart parser. This is certainly a tricky bit, but also important, due to the following improvements it could allow:

1. Writing less code for the framework. To demonstrate this, think of the command "Label visible". Today, it has its own specific VI.
   In the ideal version, the parser would recognize the structure, use the word "label" to get a reference to the label of the object you selected and then pass that reference into the "visible" VI (which was only implemented once, for the GObject class). That would mean that you only have to write the code for each action once. Now that you've written it, "visible" would work all objects, their labels, their captions, etc.
2. The parser could allow parameters. For example, "caption align left" would obtain the reference to the caption, then call the "align" action and pass "left" as a parameter. Immediately you have only one action instead of several.
3. Synonyms - imagine if you could say "label visible" or "visible label" or "show label" or "label show", etc. That would make it a lot more "natural", although I admit it's a secondary objective.

As I said, writing such a parser would be tricky. It's possible that maybe an initial version could use some explicit vocal separators to break the different parts of the text.

Inside of the MS SAPI we have the ability to do this, although it requires a better understanding of it and more testing that I never got to.

Actually Darren is hitting me up for this exact thing....

I just have a hard time seeing how much more that will simplify things on one side for the drawbacks I can see on the implementation side.

Can you give me 3 solid use cases, because the detect 'Align' then detect 'direction' will still require a unique piece of code for each direction, and the only thing it would make simpler, is that flat top level list of commands and it would make it hierarchical. Also, unfortunately not all properties are common in the VI Server Class structure. Like label is not a GObject item, and certain things require me to detect the specific type, or iterate through the valid ones (you might see this stuff in there)

[Yair]

Can you give me 3 solid use cases

I mentioned several potential improvements, so it's hard to say which one you're refering to, but here it is:

Caption.Align.Left

Label.Visible.

Label.Change to easy to read font.

Again, part of the issue is that with a parser, once you wrote the "change to easy to read font" VI, you can call it on any piece of text. The framework will need to be responsible for getting to the text part of the object.

the detect 'Align' then detect 'direction' will still require a unique piece of code for each direction

Why? If you know it's a parameter, just pass it as one into the align VI, which will attempt to cast it into one of its allowed parameters. For example:

Also, unfortunately not all properties are common in the VI Server Class structure. Like label is not a GObject item, and certain things require me to detect the specific type

The first part is correct, although in this specific case, label is of the Text class which does inherit from GObject. You're definitely correct that finding the right type might be tricky. As I said, it won't necessarily be simple, but I think it's required in order to make it really useful (as opposed to the very useful it is now).

[TobyD]

You're using it?!!!! Really!! I've got my first official user!!!! HELL YEAH! I'm going to give you a reputation point just for that .

Yup, the alignment is about the most used one, and then probably the quick drop integration.

I'm to the point that I get annoyed if I have to code without it. That, to me, is the sign of a good tool.

[Norm Kirchner]

I'm to the point that I get annoyed if I have to code without it. That, to me, is the sign of a good tool.

But have you used 'Get Pizza' yet

[TobyD]

But have you used 'Get Pizza' yet

I can't say that I have actually used it, but I showed it to a couple of co-workers for a laugh

[BastiFantasti]

Hi Norm,

I'm trying your tool on a german windows xp sp3. But when starting the LVSpeak engine I always run into the problem, not beeing able to load the grammar in the "MS_Speech - Load Grammar.vi"

The error is:

Error 1172:

Inner Exception: System.InvalidOperationException: Die Sprache für die Grammatik entspricht nicht der Sprache des Spracherkennungsmoduls.

) in MS_Speech - Load Grammar.vi->ENGINE - LVSpeak.vi aufgetreten

in english: The language for the grammar is not the language of the speech recognition module

Did you find any solution for the grammar problem?

regards,

bastian

[Norm Kirchner]

Did you find any solution for the grammar problem?

I can't say that I've ever tried to implement it on another language.

Have you adjusted any of the settings for speech under control panel?

[EstarTroy]

Norm,

Thanks for the update on EyesonVIs. So I have downloaded all the packages, but am unclear on where/how to install the LVSpeak llbs.

I have reviewed the VIPM online help but am not seeing how to tell it where to find the LVSpeak llbs. I am also unable to get it to connect to servers outside the corporate firewall, but I seem to gather that shouldn't be necessary. Also, only using the free VIPM.

Thanks again for your time. I can clearly see the benifits during development since I am a bit of a neat freak with my block diagrams. Aligns, lable moves, etc get to be tedious.

[Daklu]

Norm,

Thanks for the update on EyesonVIs. So I have downloaded all the packages, but am unclear on where/how to install the LVSpeak llbs.

I have reviewed the VIPM online help but am not seeing how to tell it where to find the LVSpeak llbs. I am also unable to get it to connect to servers outside the corporate firewall, but I seem to gather that shouldn't be necessary. Also, only using the free VIPM.

After you download the LVSpeak packages, you need to install them using VI Package Manager before starting Labview. In VIPM, go to File -> Add Package(s) to Package List... Once the LVSpeak packages are listed in VIPM you can install them like any other package.

[EstarTroy]

Ah. Now we're getting somewhere. I attempted that, but the problem appears to be that the zip files do not have any .ogp, .vip, or .vipc files in them.

The zip files I downloaded from the links at Eyes only contain a File Group 0 folder (with vis or .llb), a file called spec with no extension, and a bitmap/icon. The QD also has two .vis called PreInstall and PostUninstall

I tried directing VIPM to the "spec" file and it claims there are no packages found to add to the library.

Are the zips imcomplete or am I missing something?

[Yair]

The files you download ARE the VIP files. Some browsers recognize them (correctly) as ZIP files and saves them using that extension.

[EstarTroy]

Yair,

Thanks. I see that when IE saved them it changed the extension to zip from OGP. Once I changed those back to OGP, VIPM recognized them.

Thanks for the help. Much appreciated.

Now I can see if we can get LVSpeak to function. Will let you and Norm know if I run into further problems. Thanks again.

[BastiFantasti]

I can't say that I've ever tried to implement it on another language.

Have you adjusted any of the settings for speech under control panel?

Hello Norm,

yes, I've play around with the settings in control panel but without any success. I'll try the lvspeak on a german vista pc.

When did you, or anybody, get the dictionary error?

regards

bastian

[jzoller]

Hrm, no QuickDrop support on my XP (recognizes QuickEdit words, just not the words in QuickDrop).

It looks like the problem might be occurring within the QuickDrop Launch Window.vi. Sadly, it's locked.

Any thoughts?

Thanks,

[Norm Kirchner]

Hrm, no QuickDrop support on my XP (recognizes QuickEdit words, just not the words in QuickDrop).

It looks like the problem might be occurring within the QuickDrop Launch window.vi. Sadly, it's locked.

Any thoughts?

Thanks,

Well unfortunately, you may be experiencing a bug that only seems to happen on some peoples PCs (without reason) which causes the loading of grammar from within the QuickDrop environment fail.

The re-architect that I'm working on will hopefully work around this.

But the one difference is that you say it's locked.

That's the confusing part.

is the QD window locked? or is the whole LV environment locked, is it every time you call up QD now that it's installed?

If you want we can try to isolate what's going on over IM if you've got some time

~,~

[Michael Aivaliotis]

But the one difference is that you say it's locked.

That's the confusing part.

He means the Block Diagram is password protected.

[Norm Kirchner]

He means the Block Diagram is password protected.

duh

Joe,

Close and re-open LV (just to clear all stuff from memory)

Enable LVSpeak

open this VI <LV>:resourcedialogQuickDropEnable Voice Command.vi

Then open QD and see if an error is tossed on the output of that VI

If it is as I suspect, then I don't have an immediate fix, but I am working on the re-architecture of LVS as I write this so that patch will be coming soon.

Sorry about the error, but it's something insidious in the bowels of LV that seems to be causing it and I haven't been able to dig that deep into the beast.

[jzoller]

duh

Joe,

Close and re-open LV (just to clear all stuff from memory)

Enable LVSpeak

open this VI <LV>:resourcedialogQuickDropEnable Voice Command.vi

Then open QD and see if an error is tossed on the output of that VI

If it is as I suspect, then I don't have an immediate fix, but I am working on the re-architecture of LVS as I write this so that patch will be coming soon.

Sorry about the error, but it's something insidious in the bowels of LV that seems to be causing it and I haven't been able to dig that deep into the beast.

Error 1172: Error calling method System.Speech.Recognition.SpeechRecognitionEngine.LoadGrammar of ObjectId handle: 0x8A81160 for obj 0x23C0C5D[system.Speech.Recognition.SpeechRecognitionEngine] in domain [LabVIEW Domain for Run] and thread 5264, (System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.

Inner Exception: System.InvalidCastException: Unable to cast COM object of type 'System.__ComObject' to interface type 'System.Speech.Internal.SapiInterop.ISpRecoContext'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{F740A62F-7C15-489E-8234-940A33D9272D}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).

) in MS_Speech - Load Grammar.vi->Enable Voice Command.vi->QuickDrop Launch Window.vi->QuickDrop Launch Window.vi.ProxyCaller

Never let it be said that MS didn't give you plenty of information... maybe not useful information, but there's lots of it

Still poking around, will post if I find anything.

Thanks all,

Joe Z.

[jzoller]

A little bit more information: The error is occurring in the SpeechRecognitionEngine.LoadGrammar(grammar) method in MS_Speech - Load Grammar.vi.

All of the references are valid at the time the error is thrown.

http://msdn.microsof...521(VS.85).aspx - QueryInterface doc

http://msdn.microsof...marbuilder.aspx - GrammarBuilder doc

http://msdn.microsof...oadgrammar.aspx - LoadGrammar doc

Not sure I'm ready to speculate why this interface cast is failing on this invocation, but not on others.

---

A little more test data: modifying the dictionary that's loaded has no effect on the error: it's thrown for 260 elements, or 1 element. It's not the content being loaded.

The SpeechRecognitionEngine ref on the LV side remains the same between all calls: it's not being switched to a different instance of the engine for some oddball reason.

Okay, back to paying work. Have fun

Joe Z.

Edited August 16, 2009 by jzoller

[jzoller]

Simple workaround:

Load the quickdrop commands within the "Engine - QuickEdit.vi", immediately after loading the standard QuickEdit commands. I'm just loading them from a constant array.

It looks like there's something on the timing of the standard QuickDrop command load that causes the problem I'm seeing.

Joe Z.

[Norm Kirchner]

Joe,

Thanks for doing some digging down the rabbit hole, but as expected there is no good reason for the failure at the moment.

What I do suspect and have currently re-architected around is the fact that because the Quick Drop VI is dynamically loaded, it must somehow not be a valid accessor of the .NET references even though it's through a VI that is still reserved and resident in memory.

Yes, you can easily farm this list of quick drop items outside of the quick drop window and load them elsewhere and it should work.

The re-architecture is instead LVSpeak Engine now a receiver of all commands and is in charge of registering them within the MS SAPI which should resolve any issue with someone else registering commands outside of the VI that owns the .NET constructor.

All is going would love to use you as a guinea pig tester if you're up for it.

Open invitation to anyone else encountering the same issue.

[jzoller]

Joe,

Thanks for doing some digging down the rabbit hole, but as expected there is no good reason for the failure at the moment.

What I do suspect and have currently re-architected around is the fact that because the Quick Drop VI is dynamically loaded, it must somehow not be a valid accessor of the .NET references even though it's through a VI that is still reserved and resident in memory.

Yes, you can easily farm this list of quick drop items outside of the quick drop window and load them elsewhere and it should work.

The re-architecture is instead LVSpeak Engine now a receiver of all commands and is in charge of registering them within the MS SAPI which should resolve any issue with someone else registering commands outside of the VI that owns the .NET constructor.

All is going would love to use you as a guinea pig tester if you're up for it.

Open invitation to anyone else encountering the same issue.

I'd be glad to guinea.

Joe Z.

[EstarTroy]

Norm, I too am experiencing no Quick Drop commands under XP (both at the office and at home). I looked at the VI you suggested to Joe, and I am not getting an error. It also looks like the VI never really runs when the Quick Drop is brought up (grid goes away like it is running though so maybe my ignorance on how this works). Thoughts?

I would also be willing to be a "guinea pig" for the redo.

[Norm Kirchner]

Ok all you willing furry animals.

Attached to this you will find the new architecture.

I do not have a quick drop package just yet, but give me a day.

But for now I need to validate that this install goes smoothly and still is functional on all of you're PCs

The interface is still the nasty 2 floating windows and I'm up for recommendations on how you all think this would best be shown (1 detected speech window, for LVS core or QE, no window, how big, where to put, what options on the window and on and on.

It should install to 2 primary locations

LVS core goes in <project> (and there is a .lvproj in there as well to see how things are setup)

Quick Edit and it's plug-ins go into <resource> (videos and documentation for creating new QE plugins are on the way)

and just to make things difficult on you guys for the moment, you need to install RFSE Tools to your user.lib directory (still working on the best method of combining these into 1 install process)

So as it stands, uninstall all LVS packages, install this new one, and also extrace RFSE Tools to your user.lib directory.

Not all of the old QE commands are fully implemented yet, but there is a VI in the QE directory that shows the list of currently installed commands.

More refinement on the way soon!!

and please let me know asap of any install or getting started issues

lavag_cr_LVSpeak-0.9.1-1.ogp

RFSE Tools.zip