Jump to content
Norm Kirchner

LVSpeak - tell LV what to do

Recommended Posts

I have to say that I like the idea ever since I first saw Norm's demo of it last year, but I haven't used it all due to not wanting to install the speech recog utility or speaking all the time in the office.

I have found that I can speak very quietly and still get picked up. None of my co-workers have started calling me crazy yet, so I guess it's not bothering them.

My favorite part of this tool is the ability to quickly align code and labels in any number of ways. It saves so much time - especially when dealing with other people's code who might not be as fastidious as I am in keeping things pretty.

  • Like 1

Share this post


Link to post
Share on other sites

I have found that I can speak very quietly and still get picked up. None of my co-workers have started calling me crazy yet, so I guess it's not bothering them.

My favorite part of this tool is the ability to quickly align code and labels in any number of ways. It saves so much time - especially when dealing with other people's code who might not be as fastidious as I am in keeping things pretty.

You're using it?!!!! Really!! I've got my first official user!!!! HELL YEAH! I'm going to give you a reputation point just for that .

Yup, the alignment is about the most used one, and then probably the quick drop integration.

I have to say that I like the idea ever since I first saw Norm's demo of it last year, but I haven't used it all due to not wanting to install the speech recog utility or speaking all the time in the office.

Another thing which I think is a major hinderance is the list of options. I think that what QE needs to really take off is a smart parser. This is certainly a tricky bit, but also important, due to the following improvements it could allow:

  1. Writing less code for the framework. To demonstrate this, think of the command "Label visible". Today, it has its own specific VI.
    In the ideal version, the parser would recognize the structure, use the word "label" to get a reference to the label of the object you selected and then pass that reference into the "visible" VI (which was only implemented once, for the GObject class). That would mean that you only have to write the code for each action once. Now that you've written it, "visible" would work all objects, their labels, their captions, etc.
  2. The parser could allow parameters. For example, "caption align left" would obtain the reference to the caption, then call the "align" action and pass "left" as a parameter. Immediately you have only one action instead of several.
  3. Synonyms - imagine if you could say "label visible" or "visible label" or "show label" or "label show", etc. That would make it a lot more "natural", although I admit it's a secondary objective.

As I said, writing such a parser would be tricky. It's possible that maybe an initial version could use some explicit vocal separators to break the different parts of the text.

Inside of the MS SAPI we have the ability to do this, although it requires a better understanding of it and more testing that I never got to.

Actually Darren is hitting me up for this exact thing....

I just have a hard time seeing how much more that will simplify things on one side for the drawbacks I can see on the implementation side.

Can you give me 3 solid use cases, because the detect 'Align' then detect 'direction' will still require a unique piece of code for each direction, and the only thing it would make simpler, is that flat top level list of commands and it would make it hierarchical. Also, unfortunately not all properties are common in the VI Server Class structure. Like label is not a GObject item, and certain things require me to detect the specific type, or iterate through the valid ones (you might see this stuff in there)

Share this post


Link to post
Share on other sites

Can you give me 3 solid use cases

I mentioned several potential improvements, so it's hard to say which one you're refering to, but here it is:

Caption.Align.Left

Label.Visible.

Label.Change to easy to read font.

Again, part of the issue is that with a parser, once you wrote the "change to easy to read font" VI, you can call it on any piece of text. The framework will need to be responsible for getting to the text part of the object.

the detect 'Align' then detect 'direction' will still require a unique piece of code for each direction

Why? If you know it's a parameter, just pass it as one into the align VI, which will attempt to cast it into one of its allowed parameters. For example:

post-1431-124836844004_thumb.png

Also, unfortunately not all properties are common in the VI Server Class structure. Like label is not a GObject item, and certain things require me to detect the specific type

The first part is correct, although in this specific case, label is of the Text class which does inherit from GObject. You're definitely correct that finding the right type might be tricky. As I said, it won't necessarily be simple, but I think it's required in order to make it really useful (as opposed to the very useful it is now).

  • Like 1

Share this post


Link to post
Share on other sites

You're using it?!!!! Really!! I've got my first official user!!!! HELL YEAH! I'm going to give you a reputation point just for that .

Yup, the alignment is about the most used one, and then probably the quick drop integration.

I'm to the point that I get annoyed if I have to code without it. That, to me, is the sign of a good tool.

  • Like 1

Share this post


Link to post
Share on other sites

I'm to the point that I get annoyed if I have to code without it. That, to me, is the sign of a good tool.

But have you used 'Get Pizza' yet :lol:

Share this post


Link to post
Share on other sites

But have you used 'Get Pizza' yet :lol:

I can't say that I have actually used it, but I showed it to a couple of co-workers for a laugh :thumbup1:

Share this post


Link to post
Share on other sites

Hi Norm,

I'm trying your tool on a german windows xp sp3. But when starting the LVSpeak engine I always run into the problem, not beeing able to load the grammar in the "MS_Speech - Load Grammar.vi"

The error is:

Error 1172:

Inner Exception: System.InvalidOperationException: Die Sprache für die Grammatik entspricht nicht der Sprache des Spracherkennungsmoduls.

) in MS_Speech - Load Grammar.vi->ENGINE - LVSpeak.vi aufgetreten

in english: The language for the grammar is not the language of the speech recognition module

Did you find any solution for the grammar problem?

regards,

bastian

Share this post


Link to post
Share on other sites

Did you find any solution for the grammar problem?

I can't say that I've ever tried to implement it on another language.

Have you adjusted any of the settings for speech under control panel?

Share this post


Link to post
Share on other sites

Norm,

Thanks for the update on EyesonVIs. So I have downloaded all the packages, but am unclear on where/how to install the LVSpeak llbs.

I have reviewed the VIPM online help but am not seeing how to tell it where to find the LVSpeak llbs. I am also unable to get it to connect to servers outside the corporate firewall, but I seem to gather that shouldn't be necessary. Also, only using the free VIPM.

Thanks again for your time. I can clearly see the benifits during development since I am a bit of a neat freak with my block diagrams. Aligns, lable moves, etc get to be tedious.

Share this post


Link to post
Share on other sites

Norm,

Thanks for the update on EyesonVIs. So I have downloaded all the packages, but am unclear on where/how to install the LVSpeak llbs.

I have reviewed the VIPM online help but am not seeing how to tell it where to find the LVSpeak llbs. I am also unable to get it to connect to servers outside the corporate firewall, but I seem to gather that shouldn't be necessary. Also, only using the free VIPM.

After you download the LVSpeak packages, you need to install them using VI Package Manager before starting Labview. In VIPM, go to File -> Add Package(s) to Package List... Once the LVSpeak packages are listed in VIPM you can install them like any other package.

Share this post


Link to post
Share on other sites

Ah. Now we're getting somewhere. I attempted that, but the problem appears to be that the zip files do not have any .ogp, .vip, or .vipc files in them.

The zip files I downloaded from the links at Eyes only contain a File Group 0 folder (with vis or .llb), a file called spec with no extension, and a bitmap/icon. The QD also has two .vis called PreInstall and PostUninstall

I tried directing VIPM to the "spec" file and it claims there are no packages found to add to the library.

Are the zips imcomplete or am I missing something?

Share this post


Link to post
Share on other sites

The files you download ARE the VIP files. Some browsers recognize them (correctly) as ZIP files and saves them using that extension.

Share this post


Link to post
Share on other sites

Yair,

Thanks. I see that when IE saved them it changed the extension to zip from OGP. Once I changed those back to OGP, VIPM recognized them.

Thanks for the help. Much appreciated.

Now I can see if we can get LVSpeak to function. Will let you and Norm know if I run into further problems. Thanks again.

Share this post


Link to post
Share on other sites

I can't say that I've ever tried to implement it on another language.

Have you adjusted any of the settings for speech under control panel?

Hello Norm,

yes, I've play around with the settings in control panel but without any success. I'll try the lvspeak on a german vista pc.

When did you, or anybody, get the dictionary error?

regards

bastian

Share this post


Link to post
Share on other sites

Hrm, no QuickDrop support on my XP (recognizes QuickEdit words, just not the words in QuickDrop).

It looks like the problem might be occurring within the QuickDrop Launch Window.vi. Sadly, it's locked.

Any thoughts?

Thanks,

Share this post


Link to post
Share on other sites

Hrm, no QuickDrop support on my XP (recognizes QuickEdit words, just not the words in QuickDrop).

It looks like the problem might be occurring within the QuickDrop Launch window.vi. Sadly, it's locked.

Any thoughts?

Thanks,

Well unfortunately, you may be experiencing a bug that only seems to happen on some peoples PCs (without reason) which causes the loading of grammar from within the QuickDrop environment fail.

The re-architect that I'm working on will hopefully work around this.

But the one difference is that you say it's locked.

That's the confusing part.

is the QD window locked? or is the whole LV environment locked, is it every time you call up QD now that it's installed?

If you want we can try to isolate what's going on over IM if you've got some time

~,~

Share this post


Link to post
Share on other sites

He means the Block Diagram is password protected.

:blink:

duh

Joe,

Close and re-open LV (just to clear all stuff from memory)

Enable LVSpeak

open this VI <LV>:resourcedialogQuickDropEnable Voice Command.vi

Then open QD and see if an error is tossed on the output of that VI

If it is as I suspect, then I don't have an immediate fix, but I am working on the re-architecture of LVS as I write this so that patch will be coming soon.

Sorry about the error, but it's something insidious in the bowels of LV that seems to be causing it and I haven't been able to dig that deep into the beast.

Share this post


Link to post
Share on other sites

blink.gif

duh

Joe,

Close and re-open LV (just to clear all stuff from memory)

Enable LVSpeak

open this VI <LV>:resourcedialogQuickDropEnable Voice Command.vi

Then open QD and see if an error is tossed on the output of that VI

If it is as I suspect, then I don't have an immediate fix, but I am working on the re-architecture of LVS as I write this so that patch will be coming soon.

Sorry about the error, but it's something insidious in the bowels of LV that seems to be causing it and I haven't been able to dig that deep into the beast.

Error 1172: Error calling method System.Speech.Recognition.SpeechRecognitionEngine.LoadGrammar of ObjectId handle: 0x8A81160 for obj 0x23C0C5D[system.Speech.Recognition.SpeechRecognitionEngine] in domain [LabVIEW Domain for Run] and thread 5264, (System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.

Inner Exception: System.InvalidCastException: Unable to cast COM object of type 'System.__ComObject' to interface type 'System.Speech.Internal.SapiInterop.ISpRecoContext'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{F740A62F-7C15-489E-8234-940A33D9272D}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).

) in MS_Speech - Load Grammar.vi->Enable Voice Command.vi->QuickDrop Launch Window.vi->QuickDrop Launch Window.vi.ProxyCaller

Never let it be said that MS didn't give you plenty of information... maybe not useful information, but there's lots of it :)

Still poking around, will post if I find anything.

Thanks all,

Joe Z.

Share this post


Link to post
Share on other sites

A little bit more information: The error is occurring in the SpeechRecognitionEngine.LoadGrammar(grammar) method in MS_Speech - Load Grammar.vi.

All of the references are valid at the time the error is thrown.

http://msdn.microsof...521(VS.85).aspx - QueryInterface doc

http://msdn.microsof...marbuilder.aspx - GrammarBuilder doc

http://msdn.microsof...oadgrammar.aspx - LoadGrammar doc

Not sure I'm ready to speculate why this interface cast is failing on this invocation, but not on others.

---

A little more test data: modifying the dictionary that's loaded has no effect on the error: it's thrown for 260 elements, or 1 element. It's not the content being loaded.

The SpeechRecognitionEngine ref on the LV side remains the same between all calls: it's not being switched to a different instance of the engine for some oddball reason.

Okay, back to paying work. Have fun :)

Joe Z.

Edited by jzoller

Share this post


Link to post
Share on other sites

Simple workaround:

Load the quickdrop commands within the "Engine - QuickEdit.vi", immediately after loading the standard QuickEdit commands. I'm just loading them from a constant array.

It looks like there's something on the timing of the standard QuickDrop command load that causes the problem I'm seeing.

Joe Z.

Share this post


Link to post
Share on other sites

Joe,

Thanks for doing some digging down the rabbit hole, but as expected there is no good reason for the failure at the moment.

What I do suspect and have currently re-architected around is the fact that because the Quick Drop VI is dynamically loaded, it must somehow not be a valid accessor of the .NET references even though it's through a VI that is still reserved and resident in memory.

Yes, you can easily farm this list of quick drop items outside of the quick drop window and load them elsewhere and it should work.

The re-architecture is instead LVSpeak Engine now a receiver of all commands and is in charge of registering them within the MS SAPI which should resolve any issue with someone else registering commands outside of the VI that owns the .NET constructor.

All is going would love to use you as a guinea pig tester if you're up for it.

Open invitation to anyone else encountering the same issue.

Share this post


Link to post
Share on other sites

Joe,

Thanks for doing some digging down the rabbit hole, but as expected there is no good reason for the failure at the moment.

What I do suspect and have currently re-architected around is the fact that because the Quick Drop VI is dynamically loaded, it must somehow not be a valid accessor of the .NET references even though it's through a VI that is still reserved and resident in memory.

Yes, you can easily farm this list of quick drop items outside of the quick drop window and load them elsewhere and it should work.

The re-architecture is instead LVSpeak Engine now a receiver of all commands and is in charge of registering them within the MS SAPI which should resolve any issue with someone else registering commands outside of the VI that owns the .NET constructor.

All is going would love to use you as a guinea pig tester if you're up for it.

Open invitation to anyone else encountering the same issue.

I'd be glad to guinea.

Joe Z.

Share this post


Link to post
Share on other sites

Norm, I too am experiencing no Quick Drop commands under XP (both at the office and at home). I looked at the VI you suggested to Joe, and I am not getting an error. It also looks like the VI never really runs when the Quick Drop is brought up (grid goes away like it is running though so maybe my ignorance on how this works). Thoughts?

I would also be willing to be a "guinea pig" for the redo.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.