LVSpeak - tell LV what to do

[mprim]

Norm,

I have done a considerable amount of editing to your original LVSpeak release that has made it more stable. I have also added a few commands. Attached is a llb that includes the edited engine.

Mark

Ok all you willing furry animals.

Attached to this you will find the new architecture.

I do not have a quick drop package just yet, but give me a day.

But for now I need to validate that this install goes smoothly and still is functional on all of you're PCs

The interface is still the nasty 2 floating windows and I'm up for recommendations on how you all think this would best be shown (1 detected speech window, for LVS core or QE, no window, how big, where to put, what options on the window and on and on.

It should install to 2 primary locations

LVS core goes in <project> (and there is a .lvproj in there as well to see how things are setup)

Quick Edit and it's plug-ins go into <resource> (videos and documentation for creating new QE plugins are on the way)

and just to make things difficult on you guys for the moment, you need to install RFSE Tools to your user.lib directory (still working on the best method of combining these into 1 install process)

So as it stands, uninstall all LVS packages, install this new one, and also extrace RFSE Tools to your user.lib directory.

Not all of the old QE commands are fully implemented yet, but there is a VI in the QE directory that shows the list of currently installed commands.

More refinement on the way soon!!

and please let me know asap of any install or getting started issues

LV Speak Edited.llb

[mprim]

Norm,

I really like the second version of LV Speak. Attached is an edited copy of the project. I added several features:

1. The QuickEdit and LVSpeak engine vi's are no longer visible, they are minimized at launch. The functionality of these vi's front panels has been built into the program.

a. To list the commands LVSpeak recognizes say "list commands" and a listbox of all available LV Speak commands appears.

b. To reload the plugins say "reload"

c. To stop say "speak stop"

2. There is also now audible feedback for every command as well as the reload plugins command. Every command is echoed back to the user, the reload plugins command provides audible feedback when it finishes.

3. I also changed the Locate QEC Plugins vi so that it parses the commands in the QuickEdit Command.lvclass

4. I added a new class for property nodes and three new commands: Property Value, Property Read, Property Write.

I have several suggestions:

1. Change the format of the QEC class Execute VI such that the FP / BD case structure is inside the command name case structure. This allows for commands that are not specific to the FP or BD to be only coded once. Also it makes it easier to track whether all possible cases for a given command have been coded.

2. Filter the available command grammar such that only the ones valid for the currently selected objects are available. This would reduce the number of errors from the speech engine.

3. Add audible confirmation to the delete command.

4. Provide right click functionality such that the user could either speak the command or click it with the mouse.

Thanks again for all of your hard work. I am trully addicted to LVSpeak. Attached is a zip of the .lvproj file. If there is a better way to send you this please let me know.

Mark

LV Speak.zip

Edited September 4, 2009 by bluesky

[Norm Kirchner]

Thanks again for all of your hard work. I am trully addicted to LVSpeak. Attached is a zip of the .lvproj file. If there is a better way to send you this please let me know.

Mark

Mark,

I'm still reviewing the updates and have designed my user experience and need to still implement the changes.

I will hopefully putting the details up here soon and giving some feedback to your updates.

Very quickly though, please find the .NET components that utilize the speech to text as the activeX component does not work on my system

Thanks again for the enthusiasm and sorry for my long delays in replies.

More soon hopefully.

-Norm

[mprim]

Very quickly though, please find the .NET components that utilize the speech to text as the activeX component does not work on my system

Norm,

The Text to Speech vi is off of the NI website at http://decibel.ni.com/content/docs/DOC-2263 it uses the same Microsoft SDK.

Mark

[mprim]

Norm,

Here is an updated class template that includes my suggestions for how to handle the various Gobj clasees. I have also attached an update engine and startup vi's that run hidden and close automatically when "Speak Stop" is given. I have updated the property class and part of the text class plugins as well. They are attached. There are a couple of other updated/new vi's as well attached.

Regards,

Mark

Speak Update.zip

[Rolf Kalbermatter]

Thanks for doing some digging down the rabbit hole, but as expected there is no good reason for the failure at the moment.

What I do suspect and have currently re-architected around is the fact that because the Quick Drop VI is dynamically loaded, it must somehow not be a valid accessor of the .NET references even though it's through a VI that is still reserved and resident in memory.

Actually there is quite likely some reason. I do see that it seems not just .Net is getting involved but also COM. And from my workings with COM to interface to WIA I have encountered various strange things that seem to relate to the fact that the COM interfaces get sometimes into trouble to marshal its calls between LabVIEW and the out of process target component.

I could trigger this very reliably when debugging C calls into those COM methods when the Call Library Node was setup to be reentrant and as soon as the Visual C debugger kicked in. It could vary between COM methods failing with some marshalling errors and a complete lockup of LabVIEW and the debugger.

The solution for me was to set all Call Library Nodes to be non reentrant during debugging and set them back to reentrant for the final library. Setting all calls to be non-reentrant did not entirely fix this, but it made it possible to run at least far enough to debug the issues at hand. There were still some COM methods that sometimes (not always) failed with some marshalling errors.

I could imagine that the execution context in which QuickDrop is running might have some issues to run .Net and ActiveX methods reliably due to some synchronisation issues. Especially the marshalling of COM data is always driven by the message pump of the calling application and that is one really hairy part in LabVIEW to deal with. I'm glad I never will have to work on that code.

Rolf Kalbermatter

[dblk22vball]

Is there a list of commands somehwere?

I know in the first post it says that there is a picture of the list of commands, but I am not able to see a picture in the first post.

** EDIT *** nevermind, I looked at the source and found the case that has all of the commands. thanks

Edited September 17, 2009 by dblk22vball

[Norm Kirchner]

Is there a list of commands somehwere?

I know in the first post it says that there is a picture of the list of commands, but I am not able to see a picture in the first post.

** EDIT *** nevermind, I looked at the source and found the case that has all of the commands. thanks

Which version are you using?

The newest framework that is posted a few posts above has a VI that lists all the commands, and I am working on finishing up that new framework to actually have a command integrated that displays the list when requested.

[dblk22vball]

VIPM Says that I am using Version 0.2, Release 9

I will download the new one above and work with that one now.

Edited September 17, 2009 by dblk22vball

[Norm Kirchner]

Ok everyone. After waiting too damn long for this supposed 're-architecture' to be released, it's almost time.

I'm sorry to say, but I work at a place saturated by over-achievers and it takes most of my time just to keep up w/ all of them.

But that being said, I am going to be hosting a webinar for those of you interested this Wednesday at 5pm CST.

I will be posting the web-ex info here soon after I get some people to reply and agree upon the timing.

What I will do is review the changes, the why's and hows and get feedback for possible changes.

For reference, most of the changes came directly from you all (although, there are still no cascaded commands)

Thank you all for the feedback and please accept my apologies for not tending to this thread better.

If you have 2009 available, get it installed as I've been doing my development in it, but haven't used any 09 features that would prevent a rollback to 8.6 once done.

-Norm

[jhoskins]

Ok everyone. After waiting too damn long for this supposed 're-architecture' to be released, it's almost time.

I'm sorry to say, but I work at a place saturated by over-achievers and it takes most of my time just to keep up w/ all of them.

But that being said, I am going to be hosting a webinar for those of you interested this Wednesday at 5pm CST.

I will be posting the web-ex info here soon after I get some people to reply and agree upon the timing.

What I will do is review the changes, the why's and hows and get feedback for possible changes.

For reference, most of the changes came directly from you all (although, there are still no cascaded commands)

Thank you all for the feedback and please accept my apologies for not tending to this thread better.

If you have 2009 available, get it installed as I've been doing my development in it, but haven't used any 09 features that would prevent a rollback to 8.6 once done.

-Norm

got it on my calendar. Cant wait, see you then

[Norm Kirchner]

Oh good we've got at least 1 person.

I'll be sure to post the info for the connection ASAP

[jzoller]

Oh good we've got at least 1 person.

I'll be sure to post the info for the connection ASAP

I'll try to be there... can't guarantee, work's been, well, a lot of work lately.

Joe Z.

[Norm Kirchner]

Hello ,

Norman Kirchner invites you to attend this online meeting.

Topic: LVSpeak 2 pre-release review

Date: Wednesday, November 11, 2009

Time: 5:00 pm, Central Standard Time (Chicago, GMT-06:00)

Meeting Number: 012 338 701

Meeting Password: mhbttc

-------------------------------------------------------

To join the online meeting (Now from iPhones too!)

-------------------------------------------------------

1. Go to https://freetrial.we...mRjODI0&RT=MiM3

2. Enter your name and email address.

3. Enter the meeting password: mhbttc

4. Click "Join Now".

To view in other time zones or languages, please click the link:

https://freetrial.we...RjODI0&ORT=MiM3

-------------------------------------------------------

To join the audio conference only

-------------------------------------------------------

Call-in toll number (US/Canada): 1-650-429-3300

Access code:012 338 701

[Irene_he]

Oh good we've got at least 1 person.

I'll be sure to post the info for the connection ASAP

I would like to hear the cool stuff you did too in less than half hour.

[Norm Kirchner]

I would like to hear the cool stuff you did too in less than half hour.

Then I'll be sure to save you a 'virtual' seat.

Everything looks set so I'll see y'all in less than 15

~,~

[Irene_he]

Then I'll be sure to save you a 'virtual' seat.

Everything looks set so I'll see y'all in less than 15

~,~

OK, it didn't work with my browser, IE 7.0. I have tried at least 5 times to log in and it shows the meeting, but then I was "kicked out" back to the WebEx free trial window.

[Irene_he]

Thank you Norm. I have to leave earlier now. I finally managed to get in the meeting after I relaxed (although late, but not totally missed ), then I got it to work (with google Chrome) and I also found my glasses (my monitor is a meter away on the wall) and was able to calm down to turn off the mic, so no screaming was leaked (I hoped not) . It is very cool, although I didn't undertand "all" of it. but I got the idea. It is cool that you were able to use it to tell LabVIEW what to do, but same idea can also be used to tell your LabVIEW Application what to do. Say a scenario that your end user don't always have a free hand to press a button on your LabVIEW application, then you can talk to the application to press the "start button" etc... probably with a bluetooth ear phone. then it can be operated in a distance.

By the way, you have a nice working background. I triple checked that my video was not sent, very panic.

Thank you!

Irene

[dblk22vball]

congrats on getting LVSpeak into the NI Newsletter and on the NI website. I guess you have some more pull working for NI now

Edited November 24, 2009 by dblk22vball

[Rolf Kalbermatter]

congrats on getting LVSpeak into the NI Newsletter and on the NI website. I guess you have some more pull working for NI now

Actually if memory serves me right this is not the first time for The Captain to appear in the newsletter or the website. So not so sure it has much to do with the fact that he works for NI now. He's simply good in marketing, including his own ideas.

[Norm Kirchner]

Actually if memory serves me right this is not the first time for The Captain to appear in the newsletter or the website. So not so sure it has much to do with the fact that he works for NI now. He's simply good in marketing, including his own ideas.

HEY! Don't go telling me I'm in marketing....that's just not right.

That aside

****BIG NOTICE****

LVSpeak 2.0 has been officially released and is currently available on the dark side

http://decibel.ni.com/content/groups/labview-apis/blog/2009/11/19/labview-speak--programming-lv-through-voice-commands

There are getting started videos that detail the install/troubleshoot and commands along with the code and a presentation highlighting the process

I have submitted the material to LAVA CR un-certified and we'll see how that goes over the next few weeks to get certified

Thank you all for your feedback that has made this such a great program.

[kentwedeking]

Has anyone run into any problems with getting everything installed ... seemingly properly ... and not having the code recognize any speech? the VI's all run with no errors, but i can't make labview recognize any commands. I"m running windows 7 64 bit, and the actually windows speech recognition commands work, i can start MS word and stuff, but nothing in labview ...

[Norm Kirchner]

Has anyone run into any problems with getting everything installed ... seemingly properly ... and not having the code recognize any speech? the VI's all run with no errors, but i can't make labview recognize any commands. I"m running windows 7 64 bit, and the actually windows speech recognition commands work, i can start MS word and stuff, but nothing in labview ...

I'm not sure if the 64 bit will cause any issue, but there is one minor point that sometimes gets lost... are you holding down the ctl key when you say the command and for a few seconds afterwards? ( that I now realize I need to be something like a pop-up after start to re-inform the person)

The logic in the code is such that pressing the ctl key down does not enable speech, rather it tells the LVSpeak core that it's ok to fire the event that is associated with the speech recognized notification it is always getting from the MSAPI core. The speech recognizer is constantly translating and sending notification to LV. It's only when both speech is recognized, and then after which it looks for the Ctl key to be pressed ( thus necessitating for the purpose of debug, to ensure that you're holding down the ctl key for a long enough period after saying the command. )

Try that and see if it works. As a sanity check, under the tools menu, you should also have an entry for Quick Edit. Under that menu, you should see an option to "Show QuickEdit Window". Double check that when you select that, that another window comes up and is already running.

good luck

-Norm

[kentwedeking]

norm,

yeah, i was holding down the ctl key, and i can get the QE window to open. when i have speach recognition turned on in windows, i can get random windows voice commands to work with labview, but nothing in the LVSpeak realm ... i can't seem to get windows to allow labview to take the commands. when i watch the block diagram, i don't ever see any speech fire off the event structure. however, if i say "create control" or "create anything" i get the VI Revision window opening up and asking me to put in comments on the latest revision of the current VI i'm working in.

[Darren]

... if i say "create control" or "create anything" i get the VI Revision window opening up and asking me to put in comments on the latest revision of the current VI i'm working in.

The keyboard shortcut for the Revision History dialog box in LabVIEW is Ctrl-Y. Could there be some config option in the Windows speech settings that is making it try to convert whatever you're saying into a Ctrl-command?

Just a thought,

-D