ChatGPT and LabVIEW

[X___]

4 hours ago, gleichman said:

Interesting Demo at NI Connect today. Link starts at NIgel demo.

 

It's heavily scripted, but on face value, that looks promising.

[ShaunR]

11 hours ago, X___ said:

It's heavily scripted, but on face value, that looks promising.

I was mesmerised by the cheesy forced grin of the guy demonstrating. Why does AI type so slowly?

I agree it's looking promising. It's at the level if an intern but actually listens to you. Linear coding is never the final solution. I expect it was tuned for very specific requirements but I was more impressed with it reading the emails and specs and interpreting them in context.

Would suffice for Unit Test cases and feasibility prototypes. Looking forward to getting my hands on it.

Edited May 25, 2023 by ShaunR

[Jordan Kuehn]

The demo made me wonder if it could theoretically create drivers from spec sheets automatically. It seems like it would be a well bounded problem, create labview wrappers for the commands listed in the manual. 

[bbean]

On 5/25/2023 at 1:35 PM, Jordan Kuehn said:

The demo made me wonder if it could theoretically create drivers from spec sheets automatically. It seems like it would be a well bounded problem, create labview wrappers for the commands listed in the manual. 

you can already do this in a limited fashion in python.  This is a simple example from a single prompt but you can see where its going:

https://poe.com/s/BEWvBcTIEkrVXzTezpza

The manual was probably online when the model was trained so it already has knowledge of it in this example.  But once the size limitations of the text entry increase, you will be able to just upload a manual.

And here's a simple refactoring to make an abstract class

https://poe.com/s/MJp4t75WVrA5MR5SiS8B

On 5/25/2023 at 4:04 AM, ShaunR said:

Why does AI type so slowly?

I believe the way they work is they are predicting each character on the fly

[X___]

1 hour ago, bbean said:

you can already do this in a limited fashion in python.  This is a simple example from a single prompt but you can see where its going:

https://poe.com/s/BEWvBcTIEkrVXzTezpza

The manual was probably online when the model was trained so it already has knowledge of it in this example.  But once the size limitations of the text entry increase, you will be able to just upload a manual.

And here's a simple refactoring to make an abstract class

https://poe.com/s/MJp4t75WVrA5MR5SiS8B

Actually that makes more sense: get an actually debuggable Python code and wrap it into a Python Node call. Un-commented spaghetti code is pretty much useless...

The flat (single VI) LabVIEW generated code demo doesn't tell anything about the ability for this type of AI assistant to generate structured code.

The first step is for NI to demonstrate that they can use AI to clean up (or expand/shrink) diagrams meaningfully.

Quote

 

On 5/25/2023 at 1:04 AM, ShaunR said:

Why does AI type so slowly?

I believe the way they work is they are predicting each character on the fly

I hope not... at least if they are using the thing right.

[acb]

Hey everyone! It's Alejandro from the keynote demo. Wanted to shed some light on Nigel and gather feedback from this important forum (my first post!).

A little about myself: Stepped foot at NI in 2007 and had the honor to work on the LabVIEW language design and compiler teams alongside all the incredible folks you already know. In fact, Aristos Queue was once my mentor! Deeply fond of LabVIEW and all its users.

Back to AI --

On 5/24/2023 at 11:48 AM, LogMAN said:

Jokes aside, the copilot looks interesting but it needs much better integration to be useful for any of my day-to-day tasks. It needs much faster response time and it will have to show how well it can work with legacy and non-standard code bases that aren't developed with NI's portfolio and vision in mind.

What types of day-to-day tasks would you want Nigel to help you with?

And yes, I intend Nigel to help you with your hardware and codebases that aren't the latest & greatest.

On 5/25/2023 at 3:04 AM, ShaunR said:

Why does AI type so slowly? [...] Linear coding is never the final solution. I expect it was tuned for very specific requirements but I was more impressed with it reading the emails and specs and interpreting them in context.

It types slowly because we are using GPT 4.0 under the hood, and the demo was live on stage. GPT 4.0 is about that fast, and it varies throughout the day; at night, it seems twice as fast.

This presents a challenge that we are working on; we have lots of ideas to make the generation speed a non-factor. Nonetheless, I'm hopeful that GPT 4.0 will get faster over time, as Open AI fully transitions it out of beta, and as they buy more GPUs. It is possible to retrain Nigel with GPT 3.5 (what ChatGPT uses), but it does not perform as well.

On 5/26/2023 at 4:10 PM, X___ said:

Un-commented spaghetti code is pretty much useless... The flat (single VI) LabVIEW generated code demo doesn't tell anything about the ability for this type of AI assistant to generate structured code.

Nigel is capable of using structures, documenting VIs, and creating subVIs, but we did not show that on stage to keep the keynote short & sweet and focused on the core message of spec-to-test (going from a spec sheet to a working hardware test in seconds or minutes). Nigel can do much more than what we showed.

On 5/25/2023 at 12:35 PM, Jordan Kuehn said:

The demo made me wonder if it could theoretically create drivers from spec sheets automatically. It seems like it would be a well bounded problem, create labview wrappers for the commands listed in the manual. 

Would you share a device you'd like Nigel to create a driver for automatically? That would be a wonderful test for us.

Edited May 30, 2023 by acb

[LogMAN]

55 minutes ago, acb said:

my first post!

Welcome to the forums 🎉

55 minutes ago, acb said:

What types of day-to-day tasks would you want Nigel to help you with?

I haven't really thought about it as this was the first time I learned about Nigel (never thought anything like that is even remotely possible in LV). Still, I'm familiar with GitHub Copilot and Visual Studio's IntelliCode, which have great IDE integration.

What I'm looking for is not so much an AI that writes my code (because I know how to do that), but one that accelerates my development process by suggesting changes in the context of my code. For example, to predict what I'm going to do and provide hints in the form of grayed-out suggestions I can simply accept by pressing a key (tab-driven-development 😉).

Things like:

• When I place an "open connection" function, it suggests the corresponding "close connection" function.
• When I place several methods of a class or VISA or DAQmx, it suggests how to order them in a sensible manner (open, read, write, close).
• When I connect the terminals of a VI, it suggests to connect error wires as well.
• When I place error terminals it suggests adding an error case structure.

Of course, there are more specialized tasks where a smart AI would also be really useful:

• Creation of driver libraries, just like your example from importing a PDF to generating code.
• Configuration of CLFNs
• Beautify/Cleanup my block diagram
• Suggest changes when upgrading to a newer version of LabVIEW
• Suggest icons for my VIs 😍
• Derive VI descriptions from code 
• Apply changes to a set of VIs (i.e., renaming them)
• Point out mistakes in my code (missing cases, unhandled errors, etc.)

55 minutes ago, acb said:

And yes, I intend Nigel to help you with your hardware and codebases that aren't the latest & greatest.

That's awesome. I hope we can play with it soon

Edited May 30, 2023 by LogMAN

[ShaunR]

1 hour ago, acb said:

What types of day-to-day tasks would you want Nigel to help you with?

HTML Help files (manuals). Documentation is by far the greatest resource hog and we should be able to get away from programmer created VI descriptions. But I don't just mean documenting VI's though. I also mean generating API references, menu layouts, including external references and collating examples with descriptive comments about their function and what they demonstrate (from the code in the diagrams).

A far better DLL importer. The one we have currently is next to useless for 80% of DLL's. I'd like an AI that can do what Rolf does  . It should be able to import the DLL and create the VI's containing CLFN's for complex structures (e.g strings nested in structures), callbacks (he, he), events and error handling.

[dadreamer]

On 5/30/2023 at 1:26 PM, ShaunR said:

callbacks (he, he)

It should also automagically install Visual Studio, write the callback wrapper, compile it into a DLL and connect it to the VI with a CLFN. Moreover it would be nice to have a .NET translator from textual code to G code, as the current way of calling .NET assemblies is too awkward. It may take 10 minutes to find a good combination of Assembly -> Constructor -> Method/Property, especially when dealing with the OS native API.

Edited May 31, 2023 by dadreamer

[Jordan Kuehn]

On 5/30/2023 at 1:33 AM, acb said:

Would you share a device you'd like Nigel to create a driver for automatically? That would be a wonderful test for us.

Here's one for you. The AXS Port is a modbus device that monitors power inverters. It would be great to feed Nigel the attached document and have it generate drivers/API for communication with the device.

https://www.outbackpower.com/products/system-management/axs-port

Another one for you, in a similar vein, but a little less straightforward. This is a Banner IO-Link Master that can communicate via ethernet/IP or modbus. An ethernet/IP driver would be great that could configure the device as well as the IFM DTI434 IO-Link RFID read head also linked below with attached documents.

https://www.bannerengineering.com/us/en/products/industrial-networking---smart-i-o/io-link-masters/dxmr90-ethernet-io-link-master.html?sort=4#all

https://www.ifm.com/us/en/product/DTI434

ifm-DTI410-20190125-IODD11-en (2).pdf dxmr904k.pdf axs_app_note.pdf 229732.pdf

[acb]

Hey! Yes, we're definitely exploring what to enhance with AI, not just creating VIs. The DLL import feature already exists -- and we had a more sophisticated solution in NXG -- but has some limitations, and besides, the way you described your ideal workflow with an AI generating the wrapper VIs... really great. Your ideas welcomed... make a list, however long!

On 5/30/2023 at 2:01 AM, LogMAN said:

What I'm looking for is not so much an AI that writes my code (because I know how to do that), but one that accelerates my development process by suggesting changes in the context of my code

^ Great quote! And what we're exploring: to augment and accelerate the test engineer, whether that is a new VI or navigating a complex workflow like CLFN (like you mentioned).

We didn't show it in the keynote, but all the VIs shown in the chat are VI snippets, letting you drag and drop generated VI block diagrams wherever you want! We've explored the gray code idea, but it presents a few challenges; for example, what if the generated grayed out diagram is bigger than the diagram space it has? Do we push everything out to make room for the predicted code? Do we just clip it at the edge of the structure, move nodes, etc? We haven't dug into that yet. Ideas welcomed.

17 hours ago, Jordan Kuehn said:

Here's one for you. The AXS Port is a modbus device that monitors power inverters. It would be great to feed Nigel the attached document and have it generate drivers/API for communication with the device.

Thanks for sharing these! We will use them as test cases. Will dive into them a bit more in the coming week or two.

Edited June 1, 2023 by acb

[ShaunR]

18 hours ago, dadreamer said:

It should also automagically install Visual Studio, write the callback wrapper, compile it into a DLL and connect it to the VI with a CLFN.  

I respectfully disagree.

We should be able to use callbacks for DLL's just like the .NET callback node. However, that isn't an AI solved problem which is why I was laughing.

Edited June 1, 2023 by ShaunR

[LogMAN]

1 hour ago, acb said:

We've explored the gray code idea, but it presents a few challenges; for example, what if the generated grayed out diagram is bigger than the diagram space it has? Do we push everything out to make room for the predicted code? Do we just clip it at the edge of the structure, move nodes, etc? We haven't dug into that yet. Ideas welcomed.

Perhaps diagram zoom could be utilized or the change could be displayed in an overlay (or both combined)?

For example, there could be an icon to indicate that Nigel has a suggestion:

When hovering the icon, it could display the suggested diagram in an overlay from which I can choose to apply them:

Suggestions could also be displayed on subdiagram level, depending on the scope of the suggestion. For example, 

Once applied, the diagram grows to fit the new content.

[ShaunR]

33 minutes ago, LogMAN said:

Perhaps diagram zoom could be utilized or the change could be displayed in an overlay (or both combined)?

For example, there could be an icon to indicate that Nigel has a suggestion:

When hovering the icon, it could display the suggested diagram in an overlay from which I can choose to apply them:

Suggestions could also be displayed on subdiagram level, depending on the scope of the suggestion. For example, 

Once applied, the diagram grows to fit the new content.

Great idea. I'd prefer the second one with a slight modification - It has the icon (the bulb in your example) but you double click on it to view the suggestion. The suggestions are only displayed if a button in the toolbar is depressed (like the "Retain wire Value"). Oh. And a cancel/close button in the window.

Edited June 1, 2023 by ShaunR

[dadreamer]

3 hours ago, ShaunR said:

I respectfully disagree.

We should be able to use callbacks for DLL's just like the .NET callback node. However, that isn't an AI solved problem which is why I was laughing.

I tried to be a little ironic, but failed, it seems. To clarify, I almost don't believe, LV interaction with OS native callbacks will get same enhancements that were made for .NET. The Call Library Function Node didn't get enough love since LV6 or 7.0 and a little more attention has been given to the Register Event Callback node, introduced in LV7.0. Of course, I could propose an idea at NI Idea Exchange section, but I'm pretty sure it won't receive enough kudos to even shift the priorities in the NI's internal development list.

Edited June 1, 2023 by dadreamer

[ShaunR]

1 hour ago, dadreamer said:

I tried to be a little ironic, but failed, it seems.

The problem is that what you described is too plausible - solve the symptom rather than the problem and automate the rigmarole work-arounds we normally go through instead of giving us the language construct that is needed. Look what they (not NI) did with C++ and templates 

[kombela]

Anyone interested in a basic API to chatGPT? I created one a while ago, and thought it may be useful for others