A.I. Imaging (Stable Diffusion)

[ShaunR]

I've been playing around with an A.I. Imaging software called Stable Diffusion. It's written in Python but that's not the part that interests me...

There are a number of web browser user Interfaces for the Stable diffusion back-end. Forge, Automatic 1111, Comfy UI - to  name a couple - but the last one, comfy UI, is graphical UI.

The ComfyUI block diagram can be saved as JSON or within a PNG image. That's great. The problem is you cannot nest block diagrams. Therefore you end up with a complete spaghetti diagram and a level of complexity that is difficult to resolve. You end up with the ComfyUI equivalent of:

The way we resolve the spaghetti problem is by encapsulating nodes in sub VI's to hide the complexity (composition).

So. I was thinking that LabVIEW would be a better interface where VI's would be the equivalent of the ComfyUI nodes and the LabVIEW nodes would generate the JSON. Where LabVIEW would be an improvement, however, would be that we can create sub VI's and nest nodes whereas ComfyUI cannot! Further more, perhaps we may have a proper use case for express VI's instead of just being "noob nodes".

Might be an interesting avenue to explore to bring LabVIEW to a wider audience and a new technology.

Edited November 23 by ShaunR

[codcoder]

Doesn't sound too hard?

[ShaunR]

1 hour ago, codcoder said:

Doesn't sound too hard?

The comfyUI nodes are described by JSON in files called "Workflows" so we could import them and use scripting to create nodes. That's if we want parity. But we could support nesting which ComfUI wouldn't understand.

The WebUI's are just interfaces to create REST requests which we can easily do already. I'm just trying to find a proper API specification or something that enables me to know the JSON format for the various requests. Like most of these things, there are just thousands of Github "apps" all doing something different because they use different plugins. Modern programmers can do wonderful things but it's all built on tribal knowledge which you are expected to reverse engineer. The only proper API documentation I have found so far is for the Web Services which isn't what I want - I'm running it locally.

[Stinus Olsen]

As an experiment, this would be pretty cool - but quite an undertaking I think.
Personally though, I think I'll stick with using ComfyUI/A1111 and Kohya - they don't require me to install anything other than what I need for the task 😆

ComfyUIs nodes are a bit more versatile than you would think at first glance.
I personally like to convert a lot of the frequent parameters I use into inputs and then minimize the remaining workflow nodes to not clutter up the interface.
Inputs to the left, output to the right 😉

A fun fact is that if you save the generated images using the Image Save node instead of the Image Preview one, the workflow is stored alongside the image data stream inside the resulting PNG file.
This allows you to load anyone of your previously generated images and immediately retrieve your workflow from that image - not much unlike how VI snippets in LabVIEW works :D

[ShaunR]

On 11/28/2024 at 7:59 AM, Stinus Olsen said:

As an experiment, this would be pretty cool - but quite an undertaking I think.
Personally though, I think I'll stick with using ComfyUI/A1111 and Kohya - they don't require me to install anything other than what I need for the task 😆

ComfyUIs nodes are a bit more versatile than you would think at first glance.
I personally like to convert a lot of the frequent parameters I use into inputs and then minimize the remaining workflow nodes to not clutter up the interface.
Inputs to the left, output to the right 😉

A fun fact is that if you save the generated images using the Image Save node instead of the Image Preview one, the workflow is stored alongside the image data stream inside the resulting PNG file.
This allows you to load anyone of your previously generated images and immediately retrieve your workflow from that image - not much unlike how VI snippets in LabVIEW works

Comfy UI is a mess and I never seem to have the GB of models, extensions, clips or whatever other voodoo a workflow demands . I actually use Forge UI. That's much more manageable.

Then you can add to that that the issue that all Stable Diffusion UI's have - 50 different people created their own flavour of a plugin on Github and half of them don't bother updating it anymore because they got a real job.

More seriously though...

With ComfyUI  it is that you need to know a huge number of API's and, for want of a better description, the micro-workflows to use them. Those micro-workflows can be encapsulated just as we do a sub VI.

The main criticism I have with the ComfUI is its goups.

Why can't a group be represented as a control node? The groups are unrealised analogies to subVI's.

Where is my "Create Node" option? 

Edited November 29 by ShaunR