LVOOP wrapper around picture functions

[eaolson]

I've been working with the LabVIEW picture functions lately and am rather dissattisfied with them. Partly for that reason and partly because I wanted to play around with LabVIEW Objects, I've been fiddling around with creating a object-oriented wrapper for some of the picture functions. One of my main objectives is to completely hide that confusing "image data cluster." However, I have several ideas on how to implement the objects and keep waffling on which is the best. I'd like to ask for comments and suggestions.

1. Create a single Image object. This would handle loading and saving of image files and could be converted to/from a Labview picture datatype. It would basically be a wrapper around the image data cluster, with different case structures depending on the image depth.

2. Create a parent Image object and child Image24, Image8, Image4, and Image1 objects for 24-, 8-, 4-, and 1-bit images. This was my first thought, until I realized it's a significant duplication of effort. Internally, there is no difference between 8, 4, and 1 bit images except for the length of the colors array, so handling these separately didn't really gain me anything.

3. Create a parent Image object and child Truecolor Image (24-bit) and Indexed Image (8, 4, 1 bit) objects. This seems to me to be the most elegant solution, but I'm worried it will bite me in the butt later on. There might be some operations on indexed images that have no analogue to RGB images (i.e. changing the color palette), so I'm not sure how to handle those yet.

LabVIEW also doesn't handle transparency very well. It will load a full alpha channel for a 24-bit image file, but not an indexed image, even if it's present. Furthermore, the picture control can only handle 1-bit transparency (i.e. fully transparent/fully opaque). I wonder if that's likely to change in future versions. So I'm thinking of maintaining the 8-bit alpha channel for 24-bit images, but not for any of the indexed images.

Your thoughts/comments/suggestions/ridicule would be welcome.

[LAVA 1.0 Content]

I don't quite have the time to properly reply fully to your post, but I have dissected the picture control down to the 1's and 0's other than the 3d part of it. I have a very clear roadmap of how to rebuild the functions based upon LVOOP so I am VERY interested in getting you as much information as possible so that we can collaborate upon this. I have some pages of notes that that TST is looking to have also which will expose how to build and dissect the picture control from the ground up. I'll fill more in later, but for now I would highly suggest you put a simple line art picture together and cast it to a string control and look at the format. Also on NI forums I have put some of this information already. Lookup " nuggets and picture control "

Enjoy

[Aristos Queue]

General rule: If you have an enum or ordinal value that is set at the start of the object and is unchanged during the lifetime of the object, that should probably be a child class. I agree that your indexed image sounds best. Based solely on the information presented here it sounds like what you'd want is

Image

__|

__+-- 24-bit image

__|

__+-- Indexed image

______ |

______ +-- 8 bit image

______ |

______ +-- 4 bit image

______ |

______ +-- 1 bit image

Then you can avoid all the duplication of code for the 8, 4 and 1 cases by putting that functionality on the common parent, but you leave yourself room for specific behavior for the 8, 4 and 1 cases if you need it at some point in the future.

I don't have any info about the future of image handling in LV. That would be a question for another developer.

[eaolson]

QUOTE(NormKirchner @ Aug 16 2007, 10:27 AM)

I don't quite have the time to properly reply fully to your post, but I have dissected the picture control down to the 1's and 0's other than the 3d part of it. I have a very clear roadmap of how to rebuild the functions based upon LVOOP so I am VERY interested in getting you as much information as possible so that we can collaborate upon this. I have some pages of notes that that TST is looking to have also which will expose how to build and dissect the picture control from the ground up. I'll fill more in later, but for now I would highly suggest you put a simple line art picture together and cast it to a string control and look at the format. Also on NI forums I have put some of this information already. Lookup " nuggets and picture control "

It seems to me that there are two somewhat distinct issues: LabVIEW pictures and raster images. All the picture functions (Draw Rect, etc) simply append an instruction for drawing a geometric primitive, but don't actually draw it. Once you get more than a thousand of those instructions, rendering the picture to actually display it becomes slow and unwieldy. It looks similar to QuickDraw, which makes sense considering the Mac-origin of LabVIEW. When dealing with an external image, the picture data just sort of puts a wrapper around a raster image.

I've been working with microscope raster images a lot lately and reading them from disk (and across a serial port) becomes complicated if you want to do anything other than load and display them. The nomenclature is confusing (pixmap?) and not used consistently. That's the angle I was coming from, but it sounds like you've given this a lot more thought. I would be interested to see what you've got so far.

QUOTE(Aristos Queue @ Aug 16 2007, 10:29 AM)

Then you can avoid all the duplication of code for the 8, 4 and 1 cases by putting that functionality on the common parent, but you leave yourself room for specific behavior for the 8, 4 and 1 cases if you need it at some point in the future.

Thanks for your suggestion. I'm really starting to think that's the best direction to go.

[LAVA 1.0 Content]

You just keep setting me up for some fun tricks. It turns out that you can get +30 fps easily within the picture control when displaying a live image.

This is another one of my favorite tricks. Since you already can see how the picture control is put together, find where the raw bits of the picture data start and where they end, convert the picture control to a string through a cast, and do a string replace {inplace function :thumbup: } starting at the data start with the new frame image and then send the updated string back to the picture control.

So what you do is in the first frame, allow the std pixmap function to populate the control and then figure out where the insertion point is and only use the string replace for future frames. I was able to get 40fps on a PII 150Mhz pc.

Now w/ objects, encapsulating this functionality may be even easier, or at least cleaner.

[Tomi Maila]

My opinion is very close to Aristos Queue's view here. I'd however add a True Color class above the 24-bit image to allow 32-bit images to be added later on. I could also consider creating a single wrapper class for all LabVIEW image types and use this class as a private data member for all the other classes. For example 8-bit image could have a LabVIEW image in it's private data and delegate functionality to LabVIEW image.

p.s. AQ, do you think it would be possible in a future version of LabVIEW to make it much easier to use composition? I'd like to have an option to create new delegated method so that LabVIEW would automatically generate the method to have the same interface as the same method in a class in private data. So should I have a method jump.vi in a class in my private data, I could make LabVIEW to automatically create a method to my class that would delegate the call directly to jump.vi.

Tomi

[Aristos Queue]

QUOTE(Tomi Maila @ Aug 16 2007, 02:54 PM)

My opinion is very close to Aristos Queue's view here. I'd however add a True Color class above the 24-bit image to allow 32-bit images to be added later on. I could also consider creating a single wrapper class for all LabVIEW image types and use this class as a private data member for all the other classes. For example 8-bit image could have a LabVIEW image in it's private data and delegate functionality to LabVIEW image.

p.s. AQ, do you think it would be possible in a future version of LabVIEW to make it much easier to use composition? I'd like to have an option to create new delegated method so that LabVIEW would automatically generate the method to have the same interface as the same method in a class in private data. So should I have a method jump.vi in a class in my private data, I could make LabVIEW to automatically create a method to my class that would delegate the call directly to jump.vi.

Tomi

Yes, that does seem like something useful that NI could do. Having said that...

The LV features for "Save As on a VI" and "Add VI to library" are already public. The private functions for replacing a control are known by many folks. This is the sort of tool that LV users could write with some ease, I think.

[eaolson]

QUOTE(NormKirchner @ Aug 16 2007, 01:30 PM)

This is another one of my favorite tricks. Since you already can see how the picture control is put together, find where the raw bits of the picture data start and where they end, convert the picture control to a string through a cast, and do a string replace {inplace function :thumbup: } starting at the data start with the new frame image and then send the updated string back to the picture control.

I'm not quite clear on what you mean here. I ran into an update rate problem when working on the Game of Life that's in the CR. The displayed picture consists of many thousands of squares overlaying each other. When I just used Draw Rect a bunch of times and fed the output to the picture control in a loop, it was very slow. Scrolling the picture if it was too large for the control was disastrous; the redraw couldn't keep up. I got around it by using Picture to Pixmap and Draw Flattened Pixmap in conjunction. Basically, it forced the picture data to be rendered to a raster image.

There are two issues to this, I think. One, the instructions in the picture data must be rendered to a rectangular block of pixels. Then those pixels have to be displayed in the picture control. So the data is handed off to the UI thread at some point to update the control. It's not clear to me if the rendering happens in the UI thread or not.

[Darren]

QUOTE(eaolson @ Aug 16 2007, 05:45 PM)

Scrolling the picture if it was too large for the control was disastrous; the redraw couldn't keep up.

I ran into this problem in a recent project...I was able to work around it by changing my picture control into a 2D array of picture controls, where the picture control element in the array had no border (so the pictures in the array all sit next to each other, and it *looks* like one big picture). Turns out using the built-in scrollbars on the array control looks fantastic, with none of the redraw headaches I had before with the single picture control. The only drawback is that scrolling the array takes place in an incremental fashion. For me this isn't a big deal since the UI I was constructing was more or less a grid in the first place, so scrolling in increments of one grid space (i.e. one element of the array) actually looks pretty nice...never have to worry about half a grid row/column being visible in the UI.

-D

[Aristos Queue]

QUOTE(Darren @ Aug 16 2007, 09:39 PM)

The only drawback is that scrolling the array takes place in an incremental fashion.

So if you want smooth scrolling, just use 1x1 picture controls right? :-)

[LAVA 1.0 Content]

the reason it was slow for you is that you had to re-build the entire picture every time, which means parsing all of these, ... move point... draw square... move point... draw square...

FOR EVERYTHING

but if you know where in the bit stream the data you want to change is, such as the color of the 23rd square is at byte location 2345, then all you have to do is change 32 bits at byte 2345 and resend the data to the picture control. MUCH FASTER.

Now it becomes a simple replace bytes at offset rater than re-build entire string w/ new parameters

[Yair]

QUOTE(NormKirchner @ Aug 16 2007, 09:30 PM)

find where the raw bits of the picture data start and where they end, convert the picture control to a string through a cast, and do a string replace

Great trick. :thumbup:

But where did you find a Pentium II?

1997?

[Chris Davis]

QUOTE(yen @ Aug 17 2007, 09:24 AM)

Great trick. :thumbup:

But where did you find a Pentium II?

1997?

Its probably what Norm surfs the web and checks his email on!

[eaolson]

QUOTE(NormKirchner @ Aug 17 2007, 12:43 AM)

but if you know where in the bit stream the data you want to change is, such as the color of the 23rd square is at byte location 2345, then all you have to do is change 32 bits at byte 2345 and resend the data to the picture control. MUCH FASTER.

Ah, I see. That's very clever. It does mean that every time you redraw the picture the entire string has to be rendered. In the case of many graphical primitives, that might be slow. Flattening and unflattening the picture forces it to become an array of pixels. I imagine there is a CPU/memory tradeoff there.

Hmm. I just noticed Draw Rect has an unnecessary type coersion that doesn't seem to be in any of the other picture functions.

[LAVA 1.0 Content]

QUOTE(eaolson @ Aug 17 2007, 09:45 AM)

Ah, I see. That's very clever. It does mean that every time you redraw the picture the entire string has to be rendered. In the case of many graphical primitives, that might be slow. Flattening and unflattening the picture forces it to become an array of pixels. I imagine there is a CPU/memory tradeoff there.

Hmm. I just noticed Draw Rect has an unnecessary type coersion that doesn't seem to be in any of the other picture functions.

The nice thing is, you can send a string directly into a picture control terminal and it figures out what to do right away(as long as its formatted right), and for the picture control to string you use the cast function which didn't seem to add any overhead.

[eaolson]

QUOTE(NormKirchner @ Aug 17 2007, 10:02 PM)

The nice thing is, you can send a string directly into a picture control terminal and it figures out what to do right away(as long as its formatted right), and for the picture control to string you use the cast function which didn't seem to add any overhead.

Hmm, I can not cast a picture to a string. Do you mean Flatten to String instead?

[LAVA 1.0 Content]

QUOTE(eaolson @ Aug 17 2007, 10:37 PM)

Hmm, I can not cast a picture to a string. Do you mean Flatten to String instead?

AH. You're right, but you need only flatten the picture to string once to get your base structure, and then you use that base structure w/ the replaced data each time.

[Aristos Queue]

QUOTE(NormKirchner @ Aug 17 2007, 11:19 PM)

AH. You're right, but you need only flatten the picture to string once to get your base structure, and then you use that base structure w/ the replaced data each time.

Some of the string functions work with the picture string directly (most do not).

http://forums.lavag.org/index.php?act=attach&type=post&id=6680''>http://forums.lavag.org/index.php?act=attach&type=post&id=6680'>http://forums.lavag.org/index.php?act=attach&type=post&id=6680

[Thang Nguyen]

QUOTE (Aristos Queue @ Aug 18 2007, 01:58 AM)

Some of the string functions work with the picture string directly (most do not).

http://lavag.org/old_files/monthly_08_2007/post-5877-1187420273.png' target="_blank">

Hi,

Is there anyone discovered the format of the string pattern or the array pattern of this picture string?

I can recover the picture from the string but I failed to recover the picture with a modified string.

Thanks

[Rolf Kalbermatter]

QUOTE (Thang Nguyen @ Oct 28 2008, 09:23 AM)

Hi,

Is there anyone discovered the format of the string pattern or the array pattern of this picture string?

I can recover the picture from the string but I failed to recover the picture with a modified string.

Thanks

It's simply a byte stream like a LabVIEW string with the only difference that there is a different color on the diagram for it. It's contents are bytes, numbers and arrays packed together and the according format (for at least the published operations) can be reverse egineered frorm the diagram of the variaous Picture Control VIs. This is what Norm talks about in the begin of this thread and what he has done (just as I had about 15 years ago at some point). There is no secret about that, it's just a tedious work to do and in my case I do not have real documentation about that format anymore.

Rolf Kalbermatter

[LAVA 1.0 Content]

Have you took a look at the 'Picture Hacks' in the Lava CR by Adam Rofer?

Ton