U16 Grayscale onto U32 RGB

[orko]

I have a U32 RGB image streaming in from a webcam, and I want to overlay/merge (whatever is faster) a U16 grayscale image over the top of it. The grayscale image has 0-value pixels that I do not want to include in this "merge". Any non-zero intensity value should be included.

What would the proper IMAQ tools be to merge these together? I've tried to cast the U16 to a U32 RGB which works (loses some intensity resolution due to bringing 16bit down to 8bit color, but I can live with it). Where I'm stuck is I can't seem to find anything that would efficiently overlay/merge *just* the non-zero pixels of this casted image onto my background U32 RGB image. I know this is something silly I'm not thinking of...

I am capturing both images at 30fps and need to perform this operation in realtime. Can someone point me in the right direction of the NI-Vision palletes...or an example perhaps that will push my "ah ha!" button?

[jdunham]

Haven't played with IMAQ for a while, but this operation is called a 'Mask', and there are definitely IMAQ routines for that. You may need to make an intermediate mask image in which all of the non-zero pixels are set to 1.

[Neville D]

Maybe you can "add" the images?

Or best apply a mask to limit the pixels that will be manipulated.

Neville.

[orko]

Thanks for the prompt replies!

Neville, unless I'm mistaken, adding a 32bit-casted grayscale image to the color RGB will give me darker/lighter colors in the areas that I want to totally replace (with the actual grayscale intensity value). I wish it were that easy

JD, I took your and Neville's advice and tried to apply a mask to the color RGB image using the grayscale image casted down to an U8 grayscale image as the mask. It didn't do what I expected, instead the grayscale RGB image is replacing all of the pixels in the color RGB image. Can you take a look and see where I'm going wrong here?

Download File:post-3266-1236298441.vi (LV 8.6)

[jdunham]

It looks to me like you have your Source and Destination images backwards in the IMAQ Mask function. If you do wire a destination, it will be overwritten by the operation. Since you never put any data into the source, all you see is a copy of the mask in the output. For most of the IMAQ functions, you don't need to wire the Image Dst at all, and you just modify the source image.

[orko]

QUOTE (jdunham @ Mar 5 2009, 09:18 PM)

It looks to me like you have your Source and Destination images backwards in the IMAQ Mask function. If you do wire a destination, it will be overwritten by the operation. Since you never put any data into the source, all you see is a copy of the mask in the output. For most of the IMAQ functions, you don't need to wire the Image Dst at all, and you just modify the source image.

I guess I was confused about the nature of the mask VI itself. Reading the docs, I thought it was supposed to copy the Source *into* the Destination, but only in the pixels defined by the mask as non-zero. I think what you're saying is that the function instead makes a *new* copy of the source with the mask non-zero defined pixels...and replaces anything that was in the Destination memory location with this copy.

So if I'm understanding this correctly... how then would I replace the pixels in the RGB image with the non-masked out pixels in this new Masked image? Man, I know this is a very basic question, but the "ah ha!" switch is not quite closed yet...

Thanks for all your help.

[jdunham]

QUOTE (orko @ Mar 5 2009, 10:21 PM)

I guess I was confused about the nature of the mask VI itself. Reading the docs, I thought it was supposed to copy the Source *into* the Destination, but only in the pixels defined by the mask as non-zero. I think what you're saying is that the function instead makes a *new* copy of the source with the mask non-zero defined pixels...and replaces anything that was in the Destination memory location with this copy.

Well remember, all the IMAQ stuff is by-reference (this confused the hell out of me the first time), and images are big and copying is relatively slow, so the whole system is designed never to copy an image unless you force it to. Just about all the functions operate in-place by default. So the idea is that you wire your image to manipulate into the Source input, and leave the Destination unwired, and the routine modifies your source. Sometimes you want to preserve your original image, so you create a second image and wire it into the Dest, and then you have the before and after, but it's pretty rare to need this.

QUOTE (orko @ Mar 5 2009, 10:21 PM)

So if I'm understanding this correctly... how then would I replace the pixels in the RGB image with the non-masked out pixels in this new Masked image? Man, I know this is a very basic question, but the "ah ha!" switch is not quite closed yet...

So the normal thing is to put your original RGB image into Source, and your 16bit image into Mask, and the result should be an altered version of your RGB image with some pixels preserved and others zeroed out.

OK, I think what you want is a bit more complex that the normal mask. Let's see...

Do I have it right that the only part of the RGB image to preserve is places where the mask is zero, and everywhere else, you want the to keep the grayscale values in the 'mask'?

I think you could invert your mask with the IMAQ Compare, and then that is your mask you apply to the RGB image. At that point you should just be able to IMAQ Add that to the grayscale image, since no pixel would be non-zero in both images.

[Neville D]

Make a mask such that

U16 pixels non-zero => mask pix=1 (pixel of interest)

U16 pixels zero => mask pix =0 (not interested)

Apply this mask to the U16 image. Now merge this resultant masked image with your U32 image. Hopefully both images are the same size? (What does merge mean in your case?) Do you want the U16 to display over top of your image (hiding the original)? Displaying with varying amounts of transparency is quite complex (see attached examples) and may not be very fast.

Other approaches:

Overlays will work faster than merges (which involves actually manipulating pixels). You could transform your U16 image to a bitmap and just overlay the bitmap.

ROI tools are very powerful (though not very well documented). If you can define a ROI, you can manipulate those a lot faster. For example you can go from a ROI to a mask and back. You can combine ROI's as well.

Why is it U16? Won't U8 do? Does this overlay change or is it constant from image to image (like a logo for example)? If constant change to bmp at first then just overlay this bmp. Thats what I do with displaying our logo over the processed image on a monitor.

Attached are two interesting examples of merging with transparency. One displays an image of a brain with a tumour overlayed on that image. Its fairly complex, but works. I don't remember where I downloaded them from.

Neville.

[orko]

QUOTE (jdunham @ Mar 5 2009, 10:44 PM)

Well remember, all the IMAQ stuff is by-reference (this confused the hell out of me the first time), and images are big and copying is relatively slow, so the whole system is designed never to copy an image unless you force it to. Just about all the functions operate in-place by default. So the idea is that you wire your image to manipulate into the Source input, and leave the Destination unwired, and the routine modifies your source. Sometimes you want to preserve your original image, so you create a second image and wire it into the Dest, and then you have the before and after, but it's pretty rare to need this.

Thank you for this. It helps to clear up some confusion I had of what Source and Destination actually are.

QUOTE (jdunham @ Mar 5 2009, 10:44 PM)

OK, I think what you want is a bit more complex that the normal mask. Let's see...

Do I have it right that the only part of the RGB image to preserve is places where the mask is zero, and everywhere else, you want the to keep the grayscale values in the 'mask'?

Yes, that is absolutely correct.

QUOTE (jdunham @ Mar 5 2009, 10:44 PM)

I think you could invert your mask with the IMAQ Compare, and then that is your mask you apply to the RGB image. At that point you should just be able to IMAQ Add that to the grayscale image, since no pixel would be non-zero in both images.

HEY! That sounds doable! I'll try that!

[Neville D]

QUOTE (orko @ Mar 6 2009, 02:53 PM)

It would make sense that an overlay would work faster...I am assuming that to do this I would cast my U16 image to a 8-bit bitmap?

Yes

QUOTE (orko @ Mar 6 2009, 02:53 PM)

I'm not sure what you mean by combining ROI's and how that would apply here. My U16 image is always going to be the same size as my RGB, so I'm thinking that I probably won't be able to benefit from cordoning off an ROI.

Say for example:

you had a 3 rectangles that you wanted to keep in an image, ignoring the rest of the image (i.e a mask).

You can define each of these as a ROI using Rectangle to ROI, make an array of ROIs, then use Group ROI's to get one composite ROI.

Next use ROI to Mask to transform it into a mask. Its very fast. You don't touch the pixels in the image until the very last step and that too for a relatively small number of pixels.

N.