Jump to content

Deterministic video output


Recommended Posts

Hi,

We've to write a medical application that needs to display video (a long sequence of pictures) on a LCD/CRT screen or a projector in deterministic manner. No frames may be skipped, the timing of the frames needs to be exact within about a few ms accurancy. I tried to google about determinism of video signal but couldn't find much anything. I tried to browse NI products but no-one talks about determinism of the video output. I tried to look for the specifications of PC graphics cards but also there is no information about the determinism of the graphics cards. We experimented a little with a Windows PC running LabVIEW and it seems that some frames of image sequence get skipped for still an unknown reason.

Does anyone have experience on deterministic video displaying. Any suggestions to any direction?

Link to comment

QUOTE (Tomi Maila @ May 15 2008, 11:22 AM)

...

Does anyone have experience on deterministic video displaying. Any suggestions to any direction?

I have never done it myself so I can do is voice some ideas.

I don't think using a standard PC display with the OS driver and screen refresh characteristics is do-able since determinism is just not part of the architecture.

You may want to try seraching for a DVD player that has external control that would allow single-stepping the images. You could drive the single-step using a DO from a counter.

I have driven TV's with simulated video signals from an Arbitrary Waveform Generator. When I did that, the memory of the Arb board was limited. You may want to check on what the current mem capacity of the Arb is now. I think NI was even providing some software at that time to make this easier. (been a long time since i did that one).

Just trying to help out. If nothing else maybe one of these ideas will inspire a proper solution.

Ben

Link to comment

I've never done anything like this in a LV/Windows environment, but did it all the time years ago with dedicated DEC (dating myself) mainframes and image processing equipment. Determinism in this application would simply being able to prove that when you wanted to acquire images you could do so at a specific time and at specific intervals. That part is easy. What would be more difficult would be doing this in a non-RT, Windows environment. I have done deterministic applications in Windows, but the hardware was set up for it. In the case of my DAQ board the board itself had "bus mastering" and I had a deep enough memory that when the trigger was activated the board acquired data and took over the bus. In between scans the memory downloaded (deterministic data) and the board got ready to scan again.

This same principle could be applied to image acquisition. When you trigger the camera to acquire images all you need to know is that the camera is acquiring the images exactly when you tell it to. Now how many images and at what rate I can't be much help since I'm not too familiar with the latest and greatest technology.

Keep us posted, tho. I would love to see what is out there nowadays and what you come up with. :thumbup:

I miss image processing. (sigh)

Link to comment

Hi Tomi

I've used LabVIEW to measure the composite video signal before.

But what I never heard anybody managed to do is to generate a composite video signal from LabVIEW before :-)

So if you get the fastest analog out card, and a real fast computer you could generate your own video signal and feed it to a TV :-)

That would be something to show at NI-week!

post-941-1210881828.png?width=400

Good luck

//Mikael

Link to comment

QUOTE (MikaelH @ May 15 2008, 04:05 PM)

Agreed!

See here for NI 5431 Video Generator that used in the app I mentioned earlier.

We used that to calibrate the guns in the "old" tube type TV's. If your image is simple (small) you may e able to use the built-in looping and chage images on the fly.

Ben

Link to comment

QUOTE (Tomi Maila @ May 15 2008, 10:22 AM)

Hi,

We've to write a medical application that needs to display video (a long sequence of pictures) on a LCD/CRT screen or a projector in deterministic manner. No frames may be skipped, the timing of the frames needs to be exact within about a few ms accurancy. I tried to google about determinism of video signal but couldn't find much anything. I tried to browse NI products but no-one talks about determinism of the video output. I tried to look for the specifications of PC graphics cards but also there is no information about the determinism of the graphics cards. We experimented a little with a Windows PC running LabVIEW and it seems that some frames of image sequence get skipped for still an unknown reason.

Does anyone have experience on deterministic video displaying. Any suggestions to any direction?

Er! The big question here is for what is this good? The human eye has a very limited time resolution so what is it that makes you or your customer believe that the actual display of every single frame to a very accurate time position is so important and not just the overal speed of the movie to the original timeline?

Basically Windows is not real time and any other desktop OS neither. So they are more or less inherently unable to guarantee a whole video frame being transmitted every 40ms (25 frames per second) accurately to a time scale of only a few ms. So any normal video playing software just simply synchronizes the video stream timeline continously to the actual time, skipping frames whenever appropriate. On lower level (for instance when you control the Quicktime API directly but DirectX/Direct Play surely has similar capabilities) you can opt for frame accurate display instead of time accurate display but that usually means that the timeline of the playback is not synchronized with the original timeline anymore as it sooner or later starts to have a time lag.

I do not see any way to guarantee both frame and time accurate display of movie material on non-dedicated hardware other than simply buying the greatest and highest performance hardware components, installing a hardware video decompressor that supports your video compression and preferably has a direct link (crosswire or dedicated PCIe channel) and the meanest and leanest OS you can possible get your hands on and keep your hands crossed that no system interrupts such as network traffic or other DMA transfers will mess up your timing anyhow.

Now with dedicated hardware such as embedded systems with specially optimized RT OSes for media solutions this might be a different story.

Rolf Kalbermatter

Link to comment

QUOTE (rolfk @ May 16 2008, 05:04 AM)

Er! The big question here is for what is this good? ...

Rolf Kalbermatter

I am interested in finding a good solution to this Q since I have some widgets that can "see" and I will have to entertain them durring a test.

The last time I looked, RT platforms did not offer a user interface. Maybe that has changed.

Another way of twisting this request is rather than controlling the video output deterministicly, is to let a video run and then sync the system to the video. I believe there is some bandwidth allocated within composite video signals (before the front porch?) where digital info can be stuffed. A cRIO could watch the video signal and pull out (?) a frame count or something the that effect.

Ben

Link to comment

QUOTE (neB @ May 16 2008, 09:42 AM)

The last time I looked, RT platforms did not offer a user interface. Maybe that has changed.

Not directly, but you can access the UIs of VIs running on RT (enough acronyms for you?) through a web service. You can also use the NI TPCs with shared variable access which is pretty cool.

Link to comment

ZITAT(Tomi Maila @ May 15 2008, 04:22 PM)

We've to write a medical application that needs to display video (a long sequence of pictures) on a LCD/CRT screen or a projector in deterministic manner.

Hi, Tomi,

What means "deterministic"? And which FPS you need and which size of the image?

For example, IMAQ Draw (from IMAQ Vision) can bring up to 100-110 gray images (1024x1024, 8 bit) per second on the screen (I have NVidia 8600).

The other problem which usually occur with live image on the CRT is tearing. You need to synchronize your images with VSync signal of the monitor for getting real good live video. IMAQ Vision since ver.8 have IMAQ WindNonTearing SubVI. When you will set non tearing flag, then your FPS will be reduced according to refresh rate of the monitor. In this case some frames may skip, if IMAQ Draw will be not ready for the next VSync signal.

In my opinion makes no sense to display images with higher FPS as LCD/CRT monitor can.

If determinizm of IMAQ Draw not enough, then you can do it a little bit better if you will using VFW functions or/and DirectX from Microsoft.

hope it helps a little bit,

best regards,

Andrey

Link to comment

QUOTE (Tomi Maila @ May 15 2008, 11:22 AM)

We experimented a little with a Windows PC running LabVIEW and it seems that some frames of image sequence get skipped for still an unknown reason.

It can't gaurantee determinism, but do you need to display the movie through LabVIEW? Could you just launch media player and play it as an avi or the like?

Link to comment

QUOTE (crelf @ May 16 2008, 05:39 PM)

It can't gaurantee determinism, but do you need to display the movie through LabVIEW? Could you just launch media player and play it as an avi or the like?

Well, we have tried. Windows Media Player is everything else but deterministic, at least under XP. It may be that the playback is better under Vista, at least MS claims so. We need to synchronize the playback with measurement and that's how LabVIEW needs to be involved at somelevel. Our LabVIEW application needs to know which frame is being played...

Maybe the few ms accurancy is actually not what we need but we need to get less than a frame duration accurancy, if only possible. We can use LV realtime, if it just supports deterministic video output. Expensive PXI system could be overkill though for the application we are thinking. It should be rather cheap to reach the audience.

Link to comment

QUOTE (Tomi Maila @ May 16 2008, 10:37 AM)

Maybe the few ms accurancy is actually not what we need but we need to get less than a frame duration accurancy, if only possible. We can use LV realtime, if it just supports deterministic video output. Expensive PXI system could be overkill though for the application we are thinking. It should be rather cheap to reach the audience.

You can send images with overlays to the monitor connected to a PXI system running LabVIEW-RT. It is a little known, but very cool feature. Dual-core PXI's don't yet support this feature, but a fix is in the works at NI (a beta driver is currently available if you really need it right away).

The front panel of the application could be accessed via VI-Server/TCP-IP/Web browser to adjust params as required.

Are you playing back a series of still images?

Neville.

Link to comment

QUOTE (Tomi Maila @ May 16 2008, 12:37 PM)

I'm still not sure I can see the need for ms and frame accuracy on playback. However what we did so far was synchronization and combined storage of video and data acquisition http://www.citengineering.com/pagesEN/products/sdvd.aspx' rel='nofollow' target="_blank">Synchronized Video & Data Acquisition Device in order to have time accurate life measurements such as blood pressure, ECG and similar together with the actual video recoding of the operation so that these things can later be exactly localized in relation to the actual action taken at that moment. However playback of this video together with the data is usually not really in real time and definitly not in strict realtime as the researcher normally wants to go through the interesting sections in real slow motion.

Rolf Kalbermatter

Link to comment

QUOTE (Neville D @ May 16 2008, 08:08 PM)

Are you playing back a series of still images?

Practically yes, though the images can be converted to some video format if needed. Do the image overlays also work on a PC based realtime system if it has a SVGA compatible graphics card?

Link to comment

QUOTE (Neville D @ May 16 2008, 01:08 PM)

You can send images with overlays to the monitor connected to a PXI system running LabVIEW-RT. It is a little known, but very cool feature.

It's getting a bit off-topic, but how do you do this? I have a monitor connected to my PXI system (mostly if I need to boot into Windows), but I've never known any way to display anything on the monitor.

Link to comment

QUOTE (Tomi Maila @ May 16 2008, 06:37 PM)

[...] Our LabVIEW application needs to know which frame is being played...

You can use a counter of a DAQ card to count the frames via the V sync. And maybe you can generate an IRQ at every V sync...? With these things you can update the picture on screen according to the actual frame number.

Joris

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.