Image streaming to disk performance ?

[LAVA 1.0 Content]

Hi all,

After seeing this link on NI knowledgebase ( http://zone.ni.com/devzone/cda/tut/p/id/2909 )

I started to developp a small streaming to disk application.

Here is the list of hard/soft :

• LabVIEW 8.20
  
• PCIe 1430
  
• Basler A202k
  
• RAID 0 hard drive
  

With this configuration I found out that the limitation comes from the camera (48 fps, 1000*1000 16bit TIF images), this during a little bit more than 10 minutes ; that makes 96Mo/sec, in the document above NI claims 180Mo/sec...

I'd like to know if any of you have ever made a streaming to disk application and the max dataflow reach during a 10minutes time.

Tanks for any feedback

[Chris Davis]

Your RAID 0, how many hard drives are you using, what size are they, and what kind are they?

I'm using 8 drives setup in a RAID 0 with the Basler 504k (1024x1280x8 bits) and can get over 400 Mbs/sec to disk. I haven't tested this system in the 10 minute range, but I have tested a 150 Mb/sec camera to the same raid system in the 5, 10, and 20 minute range with no problems.

Currently using

Tyan S2895 Motherboard

Highpoint 2224 PCI-X Raid card

8 x Segate 7200.10 400 Gb Baracuda SATA II hard drives

1 x NI 1429

1 x Basler 504k

[LAVA 1.0 Content]

First, thanks for giving your feedback !

We have 4 drives of 150Go and an Intel RAID controler (631xESB Sata RAID).

Do you know which part limits you to 400Mb/sec ? the basler A504k camera or the Raid System ?

[Chris Davis]

I'll assume you are using the 150 GB WD Raptor drives. I've never used the Intel RAID controller you mention though, it looks like it is an onboard RAID controller. These are usually fine to serve 2 to 4 drives, but they don't work beyond that.

The RAID drive limits my performance, this article provide instructions on how to get upto 1 GB to disk, with enough disks of course!

To get the best RAID performance you will have to spend $ on the RAID card, and I've found Highpoint to be a good vendor. I've also found www.barefeats.com to have good reviews on RAID cards / hard drive combinations.

The 504k can run at 500 fps, producing a data rate of 650Mb/s.

[crelf]

Do you know which part limits you to 400Mb/sec ?

Are you using Firewire? If so, ~400Mb/s is a limitation of FireWire 400.

[LAVA 1.0 Content]

Are you using Firewire? If so, ~400Mb/s is a limitation of FireWire 400.

@ crelf : no, Camera link allows better performance (up to 680Mb/s)

By the way, now that this point is raised, is there a faster link between camera and framegrabber ?

I saw that the Basler A504k has 2 camera link cable... I assume this is necessary to reach 650Mb/s, correct ?

@ chrisdavis : Is your solution PC based or MAC based ?

On a complete different ballpark, do you know if NI PCIexpress 1430 is supported by MacIntels ? I was thinking about using a MacPro for another application that works at 4000 fps (1000*128 8 bit) but I'd like to be sure the hard & soft is working together.

BTW : No raid system for this application, it's all about analysing the images as the flows on.

Thanks for any information

[crelf]

By the way, now that this point is raised, is there a faster link between camera and framegrabber ?

I saw that the Basler A504k has 2 camera link cable... I assume this is necessary to reach 650Mb/s, correct ?

There are three "flavours" of CameraLink: Base, Medium (sounds like that's what you've got) and Full.

[Chris Davis]

Are you using Firewire? If so, ~400Mb/s is a limitation of FireWire 400.

Chris, maybe my acronyms were off before, but, IEEE 1394a/Firewire400 specs are 400 megabit (~50 megabytes per second) per second, not 400 megabytes per second.

There are three offical "flavors" of camera link, and a fourth unoffical version, which I call full++. The 504 uses full++, and is now supported by most, if not all, framegrabber companies (including NI and Matrox). Right now, the fastest standard off the shelf link between a camera and framegrabber is cameralink full++. NI has a good application note on what the advantages of cameralink vs gigabit vs firewire vs analog are, you should check it out if you are interested. <opinion>I personally believe the next camera connection step we will see will be a copper / fiberoptic version of PCIe.</opinion>

As far as I know, there is no support for NI's framegrabbers on the mac platform (intel or powerpc). So my solution is PC based.

The other thing that always comes up is how much $ does this cost. 8-10k for the 504k, ~5k for the computer, and ~5k for the fiberoptic extension for the camera, since most people need to position thier camera more than 5 meters (504k's cameralink cable limit) from the computer.

BTW, I know that you can buy 10 meter cameralink cables, but you can't use them with the 504k, that was probably one reason why they didn't accept basler's modifications to the camera link spec.

On a complete different ballpark, do you know if NI PCIexpress 1430 is supported by MacIntels ? I was thinking about using a MacPro for another application that works at 4000 fps (1000*128 8 bit) but I'd like to be sure the hard & soft is working together.

BTW : No raid system for this application, it's all about analysing the images as the flows on.

According to my math you'll be streaming 512 megabytes per second, which means you'll have to use a full cameralink camera to do it. Check to see if your camera is supported by NI before you start, and go ahead and invest in IMAQ, but otherwise you should be fine.

[LAVA 1.0 Content]

According to my math you'll be streaming 512 megabytes per second, which means you'll have to use a full cameralink camera to do it. Check to see if your camera is supported by NI before you start, and go ahead and invest in IMAQ, but otherwise you should be fine.

In that particular case the camera doesn't need to be far from the computer, I'll probably go for a Basler A504k (full++ camera link) + NI PCIe 1429 and a "boosted" PC.

Thanks again for sharing your experience, it's been a precious help !