Improve data transfer performance

[xive]

I am looking for any ideas for how I could improve my data transfer performance. I am running this on a 8196 controller but I think the problem is inherently data transfer. I need to transfer around 40-50 MB/s from an external device to the controller. The problem being that this just seems to be too intensive for the controller. My first approach was using a dedicated gigabit Ethernet connection and to use tcp to transfer the data. The data is split onto four ports of 12MB/s each. The controller experiences a load of 20% when receiving just one of these data channels. To receive all the data is a 80% load. If I combine all the data onto one port I still get around an 80% load. The other problem is that the external device is sending an array of integers. I now need to convert the binary string I get from the labview TCP functions into an array of integers in labview. For each of the ports this puts an extra 12% load on the controller. So if I were to get all the data and convert it that would take roughly 130% load, obviously a problem. I have also tried to use UDP and write a simple receiver in c to make a dll but both approaches have had just as bad of results, granted my c application was probably not built the best. I was wondering if any one had an idea that could maybe give me some performance increase. Maybe a tcp vi that would read tcp data as an array of integers already, thereby skipping the data conversion overhead, or some other protocol I haven't thought of. Perhaps a different physical transport system such as usb 2.0 or firewire. Any help would much be appreciated. thanks.

[Rolf Kalbermatter]

I am looking for any ideas for how I could improve my data transfer performance. I am running this on a 8196 controller but I think the problem is inherently data transfer. I need to transfer around 40-50 MB/s from an external device to the controller. The problem being that this just seems to be too intensive for the controller. My first approach was using a dedicated gigabit Ethernet connection and to use tcp to transfer the data. The data is split onto four ports of 12MB/s each. The controller experiences a load of 20% when receiving just one of these data channels. To receive all the data is a 80% load. If I combine all the data onto one port I still get around an 80% load. The other problem is that the external device is sending an array of integers. I now need to convert the binary string I get from the labview TCP functions into an array of integers in labview. For each of the ports this puts an extra 12% load on the controller. So if I were to get all the data and convert it that would take roughly 130% load, obviously a problem. I have also tried to use UDP and write a simple receiver in c to make a dll but both approaches have had just as bad of results, granted my c application was probably not built the best. I was wondering if any one had an idea that could maybe give me some performance increase. Maybe a tcp vi that would read tcp data as an array of integers already, thereby skipping the data conversion overhead, or some other protocol I haven't thought of. Perhaps a different physical transport system such as usb 2.0 or firewire. Any help would much be appreciated. thanks.

If the data is in binary form already there is no need for any conversion. Just use a Typecast function to change the incloming string into an array of integers. Hopefully you are using Network byte order because that is what the Typecast function will assume at the incoming stream side.

Of course you could write a C code function to receive the data and get it directly into LabVIEW as an integer array and if the data is in Little Endian format this would get rid of the inherent Byte and Word Swap in the Typecast function but if receiving the data already is the bottle neck, that C function most likely won't be much faster than what you have now when receiving that data as a string. After all 50MB is not peanuts for a PC system and typically will require several tens of seconds to be transmitted over a 100 Base TX connection and only slightly less when 1000 Base TX is used, loading the system CPU as well considerably during this time.