real time loops rates not consistent

[konroh]

Hi All,

I am using LabVIEW 7.1.1 with a realtime program running on a PXI chassis and a host program running on a laptop. My code consist of three main timed loops. I have a low priority communication loop for communicating with the host program running at a 50ms loop rate. I have a low priority logging loop running at 2 ms while logging, and 10 ms when idle. And I have a high priority main loop that does most of the work, running at a 1 ms loop rate (using 1Mhz clock, and 1000uSec loop time). All my loops are pretty solid when not logging (the main loop runs at about 960-1040 uSec, and the other two loops never finish late). When I start logging data (5khz data, 23 channels, averaged every 1 ms - 5 scans - by the main loop whether logging or not) I see the occasional (once every 10 seconds or so) loop that runs as high as 7-8ms. I have the main loop running PID control on a hydraulic actuator, and this size jitter is unnacceptable.

The main loops loads the data into an RT FIFO buffer every time I have an even 512byte block size (23 channels * 8 bytes per channel * 64 iterations = 11776 block size). I have no build arrays anywhere in my target code, I initialize all arrays at startup and just use replace subset. The logging loop writes each chunk of data to file using the regular LabVIEW write file.

Any ideas? I have tried all the simple ideas I could find and read up on the forums here and at NI on possible casuses for excess jitter and nothing I have tried has made any difference. Let me know if I need to give any more information.

Thanks!

Doug

[konroh]

An update: if I comment out "Write File" I can run my application for minutes at a time without a single loop finishing late, and the longest loop time I observed was 1079 uSec. Everything else is happening, including FIFO communication between the loops, just no actual write to file, the file itself is even open.

Anyone know an efficient way to write data to disk without disrupting loop times?

thanks!

[bsvingen]

I have never used a PXI chassis, but have you tried to flush the file more often? You could try to flush the file at every iteration first, to see if that helps, or the opposite - gets even worse.. I would guess that the jittering is caused by the file buffer being full and writing lots of data at once, so therefore flushing the file more often may help.

[crelf]

Hey ex-minion - can you upload your code (or at least a screenshot of it) so we can have a look?

[Mellroth]

Hi Konroh,

You could try to stream data to the host over TCP/IP, instead of streaming data to disk. Then let the host write data to disk.

I made a system that were running engine simulations at 2kHz, logging all data over TCP/IP, and never had a loop running late.

Are you using timed loops, and if you are are they using the same timing source? By using a the same timing source you have the possibility to set the loop offsets so that collissions between loop activity is less likely to occur.

/J

[konroh]

Hi Konroh,

You could try to stream data to the host over TCP/IP, instead of streaming data to disk. Then let the host write data to disk.

I made a system that were running engine simulations at 2kHz, logging all data over TCP/IP, and never had a loop running late.

Are you using timed loops, and if you are are they using the same timing source? By using a the same timing source you have the possibility to set the loop offsets so that collissions between loop activity is less likely to occur.

/J

I tried to flush the file more often, and it made no difference...does realtime use the flush command the same way as windows?

Relf, the code is pretty much just one huge block diagram (real time type code) so a screen shot would only show a small portion of the diagram. JFM, TCP/IP streaming sounds like a good idea, i am going to try it now, I'll let you all know how it works out.

thanks!

[Grampa_of_Oliva_n_Eden]

I tried to flush the file more often, and it made no difference...does realtime use the flush command the same way as windows?

Relf, the code is pretty much just one huge block diagram (real time type code) so a screen shot would only show a small portion of the diagram. JFM, TCP/IP streaming sounds like a good idea, i am going to try it now, I'll let you all know how it works out.

thanks!

There are a zillion things that can kill the determinism of a timed loop.

Could you post images of the TC loop please?

Wild guess?

Are you using any pink clusters in the TC loop?

Your diagram will help us help you!

Ben

[konroh]

There are a zillion things that can kill the determinism of a timed loop.

Could you post images of the TC loop please?

Wild guess?

Are you using any pink clusters in the TC loop?

Your diagram will help us help you!

Ben

Just finished adding the code to stream the data over TCP...and after several minutes of continuous logging the largest loop rate I observed was ~1100 uSec. (as opposed to 7000-8000 using file write) and most were much closer to 1000usec. Is file writing just not able to be done deterministically? And yes, I have one pink cluster, but it's only read from once when I start a certain mode, other than that it just sits and is not written to or read from...oh, and the error cluster...everything else is pre built arrays, or single elements, few subvi's, no globals, few locals, and lots of shift registers. I tried to grab some screen shots, but like I said, the code is too large to see anything meaningful in one screen capture...

As of now I am happy with the performance, just disappointed the 30gig hard drive on the PXI is just about useless for my application...oh well...

thanks for the help!

[Mellroth]

If you want to get even closer to the 1000us limit, check how much idle time you have got in the TC loop.

Then offset your non-TC loops by this value (if they are sharing the same timing source). This way collisions are less likely to occur and, in my experience, better performance is gained.

If you really need to log data to disk, I think it would be possible with some tweaking. But I do not know how much idle time you've got left in the TC loop.

One thing that might be important when streaming to disk in RT, is that you write data in the correct block size. I even think I have seen some discussions at NI.com about that.

If you need any more help, let us know.

/J

Just finished adding the code to stream the data over TCP...and after several minutes of continuous logging the largest loop rate I observed was ~1100 uSec. (as opposed to 7000-8000 using file write) and most were much closer to 1000usec. Is file writing just not able to be done deterministically? And yes, I have one pink cluster, but it's only read from once when I start a certain mode, other than that it just sits and is not written to or read from...oh, and the error cluster...everything else is pre built arrays, or single elements, few subvi's, no globals, few locals, and lots of shift registers. I tried to grab some screen shots, but like I said, the code is too large to see anything meaningful in one screen capture...

As of now I am happy with the performance, just disappointed the 30gig hard drive on the PXI is just about useless for my application...oh well...

thanks for the help!

[Grampa_of_Oliva_n_Eden]

... few locals... I tried to grab some screen shots, but like I said, the code is too large to see anything meaningful in one screen capture...

As of now I am happy with the performance, just disappointed the 30gig hard drive on the PXI is just about useless for my application...oh well...

thanks for the help!

The "Execution Trace Toolkit" should help ID the issues. Lacking a screen shot of the TC Loop, I do not think I can add anything further.

Have fun!

Ben

[Mellroth]

konroh,

I did a quick search on NI.com, and found the information I was talking about earlier.

Take a look at:

http://zone.ni.com/devzone/cda/tut/p/id/3746

for information regarding block size in LabVIEW RT.

/J

One thing that might be important when streaming to disk in RT, is that you write data in the correct block size. I even think I have seen some discussions at NI.com about that.