Instrument Domain with RT clients

[Dr Byte]

Greetings!

Application: Automation of manufacturing equipment with centralized data collection, multiple PXI chassis' running RT as clients on Microsort Windows 2003 domain.

Problem: I have 6 RT client clones running identical programs (each with slightly different configs). I am using dynamic DAQ loop control to monitor 21 analog channels on each client. The collected data is stored locally on each client and dumped at the end of each process step to a SQL database residing on two clustered database servers. During the execution of a step, the data collected is statistically analyzed and if deemed non compliant, the number of samples collected from key signals is adjusted to "zoom in" on the data for problem identification.

In the event that a number of channel sample rates are increased concurrently, I would like to pool system resources from other clients to contribute to the stability of other digital/analog activities such as PID loop control and timed activation of digital devices.

Is there any way to pool the resources of multiple PXI clients on an "as needed" basis based on an event on one of the clients? Some sort of data shared invoked VI that tracks process status from one client to another? I am pulling out what little hair I have!!!!

Thanks in advance

Dr Byte

[aledain]

I might be missing the point here. Could you not read the data from a central store? Either the SQL database or a new machine using Datasocket server to "pool" the resources. I would use the DS Sever because SQL might be a bit sucky if things start to happen everywhere quickly. The PXI-RT machines can then obtain their resources from the central machine (which could have some smarts in it too, like resolving conflicts, etc; but could be as simple as a host for the DS Server) . Another computer on the network is a cheap option these days.

[Dr Byte]

Thanks for your reply!

DS server running on the LAN to facilitate pooled resources with some allocation of transfer rate dependant on IP of RT client would certainly provide a sharing mechanism for process snapshot data! Perhaps using a managed switch with the DS server living behind it would allow bandwidth management for each of the RT clients without requiring any prioritization code on the DS host !?

However let me re-state the problem:

Picture 6 RT clients, each controlling 6 manufacturing machines, monitoring 21 Analog channels and collecting data on the DS server you mentioned. These clients are also controlling PID loops, communicating with hardware through RS232, analog outs and digital outs. Only one signal being monitored must never be interrupted. The PXI chassis' are fitted with the 8175 embedded controller.

All sample rates on Analog input channels are typically set to less than 5Hz. per channel. If the standard deviation of any signal becomes greater than acceptable, I am progressively increasing the sample rate in order to collect more data on the problem at hand. This puts a significant load on the 8175 and alters the amount of resources available to other tasks.

I could help reduce the problem by adding RT boards to the chassis' that have their own embedded processors, but the cost of the solution across all of the systems would go too high.

What I would like to do is use some sort of Task Prioritization to offload low priority loops to other clients in a pseudo-load balancing manner, where one client would be given a task to perform on another client through the remote clients Analog/Digital/RS232 interface. This would allow all 8175s to share the load of the increased sample rate requirement generated by the excessive standard deviation of the signal in question.

If I had a way of monitoring the performance of the 8175s and offloading tasks to those 8175s which have available resources, that would be great!!!!

Thanks again for your input!!!

Best Regards

Dr. Byte

[aledain]

All sample rates on Analog input channels are typically set to less than 5Hz. per channel. If the standard deviation of any signal becomes greater than acceptable, I am progressively increasing the sample rate in order to collect more data on the problem at hand. This puts a significant load on the 8175 and alters the amount of resources available to other tasks.

2647[/snapback]

I presume that you are using software timed loops here to adjust the sample rate. If you went to a hardware timed loop the resources would be freed up considerably. I would probably over sample and decimate the data instead, then your system does not need to ramp up in response to the signal, just the amount of decimation would decrease and the processing increase on the "affected" channel of interest. I have assumed that the 21 channels are all on the same machine so you have 6 x 21 channels? So on one machine there are 21 channels @ 5 Hz x 8 bytes (DBL) = 840 bytes/S. That's almost nothing! Well almost.

I reckon sample at 1 kHz x 21 channels x 8 bytes = 168k (which could be halved if you convert to SGL which is probably good enough for th real world). However I cannot recall the processor in the 8175 - if it's the P-200 then it might struggle a bit with this many channels (perhaps ... perhaps not), but anything over a P-500 will cope. Anyway this is only a small amount of memory.

So DAQ acquire and decimate by 200 to get your 5 Hz basic signal, any interesting signal can be "sent' to the DS server at full 1kHz rate for processing (in timestamped chunks with probably a mahine ID so you know which set point to adjust on the DS machine), adjusted/processed set points can be read back from the DS server. This now reduces 21 channels to the same loop. Your PID loop can probably still be this same loop, ie make this loop time critical, but the DS communication needs to be done in a separate loop. Alternatively I would move the PID loop to a separate time critical loop. This loop would monitor the set point set by the DS server, the current value (passed by global or whatever) from the DAQ loop with any user control available from the . I have assumed that the PID is running at 1 Hz? If the PID is faster ie >10-100 Hz then this architecture will suffer (but there are ways of beefing it up). (see diagram)

Is it still deterministic? Almost, but not truly because your value read in the DAQ loop will take time to shunt from DAQ to PID loop. Is this a problem? That will depend on your application. If it's a slow manufacturing process (seconds to minutes), I would say it's going to be negligible.

Now about the hardware DAQ, probably do a buffered DAQ rather than wait for 1k samples because a waiting type DAQ uses the CPU to monitor the arrival of the 1000 samples. A buffered DAQ will not use CPU and you can poll for your 1k samples. When you are buffering wait until >=1k is reached, shunt the remainder (N-1k) around the loop (shift register) for the next acquire and process the 1k samples as needed. You will need to tune this loop to run at better than 4 per second depending on the critical lag it introduces into the PID loop (you might be able to set it as low as 10ms therby getting an updated PID value in the PID loop within ~10-15 ms depending on the processing/decimation overhead.

These clients are also controlling PID loops, communicating with hardware through RS232, analog outs and digital outs. Only one signal being monitored must never be interrupted. The PXI chassis' are  fitted with the 8175 embedded controller.

2647[/snapback]

Other comms loops (ie RS-232) could be added as separate loops. The DS Server could also have a client on it that write to the DS Server allowing manual adjustment, debugging etc.

[Dr Byte]

Greetings!

Thank you for your reply!

I will most certainly take your advice and use buffered DAQ/aquire and decimate.

I am using three DAQ boards per chassis, a 6031E, a 6025E and a 6704.

by spreading the load, I believe I can isolate the critical DAQ from the PIDs, but time will tell!

yes I do have 6x21, and it is the P200 (hence my concern).

Timestamp/IP source would be an excellent way to tag data on the DS server. Your suggestion of a seperate client on the DS server causes me to imagine one that interrogates the data, scrubs and dumps to the sql server on the LAN when available bandwidth exceeds a min threshold.

This is a fun project with loads of opportunities for creativity and learning! Thank you for taking the time to share your thoughts!

Best Regards