AlexA

God I wish I{d seen this about 200 years ago.

Hi, Yeah, for some commands that carry with them data (specifically things that interact with the image receive a message which contains the header information and the image data), I create custom messages. The locked classes is only an issue when you haven't finalised the object data of some custom message. You're right, with good design, it shouldn't be an issue.

Hey Greg, One minor issue I've had with sending lapdog type messages over the network (especially custom message types), is that running the program to test will lock out the classes and prevent editing. As such, I use lapdog on the RT machine, but avoid sending any lapdog messages over the network.

Hi guys, thanks for the insights! A lot of good information from you guys, sorry it took so long for me to get back to it. I did a few tests to kind of side-step the issue. One of the biggest reasons that I was doing file IO on the RT system was the fact that 100 Hz at 512kb per image is ~50MB/s throughput, this was too much to just get everything onto the host computer via the network (when I was on the Uni network). I linked up my host and RT computers with a cross-over cable and ran a simple test that shows I should be able to get images over the network at a rate of about 80Hz without loss (does this seem right to you guys?). Combined with the fact that the windows machine file IO takes ~4ms to save a file and I've chosen to side step the problem like that. Sorry I didn't take it further and really investigate what could be done on the RT system to speed things up!

Ok, further to my previous testing I did some more testing of my own based on some investigation on the forums. I'm working in a PC based LV-RT environment, I think the disc is formatted as Fat32? Anyway, I ran the profiler on my code till the spikes were reaching about 400ms. It turned out that the open file operation was taking a really long time. One of the suggestions on the forums said that Fat32 doesn't handle long file names well. So I removed all the fluffy information and just wrote the pure count as the file name. That resulted in MUCH greater stability, except that now, the profiler didn't record the spikes, but they were observable on the trace. Less frequent, but still increasing in duration. Another suggestion was that you can't have too many files in a directory (as you mentioned James), so I modified the code to create a new directory every 40 images. This stopped the monotonic increase in spike duration, and reduced the random fluctuations in spikes, but I still get periodic spiking (I think it may be related to the first access in each directory as the spikes are spaced every 40 writes). Just to be clear, I'm not recording the creation of a directory in this timing information. That operation is sent as a separate command after 40 images and then the write is clocked. Anyway, the results currently look like this: I'll try preallocating for files, it was something I was doing originally but I stripped it out to benchmark a type of basic file IO. I'll put it back in and see if I can get those write times down. I'm still stumped by these spikes, they don't show up in the profiler as being due to one particular sub VI... Ok, further to my previous post, I modified the code to preallocate a file size (524800 bytes, enough for the image and the header), the results are below. Note that I haven't actually solved the monotonic spike increase as I thought, I just didn't observe for long enough. The write time is down to about 20ms but the spikes are really bad.

Hey guys, So, a quick question. Suppose I have the High-Throughput FFT in a single-cycle timed loop performing transforms on data-sets 1024 points long. For each set of 1024 points, which should be considered completely independent of each-other, do I need to trigger the reset line on the FFT? I ask because I'm seeing a weird periodicity in the results of a "find max" type algorithm which is hunting through the magnitude of the frequency spectrum. At worst, the result of this "find max" should be random, not periodic in nature. My guess is that there is some sort of index shifting going on which could be a result of a bad indexing operation (something I can check and fix myself) or as a result of some sort of accumulation in the FFT. Edit: A quick note, there is definitely no periodic shift in the data itself, it should be more or less constant. Can anyone shed light on the matter? Cheers, Alex

Hi Matin, To answer your specific question, you can set the FP.Open property to false, which means the front panel will not be opened. Yes you are on the right track if your program requires run-time variability. You can set up each data source as a SubVI which accepts an "output queue" as input to the VI. Make this queue of type "Variant Message" where Variant Message is a bundle consisting of a string and a variant. In this way, each SubVI can send a message on its output queue saying whatever you want it to in the text field and loading its data in the variant field. In your main code, you create a message handler loop which monitors this output queue (into which all your SubVIs are placing their data messages) reads the text field to figure out what to do (a string based "state machine", probably more accurately, but less commonly called a message handler), and then processes the data as you require. For each data message, you will require a Variant to Data of the appropriate type to be able to process the data. Launch each Sub VI in a separate loop using the Run method with wait until done set to false, something like below: You don't have to have an input queue if your Subs are self initialising. You may run into troubles stopping the SubVI's, there are many approaches you can take to deal with this, the simplest to use a global Stop Boolean (make sure that only one loop writes to this and all others read it).

Ahh ok, makes sense!

Just a quick question Daklu, why do the message types have an "xxxx message constant.vi"? Along with an associated type def?

Hey Daklu, Yeah you nailed it, it actually just struck me that since I don't care about losing specific frames to the UI, I can just fix the length of a single UI queue (not priority), and load the frame info into the queue with say 1-5ms timeout. If it timesout, no biggy that frame gets discarded. For more important UI messages, I'll double the timeout to say 10ms. Thanks for the link about the file permissions!

Hi Daklu, Thanks as always for the prompt answers! I did go have a look at the library in both explore (the windows directory) and in the project window. I couldn't find anything explicit in the project window to change the file privileges (I didn't look at it files view). In windows, it appeared to have modify privileges enabled, but I may have been looking at the wrong settings. I'm sure I can figure that out. The problem with trying to deploy to real-time is that you can't choose NOT to save it if you still want to deploy (while in the dev environment). The only options are save or cancel. I'm not using the messaging in any time critical applications so I'm not worried about potential for jitter, but thanks for the heads up! Yes, you have answered my questions very clearly. To expand on my intention for the priority queue, I think I am possibly using it in what you call a Work Queue. My UI update queue (the one that takes messages and sends info via network to my host code) is a Priority queue. The high priority queue is for important status updates, the lower priority queue is used to send image data. It's a fixed size, and enqueued in a lossy sense (I don't really care exactly what image the user sees, as long as it's updating at 30 FPS and correct in order). I perform important file IO on the real-time machine. The reason for this is that the connection can handle 30FPS without lagging (building up a backlog of image information), but I may want to run the camera a lot faster (potentially as high as 100Hz). The image data is pulled from FPGA in a sub-loop, then sent to the main message handling loop, and from there to the UI update loop (or simultaneously to file IO). I may be trying to step around the problem of rate limiting (down sampling) the UI update in kind of a brutal fashion, or it may be more elegant, I'm not sure..? Thank you very much!

Hey Daklu, I've bombed you with a series of messages, sorry about that! Anyway, in case you're more a forum guy than a PM guy, I'll repeat my questions here, along with another question I have. First, the most recent question (didn't send it to you via PM): I'm trying to deploy the messaging system to a real-time system. When I started placing the various VI's, it is asking to save (recompile) the NAtive Arrays and Native Types libraries, but when I tell it to accept and save, it says the library exists and I do not have permission to modify it, can you help me out with this? A Re-cap of PM questions (with the dumb ones sorted out): 1.) Is it ok to use the deprecated version of Priority queue? (I already know you have to manually create messages, which isn't a big deal). 2.) Are there plans to reinstate Priority Queue in future releases? If so, do you have an ETA on that? Kind regards and thanks for all your help!

So for anyone that's interested. The FFT will not finish a computation when input valid goes false, even if it has previously received the correct amount of data points to do so. I'm not sure why this decision was made, and frankly find it to be more than a little stupid but that was the reason for my problem.

Thanks mike, I thought it'd be something simple I hadn't understood. Thanks for the insight.

Yeah that was what I was doing, with the added difficulty that different processes could register for the same data queue (not simultaneously). Looking at my code now, I think I had a problem of too much separation of functionality. For example, almost sequential functionality such as reading in a data set from the FPGA and calculating its average, were separated, the reader read the data then passed it to a whole other process that basically existed to calculate the average and subsequently pass it to the UI. Needless complexity. I'm busily trying to create a logical construct, but its very much a learning process for me, I'm still struggling with the fundamental discussion of this thread, the problem of componentising code. I wish I could wrap my head around the whole "write a story where the nouns become objects and the actions become methods" but it just doesn't gel with me at the moment.

Yeah Daklu, after reading James post and letting a few things percolate that had already bothered me (the fact that a "cast to more specific" call feels very similar to a "variant to data" call), I've come to the conclusion that I was wasting my time. Thankfully though, I've largely gotten over my irrational aversion to passing data via the messaging architecture. Contingent on the architecture being able to keep up with my data update rates!