Conner P.

Hey James, Could you go into a bit more detail about reasoning behind the EventDVR zero-copy code? It looks to me like the message is still going through a queue until it reaches the async EventDVR Forwarder. I don't see how this provides any benefit over dequeueing directly at the recipient, since it has to go through a queue anyways. It seems the coder is still responsible for making sure that there is only one recipient of the large data packet, otherwise any additional observers necessarily create a copy which makes application of the DVR moot. Edit: I came across this thread: https://forums.ni.com/t5/JDP-Science-Tools/Notes-on-Memory-optimization/td-p/4412690 . It seems your points there imply that the DVR is more about overcoming a limitation of event based communication necessarily creating a copy whether it is warranted or not. (Me speculating) This limitation is unique to events and not an issue for queues-- is that correct?

D'oh! Got it now, thanks...

Looks like the original attachment got lost during a migration. Don't suppose you would mind uploading another copy?

The target system is for optical coherence tomography. We are pulling 2048 wide U16 arrays from a linescan camera spectrometer from a framegrabber using IMAQ. The line rate is 80khz. The data is naturally blocked into "b-scans" which are ~1024 pixels wide, but we want to increase this in the future. The IMAQ buffers reflect this. So we get a 1024x2048 U16 image every 13ms or 75 images per second. The images are then converted to doubles for processing due to the limits of the LV Analysis library. This means 1024x2048x8bytes = 16MB per 13ms or 1.2GB/s. The processing includes linearization through interpolation and FFT. We could probably optimize the hell out of it using C++ but that isn't on the table. At present we are pushing around the IMAQ references and using IPE to do the processing in place. Of course moving references around is not very heavy lifting, but since we aren't running on a RT system it is very much at the mercy of the OS and any delay quickly adds up and can lead to the IMAQ ring buffer overrunning.

I use TortoiseGit for most operations, but do frequently reach for the command line, especially when trying to hunt down specific commits. I agree with the others who say it is a pain in the ass but it is better than the alternative of not having source control. I tried SVN for a while since it is NI recomended but having already been exposed to git from line coding I just felt more comfortable with git.

Hi James (and Messenger Library fans), on page 2 of this thread you made a post showing a reconfigurable analysis pipeline. Would you mind explaining how you handled data transfer between the different analysis actors? I plan to start playing with this today, but would not mind some hints from the master. I am interested in doing something similar for really high data rates and the data transfer has been a sticking point for me. I do think that standard queues in the actor data would be the way do go, but the specific protocol for linking up the queues has me in a quandry. My idea is to have each actor be in charge of it's input queue, which it then passes to its parent actor upon instantiation. On the opposite end, each actor would receive one (or possibly even multiple-- your example does show branching) output queues during its initialization, which could be changed during runtime with a message from the parent actor. Another issue I have contemplated is doing something like a queue swap while there is still data waiting to be processed. I am thinking of giving the processing loop two states: idle and processing. Any changes to the pipeline must occur during an idle state and the transition from processing to idle be driven by a flag passed along with the data saying "I am the last one in this batch" which ensures that the queue is empty before messing with it. This flag would be set by the DAQ process either when it reaches the natural end of the acquisition or is told to stop by its parent. Of course a lossy hard-stop command that involves flushing the queue could also be implemented. Hopefully I didn't botch putting my ideas into words. Looking forward to your input. I will try to report back with some code when I have it.

Bob, I just saw your document. WOW! Thank you for contributing that to the community. I'm looking foreward to reading it!

Hi all, I have a question mostly directed at James, but others can feel free to chime in. How do you represent your actors in UML diagrams? What I have in mind is that receivable messages correspond to public methods, frames in the state machine refer to private methods, and State/Event updates to the listener registry are private and notated with <<notify>> . What do y'all do?

I PM'd James the following question regarding unit testing and the Messenger Library actors. He asked I repost it here so that he can answer publicly for the benefit of the community and it is documented with the rest of Messenger Library discussion.

Ah yea, I hadn't considered using an object. But since I was planning on passing a cluster, what difference does it make if it is an object or a plain cluster? Not much in the message passing regard. But, packing the data in an object does open the door to created class methods to manipulate the data, which in turn could use the command-pattern for the ThreadPool. With that, it wouldn't be so much passing in a set of parameters and returning a separate result, but more passing the object through the ThreadPool and "pushing a button on it"-- very assembly line like in my mind. I think I just realized the potential of the command pattern.

Hi James, how would you recommend making a message class for a specific data type? As a little preview I am trying to make a thread pool class which includes a manager actor and worker actor clones. Normally I would find the variant solution satisfactory, but as we discussed in my post on the NI forum, I have a high throughput application and don't want to give up speed I don't have to. My initial thought was to just make a subclass of MSG.lvclass like variantMSG.lvclass (sorry if the names aren't 100%, don't have LV with me atm), but then I discovered that the internal send and extract variant message methods DO have "abstract" parent versions within MSG.lvclass. I didn't want to go fussing with the library base classes, so instead I designed the send and extract methods of my customMSG.lvclass to accept a generic MSG object and then internally cast it to my custom child class, allowing the internal methods to do the extracting or writing. Is this a reasonable approach?

Holy crap you weren't kidding! That is one deep rabbit hole. I'm amazed at the internal complexity of this library. I guess you really did earn that Dr. title! Very clever solution with the asnyc call to spin-up the reference checker. Is there any reason for choosing a queue vs some other type of reference?

Hi James, I was looking through the VIs and tried searching for an answer, but I can't seem to get my head around how nested actors get triggered to shutdown when their calling actor shuts down. I saw somewhere mention of a queue references whose sole purpose was to indicate to the nested actor if the calling actor is alive, but I cannot figure out how that is implemented in the code. I understand such info is not necessary for usage of the library, but I am curious. Could you please highlight the implementation of this feature?