ShaunR

OK. Tidied up the benchmark so you can play, scorn, abuse. If we get something viable, then I will stick it in the CR with the loosest licence (public domain, BSD or something) You can grab it here: Prototype Circular Buffer Benchmark It'll be interesting to see the performance on different platforms/versions/bitnesses. (The above image was on Windows 7 x64 using LV2009 x64) Compared alongside each other. This is what we see.

It doesn't tell me when I upload (just says uploading is not allowed with a upload error). When it first happened, I deleted 2 of my code repository submissions just in case that was the problem (deleted about 1.2 MB to upload a 46KB image). It didn't make a difference. There used to be a page in the profile that allowed you to view all the attachments and uploads and monitor your usage. That seems to have disappeared so I can't be sure that deleting more from the CR will allow uploading..

The images are just inserted images that have to reside on another server (using the insert image button in the bar). Usually I upload the images to lavag and insert them. Obviously I cannot do that at the moment so this way is a work-around. I can do something similar for the files I want to upload, but then they won't appear inline in the posts as attachments (and presumably I cannot put stuff in the code repository for people). You will have to be redirected to my download page to get them (not desirable).

For a while now (ever since the last Lavag.org crash). I have not been able to post pictures or upload files, The upload section just states "Uploading is not allowed" and there is no real indication as to why this is so.

You want it 40 usecs because because 40+50 <100? Put your acquisition and processing (50us) in a producer loop and the TX in a consumer loop. Then your total processing time will be just the worst of the two (70us) rather than the addition of both.

It depends where your bottleneck is. 24xDouble precision numbers @ 10k is about 2MB/sec. Doesn't sound a lot to me. Are we talking PXI-RT or PXI-Windows7? How are you acquiring and how are you transferring (TCPIP, MXI?).

Well. Another weekend wasted being a geek So I wrote a circular buffer and took on board (but didn't implement exactly) the Disruptor pattern. I've written a test harness which I will post soon once it is presentable so that we can optismise with the full brain power of Lavag.org rather than my puny organ (fnarr, fnarr). In a nutshell, it looks good - assuming I'm not doing anything daft and getting overruns that I'm not detecting.

It looks to me like the data is already processed. If you just plot the data directly (and change the graph scale to logarithmic) you will get: If you want to smooth it, use the Interpolate 1D.vi and select spline (ntimes= 10).and you will get:

Resolved via email.

You can always just return an array with one element for single values (if a task returns a single value, just use the build array to convert it). Then all your companes are the same. If you really want to, you can wrap that that into a single value polymorphic VI to return just element 0. That way you won't get a run-time error.

Reference: LMAX Technical paper

If I remember correctly. The Array Subset only keeps track of indexes (sub-array type). Would this avoid the copy? Not sure where "parallel" queues came into it. If we were to try parallel queues (I think you are saying a queue for each read process). Then data still has to be copied (within the labview code) on to each queue? Would you not get a copy at the wire junction at least if not on the queues themselves? This scenario is really slow in LabVIEW. I have to use TCPIP repeaters (one of the things I have my eye on for this implementation) and it is a huge bottleneck since to cater for arbitrary numbers of queues, you need to use a for loop to populate the queues and copies are definately made. I think it will be impossible to see the same sort of performance that they achieve without going to compiled code (there is a .NET and C++ implementation) and if our only interest is to benchmark it to the LV queue primitives, we aren't really comparing apples (compiled code implementation of queues in the LV runtime vs labview code). However, the principle still stands and I think it may yield benefits for the aforementioned scenarios (queue-case for example), so I will certainly persevere. Of course. It'd be great if NI introduced a set of primitives in the LV kernel (Apache 2.0 licence I believe )

In general, I think it will not realise the performance improvements for the pointer reasons you have stated (we are ultimately constrained by the internal workings of LV, which we cannot circumvent). I'm sure if we tried to implement a queue in native labview, it wouldn't be anywhere near as fast as the primitives. That said... There a lot of the code seems to be designed around ensuring atomicity. For example. In LabVIEW, we can both read and write to a global variable without having to use mutexes (I believe this is why they discuss CAS). LabVIEW handles all that. Maybe there are some aspects of their software (I haven't got around to looking at their source yet) that is redundant due to LabVIEWS machinations........that's a big maybe with a capital "PROBABLY NOT". I'm not quite sure what you mean about "is going to have to copy data out of the buffer in order to leave the buffer in tact for the next reader". Are you saying that merely using the index array primitive destroys the element? I'm currently stuck at the "back pressure" aspect to the writer as I can't seem to get the logic right. Assuming I have the logic right (still not sure) then this is one instance when a class beats the pants off of classic labview. With a class I can read (2 readers) and write at about 50us, but don't quote me on that as I still don't have confidence in my logic (strange thing is, this slows down if you remove the error case structures to about 1ms ). I'm not trying anything complex. Just an array of doubles as the buffer. DVRs just kill it. Not an option, So it makes classes a bit of a nightmare since you need to share the buffer off-wire. To hack around this, I went for a global variable to store the buffer (harking back to my old "Data Pool" pattern) and the classes just being accessors (Dpendancy Barrier?) and storing the position (for the reader). I should just qualify that time claim in that the class VIs are all re-entrant subroutines (using 2009, so no in-place). Not doing this you can multiply by about 100. Which method did you use to create the ring buffer? I'm currently trying the size mod 2 with the test for 1 element gap. This is slower than checking for overflow and reset, but easier to read whilst I'm chopping things around.

Well. The title was really to place the discussion in the queue producer/consumer pattern vs a ring buffer producer/consumer. Whilst queues generally are just a buffer behind the scenes (some can be linked lists) there is a domain separation here. Queues are a "Many to One". Their real benefit is having many produces and a single consumer. In the one producer, one consumer, this is ok, but the example isn't really a one-to-one although we would shoehorns it into one in LabVIEW such that we have one consumer then branch the wire. Additionally. Looking at the classic dequeue with case statement which many messaging architectures are based on including mine. This is mitigating concurrency by enforcing serial execution. The Disruptor or ring buffer approach is a "One to Many". So it has more in common with Events than queues.Events, however, have a lot of signalling and, in labview, are useless for encapsulation. I've only breached the surface of the Disruptor pattern. But it doesn't seem to be a "Data Is Ready" approach since its premise is to try and remove signalling to enhance performance. The "Write" free wheels at the speed of the incoming data or until it reaches a piece of data that has not been "consumed". By consumed, I do not mean removed, simply that it has been read and therefore is no longer relevant. A "Reader" requests a piece of data that is next in the sequence and waits until it receives it. Once received, it then processes it and requests the next. So. If it is up to the latest, it will idle or yield until new data is incoming. The result seems to be self regulating throughput with back-pressure to the writer and all readers running flat out as long as there is data to process somewhere in the buffer. It also seems to be inherently cooperative towards resource allocation since the fast ones will yield (when they catch up to the writer) allowing more to the slower ones. Here's a pretty good comparison of the different methods. There's also a nice latency chart of the Java performance And finally. Some hand drawn pictures of the basics

It doesn't solve the garbage collection issues in Java. They "alleviate" that by having custom objects and reboot every 24hrs . The advantage of this technique is that M processes can work on the same data buffer in parallel without waiting for all processes to finish before moving to the next. As an example. Lets say we have a stream of doubles being written to a Queue/buffer of length N. We wish to do a mean and linear fit (Pt by Pt). We will assume that the linear fit~ 2x slower and that the queue/buffer is full and therefore blocking writes. With a queue we remove one element, then proceed to do our aforesaid operations (which in LabVIEW we can do in parallel anyway). The queue writer can now add an element. The mean finishes first and then the reader has to wait for the linear fit to finish before it can de-queue the next element. Once the linear fit finishes, we then de-queue the next and start the process again evaluating the mean and linear fit. From what I gather with this technique the following would happen. We read the first element and pass it to the mean and linear fit. The mean finishes and then moves on to the next data point (doesn't wait for the linear fit). Once the linear fit has finished, the next value in the buffer can be inserted and it too moves to the next value. At this point the mean is working on element 3 (it is twice as fast)The result is that the mean travels through the buffer ahead of the linear fit (since it is faster) and is not a consideration for reading the next element from the buffer. Additionally (the theory goes) that once the faster process has reached the end of the data, there are more processing cycles available to the linear fit so that *should* decrease its processing time. Now. They cite that by reading in this fashion, they can parallelise the processing. We already have that capability so I don't think we gain much of a benefit there. But leveraging processing of a function that would spend most of it's time doing nothing due to data being unavailable until the slower process finishes seems like it is worth experimenting with.

I happened to come stumble upon what, was to me, an interesting presentation about high speed transaction processing (LMAX presentation). Their premise was that queues, which are the standard approach, were not the most appropriate for high throughput due to the pipeline nature of queues. To achieve their requirements, they have approached it from using ring buffers which enable them to parallel process data in the ring buffer, thus alleviating, but not eliminating pipe-lining (if the readers are faster than the writer, they still have to wait for data). The "classic" producer consumer in LabVIEW heavily relies on queues and, one of the problems we encounter is when the reader is slower than the writer (we are concerning ourselves with a single write only). Because we can only process the data at the head of the queue, we have a similar throughput problem in that we cannot use LabVIEWs parallelism to alleviate the bottleneck. So I was thinking that the alternative design pattern.that they term the Disruptor might be worth discussing even though we are contained by how LabVIEW manages things in the background (it will probably pan out that LabVIEWs queues will out-perform anything we can write in LabVIEW-parallel or not). Thoughts? (apart from why can't I upload images )

Just append the SENDER (as received by the handler) as part of the ACK message

That's what the Just inspect the SENDER part of the message

The controls have different names from one picture to the other. They are the same controls, right?

Well. It looks like images aren't in the supported datatypes for that package

Well. It's fairly easy to eliminate if that is a suspected problem. Just put a check path exists and log the path before the dialogue is invoked. Then, if you see the dialogue again, you will be able to see whatf the path was and whether the app thinks it exists. You can them paste it into a browse window and see if explorer complains.

What is the default path or current path? Is it pointing to a networked drive? Are all drives actually available that are mapped? I've seen in windows (not specifically LabVIEW) that file dialogues can be "flakey" when network locations are invoked on drives that are unavailable or NAS locations that are asleep. Basically any network access failure will bring windows to it's knees and is usually facilitated by a file action (browse or open/save). When a file dialogue is invoked, it will try to enumerate drives and you only need one to not respond. If you look carefully, you will see that all the indicators are there, they are just not populated (they appear as "discolourations")-classic symptom of struggling to enumerate the drive list. You may find something in the windows event log.

Not from Europe (not sure about NK; cannot find any relevant EU sanctions) . Sanctions to Iran basically cover finance, transport and energy although if you had a "nuclear processing plant.vi", they might group that under "energy". LabVIEW is, apparently, also available in Iran, so I wouldn't be too quick to jump to conclusions about the OPs country of origin. Maybe it just cannot be bought online from wherever he resides. Congratulations! Lavag.org made it on the spooks radar. Three more keywords and they break your door down

Elimination of the case structure isn't that important. The encapsulation of the state-machine is the important bit which is what you have achieved but It is only the equivalent of "create Sub VI" on the case. From your description, you are still "driving" the machine with external, module specific messages which causes you to require the application state-machine (execution engine or sequencer) to know what module specific messages to send and in which order. That's a dependency that I don't like. So. Keep the class and keep the case structure and hard-code the "Message X" and "Message Y" in a couple of frames (probably multiple hard-coded messages in one frame) and we are back to the API. The execution engine only has to worry about application stuff and not what language the state-machines speaks (I just feel that if the messages are hard-coded, then there isn't any point to them). At that point, you can rationalise the interfaces to the modules (same messages) and swap entire modules in and out with the same execution engine (same as switching out the actor, I suppose) OR swap the execution engine with another. The end result of breaking message interdependence is is that you get swappable modules, swappable execution engines and swappable user interfaces and whole sections of a project become reusable (which is why I'd love to see the project manager cope with nested projects).