A simple method to buffer input

[Taylorh140]

I find myself frequently looking for a good pattern for collecting a pool of array elements until they reach a certain size and then removing the oldest elements first. I have used very stupid methods like a bunch of feedback nodes being fed into an build array node. But today I thought up one that I really enjoyed and I thought that I'd share it. Its a simple pattern and no crossing wires . Perhaps someone has thought of something better, if so don't hesitate to share.

[infinitenothing]

I've never done performance testing but I've always used rotate and replace to avoid the build array. It should be very easy to make a malleable VI out of it to further reduce block diagram clutter.

For example:

Edited November 15, 2017 by infinitenothing

[crossrulz]

There is also the Data Queue VI in the PtByPt palettes.  Personally, I found a fixed sized lossy queue to work very well as a circular buffer.

[hooovahh]

Very neat and postage stamp sized.  There are apparently lots of different ways to perform this and I have yet another way that I've been doing.  This is part of my Force 1D Array Min or Max Size VIM that is part of my array package found here.  I am curious what is the fastest, and under what conditions one works better than another.

 

Also somewhat relevant is my Circular Buffer XNodes I made here.  This one uses the shifting method, but tried to be a little more efficient with things like not shifting if it wasn't needed, and if a shift was needed to keep the shifted data so a consecutive read wouldn't require a shift or split and build.

[Taylorh140]

So I did a quick run, and I probably need to state that the above methods cannot really be compared apples to apples. Partially for the following reasons:

• Some methods only support one element at a time input. If you need to enter 1000 pts at once these methods will probably be slower and involve more operations.
• Some methods like the circular buffer are much more useful in certain situations like where the buffer is needed in different loops or are acquired at different rates. 

here are numbers for single point(one element at a time) inputs:

How long does it take to process 10000 input samples with a buffer size of 1000 on my computer?: 

• Taylorh140 => 8.28579 ms 
• infinitenothing => 2.99063 ms  (looks like shifting wins)
• Data Queue Pt-by-pt => 9.03695 ms (I expected that this would beat mine)
• hooovahh Circular Buffer => 8.7936 ms (Nearly as good as mine and uses DVR)

I would consider all these to be winners, except maybe the Data Queue pt-by-pt (but it is available by default which gives it a slight edge), Perhaps ill have to do another test where inputs are more than one sample.

Note: if you want to try the source you'll need the circular buffer xnodes installed.

buffer.zip

[ensegre]

How often you write new elements versus how often you have to read the whole array may also skew the preference given to one scheme. For example, if you write often a single element, just replacing it in place at (i mod N) may be efficient, whereas the readout which involves an array copy may be left with a more expensive solution. This is my go at it, not double checked, may be bugged.

[infinitenothing]

That VI always returns an array of all zeros. If I use a feedback node as shown it's 10% faster than my first example:

 

[ensegre]

Yes I wasn't diligent at all, I just sketched it to make the point.

[Stagg54]

On ‎11‎/‎14‎/‎2017 at 7:08 PM, crossrulz said:

Personally, I found a fixed sized lossy queue to work very well as a circular buffer.

That is what I typically use... Generally I wrap in a class.

[Jared Breiter]

What would these types of buffers be called? I see that these and circular buffer are both FIFO, but the array in a circular buffer is not in time order (the index of the recent value is last vs: always moving right?)

[Neil Pate]

53 minutes ago, Jared Breiter said:

What would these types of buffers be called? I see that these and circular buffer are both FIFO, but the array in a circular buffer is not in time order (the index of the recent value is last vs: always moving right?)

In the case of a circular buffer your "get data" method would deal with the wrap around so would return to the caller data in time order (otherwise what would be the point).

This is a nice use case for a class as you don't really want to expose the implementation to the calling code.

I have something like this