Merging overlapping/interlaced time series / XY arrays

[Mads]

To analyze large time series remotely I have a client application that splits the data transfer using two methods; It will break down the periods into subperiods that is then assembled into the full period by the receiver *and* if the subperiods are too large as well they are retrieved gradually (interlaced) by transferring decimated sets with a decimation offset. So, as a result of this  I need to merge multiple overlapping fragments of time series (X and Y array sets).

My base solution for this just concatinates two sets, sorts the result based on the time (X values) and finally removes duplicate samples (XY pairs) from it. This is simple to do with OpenG array VIs (sort 1D and remove duplicates VIs) or the improved VIM-versions from @hooovahh, but this is not optimal. My first optimized version runs through the two XY sets in a for-loop instead, a for loop that picks consecutive unique values from either of the two sets until there are no such entries left. This solution is typically 12-15 times faster than the base case (<10 ms to merge two sets into one set with 250 000 samples on my computer e.g.).

Has anyone else made or seen a solution for this before? It is not as generic as the array operations covered by the OpenG array library e.g., but it still seems like something many people might need to do here and there...(I had VIs in my collection to stitch consecutive time series together with some overlap, but not any that handles interlacing as well).

It would be interesting to see how optimized and/or generalized it could be solved. I have attached the two mentioned examples here , they are not thorougly tested yet, but just as a reference (VIMs included just in case...). Below is a picture of the front panel showing an example of the result they produce with two given XY sets (in this case overlapping samples do not share the same value, but this is done just to illustrate what Y value it has picked when boths sets have entries for the same X (time) value):

Merging XY series LV2022.zip

Edited Friday at 02:03 PM by Mads
Example VIs added

[hooovahh]

I don't have anything to contribute to the development here.  Only to say that I really like this type of function, and looking at your source it sure looks efficient.  Thanks for sharing.