Aligning two waveforms

[infinitenothing]

I have xy data for two waveforms (on the left). I'd like to figure out what's the offset between them ignoring any points that aren't in the waveform. Right now I'm just grabbing the peak but if the signal get noisy that might not be a great idea. Is there a way to take into account the whole functions?

 

[hooovahh]

I wasn't sure what version of LabVIEW you were using.  The snippet says 2009, but it forced me to open it in 2016 for some reason.  Anyway attached is my quick attempt.  It will take the last N points (in my case 10) from the first plot, and then try to find a match with a tolerance (in my case 0.5) in the second waveform.  If no match is found maybe you could look into adjusting the tolerance.  I also wasn't sure if the second waveform would always continue the first, so maybe grabbing the middle points from the first waveform would be better if you know there is that much overlap.

Detect Waveform Alignment.vi

[infinitenothing]

Thanks for the input. I'm using 2016 but I used the code capture tool to try and backsave it in 2009 incase someone wanted the dataset.

Here's some potential issues I'm concerned with. (last minute requirements ):

• The size of 10 points for the pattern seems arbitrary.
• 0.5 tolerance also seems arbitrary
• The X data isn't really used. For example, your method assumes that the two data sets will be evenly spaced and equally spaced. Fortunately they are pretty close in my example but I'd rather the algorithm could elegantly handle the data if the second data set had 25% more space between the points.
• You seem to only be moving the index in increments of 1. That means that I could have an error of up to 0.5 dx.

[hooovahh]

You're beginning to sound like a customer.   10 Points is arbitrary, 0.5 tolerance as well I figured those could be adjusted as needed.  I was just trying to come up with a slightly more robust solution while making some assumptions until more information came in.  The non-even X would complicate things for sure.  Having to perform more of an integration between times, and then look for when the subtracted integration between two lines at moving X is the minimum.  But again I'd need more information about the types of overlap we are expecting.  Can we always assume the second half of the first waveform is intended on overlap the second waveform?  That would help simplify things a bit.

Honestly most of my assumptions were based on the idea that these were two sets of data read from some analog device with hardware timing.  Like an AI DAQ device, which is why I did it the way I did.  The integration solution is probably a better way to go since it sounds like that is not true

[infinitenothing]

I appreciate your requirement gathering.

Ideally the algorithm would use as many points as possible so if there was only maybe 10% overlap (see image below), hopefully it would be able to work, and if there was 95% overlap, ideally, it would use 95% of the points for the match. The offset won't be bigger than ±0.02

I think I understand the solution you suggested. You mean interpolation right? I can probably make that work but I was hoping that someone knew of a more standard way of solving my problem. No reason to reinvent the wheel right?

[Taylorh140]

I think you might be looking for the cross-correlation function.

https://en.wikipedia.org/wiki/Cross-correlation

 

Edited May 9, 2017 by Taylorh140
wrong information

[infinitenothing]

Does it take xy data or just waveforms and intensity plots?

[hooovahh]

No I meant integrate, which is the area under the curve.  So comparing the area under the curve for some set amount of X time on the first waveform to the same amount of time on the second waveform, and perform the integration then subtract the two and you'll get the difference between the two waveforms for that period of time.  When that number is the minimum for a range of values that that is where the two should line up.  I think the amount of X time can just steadily increase.  So look at the first 0.01 of waveform 2 and the last 0.01 of waveform 1.  Perform integration and see the difference, now the first 0.02 of waveform 2 and the last 0.02 of waveform 1 and do the same.  Of course you can do smaller slices of time. 

Attached is the code that I think does this operation.  It looks at the length of time the second waveform has, and performs the integration on the two waveforms 100 times, with ever increasing X length.  Then it finds when the two waveforms had most similar integral on that period of X.  This 100 is again arbitrary but can be easily increased if you want more precision, it will just take longer the more slices you are using.

Detect Waveform Alignment 2.zip

[Zyl]

Hi InfiniteNothing,

I think Taylorh140 pointed the right tool. Correlation is exactly what you need since it returns the a 'resemblance' ration between X and Y. You may have to slide X array onto Y and remeber where is the highest value from the correlation function. It should give you the index on where to align your arrays.

Using waveforms can also help, because you'll be able to modify the t0 value of one waveform to get the traces aligned on a graph.

[infinitenothing]

4 hours ago, hooovahh said:

No I meant integrate, which is the area under the curve.  So comparing the area under the curve for some set amount of X time on the first waveform to the same amount of time on the second waveform, and perform the integration then subtract the two and you'll get the difference between the two waveforms for that period of time.  When that number is the minimum for a range of values that that is where the two should line up.  I think the amount of X time can just steadily increase.  So look at the first 0.01 of waveform 2 and the last 0.01 of waveform 1.  Perform integration and see the difference, now the first 0.02 of waveform 2 and the last 0.02 of waveform 1 and do the same.  Of course you can do smaller slices of time. 

Attached is the code that I think does this operation.  It looks at the length of time the second waveform has, and performs the integration on the two waveforms 100 times, with ever increasing X length.  Then it finds when the two waveforms had most similar integral on that period of X.  This 100 is again arbitrary but can be easily increased if you want more precision, it will just take longer the more slices you are using.

Detect Waveform Alignment 2.zip

Thanks for the code. I get what you mean now by integration. That's a neat idea. I graphed the input into the integrals and I noticed that XY were switched and the interpolation function was outputting something weird. I also tried deleting the first 3 xy pairs out of the first waveform and the alignment is really off though that might be a function of previous issue. Still, the ideas were useful and I might be able to take it from here.

 

 

 

[infinitenothing]

4 hours ago, Zyl said:

Hi InfiniteNothing,

I think Taylorh140 pointed the right tool. Correlation is exactly what you need since it returns the a 'resemblance' ration between X and Y. You may have to slide X array onto Y and remeber where is the highest value from the correlation function. It should give you the index on where to align your arrays.

Using waveforms can also help, because you'll be able to modify the t0 value of one waveform to get the traces aligned on a graph.

My problem is that correlation takes in two arrays where I have 4: X orig, Y orig, X shifted, Y shifted. I can't figure out how to get 4 into 2.

[Tim_S]

32 minutes ago, infinitenothing said:

My problem is that correlation takes in two arrays where I have 4: X orig, Y orig, X shifted, Y shifted. I can't figure out how to get 4 into 2.

Wouldn't you really want the convolution of the two XY signals? Getting the 4 into 2 would be a matter of converting the XY pairs to a waveform (untested VI attached).

Convert XY to waveform.vi

[infinitenothing]

Thanks for the code. I made it into a subVI and used it as I think you wanted but the answer it's giving me is way off the expected 0.006 or the shift of 0.012 if I ignore T0

[Tim_S]

Your snippet is getting identified as LabVIEW 2016, so it's not loading into 2015 for me.

Looking at the X data sets, I expect the dt are different, which would mess up the convolution (though possibly not so much).

The Xcor Peak should be from... uhm... the length of the convolution is the sum of the length of the input functions, so I believe that "0" is actually the sample the length of the Y data set.

It's been a few uhm... lets say years... since had to work with convolution. Talking it out a little, convolution takes the one waveform and slides it across the other... The first point of the output is the sum of the product of 1 point of overlap of the waveforms (the last point of one waveform and the first point of the other waveform) then the second point is two points (last two of one waveform and first two of the other) and so on. My thinking was at the point where the two overlap you should get the sum of each point squared which should produce a peak value like in the animated image of the two squares example. I just tried this with a sine and cosine centered around zero and got about what I would expect (peaks near +/- 0.25 from center for a 1 Hz, 1 second signals) but got something very different when I put in an offset (1 for each).

 

[infinitenothing]

Huh, not sure why the snipping tool isn't working. I got a peek value as shown in the upper right of the graph but that peak doesn't seem to correspond with anything. I backsaved the projected the old fashioned way and attached it as a zip

Find lag example with correlation.zip

[Tim_S]

Was able to play with your data a bit and haven't managed good results with convolution. Not sure why because it is a means to determine phase offset.

[infinitenothing]

I'm not sure but does convolution rotate the array so it wraps around? If so, there might be some issues with a nonsinusoid. In my case, when the waveforms are perfectly aligned it would be comparing the first signal's segment that looks like this: / to the second signals segment that looks like this: \ making for a bad match.

Edited May 11, 2017 by infinitenothing

[Tim_S]

It doesn't rotate, but does slide of the Y across the X starting with the last point of the Y and the first point of the X, and ending.with the first point of the Y and last point of the X (Wikipedia has a good gif of this).

It is possible convolution works well with sinusoidals and not this data set. I was only getting 35 points in the waveforms when I specified a dt (0.001 I think), so I wonder if the sample size isn't large enough.

[GregSands]

One approach that is similar to, but a little more robust than, your original method is to use the Peak Detection vi which fits a quadratic to the data, and returns a fractional index.  Here I've used it just to shift each waveform so that the peak is at zero, but you could use the shift information in different ways.

Here's a noisy signal, and increasing the width of the fit seems to cope with it ok.

Edited May 12, 2017 by GregSands
Added noisy signal