Std Deviation and Variance.vi outputs erroneous value in corner cases

[X___]

 

Feed an array comprised of non trivial equal numbers (e.g. a 1,000 values of 1.2345) to the Std Deviation and Variance.vi of the Mathematics>>Probability & Statistics palette and look at the standard deviation output using a display format such as that shown below:

 

You will be surprised to see that the standard deviation is non zero and within the range of the precision for the type (~1E-15).

While this might not be an issue in most use cases, it is still not an optimal result, and suggests that the calculation done in the called DLL is not optimal (it probably uses the standard formula, but because of round-off errors, the outcome is non-zero).

I am posting this in the "bug" category, although it is not strictly speaking one, just to warn unsuspecting users (like me), that dealing with floating point numbers comes with its load of gotchas.

 

Here is a snippet to experiment with the phenomenon:

Again the display format of the standard deviation indicator needs to be chosen wisely to reveal the round off error.

 

 

Edited June 21 by X___
typo correction

[mcduff]

I do not think it is a bug, just floating point math. You don't have infinite precision. See below for another example.

 

[infinitenothing]

Yes, as a general rule, it is fairly unusual to stick an equal sign on a floating point number. This applies to most programming languages, not just LabVIEW. So, I would never stick an equals zero on the output of std dev but I have once or twice broken the rule like this:

[Rolf Kalbermatter]

On 6/21/2024 at 8:40 PM, X___ said:

You will be surprised to see that the standard deviation is non zero and within the range of the precision for the type (~1E-15).

Even though many mathematic operators are commutative (meaning the order of execution of those operators does not change the result) this only applies if you have infinite resolution. But in order for a computer to have infinite resolution when doing numerical calculations, it needs also infinite amount of resources such as memory. Obviously that can't be done! 😁