X___ Posted June 21 Report Share Posted June 21 (edited) Feed an array comprised of non trivial equal numbers (e.g. a 1,000 values of 1.2345) to the Std Deviation and Variance.vi of the Mathematics>>Probability & Statistics palette and look at the standard deviation output using a display format such as that shown below: You will be surprised to see that the standard deviation is non zero and within the range of the precision for the type (~1E-15). While this might not be an issue in most use cases, it is still not an optimal result, and suggests that the calculation done in the called DLL is not optimal (it probably uses the standard formula, but because of round-off errors, the outcome is non-zero). I am posting this in the "bug" category, although it is not strictly speaking one, just to warn unsuspecting users (like me), that dealing with floating point numbers comes with its load of gotchas. Here is a snippet to experiment with the phenomenon: Again the display format of the standard deviation indicator needs to be chosen wisely to reveal the round off error. Edited June 21 by X___ typo correction Link to comment

mcduff Posted June 21 Report Share Posted June 21 I do not think it is a bug, just floating point math. You don't have infinite precision. See below for another example. Link to comment

infinitenothing Posted June 25 Report Share Posted June 25 Yes, as a general rule, it is fairly unusual to stick an equal sign on a floating point number. This applies to most programming languages, not just LabVIEW. So, I would never stick an equals zero on the output of std dev but I have once or twice broken the rule like this: 1 Link to comment

Rolf Kalbermatter Posted June 26 Report Share Posted June 26 On 6/21/2024 at 8:40 PM, X___ said: You will be surprised to see that the standard deviation is non zero and within the range of the precision for the type (~1E-15). Even though many mathematic operators are commutative (meaning the order of execution of those operators does not change the result) this only applies if you have infinite resolution. But in order for a computer to have infinite resolution when doing numerical calculations, it needs also infinite amount of resources such as memory. Obviously that can't be done! 😁 Link to comment

## Recommended Posts

## Please sign in to comment

You will be able to leave a comment after signing in

Sign In Now