Jimmy Chretien Posted November 20, 2014 Report Posted November 20, 2014 Hello everybody, is someone able to tell me why the Min-Max functions doesn't return an exact same value as connected on the inputs? Attached an example you can easilly reproduce at home. Cheers, Jimmy Quote
Darin Posted November 20, 2014 Report Posted November 20, 2014 Put an Equals? comparison and you will see that it is in fact a bit-for-bit replica of the input. Right-click your 50.01 constant, select Display Format and up the number of SigDigits to 17-18 and compare. 50.01 simply does not have an exact DBL representation. 1 Quote
Jimmy Chretien Posted November 20, 2014 Author Report Posted November 20, 2014 Yes you're right, I wrote to quickly. Quote
JKSH Posted November 20, 2014 Report Posted November 20, 2014 See http://floating-point-gui.de/ -- it explains why floating point numbers lose precision. Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.