Bjarne Joergensen Posted October 25, 2007 Report Share Posted October 25, 2007 Hi I got a problem with the compare function "Greater?". I use it the way you can see in the VI. With the controls set as default there is no problem. The current value cycles between 8 and 4 with 0.2 steps. The problem is that when I set the Start value less than 7.6, current value never reach 8.00. I can't what's wrong with my VI, or is it the Greater? funktion that got a bug ? best regards Bjarne Jørgensen PS: It's the same behaviour in LV6.1 and 8.2 The Vi is in 8.2 Quote Link to comment
Aitor Solar Posted October 25, 2007 Report Share Posted October 25, 2007 Well, the problem arises in the "7.8" step, when you compare two "7.8" values and the "greater?" function returns true. I'm not an expert on this but I believe is usual for floating point numbers not to be exactly the expected value, due to accuracy limits (number of bytes, etc). Surely someone else can explain better. Anyway, it's better to avoid those risks. For example, you can substract both values and check if the difference is greater than 0.1, or work with integers and convert then to double in the end. Saludos, Aitor Quote Link to comment
Bjarne Joergensen Posted October 25, 2007 Author Report Share Posted October 25, 2007 QUOTE(Aitor Solar @ Oct 24 2007, 11:47 AM) Well, the problem arises in the "7.8" step, when you compare two "7.8" values and the "greater?" function returns true. I'm not an expert on this but I believe is usual for floating point numbers not to be exactly the expected value, due to accuracy limits (number of bytes, etc). Surely someone else can explain better.Anyway, it's better to avoid those risks. For example, you can substract both values and check if the difference is greater than 0.1, or work with integers and convert then to double in the end. Saludos, Aitor I have tryed with integers and then there is no problem. If I set the procision to 20 digits for a DBL-control I see a strange behaviour. If I type 0.5 then i display 0.50000000000000000000, but if I type 0.2 then it display 0.20000000000000001100 Why is that ? Bjarne Update: I found this on Lava http://wiki.lavag.org/Numeric This explain the problem Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.