twstedsurf Posted July 28, 2005 Report Share Posted July 28, 2005 Hey all, i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do. any help would be much appreciated, as this is driving me crazy. thanks, jesse Quote Link to comment
Ed Dickens Posted July 28, 2005 Report Share Posted July 28, 2005 Hey all, i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do. any help would be much appreciated, as this is driving me crazy. thanks, jesse 5438[/snapback] Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display. Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal. Ed Quote Link to comment
twstedsurf Posted July 29, 2005 Author Report Share Posted July 29, 2005 Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display.Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal. Ed 5440[/snapback] yes, but even if it's padded with 0's, shouldn't "=" still return true? Quote Link to comment
Ed Dickens Posted July 29, 2005 Report Share Posted July 29, 2005 yes, but even if it's padded with 0's, shouldn't "=" still return true? 5455[/snapback] If it's all zeros out to the precision of the number type, then it should equal. The problem comes in looking at what's displayed and what it really compared. The front panel display might be showing 1, but what's really contained in the control could be something different depending on what the display properties are set to. Try this. Create a new VI and drop a standard numeric control on the front panel. Now type in "1.000001" (that's 5 zeros) and see what happens when you click somewhere else on the panel to enter the number. It's now just displaying a 1 instead of the entire number you typed in. The entire number is still there, it's just not being displayed correctly because the default display properties only show 6 Significant Digits and hide the trailing zeros. (a bad default choice by NI IMO) Now create another control and just enter 1. Compare these two with the equal function. Should they be equal? Remember, the display properties are not the same as what the diagram wire is carrying. Engineering calculators work the same way. You can set the display to just show 3 decimal places, but the operations work on the entire number, not just what is displayed. Ed Quote Link to comment
kel712 Posted August 7, 2005 Report Share Posted August 7, 2005 It's all to do with rounding errors with floating point numbers. So instead of using "x = y" use "|x-y| <= 1e-14" which checks if the difference of the two numbers is close to 0. See attachment. This is not just a LabVIEW problem either. Kelvin. Download File:post-2707-1123402451.vi Quote Link to comment
pandaman Posted August 18, 2005 Report Share Posted August 18, 2005 It's all to do with rounding errors with floating point numbers. So instead of using "x = y" use "|x-y| <= 1e-14" which checks if the difference of the two numbers is close to 0. See attachment.This is not just a LabVIEW problem either. Kelvin. 5569[/snapback] LabVIEW should integrate this into their comparisons when dealing with floating-point numbers...I don't see the use of preserving traditional exact comparison in this circumstance (even when comparing two different numerics, since one is casted to the other's type anyway). Maybe when LabVIEW 8 comes out :clock: Quote Link to comment
Michael Aivaliotis Posted August 22, 2005 Report Share Posted August 22, 2005 Also, see this FAQ entry: http://forums.lavausergroup.org/index.php?showtopic=250 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.