Equals not working?

[twstedsurf]

Hey all,

i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do.

any help would be much appreciated, as this is driving me crazy.

thanks,

jesse

[Ed Dickens]

Hey all,

i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do. 

any help would be much appreciated, as this is driving me crazy.

thanks,

jesse

5438[/snapback]

Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display.

Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal.

Ed

[twstedsurf]

Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display.

Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal.

Ed

5440[/snapback]

yes, but even if it's padded with 0's, shouldn't "=" still return true?

[Ed Dickens]

 

yes, but even if it's padded with 0's, shouldn't "=" still return true? 

5455[/snapback]

 

If it's all zeros out to the precision of the number type, then it should equal.

The problem comes in looking at what's displayed and what it really compared. The front panel display might be showing 1, but what's really contained in the control could be something different depending on what the display properties are set to.

Try this.

Create a new VI and drop a standard numeric control on the front panel. Now type in "1.000001" (that's 5 zeros) and see what happens when you click somewhere else on the panel to enter the number. It's now just displaying a 1 instead of the entire number you typed in. The entire number is still there, it's just not being displayed correctly because the default display properties only show 6 Significant Digits and hide the trailing zeros. (a bad default choice by NI IMO) Now create another control and just enter 1. Compare these two with the equal function. Should they be equal?

Remember, the display properties are not the same as what the diagram wire is carrying. Engineering calculators work the same way. You can set the display to just show 3 decimal places, but the operations work on the entire number, not just what is displayed.

Ed

[kel712]

It's all to do with rounding errors with floating point numbers. So instead of using "x = y" use "|x-y| <= 1e-14" which checks if the difference of the two numbers is close to 0. See attachment.

This is not just a LabVIEW problem either.

Kelvin.

Download File:post-2707-1123402451.vi

[pandaman]

It's all to do with rounding errors with floating point numbers. So instead of using "x = y" use "|x-y| <= 1e-14" which checks if the difference of the two numbers is close to 0.  See attachment.

This is not just a LabVIEW problem either.

Kelvin.

5569[/snapback]

LabVIEW should integrate this into their comparisons when dealing with floating-point numbers...I don't see the use of preserving traditional exact comparison in this circumstance (even when comparing two different numerics, since one is casted to the other's type anyway). Maybe when LabVIEW 8 comes out :clock:

[Michael Aivaliotis]

Also, see this FAQ entry:

http://forums.lavausergroup.org/index.php?showtopic=250