Jump to content

Equals not working?


Recommended Posts

Hey all,

i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do.

any help would be much appreciated, as this is driving me crazy.

thanks,

jesse

post-2536-1122563519.jpg?width=400

Link to comment
Hey all,

i set up a program to read in from an excel file(using tab-delimited), and it seems to read into the arrays just fine. However, i need to look through the first column(of two) and pick out two values. I altered the read in vi for pick out the column, and it seems to be when i go through it after the vi runs. The problem with the vi occurs when it is comparing the values in the column array to the entered numbers. I have set probes to check the numbers being sent(and while it says that they are the same) the "=" doesnt recognize them as being the same. Attached is an example of what im trying to do. 

any help would be much appreciated, as this is driving me crazy.

thanks,

jesse

5438[/snapback]

Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display.

Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal.

Ed

Link to comment
Comparing fractional numbers doesn't always return what we think it should. The decimal presision you are seeing in the probe is not the entire number, it's just it's set to display.

Try creating a couple indicators on the front panel and set their properties to show 10 decimal places (or more if needed) and see if they really are equal.

Ed

5440[/snapback]

yes, but even if it's padded with 0's, shouldn't "=" still return true?

Link to comment
 

yes, but even if it's padded with 0's, shouldn't "=" still return true? 

5455[/snapback]

 

If it's all zeros out to the precision of the number type, then it should equal.

The problem comes in looking at what's displayed and what it really compared. The front panel display might be showing 1, but what's really contained in the control could be something different depending on what the display properties are set to.

Try this.

Create a new VI and drop a standard numeric control on the front panel. Now type in "1.000001" (that's 5 zeros) and see what happens when you click somewhere else on the panel to enter the number. It's now just displaying a 1 instead of the entire number you typed in. The entire number is still there, it's just not being displayed correctly because the default display properties only show 6 Significant Digits and hide the trailing zeros. (a bad default choice by NI IMO) Now create another control and just enter 1. Compare these two with the equal function. Should they be equal?

Remember, the display properties are not the same as what the diagram wire is carrying. Engineering calculators work the same way. You can set the display to just show 3 decimal places, but the operations work on the entire number, not just what is displayed.

Ed

Link to comment
  • 2 weeks later...
  • 2 weeks later...
It's all to do with rounding errors with floating point numbers. So instead of using "x = y" use "|x-y| <= 1e-14" which checks if the difference of the two numbers is close to 0.  See attachment.

This is not just a LabVIEW problem either.

Kelvin.

5569[/snapback]

LabVIEW should integrate this into their comparisons when dealing with floating-point numbers...I don't see the use of preserving traditional exact comparison in this circumstance (even when comparing two different numerics, since one is casted to the other's type anyway). Maybe when LabVIEW 8 comes out :clock:

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.