Help debugging

[DogOnWood]

I have a project that will need to run 24/7 for weeks on end (power cycling parts) problem is it is dying at the 34-ish hour mark running under LV10, 79-ish hours running under LV9.

No Errors or warnings when compiled.

PC it is running on is running XP, 3GB ram. installed is an PCI-5402 for clock generation and a PCI-6353 A/D to capture data. It uses GPIB to setup several power supplies and get heat sink readings from a Agilent 34907a.

It uses a NI USB to RS232 to control and monitor a chiller. all data is being saved to a network based Access database.

so far I've tried isolating the some of the IO's as stand alone apps (GPIB reads, USB reads, PCI access) using as much of the actual VI as possible. initial try ran for 40 hrs without incident.

when I went from VL10 to LV9 the run time before failure increased, also after the run I got a warning about and insane object. I then recreated the object (large type def cluster used in an array in a functional global, size set at runtime) and re-ran the test under LV10, test ran for 65 hours before failing.

the failure mode is no error, complete freeze, I was able to run trace execution during a failure, nothing, no indicators as to why it died.

I do have a support session going with NI, they even have my code, but so far nothing solid and no real progress.

I'm hoping someone in the community has seen a similar issue or have some suggestions as to what to try in order to isolate the problem, any and all ideas are welcome.

Of courser this is a high profile project with a time constraint and customers screaming for data (is there any other kind?)

thanks,

-Doug

[SuperS_5]

Have you checked your system logs to see if LabVIEW left any error messages. Often there will be an error left there, even when no dialog box shows up. In any case, it kind of sounds like the computer is running out of resources. My first guess would be RAM. Check to see if your endlessly building an array, or some other memory leak. You did not post any code for us to look at, so that will make things difficult. (I am talking about sterilized code of course.)

[Wire Warrior]

You might try running the Desktop Execution Trace Toolkit (DETT) when testing your program. Even without the inclusion of user messages it can provide some insight to your issue. I have to agree with SuperS_5 it does sound like you have a memory or resource leak problem. Check to make sure every refnum you open you close. I would also make sure if you're doing any file access that you are not running into a problem with the hard disk slowing down the system and causing an issue that way. If you haven't done it you can bring up the task manager (assuming windows here) and then go to the processes tab to check the memory and CPU usages for your program. That could give a clue.

Oh and one last thought.....is there a virus checker on the machine? We have had issues with that. If the virus checker runs a scan it can cause a resource starvation on your computer.

Good luck,

Jason

[Ton Plomp]

Perhaps you are maxing out on handles. Windows can only give a dedicated number of handles for each application (link, I have not studied it). But it sound to me that even if you close all references, you will get errors eventually, so be very careful in case of opening handles in a loop. Perhaps you can create your own handler of handles.

Ton

[DogOnWood]

Thanks for the input, My initial thought was memory issues(lack of) as the machine originally only had 1GB of RAM, resource manager only showed it using a quarter gig when it died but I bumped it up to 3GB anyway.

No change in behavior.

Hadn't thought of the system logs, will take a look at that if and when the current run fails.

The Refnum idea is a a good one, I try to make sure I pair all calls but it is possible that I'm not closing something, will dig into that one today. Will also check on a virus checker.

I was able to run the Desktop Execution Trace Toolkit (DETT) and capture at failure time, unfortunately it showed nothing, even sent ti to NI and they said the same thing.

all my arrays are defined at program start, nothing new is created or added too once the test starts, I am however using several queues.

there is no disk access either, unless there is a failure to write to the db which is on a remote server.

will explore the Handles Idea as well.

Thanks again for the help, got a couple new things to try.

-Doug

[DogOnWood]

I fired up the Process Explore and starting looking at the handles, while I seem to be creating and destroying handles equally, I am getting a number of "Invalid handle" and some "access denied".

[sertol]

Hi:

Since you are saving the data into an Access database located in the network make sure no one has opened the database while the test is running. If the database has been left open it would cause an access denied this would cause your computer to use all your system resources and eventually lock-up the computer.This can be solve by creating a copy of he database so it can be access by multiple users.

Another reason for this to happen is if the network is down or is back mode of the anti-virus is checking the network or if big files are been downloding slowing down thew network to much.

If that is the case what you can do is get your self a 2TB hard drive install it in the test computer and save he data to it.

PS. Make sure not to send NULL values to Access it does not likes them.

When you solve your problem lets us know what was the cause and solution. Thanks.

[DogOnWood]

Turns out the "invalid handle" and "access denied" where a non-issue - self inflicted - the NI help had me remove all the delays in the program - one loop updated a time field in the db, well as full speed it was running into issues, once I slowed that portion done those errors went away - false lead.

digging through the code, found a functional global that has several multidimensional arrays that was supposed to get the size set and initialized when the number of DUTs was set - it wasn't.

Seems that was the issue - have no idea why it would work for 64 hours then die, but that seems to be the case - current test is at 120+ hours since I made that one change.

Thanks

-Doug

oh, when I change the system fro XP to Win 7 it went from running 64ish hours to only running <18 before dieing, it also became more random. On XP it was fairly consistent when it died, under Win 7 it became very inconsistent and would die rather randomly time wise. (moved from XP to win 7 to try an eliminate the OS and some pre-installed bloatware as a culprit and IS won't do fresh XP installs anymore - Mordac FTW)