File Logging in RT

[JustinThomas]

I have a RT application which logs some events during the execution of my application. This is an event log which i later retrieve to examine in case of improper execution. This file is opened as soon as the application starts executing and is never really closed, as i expect events to be logged till the RT system is powered off. The problem is that once in a while I get strange values for the file attributes, and am unable to open the file. The most common problem is that the file size returned by labVIEW is a very large number, which does not match the actual file size.

I don't get the problem if I explicitly close the file. This means the problem is because I do not close the file before power off. Any help would be appreciated.

Regards,

Justin Thomas

[LAVA 1.0 Content]

CITATION(JustinThomas @ May 10 2007, 09:24 AM)

I don't get the problem if I explicitly close the file. This means the problem is because I do not close the file before power off. Any help would be appreciated.

Hi Justin,

Why do you not want to close the file after each write ?

If you end up with very big log file, you may add maximum file length - eg : 10k lines - and create a new log file whenever the max size is reached. This would be quite easy if you turn your "write event to log file.vi" into a FGV that increment a counter each time you call it to log an event.

Hope this can help

[Herbert]

When you open a file and write to it, the values are not immediately written to disc. They are cached in memory until a certain amount of data has accumulated, then the operating system will "flush" the cache to disc. If you need an event log for debugging purposes, you have two options:

• Open, write and close everytime you write. Close will force the operating system to flush your data to disc.
  
• Use the "Flush" function from the "Advanced" sub palette in file i/o. That essentially does the same thing, but it performs a little better, since you save the overhead for re-opening the file.
  

If your system powers off without the OS properly shutting down, and that happens at a point in time when the system is flushing it's disc buffers, you might end up with a corrupted file. In that case, I would recommend to narrow down what causes the error, and then write one log file for each message from the code that appears to cause the problem. This will obviously result in tons of files, but it will tell you when your system bailed out.

Hope that helps,

Herbert

[JustinThomas]

QUOTE(TiT @ May 10 2007, 01:23 PM)

Why do you not want to close the file after each write ?

I would like to reduce the file opening overhead each time, so the file is kept open for the duration of logging.

QUOTE(Herbert @ May 10 2007, 08:16 PM)

If your system powers off without the OS properly shutting down, and that happens at a point in time when the system is flushing it's disc buffers, you might end up with a corrupted file.

I thought in the RT system there was no shutting down sequence, it was just power off.

Right now the solution I have come up with is to use a digital input wired to the power off in my main system as an input to my RT device. The RT device is powered off after two seconds of the digital line going high.

I check for this input in my RT application and perform my clean up operations like file closing.

I would like to know if there is a way to do this through software. Does the RTOS generate any event before switch off which I can use?

Regards,

Justin

[wallyabcd]

Hi;

I presume yo are using the binary mode to log the data; otherwise you would not have this problem !

Logging the data in ascii solves this problem; but then has some overhead.

I have been doing this for the last one year and with proper thread management I get good

performance on the RT using a highly distributed program with heavy data logging.

If you want to go with the binary road, things are more difficult as in an embedded operations, your instrument

needs to recover automatically in case of power failure or unprepared power down.

If you don't close the file properly by flushing the stream, then you'll get an error opening the file under windows...

In this case like one of the users mentioned you have to flush after every write to force the system to commit

the data to the drive. This also has some small overhead too.

In general I find that unless you are doing data acquisition, using the ascii format is worth the price in most cases.

I was very hesitant to do this at the beginning, but I appreciate being able to look at the log files with any simple viewer.

I regular produce text files of 30 megabytes without problems either or overly slwing down of the process.

Walters

Spinx

QUOTE(JustinThomas @ May 11 2007, 06:55 AM)

I would like to reduce the file opening overhead each time, so the file is kept open for the duration of logging.

I thought in the RT system there was no shutting down sequence, it was just power off.

Right now the solution I have come up with is to use a digital input wired to the power off in my main system as an input to my RT device. The RT device is powered off after two seconds of the digital line going high.

I check for this input in my RT application and perform my clean up operations like file closing.

I would like to know if there is a way to do this through software. Does the RTOS generate any event before switch off which I can use?

Regards,

Justin