Logging text status messages without losing data

[m3nth]

I am looking for suggestions on how to log status messages efficiently to disk without losing them and without having to open and close an entire log file to make sure it saves.

Currently I have a subvi that prints the message both to a message window and also saves it to a file using the 'Write Characters to File.vi' with the "Append?" flag set True. This uses a lot of resources when closing the file once it gets big, because the file is opened and closed for each message. With lots of messages right in a row, this subvi gets banged quite a bit during program execution

I was wondering if using a queue and an external logging loop would be helpful, so disk saving would not be a serial event in the chronological execution of the program, and also if that would be lossy if the program crashed--Which I think revolves around what type of loop you're using to write things to disk, and whether the file remains open. I am under the impression that if you don't close the file and you're writing to it, that it might be cached and then you would lose status messages that should have been saved to disk, if the machine were to go down.

I think that my routines for logging messages work ok, but that they could also be made much more efficient. I'm just wondering if that efficiency would come at a cost to reliability, and if anyone else has had experience in this arena.

[didierj]

I am under the impression that if you don't close the file and you're writing to it, that it might be cached and then you would lose status messages that should have been saved to disk, if the machine were to go down.

2303[/snapback]

In the palette "File I/O">>"Advanced File Functions" there is a function named "Flush File". It is probably an "antique" function, when computers weren't that fast and writing to disk stopped every other execution (including the mouse) on your computer.

Open once the file and keep the reference open as long as your app runs. Write to the file as wished and every time after a write add "Flush File".

Didier

[James McAnanama]

You should consider logging to an SQL database. MySQL is very powerful and you can't beat the cost. You could log your status and define another column in the db table as a timestamp. Every time you write a status message, the timestamp gets added for free. You can access the db remotely, and can of course query it for specific message etc. Once you start using a SQL db, I believe you will find more and more uses for it.

:2cents:

James

[Michael Aivaliotis]

...Currently I have a subvi that prints the message both to a message window and also saves it to a file using the 'Write Characters to File.vi' with the "Append?" flag set True.  This uses a lot of resources when closing the file once it gets big, because the file is opened and closed for each message.  With lots of messages right in a row, this subvi gets banged quite a bit during program execution...

2303[/snapback]

Really? Have you benchmarked this? My impression of the append mode is that it is very fast. LV is smart enough to know that it does not need to open the whole file when appending. The only time it loads the whole file in memory is when you need to read the file and put the data on a wire. It could be the way that the 'Write Characters to File.vi' is programmed. Try using the lower level functions, see attached image. The image assumes the text file already exists.

[m3nth]

First... I'll definitely check out SQL logging... SQL is good but I've not looked into what I need to use it with LabVIEW. Remote logging could be extremely helpful.

Second... I haven't benchmarked file logging necessarily but every time I profile by the amount of time spent, Close File.vi (which is used in "Write Characters to File.vi"), always racks up the most time (or is high up on the list). I haven't looked at it recently... maybe I'll try and run some tests.

[Michael Aivaliotis]

Second... I haven't benchmarked file logging necessarily but every time I profile by the amount of time spent, Close File.vi (which is used in "Write Characters to File.vi"), always racks up the most time (or is high up on the list).  I haven't looked at it recently... maybe I'll try and run some tests.

2330[/snapback]

I guess the point I was really trying to make is that the time allocated to the write function will be the same regardless if the file is 1k or 10MB. If it is the longest time compared to all of your other code is another issue. By the nature of it, it will be longer since it's disk IO. The best would be to buffer it in a functional global and save it when the program exits or at a time when the software is idle. Of course if the computer crashes.... your outta luck.