ETS Filesystem caching bug

[Jeffrey Habets]

Found this on a cRIO system and NI reproduced it also on a cFP controller, so this problem probably exists on all ETS RT targets.

The filesystem cache seems to get committed to flash every second or so. With each files access you do, the timer used for this is reset. So, if you (like me) for example call the volume info primitive in your idle case and your loop rate is < 1s, this timer is constantly reset and thus all data your write to the flash (either from your app or through ftp) is lost on system reset or reboot unless you gracefully end your application using a stop button and ending your loops (which is mostly not the case on an embedded system).

Call the flush primitive you might say.. Well, this doesn't seem to work on the target.

So basically I see at least two bugs here:

1. The flush should simply work

2. The calling the volume info primitive should not reset the cache timer

[Michael Aivaliotis]

Are you saying that after you do a write to flash, you will lose that data if you do any file IO before the write timer expires? This is odd.

[Jeffrey Habets]

Are you saying that after you do a write to flash, you will lose that data if you do any file IO before the write timer expires? This is odd.

If you keep that up during the lifetime of the program, yes.

I first ran into this because I was constantly (I think it was at a 500 ms rate or so) was reading volume info to provide realtime insight into available diskspace on the flashdevice. After I added the code to do that, all data I thought was written to flash mysteriously disappeared when the device was rebooted. My first thought was something was wrong with the hardware (defective flash e.g.). It took me over a day to figure out what actually was going on.

[Louis Manfredi]

Seems like Flash memory is tricky in general... A client running a non-LabView RTOS in a custom hardware system had many units sold, in the field & running fine for some time. Suddenly new units started hanging.

After much pain & troubleshooting, eventually turned out that the flash card maker had made a minor change to this supposedly fully interchangable commodity card normally used in cameras. The conditions under which the flash card sent interrupts to copy from buffer to non-volatile memory were subtly modified. So a timing error caused an entry to the error log file (stored in flash), which caused a flash interrupt, which caused a new timing error which... :headbang:

Going to another flash manufacturer made the problem go away. (for a while at least...)

Kind of neat gadgets these flash cards, but like anything else, gotta be cautious...

Louis