Jump to content

FYI: cRIO VxWorks Bootup Slowdown Issue


mpodhor1

Recommended Posts

FYI:

For those who use cRIO controllers, there is an acknowledged bug in LV8.5.1 (& possibly 8.6.*) which causes an incremental bootup delay for every boot of the system. These are the conditions under which this occurs:

  • Your executable code makes a reference to vilib (signal processing, math, pt-by-pt, etc.); almost anything really.
  • Every bootup causes an lvtxxxxxxxx.xml (where xxxxxxxx is unique for every boot) file to be written to /tmp on the cRIO controller. This .xml file has no purpose. I suspect it is some debugging artifact to show what vilib elements are referenced.
  • These .xml files will accumulate in your /tmp directory. Though they are not large; they really mess-up VxWorks when enough of them are present. If you manually ftp from the command line to your processor and do an ls on /tmp it will sometimes take a minute to get a response. Your ftp client may timeout or lockup. I suspect the TCP/IP stack for this implementation of VxWorks was not tuned for dealing with this many files.
  • Upon subsequent bootups; VxWorks; during initialization performs some type of verification (fsck?) on the flash file system. Since the presence these .xml files in /tmp cause this verification to take longer. My measurements show about a 1 second bootup delay for each 100 .xml files in /tmp. This means your cRIO processor will take 1 second longer to boot for every 100 bootup cycles. This will grow and grow until your controller will no longer boot. A sort of timebomb.
  • I suspect in the VxWorks startup code; the file checking / verification routine is first run; then after a max delay your app is started. Thus after a critical number of .xml files are present; the file checking routines will still be running while your startup app begins to run. This has the potential of causing all sorts of problems with your real-time app; as the processor and memory load are all of a sudden different from when you developed and debugged your app.

The solution:
  • In your LV code; the very first thing to do is call a subvi to delete all .xml files in /tmp. I also call request deallocation to release any memory resources as a protective measure.
  • This will prevent the buildup of .xml files and stop the entire delayed time-bomb effect.

In my case; I was using these controllers in an automotive radar development application which started showing unusual behavior only after a month or two in the field. It took a while to figure out what was happening and why. NI is aware of this issue and I trust it will be fixed in LV2K9; but for all those developers with systems in the field this is one nasty bug to watch out for. My cRIO controllers usually boot in about 15 seconds or so; but I have measured a few that were taking 80+ seconds to boot. The seriousness of this bug for developers will depend on how often their controller gets powered up. For some lucky enough to have the controller always on; possibly a few interruptions per year; no big deal. With other applications that bootup once or more per day; this may be a serious issue.

NI

  • Shame shame. This kind of bug should have been squashed long long ago.

Link to comment
  • 5 weeks later...

Hi..

I m facing the same problem with my digital dyno controller CRIO... The CRIO boot up time slows down with every wake up frusty.gif ..

Can you tell me how can I delete the .xml files from the tmp folder programmitically..

Please help..

Regards,

Bug smile.gif

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.