Pierre,
We do have a customer application with similar statistic (6000+ VIs but only around 100+ classes) and the build does take about 15 hours. Like you we have a lot of problems with the build (it is very easy to brake it for no obvious reason and when it take so long to do a rebuild this can very quickly eat weeks of your time).
Here is some info that might help you.
We notice that adding xcontrol can brake our build (ex: adding the 3dsurface plot xcontrol did that [took us 2 weeks to find that this was the issue]).
Make sure your build output is as close as possible to the root folder (ex: C\build) so you wont hit the "path too long" issue that plague windows OS.
Build that fail do sometime succeed when we do a control+shit+run arrow (re-compiles everything in memory) and a resave.
Build that do fail on "regular" 32-bit OS do sometime succeed on 64-bit OS.
Sometime adding more memory to the build machine helps.
When none of the above succeed we have to try to figure out what was change since the last successful build. Note: we try very hard to do build very frequently to alleviate these issues.
We have several machine (for instance Virtual Machine) where we can try several builds in parallel to speed up the debugging.
So my first suggestion would be to let the build go until you get an error as it can take a very long time. Also, if you have not done so, make sure you can try the build on several machines (using VMs [Virtual Machine] is really ideal for this tasks).
I feel your pain, and I wish you the best of luck. Please report back here when you found out what was the issue.
PJM