So many binary file options!

[torekp]

QUOTE(Ben @ Dec 18 2006, 05:31 PM)

Advice on speed.

Pre-write your files.

Hey Ben, or anyone, does pre-writing of files help with datalogs?

I'm pre-writing a ton of data, then setting the # of records = 0. But perhaps this defeats my purpose. Is it possible/better, after pre-writing the phony data, to simply set the file position at 0 records from start? Will the datalog then correctly overwrite the phony data with real data?

[torekp]

QUOTE(torekp @ Apr 11 2007, 08:00 PM)

Further update: like datalogs, TDMS also always appends; no way to pre-write the files. I wonder then: Ben's method of pre-writing a plain old binary, vs TDMS, which is faster for a file of known maximum size?

Answer: they're about the same. Attached: my testing VI (8.2). I learned something very interesting about Profile Performance and Memory - the time taken to actually write to the files, apparently "doesn't count". According to my tick counts embedded in the code, it took over 2 seconds to write to the files ten times (not counting the initial 5 writes). According to the Profiler, the testing VI used only 500 ms. Liar, liar.

[Herbert]

QUOTE(torekp @ May 17 2007, 08:43 AM)

Liar, liar.

I'm almost sure you've seen it, but just in case ... I posted some more details on how we benchmark file formats at NI on http://forums.lavag.org/index.php?s=&showtopic=7939&view=findpost&p=30185' target="_blank">this thread, including prerequisites and the actual VIs we use to run our benchmarks.

For relatively short periods of writing, the profiler returns only the time it takes to shove your data into the Windows buffer, but that doesn't mean it's on disc yet. Don't yell at it - the poor thing doesn't know any better

Herbert

[torekp]

QUOTE(Herbert @ May 17 2007, 05:36 PM)

For relatively short periods of writing, the profiler returns only the time it takes to shove your data into the Windows buffer, but that doesn't mean it's on disc yet. Don't yell at it - the poor thing doesn't know any better

Herbert

Yeah well, I restrained myself from adding "pants on fire" to "Liar, liar."

The thing is, the Labview thread doesn't immediately continue after "shoving your data into the Windows buffer". If it did, the elapsed time from the difference of the two tick counts would match what the Profiler says. Do you know if this cursed behavior is peculiar to Windows? Can Linux do this much better? How about a PXI box?