Jump to content

So many binary file options!


Recommended Posts

QUOTE(Ben @ Dec 18 2006, 05:31 PM)

Advice on speed.

Pre-write your files.

Hey Ben, or anyone, does pre-writing of files help with datalogs?

I'm pre-writing a ton of data, then setting the # of records = 0. But perhaps this defeats my purpose. Is it possible/better, after pre-writing the phony data, to simply set the file position at 0 records from start? Will the datalog then correctly overwrite the phony data with real data?

Link to comment
  • 1 month later...

QUOTE(torekp @ Apr 11 2007, 08:00 PM)

Further update: like datalogs, TDMS also always appends; no way to pre-write the files. I wonder then: Ben's method of pre-writing a plain old binary, vs TDMS, which is faster for a file of known maximum size?

Answer: they're about the same. Attached: my testing VI (8.2). I learned something very interesting about Profile Performance and Memory - the time taken to actually write to the files, apparently "doesn't count". :o According to my tick counts embedded in the code, it took over 2 seconds to write to the files ten times (not counting the initial 5 writes). According to the Profiler, the testing VI used only 500 ms. Liar, liar.

Link to comment

QUOTE(torekp @ May 17 2007, 08:43 AM)

I'm almost sure you've seen it, but just in case ... I posted some more details on how we benchmark file formats at NI on http://forums.lavag.org/index.php?s=&showtopic=7939&view=findpost&p=30185' target="_blank">this thread, including prerequisites and the actual VIs we use to run our benchmarks.

For relatively short periods of writing, the profiler returns only the time it takes to shove your data into the Windows buffer, but that doesn't mean it's on disc yet. Don't yell at it - the poor thing doesn't know any better :blink:

Herbert

Link to comment

QUOTE(Herbert @ May 17 2007, 05:36 PM)

For relatively short periods of writing, the profiler returns only the time it takes to shove your data into the Windows buffer, but that doesn't mean it's on disc yet. Don't yell at it - the poor thing doesn't know any better :blink:

Herbert

Yeah well, I restrained myself from adding "pants on fire" to "Liar, liar." ;)

The thing is, the Labview thread doesn't immediately continue after "shoving your data into the Windows buffer". If it did, the elapsed time from the difference of the two tick counts would match what the Profiler says. Do you know if this cursed behavior is peculiar to Windows? Can Linux do this much better? How about a PXI box?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.