Jump to content

Choice of logging format/style [TDMS, SQLite, some combination?]


Recommended Posts

1 minute ago, drjdpowell said:

Sorry, what “enumeration”?    

 

On 10/19/2016 at 11:08 AM, ShaunR said:

How about enumerating these alleged design cases for the OP and comparing the practicalities?. I'll start you off........

  • TDMS is designed for high speed disk writing and can write to disk at more than 400MB/s with the right disks. SQlite can only manage about 0.5-1MB/s.

 

Link to comment
1 hour ago, ShaunR said:

SQlite is single writer, multiple readers as it does table level locking. If drjdpowell had followed through with the enumeration it would (or should) have been in there. The high level API in the SQLite API for LAbVIEW insulates you from the Error 5 hell and "busy" handling (when the DB is locked) that you encounter when trying simultaneous parallel writes with low level APIs. So simultaneous parallel writes is not appropriate usage.....sort of ;).

Oh, that enumeration.  Not sure that would be on the list as only one thing can be written to a file at one time, even by TDMS.  SQLite is by default ACID compliant (now that would be on the list), but one can turn that off to get asynchronous disk writes, just like TDMS.  And as long as you writes are faster than your busy-handler timeout (I set that at 5 seconds, adjustable) then there are no Busy(5) errors.  The issue is just write speed, where TDMS wins.

Link to comment
20 hours ago, Christian Butcher said:

Is this (1%) an approximate order of magnitude? Or just an example?

It's an example, but there is a an actual real number I can give you when I'll have taken the time to dig into my Excel sheets...

20 hours ago, Christian Butcher said:

What I'm meaning to ask is, you write an XY-array style, rather than using the waveform data type or an array of waveforms?

Exactly. I don't use Waveforms. Each sample has 3 columns for the time stamp and then X columns for the X channels. So X+3 columns total.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.