drjdpowell

Interesting discussion.

My bias against a middle C layer is partly related to my lack of C coding skills.  I can use a third-party library with the confidence that I can fix bugs or add features to the LabVIEW part of it, and with some confidence that the underlying dll (such as HDF5) is widely used and thus probably well tested and near complete.  But issues with the middle layer leave me stuck.  

It also matters how complex the library being wrapped is.   My understanding is that netCDF is a simpler API than HDF5 (with the tradeoff that it cannot do everything that HDF5 can do), and I haven’t found the LabVIEW code needed to call it directly is that complicated.

Regarding the Unreserve() and Abort() callbacks, I would much rather be able to register a VI callback when a VI hierarchy goes idle.  It could then do any cleanup actions like closing things.  Perhaps I should suggest this to NI.  I can, and sometimes do, use such cleanup functions using the “watchdog” asynchronous actions in Messenger Library, but I can’t add Messenger Library as a dependency for a reusable wrapper library (also, asynchronous watchdogs are problematic compared to synchronous callbacks due the possibility of race conditions).  

On December 16, 2016 at 6:45 PM, Stobber said:

djpowell, could you add support for extended error codes to this API? I'm struggling with a SQLITE_CANTOPEN error on a cRIO that never used to appear, and I don't have enough context to know what the heck is wrong.

Also, could you modify SQLite.lvlib:Format Error.vi to yield the full call chain? Figuring out where an error came from without it is sometimes really hard. If you don't want to stringify the call chain all the time, maybe make it an option I can toggle on Open?

See the 1.6.5 version now in the CR.   I added the extended code in the Error Description, after the standard error description.   So SQLITE_CANTOPEN(nnn), with nnn the full code, which you can lookup at http://www.sqlite.org/rescode.html.  At some point, I’ll implement the Extended Error names, but I don’t have time now.

On 19/12/2016 at 8:11 PM, GregSands said:

 The HDF5 libraries are mostly wrapped directly, with a small interface dll just for handling some of the memory allocation issues.

Just a side comment, but I think it is better to avoid a middle-layer dll if at all possible.  I have yet to find a memory allocation issue that could not be solved with LabVIEW.exe:MoveBlock.

So, no one has used netCDF then?  I'm trying to see if there would be any interest in a netCDF LabVIEW API.   The API I've made is incomplete, as I can only justify implementing the minimum needed for the client who's paying for it.   Seems like it's better to just use HDF5 directly, unless one already uses netCDF (which I think is mainly just the Atmospheric Science community).  

17 hours ago, Stobber said:

djpowell, could you add support for extended error codes to this API? I'm struggling with a SQLITE_CANTOPEN error on a cRIO that never used to appear, and I don't have enough context to know what the heck is wrong.

Also, could you modify SQLite.lvlib:Format Error.vi to yield the full call chain? Figuring out where an error came from without it is sometimes really hard. If you don't want to stringify the call chain all the time, maybe make it an option I can toggle on Open?

I will do both those things.  

17 minutes ago, shoneill said:

IIt seems the WAL file does not operate in synchronous mode at all.  I wonder if that affects fobustness at all?

You mean with PRAGMA synchronous=OFF?   With "NORMAL" that you are using I think there is a chance that the WAL will be corrupted on power loss, but that just means the loss of the latest few transactions.   On the default "FULL" setting each transaction to the WAL is synced.  

I would hazard a guess that it's something to do with the Write-Ahead-Log (WAL).  Perhaps an UPDATE is done on the main file normally, but if the main file is locked by a reader the UPDATE is recorded in the WAL file (which might be faster).   Then the completion of the UPDATE happens at the next WAL checkpoint.  I don't know, but regardless, WAL checkpointing makes benchmarking trickier; you might want to manually checkpoint by calling PRAGMA wal_checkpoint(FULL) as part of your performance test (just after the COMMIT) to make sure you are measuring the full cost of your UPDATEs.

I'm developing, for a client, a wrapper for netCDF4, a format that used for large atmospheric-science data sets that is implemented on top of HDF5.  Seems quite nice so far, and I was wondering if anyone had any experience with it?  Or HDF5 in general (which I have never used)?  It's kind of TDMS-like, but with n-dimension arrays.

8 hours ago, maxout said:

Another question is it possible to generate .db file with specific columns and tables using the wrapper you provided?

Tables are created using standard SQL statements.

Thanks from me also.

Sorry, Shaun, I didn’t really follow (though I happily nod along with you bringing up “execution state”).  I still don’t know what this has to do with OOP (or “services” or “non-reentrant subVIs” for that matter).  

3 hours ago, ShaunR said:

Why would you want a dataflow construct when the language supports it implicitly, then?

It doesn’t fully support it as message flow between loops.  A message to a loop, that prompts further messages from that loop, is like a subVI call in regular LabVIEW programming, in that the action waits for the “input”to be available before it happens.  However, this only works for ONE input; a subVI can wait for multiple inputs before it executes.  My primary use of Futures is to extend this such that a loop can take an action only when multiple input messages are received.   It also helps with ordering arrays.  If I call three subVIs in parallel and combine their outputs in an array, that array has a defined order.   But if i send a message to three loops, the reply messages can come in arbitrary order, unless I use Futures to specify the order.

There are no OOP concepts involved.

I didn’t get the OOP reference, and don’t want the casual reader to think that “Futures” are an OOP concept.  They’re a Dataflow concept.  We don’t have “Futures” explicitly in LabVIEW, because every wire is implicitly a future.

My “reply addresses” in “Messenger Library” are also actually callbacks.  Callbacks are very flexible.

Just as a note, my experience with “futures” (or “Write-once, destroy-on-reading Queues” as they are implemented as) is that I mostly use them to easily implement the Scatter-Gather messaging pattern, plus occasionally a Resequencer.  Both come up when one is interacting with multiple loops, and reply messages can arrive in arbitrary order.  “Future tokens” allow enforcement of a specified order.

12 hours ago, Daklu said:

I can't speak to the attitudes of the larger community, but my terminology preference is "future token" and "redeem."  I do agree with Shaun though, it probably doesn't matter as long as you communicate it effectively and are consistent.

Searching “Futures” implementations in other languages, I couldn’t find a standard set of terms.  Some even used the generic and non-evocative “Get” for the part where one waits for a future to become an actual thing.  So I’ll stick with redeeming my tokens.  Thanks.

Does anyone else use “Futures”-like features in their code?   If so, a question about terminology.  Four years after this conversation “Future Tokens” are a significant part my reuse code (“Messenger Library”) and I use them often for handling multiple “actor” loops as arrays.  I was just reading the docs for the Akka actor extension for Java and Scala and they use slightly terminology for their equivalent feature.    They “await” a “future”, while I “redeem” a "future token”.  Which is better or more intuitive?

I like “token” because this is a physical thing that represents something else (a message) that has yet to exist.  But I like “await” because it stresses the fact that we block until the message actually arrives.

Yep.  Right-click on Combo box (inside cluster) and Create>>Property Node>>Strings[]

I just tried it and I had no problem making property nodes for a combo box inside a cluster.

The lock will be being caused by something else than the code being blocked, so you should post the rest of your code.  Are you properly closing the “SQLite Connection” when finished with it?  Leaving a transaction uncommitted?   A Prepared Statement unreleased?

BTW, you don’t need the BEGIN/COMMIT VIs in that code, as it’s only one SQL statement.  

All the functionality is in those two poly VIs (personally, I’d prefer a more spread-out pallet).  There are a few examples installed with the package.

I’ve been doing Approach #1.  I haven’t encountered the problem you mentioned, though.

2009 is what OpenG is currently in, I think, so recursion should work.

Google suggests 8.5 for recursion with Dynamic Dispatch VIs and 2009 for any VI.

But I have direct recursion, not via VI Server, in packages in 2011 (example: Shortcut Menu from Cluster), and, at least in LVOOP classes, I had recursive calls in LabVIEW 8.6.  OpenG uses VI Server because it was written well before that.

Does it require 2012?   I thought recursion worked well before that.