Leaderboard

Hi all, I developed h5labview for storage and analysis of large, rapidly acquired scientific datasets during my PhD, because of a lack of alternatives at the time. It's been stable since 2012 and provides a robust interface to the HDF5 library. In my case, the requirement for an intermediary library was motivated by (1) need to ensure file handles were closed - especially to prevent locking of networked files, (2) to handle translating between HDF and LabVIEW memory layouts and (3) to implement library callback functions. For simple N-dimensional numeric datasets the interface is trivial, but for arrays of strings, and also for clusters, it is complex. Handles and "Pascal" strings are not HDF-native and there are significant headaches that get introduced for aligned memory on 64-bit platforms. I devoted significant time to working out the complex issues that arise for heterogeneous datatypes, and the polymorphism necessary to handle arbitrary dimensional datasets and arbitrary clusters. The result is available for all to learn from as the project is open-source, although I could probably distill the information gleamed from trial-and-error into a better document! I'm still adding features as I have time to (waveform support is about to be released) but am definitely interested in other developer feedback. My understanding is that netCDF-4 is an HDF5 file with specific applied structure, although I've never used netCDF myself. But I would encourage you to look at the examples on my project webpage and am happy to answer why I did things the way I did. Much of it is because of "pointer nirvana" as Rolf called it, and cluster memory alignment magic. Also, XNodes for arbitrary type adaption without requiring slow flattening-to-string - which is simply too costly for large datasets. Cheers, Martijn

It's no magic really, although I haven't used it myself yet. I make use of other features related to so called UserDataRefnums that are although not really documented a bit more powerful and flexible than the (IMHO misnamed) "DLLs Callbacks". Basically each Call Library Node instance has its own copy of an InstanceDataPointer. This is simply a pointer-sized variable that is associated with a specific Call Library Node. You have the three "Callback functions" Reserve(), Unreserve() and Abort(), each with the same prototoype MgErr (*proc)(InstanceDataPtr *instanceState); So each of them gets a reference to the the Call Library Node instance specific pointer-sized variable location.You could store in there directly any 32 bit information (it's of course 64-bit on 64-bit LabVIEW but you do not want to store more than 32-bits in there for compatibility reasons for the case where you might need to support 32-bit LabVIEW and OSes, such as Pharlap, VxWorks and NI Linux ARM targets) but more likely you will allocate a memory block in Reserve() and return the pointer to that memory block in this parameter. In addition you should make sure the memory is initialized in a meaningful way for your other functions to work properly. The Unreserve() callback is called before LabVIEW wants to unload the VI containing the CLN in order to deallocate anything that might have been allocated or opened by the other functions in the InstanceDataPointer including the InstanceDataPointer itself. Abort() obviously will be called by LabVIEW when the user aborts the VI hierarchy. Now these three functions in itself are not very helpful on their own but where it gets really useful is when you add the special function parameter "InstanceDataPointer" to the parameter list in the Call Library Node configuration. This parameter will not be visible on the diagram for that Call Library Node. Instead LabVIEW will pass the same InstanceDataPointer to the library function as what is passed to the three callback functions. Your function can then store extra information during execution of the function in that InstanceDataPointer that Abort() can use to properly abort any operation that the function itself might have started in the background, including closing files, aborting any asynchronous operation it started, etc, etc. Depending on the complexity you can probably even get away with not implementing the Reserve() function specifically but instead have each function invocation check if the InstanceDataPointer is NULL and then allocate the necessary resources at that point. It may be a performance optimization in not allocating an InstanceDataPointer on load of the VI but only on first execution, so if someone only loads the code without ever starting it, you won't unnecessarily allocate it. If you ever had the "joy" of using Windows API functions with asynchronous operation you will recognize this scheme from the LPOVERLAPPED data pointer those functions use. Remains to stress the fact that every Call Library Node instance has its own private InstanceDataPointer. So if you have 10 Call Library Nodes on your diagram all calling the same library function you still end up with at least 10 InstanceDataPointers. I say here "at least" since this would be multiplied with the number of clones that exist for this particular VI when you have a reentrant VI. As to providing ready made samples with code, that is a crux with this kind of advanced functionality. As it involves asynchronous programming it really is a rather advanced topic. Anyone who understands the explanation as above will pretty readily be able to apply it for their specific application and others who don't won't be helped much with an example that doesn't match their specific use case almost perfectly. Even I get myself regularly lost in the pointer nirvanas where an asynchronous task is accessing the wrong pointer somewhere that the debugger is having a hard time to reach into.