drjdpowell

Here you go.  Afraid I've not been able to work on it recently, as other priorities keep intruding.

jdp_science_postgresql-0.1.1.8.vip

This makes it not realy intutive... but let me play around with it a little mybe I missunderstand your suggestions at the moment

Reread my suggestion about the cluster.  Don’t do what you just suggested.

Quote

 It was not possible for me to dispatch the object reference dynamically.

Try an Object in a Cluster for your DVR, rather than an Object directly.  There are special issues around Objects in DVRs that are possibly causing your problems.

Yes.  With snapshot off, the image redraws from the buffer whenever it refreshes.  Writing to a terminal triggers a refresh**, but you can also do it by the "Refresh Image" Method.

** Note: LabVIEW controls are running in the UI thread and are asynchronous to their terminal, so writing an Image ref to a terminal only schedules an update for possibly several milliseconds in the future, by which time the image may have been overwritten.  I think this may also be a big source of confusion, as with by-value data the delay is never a problem, but with by reference data it is.

Well, to dequeue from a Queue, you have to provide the Queue reference every time.  You can't call dequeue once and have it automatically keep happening.  The confusing thing is that it is we write the Image Ref to a Control Terminal, rather than a VI.  That's what fools us into thinking of passing images by-value.

1 hour ago, Neil Pate said:

Another related question, in my particular use case I have a piece of code generating a user event with an image as the data at some arbitrary fast rate. The act of Generating the user event creates the a copy, right? 

I always find images annoying because the are by-ref.  You send a copy of the reference to the image, not a copy of the image.  

Cast it to a U64 instead of an I32; then the numbers are different.

The “Create DVR” creates a special memory place, that is not the same as your “Numeric” cluster element or Control.  You are setting that new memory to be equal to your “Numeric” on creation, but they are not thereafter connected.  Those orange wires are by-value.  To get the value inside the DVR, either use the IPE structure or it is provided as an output to the DVR destruction primitive.

Indexing tunnels (conditional or not) follow a preallocation strategy of filling a larger-than-initially-needed array (later cutting the unneeded elements), while the “Build Array” primitive allocates a new array of exactly the right size on each iteration.  So there is a lot fewer calls to the memory manager with an indexing tunnel.

12 hours ago, Manudelavega said:

Uhm, If I reall y want/need to understand this I guess I need to dig into type descriptors and so on?

Type descriptors?  No.   There are VIs such as  “Get Type Information.vi”, which give you the info.

Type descriptors come with flattening data, and flattening/unflattening data is slow.  The various Variant functions are much faster.  

A cluster is a fixed-size array.  I often use this to convert clusters (of mixed types) into an Array of Variants.

What’s a “cracked” module?

Anyway, back to netCDF.   Looks like no-one has used it, and, though it looks like a nice data storage API, it doesn’t offer any improvement over the existing HDF5 labview implementations.   Unless one needs compatibility with existing netCDF tools, but I think that is only if you are in the Atmospheric science community.  The MDSplus library I mentioned above is even more niche; only used by those doing research in nuclear fusion.

7 minutes ago, ShaunR said:

Multi-tasking <> multithreading. You seem to be conflating many different aspects of application execution.

I'm only interested in LabVIEW applications executing correctly when one is calling the same library from two parallel loops.  If libraries fail that, then I do not care if they are "multithreaded" by some technical definition.

I would first try copying your 4Gb file to a file system that can handle larger files, then try opening it.

9 minutes ago, ShaunR said:

The HDF5 library uses recursive locks and keeps a separate error stack for each thread. 

The “multithreaded” version of HDF5, you mean.  The “separate error stack” is actually an example of something you need a LabVIEW-level lock to solve.  LabVIEW does multi-tasking in the UI thread, so if one queries the error information after a function throws an error, it is quite possible that LabVIEW has called some other function from a parallel loop which has overwritten the error stack.  Using Thread-Local Memory to allow “multithreading” is based on the assumption that the threads are not multitasking.  Instead, you need to have a lock held over the original (error-throwing) function call and the secondary call to read the error information.

49 minutes ago, rolfk said:

Another way is to use a semaphore. That can be done explicitedly by the caller (what drjdpowell describes) or in the library itself but the later has the potential to lockup if the library uses multiple global resources that are each protected by their own semaphore.  OpenSSL which Shaun probably refers to, requires the caller to install callback functions that provide the semaphore functionality and which OpenSSL then uses to protect access to its internal global variables.

Ahh, so the callback is the locking mechanism of your choice.  Having the library do the lock means a lock is only made when necessary, in contrast to my Semaphore in LabVIEW, where I always lock before any call, even where it might not be needed.

I don’t have LabVIEW 2016 and can’t open your code, but you might be trying to change a parameter directly.  C calls pass parameters by-value, not by-reference, so changing any of the parameters in the function will not affect their value in the calling code.  

2 hours ago, ShaunR said:

But thread safety isn't about one problem. It is an umbrella term for multiple undefined behaviours when using multiple threads, including race conditions, pointer overflows, memory dereferencing, counter overruns,  and lots of nasty stuff that will crash LabVIEW-probably randomly and when already deployed to the customer site.

All those issues are about parallel actions; they are not specific to OS “threads”.  Only thread-specific memory is something that would require LabVIEW to use the UI thread.  Otherwise one can serialize prevent parallel calls by any number of means.

Well. Serialization means writing binary as strings to me (like the flatten functions,XML etc). So I'm already floundering with your question. 

Serialization is the act of serializing, which is to make something into a serial form, which is a set of things one following another.  Serial is often contrasted with parallel.  Used in multiple contexts in computer equipment, not just in converting memory data structures into serial bytes.

An HDF5 reference I found discussing its issues with thread safety.  From it I gather serialization would probably be sufficient.

especially for recursive mutexes

I used a mutex count for that, releasing the semaphore when the count reached zero.   That allowed me to call locking functions from inside other locking functions.

8 hours ago, ShaunR said:

You cannot mitigate thread safety in LabVIEW.

Depends what the issue is.  The MDSplus library I mentioned uses a global variable.  It doesn’t matter if function calls are made in different threads; it just matters that critical code sections that use that variable be protected from parallel access, and thus I used a semaphore for serialization.  So I may have been using “thread safety” imprecisely to include race condition issues.  What’s the proper term for a library that requires serialization but not use from a single OS thread?

That means netCDF isn’t thread safe either.  In a previous wrapping of a non-thread-safe library, MDSplus, I used a semaphore to serialize access, rather than the UI thread.  

...but am definitely interested in other developer feedback.

Hi Martijn,

Perhaps we should start a new topic on your library; I’ve had a look at it and could make some comments/questions.  For example, why are you making your dll calls in the UI thread?  Is HDF5 not thread safe?  

— James