drjdpowell

I suspect the terminology is a holdover from the programmers of LabVIEW, who were loading into and unloading from the execution system, rather than memory.

Bad terminology then; it should  be “Reserve and retain on first call” or similar.

For comparison, I have a 2GB test SQLite database with 660k spectra, each of about 140 wavenumber (WN) readings, a total of 91M rows of Time, WN, Value.   These are stored in a "Without ROWID" table with primary key (Time,WN) I select regions of interest (ROI) in WN and Time, and I average spectra over time intervals (every 100 seconds, say), then take some statistic like a maximum of an average over each interval.  With WN ROI of about 30 points, and a time range of a few percent of the data, update time is about one second.  Doing the full time range takes 30 seconds.  

I don't understand why MySQL and Postgres are not able to get comparable results.  But I would suggest trying (Time,id) as your primary key.

Notes:

-- the 2GB file is small enough to fit in the Windows File Cache, and so there is no disk access involved.

-- SQLite is only using a single CPU for this, so it doesn't matter how many processors my computer has

Remind me.  What Primary Key do you use?  Is it (id,time) or (time,id)?

Oh, and what hardware are you running on?

This example gathers references to all non-indicator controls on a Tab.   Controls whose names end in "_IGNORE" are ignored.

All Controls on a Tab Control.vi

LabVIEW 2015

1 hour ago, ASalcedo said:

However how can I write and read the whole VI except the image? is there a shorcut way? Because it will be really tedious to put every indicator and controls except the image in the block diagram to reaad/write...

There’s a middle ground between those two extremes, where one operates on arrays of control references.   You either create the array of references explicitly (lot less work that coding it), or you get it programmatically from either the pane reference or tab-control tab references.  You can even do things like use prefixes in the control’s name to exclude some controls.

looks like they don’t have the feature you want.   Re your original problem, I wonder if the issue is some very large data that you may be saving in the config file.  How large are they?

12 hours ago, ASalcedo said:

Maybe there is another library that I can use?

You might see if the MGI Library has similar functionality.

I'll be interested in learning your results, as I'm considering Postgres for a data-recording application.

Note:  0x40 isn’t to open a ref to a re-entrant VI; it’s to open a common reference to a pool of shared clones.  If you just want a single clone then don’t use 0x40.

Dr. Powell:  While debugging, we are opening the Front Panels.  How?  Well... they opened automatically as set in the VI properties (to open when called).  We also used an invoke node to open the front panel.  It did not change anything.

If you’re using “Open when Called” then the FP will be loaded before anything else happens, so that can’t be the source of your problem.  Be wary of using an invoke node on the VI block diagram, as the FP doesn’t exist until after that node executes.

Note, BTW, that I haven’t worked on INSERT speed yet (as my application doesn’t require it) and the example only inserts one row at a time.  Better speed comes from multi-row INSERTs, and even faster is likely the COPY command, which I intend to support with the toolkit at some point.

Also, from my reading, I wouldn’t expect MySQL to be slower than Postgres (except possibly for complex queries), so I think there must be something wrong with your MySQL benchmark.

Also, make sure your FP is loaded before your indicator is written.  Your input Controls will show default values if the subVI was run before its FP was loaded, even though the proper values were passed.

How are you opening the Front Panels of your clones?  If you’re trying to use those references opened with 0x40, you will be opening a different clone that the one you started (someone on NI.com had this problem once).

Here you go.  Afraid I've not been able to work on it recently, as other priorities keep intruding.

jdp_science_postgresql-0.1.1.8.vip

This makes it not realy intutive... but let me play around with it a little mybe I missunderstand your suggestions at the moment

Reread my suggestion about the cluster.  Don’t do what you just suggested.

Quote

 It was not possible for me to dispatch the object reference dynamically.

Try an Object in a Cluster for your DVR, rather than an Object directly.  There are special issues around Objects in DVRs that are possibly causing your problems.

Yes.  With snapshot off, the image redraws from the buffer whenever it refreshes.  Writing to a terminal triggers a refresh**, but you can also do it by the "Refresh Image" Method.

** Note: LabVIEW controls are running in the UI thread and are asynchronous to their terminal, so writing an Image ref to a terminal only schedules an update for possibly several milliseconds in the future, by which time the image may have been overwritten.  I think this may also be a big source of confusion, as with by-value data the delay is never a problem, but with by reference data it is.

Well, to dequeue from a Queue, you have to provide the Queue reference every time.  You can't call dequeue once and have it automatically keep happening.  The confusing thing is that it is we write the Image Ref to a Control Terminal, rather than a VI.  That's what fools us into thinking of passing images by-value.

1 hour ago, Neil Pate said:

Another related question, in my particular use case I have a piece of code generating a user event with an image as the data at some arbitrary fast rate. The act of Generating the user event creates the a copy, right? 

I always find images annoying because the are by-ref.  You send a copy of the reference to the image, not a copy of the image.  

Cast it to a U64 instead of an I32; then the numbers are different.

The “Create DVR” creates a special memory place, that is not the same as your “Numeric” cluster element or Control.  You are setting that new memory to be equal to your “Numeric” on creation, but they are not thereafter connected.  Those orange wires are by-value.  To get the value inside the DVR, either use the IPE structure or it is provided as an output to the DVR destruction primitive.

Indexing tunnels (conditional or not) follow a preallocation strategy of filling a larger-than-initially-needed array (later cutting the unneeded elements), while the “Build Array” primitive allocates a new array of exactly the right size on each iteration.  So there is a lot fewer calls to the memory manager with an indexing tunnel.