drjdpowell

Yes, Microsoft is a bitch when it comes to protecting precious data. The memory management standards are now insane compared to XP.

Actually, turns out the client was running XP when they had an error.  But, my XP is running on a virtual machine under Parallels on a Macbook, so perhaps that changes the memory protection and is why I have never had this error in testing.

Attached is a new VIPM package (beta) with modifications to how “Prepare" works to avoid reading outside the allocated memory.  I do not know if this will solve the problem, but as an added benefit the new method will give improved performance of “Execute SQL” for multiple SQL statements.  Please give this a try and report if it eliminates the 1097 errors.

drjdpowell_lib_sqlite_labview-1.2.0.28.vip

Interesting query. Is the -- comment valid SQL syntax?

 

Could a badly formed query be throwing the exception?

“Badly formed query”?  Excuse me?!?       Yes, that comment form is valid SQL.

Beta version, seems to be working:

Does the LabVIEW string to C string conversion explicitly, allowing the difference between the two pointers to be used to calculate the number of bytes of remaining SQL.

Well, this has just become a lot more relevant for me as a customer just emailed me this: 

First time I’ve seen it.  I use Windows XP while the client uses Windows 7; I wonder if 7 is more unforgiving of “reading someone else’s memory”?

 

I don't know the SQLite code at all, but a quick look at the image (and the docs¹) brings up the following question²: if the pointer is supposed to return a pointer to "the rest of the SQL", is it OK that you're giving it the full length of the SQL string as the max length? Won't that cause it to read past the end of the SQL string? Is the SQL string guaranteed to have a NULL terminator? Even if it does, does MoveBlock respect that or will it copy whatever's there even if it is a NULL (I'm assuming it would and that then you're potentially reading someone else's memory, which could explain the exception).

In this code I had the problem that the SQL statement is passed into the “prepare” dll function by pointer, but I don’t have access to that pointer, so I can’t calculate the length of the remaining part of the statement.  So I instead copy the maximum amount, including any invalid junk off the end, and then walk the string till I hit the null terminating byte (done inside “pointer to C string”, where the junk bytes are discarded).  

Can “reading someone else’s memory” cause an exception?  If so, I’m surprised I’ve never had this error myself.  If someone can confirm this as the problem, then I can rewrite “Prepare” to convert the LabVIEW string to a pointer first, then do things by pointer.

I am receiving the same error on the odd occasion using this library. So far it has only occurred when running a built EXE and not when running from source. I have yet to figure out a reproducible way to trigger it:

I still cannot see what might be causing this error.  You should defiantly have a shift register in your code, to guard against the case where the for loop is called zero times (is there any LabVIEW developer out there who didn’t learn this the hard way?).  But I can’t see how this could cause the described error.  

Here is the code of the “Prepare” method:

The error can only happen if:

1) sqlite3_prepare_v2 dll call runs without error.

2) “Statement” handle is returned greater than zero.

3) “Remaining SQL tail” pointer returns zero or invalid.

I would expect most problems (invalid dll path, for example) to violate (1) or (2).

Do you use multiple levels of abstract classes?  File.lvclass<— LVfile.lvclass<— VI.lvclass, for example?  A HasBlockDiagram method would be defined in the LVfile class (default result FALSE) and would be overridden in VI.lvclass to return TRUE (but not overridden in LVLIB.lvclass).  

In my case, I am implementing a subscription message system.  The system needs to allow for multiple subscriptions and each subscription needs to keep track of all of its subscribers and the last message sent.  That allows me to broadcast to all active listeners and for new listeners to request the current value of they subscribe between broadcasts.

Funny, I use variant attributes for much the same thing, and asked about copies in this conversation.  I also allow new subscribers to get a copy of the most recent message on subscription (though not always; I distinguish between “State” notifications (save the most recent message) and “Event” notifications (don’t save).

I’ve long been meaning to improve the performance, but haven’t got round to it.

Another idea is to keep all data in a single flat structure such as an array, with the index to that structure stored in the variant tree.  Lookup the index then index the array in place.  Then at least all the copying will be of small integers.  Or one could try a tree containing DVR references.  But if we are taking very large data sizes then mje’s ultimate solution of a database might be best.  I’ve even considered using in-memory SQLite databases in place of variants for small data structures, just to have the increased flexibility of SQL, but the performance is significantly worse than variants when the data size is small. 

LVOOP objects are just a (nice) wrapper; you still need an underlying storage method. 

It is not possible to make the first 'get attribute' inplace since I am getting an attribute value by name and replacing it.  The IPE only provides access to the node data, not its attributes, as pointed out in the Idea Exchange link you provided in your first comment to the thread.

Oh yeah, duh.  I’m working too hard.  Liking my original suggestion better.

Thanks for the trick.  I will try that next.

I just tried the re-factor using the IPE instead of the for loop to preserve the child attributes.  The result is a 31% DECREASE in performance.  I must admit I am very surprised by this.

update middle variant data using IPE.png

 

Assuming I did this right, I think your other solution sounds better.  I'll post my results once I complete the edits...

I’m confused because I couldn’t get the IPE to work with a variant for the type input (LabVIEW 2011); I have to supply the actual type (which limits its applicability).  Using a variant only works on variants inside variants (did you unknowingly do that? would reduce performance).

BTW, the first “Get Attribute” that is copying an entire branch of your tree, so that’s also needs to be inplace (if you can get that solution to work).  

3. I often extract all child attributes from one of the top attributes and then perform an action on each of those.  If the node is now an attribute, I would need to remove that item from the array of attributes before I process them.  There are two ways I can think of doing this.  The first is to test the attribute name inside the process loop and skip the one named 'NodeContent'.  The second is to search the array of attribute names for the element named 'NodeContent' and then delete it from the array before I process the array in a for loop.  Any opinion on which solution would be faster?

I believe attributes come out ordered, so name your content attribute a single zero byte (Hex 00) and it will always be the first element (which can be skipped at minimal cost with an array subset).

Another thing to look into is the In-Place Structure’s Variant To/From Element structure, which does preserve attributes.  But I think it allows one to change the type of the Element.

Make sure you’ve kudoed "In Place Element Structure Support for Variant Attributes”.    It’s the copy at the first “Get Attribute” that is killing you.  

One way around this is to not use the actual value of your storage variants at all; instead keep the "content" of that tree node as a “NodeContent” attribute.  Then your above code becomes just a “Set Attribute”.

 I thought that you could use the "Default Value" of your get variant attribute node to do the casting for you. However this appears to not work.

Seems to work, at least in LabVIEW 2012.

An experiment in crude pseudo-UML using the IDE (for a selection of classes, and subset of methods, in my current project):

It looks like you haven't tried Symbio's UML Modeller ;-)

I have not. I should have a look 

Well, I don't know how seriously this is intended, but I think it questions the point of doing software modeling at all.

Oh, I didn’t mean to cast doubt on the use of modeling; just noting the handicaps of using tools designed for languages with an X=F(A,B,..) syntax.  It would be very nice if there where a UML tool that could interface with LabVIEW to present a class with all the methods as icons, with ctl-H bringing up the standard window showing terminals and descriptions.  Modeling before coding might be slower, as you’d had to create shell VIs with terminals and rough-draft icons, but at least this will save you time when it comes to actually coding.  And in return you get a much clearer visual diagram.

Not used UML myself, but it seems to me that the terminals of a VI are much better specified in this way (not that available UML tools will support this   ):

You only have one subpanel, and a subpanel only holds one VI at a time (dropping the previous VI if you insert a new one).  You also seem to be only creating one copy of your subVI (but you opened four separate references to it).  If you want four copies (“clones”) of your SubVI, you need to use option 0x8 (Reentrant run) in the Open Reference, and you need four subpanels.

Making a new “MoveStrategy” class tree is my first thought also, but another possibility is to create a HysteresisCapableAxis child class that by default has a Move method that just calls the parent method, but that can also enable hysteresis (via some “Enable” boolean).  You can then enable or disable hysteresis without changing the class.

Nope. It's far too slow. 

You mean it is too slow when the reference is invalid?  When it is valid then this method takes less time than testing for an invalid refnum and then using it.

Agreed. You are much better of testing for an invalid refnum instead. Then it will be created on the first call and if it ever becomes invalid - recreated.

Even better than testing for an invalid refnum is actually trying to use the reference.  Recreate it if you get an error.  There is some post somewhere by AQ that points out the race condition in testing for refnum validity.

Here’s an example of a “refnum created and maintained inside a non-reentrant VI” from one of my projects:

You can use the first call primitive to obtain the ref the very first time it is called, which is nice as then you do not need to care who called it first (parallel loops). See picture, in the false case the ref is just wired straight through.

A minor warning about that technique.  A reference is destroyed if the top-level VI under which it is created goes idle.  The “First Call” primitive is reset when the subVI it is contained stops being reserved for execution.  If you are working with multiple top-level VIs (such as if using dynamically “launched” VIs) then it is possible to invalidate the reference without resetting the first call.