Jump to content

Matt W

  • Posts

  • Joined

  • Last visited

  • Days Won


Matt W last won the day on December 14 2014

Matt W had the most liked content!

Matt W's Achievements


Newbie (1/14)



  1. I found some more info http://zone.ni.com/reference/en-XX/help/371361H-01/lvexcodeconcepts/array_and_string_options/ In the section Determining When to Resize Array and String Handles in Shared Libraries It sounds like you're not supposed to use NumericArrayResize for string data. I'm guessing the problem is alignment since it looks like in LV64 on windows the string size would be padded with NumericArrayResize, when it shouldn't be padded. On LV32 on windows there is no padding so I guess it wouldn't make a difference in that case (or in any case where alignment is less than or equal to 4 bytes).
  2. I think I found the answer to my second question http://zone.ni.com/reference/en-XX/help/371361H-01/lvconcepts/how_labview_stores_data_in_memory/ Sounds like with strings it is either the handle or pointer that can be null, but with arrays it is just the handle.
  3. Thanks, that helps clarify some things. I haven't seen any nulls during my basic testing, so I'll add some code to check for that. Is there a way to encourage LabVIEW to use nulls, I would like to double check that the null handling is correct. Which part of the handle can null? Only the handle, only the pointer or either? If I pass a Handle by value into a CLN can I assume that the passed in handle will not be null (otherwise I can't resize that handle, and I would have no way to return a new handle).
  4. I have a method for handling syncing with DVR classes, but it's a bit messy and I haven't used it behind checking that it seems to work. You basically wrap the class in a temporary dvr for the synced operations (It can implemented much cleaner with SEQ Classes). Just throwing it out there in case it gives someone an idea.
  5. If you make sure the indicators are updated after the benchmark (run their wires through all the remaining sequence frames) then array will win, with your current bench mark the winner will depend on which test is run first (I suspect the difference is due to pressure on the memory allocator). Personally I use also use an always copy node to copy any input data before entering it into the first sequence frame to make sure I'm not measuring a data copy (just wiring into the first frame maybe sufficient but I like being sure). That wont matter in this case since you're not modifying the U8 array. A couple side comments. A string to U8 array can be done in place so that it doesn't take any time (a u8 array and string have identical structures in memory so LabVIEW just has to reinterpret the data) You're greatly underestimating the size of the string array. Each string takes 6 bytes 4 for size 2 for data. An array of strings is an array of handles. 4 bytes (array size)+5.5 million * (4 bytes [string handle]+4bytes[string pointer]+6 bytes [the string itself] ) so a total of ~ 73.4 mebibytes. In 64bit the handles and pointers would double in size, I'm unsure if the sizes would double or not. If you avoid the string array (and it's 5.5 million allocations) things go much faster On the first run with 5.5 million elements I get For Loop: 4725 Array: 2758 Mine: 134 On the second run (where LabVIEW can reuse the large arrays allocated in the first run). For Loop: 2262 Array: 2279 Mine: 127
  6. Using DETT and a test VI LabVIEW did deallocate strings that I allocated and put into a LabVIEW allocated array. The deallocation happened before the vi terminated, since all DS* allocated data would have been cleaned up by then anyway. I'm forcing deallocation by reshaping the array. So it looks like LabVIEW will deallocate handles placed properly into aggregate data types. I'm not sure how uncontained handles are dealt with (my guess is that those would need to be removed by hand), but that isn't relevant for my issue. I don't suppose anyone knows where this stuff is documented (I assume the people who used to write CIN's would have experience with this). Testing LabVIEW this way would be good for double checking the documentation does what I think it says, but seems error prone to figuring out what's exactly going without documentation. I'm going to assume my third approach will work, unless someone knows of some issues with it.
  7. I'm currently adding a scripting support via LuaJIT to a program I'm working on. The script is working with potentially large arrays (1 million+ elements) of doubles and strings. I pass both types via Adapt to Type Handles by value and I am able to read both of them without issue. I can also modify values within the double array. Now my problem is I want to output arrays of doubles and strings from my script, but their size is unknown until I run the script (I do know a maximum size, but that will be far larger than needed in the majority of cases). I can think of a few approaches but since I can't find a good description of the semantics of the LabVIEW memory manager and especially it's interaction with DS* functions I'm unsure if what I do will produce leaks or deallocate possibly in use data. I'm just going to list some of the approaches I have come up with with string arrays since double's should be a much easier problem. The safe method: Run the script have it store the data separately then read the string array one element at a time into the LabVIEW array. While this is certainly safe it's also horrendously slow. The I think might be safe method: Have LabVIEW pass a handle to a string array that's been preallocated with empty strings, the length of the array is the maximum amount of strings I could possibly need. Now use DSSetHandleSize on all the string handles I'm writing to to make space and write the string data and string size. Next use DSDisposeHandle on all of the unused strings (I assume I'm responsible for deallocating them since I'm throwing them out them), then DSSetHandleSize on the array handle and set the array size. With this method I'm assuming LabVIEW is still responsible for cleaning up all the resized handles. The problem with this that the array is probably several orders of magnitude too large, so I'm putting extra pressure on the memory manager. The method I want to use: Have LabVIEW pass a handle to an empty string array. Use DSSetHandleSize on the array handle to make space for the strings I'll need and set the array size. Then use DSNewHandle to create new strings to fill up and set their sizes. Now typically the thing responsible for allocating is also responsible for deallocating, is this still true when I'm putting the string handles in the array that LabVIEW is already handling (in my previous approach I deallocted handles that LabVIEW made, would LabVIEW do the same when it's done with the ones I allocated). If I need to deallocate the strings myself do I need to be extra careful that LabVIEW isn't currently using that data already (IE use the always copy node on anything that comes out of the string array, before I pass the string array back in to the script to close it). There's a decent change I might end up using arrays of single element clusters of arrays of strings. If there's some reason the concepts from the above wouldn't apply to that. Matt W
  8. If you use a reference to the subpanel with a register for events node, you can switch the state when the mouse enters and leaves the subpanel. But when I tried it with the busy state on the leave event was only triggered when I left the vi and not the subpanel. Which I think is a bug in LabVIEW, but I'm not certain how the busy state is supposed to affect events.
  9. With memory it's cheaper to have one array. Assuming the difference between times is constant then it's even cheaper to use waveforms which encode times as a starttime and the difference between times. Speed wise it depends on what you're doing to the data but it's probably going to be best as waveforms, then a single arrays then multiple arrays.
  10. I was mistaken you can use recursion, instead of messing with the VI server. I made an example in the attached zip (note poorly tested, and poorly documented since I couldn't spend too much time on it). In the example URLS of "slow" take 500 ms, "timeout" takes what ever the timeout limit is, and anything else takes 100ms. The simplest and fastest way (although it uses the maximum amount of connections), is to set C to 1 and P, as well as the generated number of loops, to the maximum amount of connections you can have at once. Multiread Recur Example.zip
  11. If you configure the parallelism in the for loop you can have up to 64 parallel instances. I meant this (note it's untested, so could have bugs). The max session limit is a setting on the router, looking up that message on google turns up Zywall routers as having a rather low limit. Some info on fixing that type of router (assuming that's what you have). http://www.broadbandreports.com/forum/r9235388-How-many-is-the-max.-number-of-session-per-host- http://www.broadbandreports.com/forum/remark,9094710~mode=flat?hilite=sessions http://www.broadbandreports.com/forum/r17252762-ZyWall-2-max.-number-of-session-per-host- I'm not sure if the datasocket caches it's connections, if it does explicitly opening and closing the datasocket might help.
  12. The simple solution is to adjust the parallel for loops settings like so. The problems with this are your attempting P network connections at once, which might cause problems if P is really high, but if the amount of troublesome links is larger than P then It'll slow down some, also if a slow page finishes then that loop instance gets another slow page it might be too slow. Since your time out is smaller than the loop time, P doesn't really need to be large than the number of troublesome pages. I think ceil(numberofpages / floor(looptime/timeouttime)) for the P value (with the generated loops set to value larger than P will ever be) should work. You could also set C to floor(looptime/timeouttime) for some performance, but I don't think you'd notice the difference. The advanced solution is dynamically launch multiple copies of a VI that handles a request and dynamically launch another when a request seems to be slow.
  13. I haven't tried it but there might be another way of going about it with the import and export string commands. There's an app reference invoke node that can batch all the different VIs together into one text file. http://zone.ni.com/devzone/cda/tut/p/id/3603
  14. I meant move the column count out of the while loop. You might want to put a note in the documentation on which execution system you're using. I've just been using the same as callers in mine. I haven't really experimented with tuning priorities in LabVIEW. I didn't find it that to be much of a problem (you would just need to add it to your SQLite_Error VI, and maybe call it a bit more often). It helps a lot with debugging, of course I'm working with raw SQL strings far more often than you are.
  15. Mine is heavily dependent on inlining for speed, you're probably much faster in LV2009 due to that. If I needed LV2009 support I'd probably switch to subroutines. VI nesting wise we're pretty similar (if we ignore the higher level abstractions on yours), I have only three VIs where I have more than one DLL call (one to copy a query string to a pointer, the other is handle strings with /00, the last is to read a column based on it's type). But I avoid the speed lose due to inlining. I added a repeated query interface to mine (I use it to run stuff like mass inserts) and In the benchmark I gain about 10% on the insert due to inlining (even though it's only called once). LabVIEW cuts of all the dead call for storing the results of the repeated query (since I don't wiring the result data on the insert). If I wire the results to something I loose my 10% gain. The dead code elimination and inlining makes writing a performant interface much easier since I don't always to write multiple version of the code for different requirements (which also simplifies the interface). If I modify your fetch all on LV2010 64 on win7 I gain about 13 ms on a select with 100000 records. A few more suggestions on your version. Drop the high priority execution setting, typically disk IO is considered low priority (starving data collection would be bad). You should support for sqlite3_errmsg in your error handler, since that'll give a much explanation of the error (stuff like table names or which command is misspelled). You'll probably want update to use binding, since currently you can't update to a blob value.
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.