ShaunR

Reading, yes. Writing no. Incidentally. It's the same with INI files etc. You don't need to keep a list in memory or have lookup code for it. Just read and write directly to the the file. Not sure about other platforms, though.

SQlite is single writer, multiple readers as it does table level locking. If drjdpowell had followed through with the enumeration it would (or should) have been in there. The high level API in the SQLite API for LAbVIEW insulates you from the Error 5 hell and "busy" handling (when the DB is locked) that you encounter when trying simultaneous parallel writes with low level APIs. So simultaneous parallel writes is not appropriate usage.....sort of . You can mitigate some of these operational aspects as you are attempting to do but in reality you are now just starting out the process of proving my rule of thumb:

Only with Pragma Synchronous = OFF

Yes. The actual performance is dependent on a number of factors, two of which are the version of SQLite used and how the binaries are compiled. The figure I quoted is a nominal value for SQLite in general rather than with the binaries I used since the intended message is that TDMS is orders of magnitude more performant for writing in the right circumstances. There is a benchmark included as an example with which you can ascertain the actual performance on your system with arbitrary columns and transaction numbers. You can download it from that page to try it as it is open source and free for non-commercial use. It is a single line SQL query so the complexity for decimation is trivial - unlike handling the data in LabVIEW with or without TDMS. I got bored after 1 million data points but if you do more (that VI is one of the examples), please post the results. Sweet!

Are there bench marking graphs or data you can post? I did a lot with SQLite - some of which you can see under the "Performance" heading. It might be nice for some graphical comparisons. SQLite should blow away TDMS for queries but TDMS should blow away SQLite for Writes. The data logging example in the API (which you can play with) demonstrates decimation and takes about 150ms to retrieve the 500 decimated datapoints when there are 1 million [DBL] data points in total (IIRC).

Hmm. That's not really saying anything and a variant on "use the right tool for the job". How about enumerating these alleged design cases for the OP and comparing the practicalities?. I'll start you off........ TDMS is designed for high speed disk writing and can write to disk at more than 400MB/s with the right disks. SQlite can only manage about 0.5-1MB/s.

But memory [re]allocation is Rule of thumb: If you want to search the data, use SQLite. If you want high speed bulk data written to disk then use TDMS. If you want high speed data that is searchable then use both with just-in-time post processing.

The purpose of my example was to show how to get the managers fighting amongst themselves for ridiculous requirements rather than making it your problem I call it upward delegation. Just light the touchpaper of an interdepartmental process, grab some popcorn and watch the shots fly over your head

So Boss. You question the suppliers QA? I'll need you to sign this request in order that Goods In switch to 100% inspection sampling and I'll hand it personally to the Goods In manager along with your extension number When will you be at your desk?

I didn't create screws from bars of metal with a lathe but components were chosen, stripped, cleaned, inspected and modified to suit which ensured that tampering would have been spotted and rectified. A car designed to kill me would try to hide or prevent me from taking safety measures or circumventing it's goal of killing me. From a design point of view it is the risk I discover the purpose as opposed to overlooking and not discovering a rare defect. These are intuitive to me. I guess they are not to you. With those levels of paranoia the thing that should be striking fear into you is that LabVIEW is a closed source compiler and IDE so how do you prove to your customer that the language itself that you have chosen to solve their problem is not malicious and the code generated from your diagram is only doing what you have coded. I suppose the point is that you have to convince yourself that something is safe or non-malicious before you try to convince others. Trust isn't transferable so if you trust a toolkit then you have to accept that others may not unless you vouch for it and the implications that arise from that. The process and facts discovered whilst convincing yourself will be the evidence to the customer and that is much easier with your own code because the design goals, implementation specifics and desires/intent are implicitly known. The question is. Does the customer trust you!

This is demonstrably incorrect. We used to make petrol go-karts as kids out of lawn mowers and I built my own [hard-tail] motorbike when I was 17. A quick perusal of youtube will show home made drag racers, off-roaders, dune buggies, monster trucks and even tanks, boats and aeroplanes. The only requirement to drive a car of any description on a road in the UK is that they pass an MOT, which is a government test that says "it probably won't kill you or other people [today]". So yes. I would drive my own car and when (not if) it passed the MOT I, along with the police would be pretty happy that it was safe to do so.

It's a discipline in its own right and mainly in the defence industries. This quite often happens when offloading responsibilities onto a supplier. The choice is really to employ a Security Engineer or decline to quote. It's similar to declaring your software fit for medical use or for use in explosive environments. People are too used to the "App Crap" of IOS and android where bugs and vulnerabilities are considered an ordinary and expected part of the development cycle rather than a testament to incompetence and/or lack of discipline so they expect every programmer to have in depth knowledge of dangerous domains because........it's only software, right?.

A) will definitely kill you wheras B) Probably won't since it is a design requirement not to kill you. If that isn't obvious then I despair Or. Looking at it another way. The goal of writing software is to succeed and realise the requirements. Would you prefer A) or B) to be successful and the contingencies employed to ensure success to drive towards A) or B)?

I said it was intuitively obvious that one was a higher risk than the other. Someone trying to kill you poses a higher risk that you will die than the risk from someone that isn't and I don't need to spend 3 weeks investigating to come to that conclusion unless I want a few decimal points on that binary result. If you actually look at the security assessments you sill see things like mitigating controls exist that make exploitation of the security vulnerability unlikely or very difficult. Likelihood of targeting by an adversary using the exploit (my emphasis) So even the formal appraisals require intuitive judgement calls.

I've thought long and hard about your question. While it is intuitively obvious, I'm unable to quantify it outside of the formal risk assessment process - which itself can be fairly subjective. (You do risk assessments, right?) When I say "obvious". I mean in the same sense that the prospect of being beaten senseless on a Friday night is intuitively worse and riskier than tripping over and possibly hurting yourself even though I don't know the probabilities involved. I'm aware of formal processes for mitigation of malicious code - which tend to be fluid and dependent on the particular adversary. However I'm not sure of how you would apply those processes to incompetence.

Indeed. The compatible with LabVIEW program is multi tier and intended to standardise the out-of-the-box experience of addons and herd the cats. The lowest level basically states you have followed the style and submission guides whereas the silver and gold simply attests that you have had positive feedback reviews from real customers as to levels of support (responses within 48 hrs etc). Once a product is on the Tools Network and approved, the same rigor of checks required to gain acceptance is not applied for every release so I would be very wary of a supplier offering this as proof in isolation.

Citation?

Don't use open source with this project (and that is not a glib comment.). Their procedures may let you get away with verifying a PGP key against a distribution but the legal ramifications are not worth it unless you have a dedicated legal team. Do you have a dedicated legal team? If not then bite the bullet and refactor the opensource components out. (Then apply to become one of their "preferred suppliers" )

Yes. I have looked at the RT Images directory now and it seems fairly straight forward. There seems to be a couple of ways to utilise the deployment framework but the only thing I'm not sure of is the origin of the GUID strings. I haven't looked at the zip package as yet. I get around the elevation by using my own VI that is invoked as the pre-install VI of VI Package Manager. If it fails, it tells the user to run the LabVIEW IDE using the "Run As Administrator". Not perfect but users seem to have no issue with this process and it works on other platforms (like Linux and Mac) too - of course changing the appropriate language (su, kdesu etc). Since I have all the tools, I'm also thinking of SFTP from a LabVIEW menu instead of the NI deployment. Max and I don't really get along and Silverlight brings me out in hives It would be great for the Linux platforms and avoid most if not all all the privilege problems since I would be able to control the interaction . I'm just mulling over whether I want that knowledge and can be bothered to see where it goes

It this process documented anywhere?

For completeness. Here is how you could have used the flatten function from your first try. python example.zip

You can have your cake and eat it if you wrap them in a VI Macro

I'm using 2009 I'll check the others later. ...a little later... ok for 2014 too (it must be operator error on your machine )

Well. The TCP Write can accept a U8 array so it's a bit weird that the read cannot output to a U8 indicator so simple adapt-to-type on the read primitive would make both of us happy bunnies.. Ahem. "\r\n" (CRLF) instead of EOF (which means nothing as a string in LabVIEW) is the usual ASCII terminator then you can " readlines" from the TCP stream if you want to go this route. You will find there is a CRLF mode on the LabVIEW TCPIP Read primitive for this too. Watch out for localised decimal point on your number to string primitive which may surprise you on other peoples machines (commas vs dots) and be aware that you have fixed the number of decimal places or width. (8) so you have lost resolution. You will also find that it's not very useful as a generic approach since binary data cannot be transmitted so most people revert back to the length byte sooner or later.

I don't? (write shared libraries). This issue was solved a long time ago and there are several methods to determine and correct the endianess of the platform at run-time to present a uniform endianess to the application. I have one in my standard utils library so I don't see this problem - ever. Don't get me started on VISA...lol. I'm still angling for VISA refnums to wire directly to event structures for event driven comms Strings are variants to me so I see no useful progress in making the TCP read strictly typed-it would mean I have another type straight-jacket to get around and you would find everyone moaning that they want a variant output anyway.