Jump to content

JackDunaway

Members
  • Posts

    364
  • Joined

  • Last visited

  • Days Won

    39

Posts posted by JackDunaway

  1. Are all your variables bound to your prepared statement?

     

    For example, rather than

    SELECT foo FROM table1 WHERE col='bar'

    the statement 

    SELECT foo FROM table1 WHERE col=@value_to_match -- then later binding 'bar' to @value_to_match

     is going to be more performant (over multiple queries, especially), more precise (for floating-point numbers), and more secure (against SQL Injection Attack, assuming that the original `bar` was inserted at run-time using the likes of the labview `Format to String` function or C `snprintf()`).

  2. Do you have specific issues with using the NI webserver on public networks?

     

    Unless something has changed, both the Embedded Web Server and Application Web Server are AppWeb by EmbedThis. That web server in and of itself seems pretty capable and pretty sweet, but the wrapper that LabVIEW puts around it strips and sanitizes virtually all of the configuration file options: https://embedthis.com/appweb/doc/users/configuration.html

     

    Also, unless something has changed, setting up `SSLCertificateFile` with a private key signed by a bona fide CA (rather than self-signed) took some jumping-thru-hoops.

     

    You can poke around and modify the `ErrorLog` and `Log` specs, but only a couple security tokens are honored by not being sanitized away (https://embedthis.com/appweb/doc/users/security.html and https://embedthis.com/appweb/doc/users/monitor.html)

     

    Also -- unless something has changed -- the developer experience between dev-time and deploy-time is vastly-different, the way the web service actually works. At a very high level, the problem was this: the concepts of who calls `StartWebServer()` (maRunWebServer) were hard-coded or something into the way the RTE loads applications, and so therefore was not invoked when you just run your application from source in the IDE. In that case, there was this weird out-of-band deployment via a Project Provider, where it used the application web server rather than the embedded web server. Where of course, managing the configurations of these two disjoint deployments didn't work at all.

     

    YMMV, but I don't use it for anything. libappweb always felt pretty awesome, but it was just shackled (oh; and while libappweb continually improved, the bundled version with lv stayed on some old version)

     

    On a side note, the AppWeb docs site got a huge facelift -- looks like Semantic UI; looks/feels spectacular.

    • Like 1
  3. So is it useful to add yet another "buffer limit" on the higher level protocol layers? Aren't you badly muddying the waters about proper protocol layer respoinsiblities by such bandaid fixes?

     

    Exactly.

     

    If we're talking TCP here (we are), rate limiting thru backpressure is what you want by setting `SO_RCVBUF` with `setsockopt()`

     

    Buried in vi.lib you can find some methods that take a TCP connection reference and expose the underlying file descriptor -- presumably, you could set such options with kernel APIs (I've not tried) -- but in reality using the labview TCP API is kinda jokey if you're concerned about security (or high performance). Granted, best-in-class security and also high-performance are not the presiding requirement for many applications, in which case the TCP API is perfectly fine and convenient.

     

     

    I still don't really get who you are trying to secure yourself against.

    This seems like a lot of work for very little real reason.

    I suppose in response to your OP, I am in the don't care category. All my systems that have some kind of network comms are on an intranet, and if somebody else on that network is trying to maliciously ruin my day it really is not my problem.

    Where I need to expose things to the Internet I would never use raw tcpip.

    In those circumstances I use Web Services with their built in security features.

     

    Response to each sentence: Yikes; yikes; fair enough; seems like a reasonable; only a 100% maniac would do this; YIKES NO NO NO.

     

    In regards to all instruments/control systems being accessible remotely -- whether intranet, or especially public internet -- one must analyze the risk profile of what happens in the physical word if the controller goes up in smoke, and how easy/likely is that to be triggered remotely?

     

    The controller is a finite resource. If demanded, it can spend 100% of its finite power either controlling the process, or 100% of its power fending off an DoS or fuzzing attack. Each can starve the other, and both can starve "well-behaved", authorized remote clients/peers attempting to bring order to a railed-out controller.

     

    (Here, "attack" is a word shrouded with improper connotation. It's likely that we are ourselves (and our colleagues) are our own (and colleague's) biggest attackers, just by writing dumb bugs. Networked communication is an *excellent* manner of flushing out bugs we were not even aware we were so good at writing, having become accustomed to the reasonably deterministic and reliable medium of memory and CPU for in-process communication, not always programming against such lossy mediums as the physical jankiness the IP stack must deal with.)

     

    All this said; the labview web service is woefully incapable of configuring security and QoS parameters for clients. It's reasonable to use behind VPN on an intranet, but I would strongly recommend against putting it on the internet without sitting behind a production-quality HTTP server acting as a reverse proxy.

  4. When it comes to two event structures sharing the same static event, I think things are inconsistently handled.  By that I mean if you have two event structures, handling the same boolean Value Change control, some times they both get the event, sometimes the first structure will get it and the other won't, or the sometimes the second structure gets it but not the first.  Basically avoid this situation in all LabVIEW code.

     

    This one bit is not quite correct. You're thinking about multiple Event Handler Structures binding to a shared queue created by a single Register For Events Node. That's the scenario that produces undefined behavior. (Not just unpredictable behavior; truly undefined behavior)

     

    Multiple Event Handler Structures bound to a "Statically-Registered Event" are OK, but as hooovahh suggests, not usually recommended.

     

    ---

     

    "Statically-Registered Event" -- contrasted with "Dynamically-Registered Event" -- is just a silly terminology for "magic wires that you don't see that you just 'know' must exist".

     

    The original post is an excellent case study for the mental leap required to learn/believe/trust/understand Static Event Registration.

     

    For some reason, in LV world it's taught that Static Event Registration is "easier" than the "advanced" topic of Dynamic Event Registration. Maybe if Dynamic Event Registration were taught like this -- it's wires you see, in the code you wrote, that code in front of you on the block diagram; that's what happens and why it happens. And lifetimes of both the event sources and event sinks -- yeah, that's kinda just wires too -- the wires you see in front of you, on the code you wrote. Understand this, and then the concept of "Static Event Registration" might help you by replacing a little syntax with some accepted, implicit convention.

     

    Said another way -- Dynamic Event Registration is syntax where the mental model maps to the visual source model, whereas Static Event Registration is just a shortcut that hides implementation details.  (Yes, there do exist some notable functional differences between the two, but that's just a shortcoming of both, not a fundamental difference in the model)

     

    @Scatterplot -- as an excercise, can you create the same diagram, just with a static reference to `Value 1`, wired into a `Register for Events`, wired into the Event Handler Structure? Post that snippet, then let's use that as an explicit example to the "wireless" model of Static Event Registration.

    • Like 1
  5. Really good insight here, I am quite envious of the technical grokking that you fellows have on these quite arcane issues.

     

    Envious should be the wrong word here. Such "arcane issues" should have been made more developer-friendly long, long ago -- these topics definitely feel "foreign" coming from labview, but "arcane" -- we're on the far other end of the spectrum from "arcane" as far as C is concerned here -- this knowledge should be some straightforward day-to-day interop. But it's not, unfortunately.  :(

     

    That said ... you'll find relatively few questions unanswered, and will rarely be disappointed, appending "RolfK" to your searches on these topics :-)

     

    Jack, this may be obvious to you, but you say the data is posted on the stack (and not the heap).

     

    I had started to type a response here, but it got too long -- this is actually one topic with many excellent explanations findable in a search, and to help guide your search, it might be helpful to also look for "data segment" (since this is sometimes where other types of static memory are compiled into the object file).

    It is this that means that the function calls must be run in the root loop (orange node).. That is really, really bad for performance..

     

    If you passed the ref into the callback function as a parameter then you could turn those orange nodes into yellow ones. This means you can just call the EventThread directly without all the C++ threading, (which is problematic anyway) and integrate into the LabVIEW threading and scheduling. The problem then becomes how do you stop it since you can't use the global flag, b_ThreadState, for the same reasons. I'll leave you to figure that one out since you will have to solve it for your use case if you want any thread operation :P:book:

     

    It took me a minute to understand why you had to bring the orange banner into the mix -- but by all means, declaring `dw_LabViewEventRef` as a global variable *might* be justified as a gross simplification for an example program, but this is by no means production-quality.

     

    I think this advice holds pretty much absolutely in this domain -- if you find yourself using the orange banner to "solve" problems, it's going to be a long, cold day.

     

    But that said -- and to refine what @ShaunR is saying here -- you can't pass the ref into the callback function, since it's the main library who invokes and passes parameters into your callback function. When telling the main library what callback function to use (e.g., I would expect the library has a function with a name along the lines of "register callback function"), does it also give you an opaque data pointer you can define yourself that it will pass to your callback? This is where to initialize the main library with the UserEventRef from labview.

     

    That said ... the next topic would be how and when to allocate and deallocate that memory on the heap  :book:

  6. Yes Jack, it is all soooooooo 1900 but that is the century when LabVIEW was developed. And something like byte alignment can't be changed later on on a whim without rendering almost every DLL interface that has been developed until then incompatible. The problem will however soon solve itself with the obsoletion of the x86 platform in general and in LabVIEW especially.  :D

     

    This part actually still fascinates me, how labview provides such a clean abstraction over the underlying memory ...

     

     

    Your other remarks do sound more angry than jaded, Jack!  :cool:

     

    Yes I also feel the pain from extcode.h which is in some ways a bit dated and hasn't really seen much of development in the last 20 years. PostLVUserEvent() was one of the few additions in that timeframe and it wasn't the greatest design for sure. Incidentially NI doesn't really use it themselves but they rather use the undocument OM (Object Manager) API which supports also event posting (and custom refnums like DAQmx, IMAQdx, etc) but uses an API that is basically impossible to use without further detailed insight into the LabVIEW source code, despite a documentation leak in the 8.0 cintools headers for some of them.

     

    ... and you pretty much were able to hit every facet from which my frustrations stem. 1) underpowered interface that 2) never changes that 3) was not dogfooded so 4) it totally blows and therefore so does my developer experience while trying to do something meaningful with the API while 5) NI has their own far more powerful interface, and are therefore unconcerned day-to-day with 1)

  7. As if by magic, the shopkeeper appeared. :P

     

    Thanks for actually running this and confirming my suspicion; i was just poking thru source code on a 13" macbook air and xcode, and of course couldn't even load the VI or C without compiler error  :lol:

     

    I seem to recall @Darin.K commenting before on the quality of the PRNG in LabVIEW in the past; @ShaunR, perhaps you have demonstrated a better one here!  :P

  8. Jack, can you please reply to the post #23? After rereading your suggestion, I see it is the same as mine and I just didn't understand the text after first glance.

     

    No prob.

     

    First, your statement "nice simple example from NI engineer" threw my BS detector into red-alert mode  :lol:

     

    "space heater" means CPU, polling without any sort of throttle, or more appropriately interrupt-driven procedure, turns into this: https://www.google.com/search?q=%22space+heater%22&tbm=isch

     

    Time bug -- publisher waits only long enough to allow "the thing" on the other side of `PostLVUserEvent()` to make a memory copy, but not to reasonably ACK that the event has been processed and the handler is effectively ready for the next event. In other words, publisher is asynchronized from subscriber -- or even bigger problem, subscribers (plural). 

     

    Space bug -- subscriber is unable to put a cap on its recv queue size in labview, unlike, say, a BSD socket with a cap on recv buffer that has `connect()`-ed to a `bind()`-ed publisher. In that scenario, TCP backpressure from subscriber to publisher is capable of effectively enforcing timeouts and message drops with EAGAIN or ETIMEDOUT on the publisher -- meaning the lack of specifying and limiting space on the subscriber side in labview (in the form of limitign the queue size of an `Register For Events` node) severely limits the publisher to act sanely (since, no backpressure on `PostLVUserEvent()`). You have to jump through totally bogus, undocumented, unsupported, uncharacterized, and unintuitive hoops to hack together a viable solution using `PostLVUserEvent`, `Register Callback VI` node, LabVIEW queues, and and additional User Event to grossly approximate a decent `recv` buffer limit. I do not say this proudly -- I say this defeated and deflated -- but I may have accidentally characterized the only solution in existence to such a fundamental, gaping hole in `extcode.h` -- the ability for C to synchronously call into *LabVIEW* and synchronously wait for a response on the call stack (rather, than the other way around). Summarized: it's easy to create memory leaks when the `send` endpoint cannot detect the `recv` endpoint is not processing quickly enough.

     

    Basically, `PostLVUserEvent()` makes wildly dumb assumptions about both space (memory) and time (synchronization between `send` and `recv` endpoints), and perhaps nobody on earth is using it properly, except for serendipity of an application domain where it happens to work, and/or hearty `sleep()` sprinklings.

     

    I'm not bitter, just jaded and irritated with `extcode.h`

  9. To be perfectly honest, I thought, that you really do not need to clean that up, since it will get a record in the global memory table and LabVIEW will clean it up.

     

     

    To be fair, your expectation here is utterly reasonable and desirable. It's the C<->LabVIEW FFI is utterly unreasonable and undesirable, and virtually unchanged since the 1900's.

  10.  

    Over here is a nice simple example from NI engineer. Makes a thread that sends simple structure to labview. Notice, that he doesn't use malloc or new to make a data copy, since PostLVUserEvent makes a copy of the top level data structure.

     

    http://forums.ni.com/t5/LabWindows-CVI/using-PostLVUserEvent-function-from-Labview-dll/td-p/2510908

     

     

    OK, having downloaded that example; the data posted is allocated on the stack, not on the heap. Major difference. That's why that particular example you cite has no memory leak.

     

    But creating a new DSHandle without freeing it -- definitely a memory leak.

     

    Now, ideally, `LVPostUserEvent()` would have an additional (boolean) parameter that indicates "take ownership of this data" -- to avoid caller bookkeeping and provide vastly superior performance by not bring the Memory Manager into the mix making a new allocation -- but the LVRT is back in caveman ages, so we are relegated to caveman practices.

    *edit: And a couple minutes more code review on that example: alignment brittleness, turns CPU into a space heater, assumes receiver is capable of pacing publisher (I sincerely doubt this is true; my hunch is that a slow memory leak would eventually crash LabVIEW, sicne publisher is not throttled in time and recv is not throttled in space; that said, no "substantial" work is done in event handler case, which *might* mean this contrived example itself would not crash, but that certainly doesn't extrapolate to reader-application-domain-du-jour). It's not a horrible example, just misleading and kinda bad. And another race condition on labview side in registration.

  11. I wouldn't dive into doing some memory copies into memory preallocated by LabVIEW. Just make a LabVIEW "aware" data in your DLL code. It is the cleanest solution. You do not need to dispose any handles, just make a new one for the data, show it into PostLVEvent and harvest it in event structure with native LabVIEW code. LabVIEW will take care of all cleaning up (you do not need to dispose the handle you made).

     

    I directly contest these statements. Provide an executable test case here that indicates what you say is true.

     

    Am I missing something here? In your example you allocate a handle each time you send an event but never deallocate it. And right below that you say that it's not necessary to copy the original data since PostLVEvent will create its own copy of the data!

     

    *edit: "what RolfK said"

  12. Good luck here! Report back if/when you run into trouble.

     

    BTW, I'd tend to lean toward static linking here; it better matches the risk/reward profile in the circumstances you lay out.

     

    In terms of performance, static versus dynamic linking is utterly negligible in virtually all application domains -- it's a mostly a matter of your deployment/distribution/upgrade scenarios required by the application.

  13. I wan't to try and keep things as simple as possible, and having dealt with memory allocations in the DLL before I would like to try and avoid it if possible.

     

    It's unavoidable, unfortunately.

     

    Consider this race condition --

     

    Let's say your wrapper library uses `PostLVUserEvent()` inside the function callback that the main library calls, in order to send the structure represented by `struct tyMessage` (let's ignore alignment and padding for now; do a little hand-wavy thing knowing it's a surmountable prob).

     

    What happens on the LabVIEW side? Let's say this goes into a Register For Events queue and is handled by an event structure. This means, the handling of the event is asynchronous from the callback from the main library.

     

    Feel free to pause the tape now and consider the problem here; press play once you think you have an answer ...

     

     

    .... that's right -- race condition, where if you lose the race, the kernel is very likely going to throw an Access Violation (if you're lucky) or you're going to be accessing valid, but undefined memory space.

     

    Because, my best guess at how the library works is this:

     

    ```

    two malloc's to represent the non-fixed width data in struct tyMessage

    invoke your callback, based on function pointer you had passed in Init()

    on return of function, perform two free's on those two buffers, having expected the callback to eagerly make its own copy, or otherwise perform all processing before not needing it anymore

    ```

     

    So the other option is to instead of using Register for Events, use a Register Event Callback instead. Inside, you could `MoveBlock()` into a preallocated LStrHandle, but this the least performant, least maintainable, least reliable, and most moving parts. That callback would then need to work as a Flaky Delegate in some manner to broker that information back to your application.

     

    The final option (my recommendation, considering information so far) is to create two LStrHandle's (or some other more sensical structure, based on whatever cType and cData actually represent) within your callback, use `PostLVUserEvent()` to get those back into LabVIEW, then destroy the handles, allowing your callback to return control to the main library.

     

    In precisely the same vein as our callback here making its own copies, whatever queueing mechanism within LabVIEW uses to put UHandles into an Event Queue, the LabVIEW Memory Manager ensures that a "copy" is made, such that the original entity calling `PostLVUserEvent()` is able to immediately deallocate all resources it just sent -- conceptually, at least, this maps to the mental model.

     

    `DSDisposeHandle()` is woefully lacking in documentation how/whether the Memory Manager reference counts the underlying buffer pointed to by a UPtr, but anecdotally this is how I observe it behaves. Said another way -- even though your callback is responsible for the initial allocation (ref count == 1), `PostLVUserEvent()` then allows the enqueuer to make another claim on that underlying buffer (ref count == 2), then your callback calling `DSDisposeHandle()` will atomically* decrease the count (ref count == 1) without the Memory Manager actually `free()`ing the buffer. Later, in the Event Structure, once that data has been accessed via the left-hand Event Data node and the frame finishes executing, the data goes out of scope, at which point ref count drops to 0 and an actual `free()` (or at least, a conceptual release of that resource) is done.

     

    Clear as mud?

  14. This is good news as I was not really very excited to wrap every single other function exported by the DLL.

     

    Not so fast, friend ...

     

    First, can you provide some representative examples of say, some of the more complex functions you intend to invoke from LabVIEW?

     

    Also, it would help to know some basic timing diagrams and synchronization requirements between lv and the shared lib; e.g., are function callbacks spray-and-pray, or does the library routine invoking them expect synchronous return values or manipulations of arguments passed as pointers?

     

    Ideally, provide a header file, or a good smattering of function declarations, anonymizing variable names into `a`, `b`, etc. if necessary.

     

    Also, any cross-platform requirements? Is the DLL C calling convention? a CLR assembly? (obviously not, but `.dll` alone does not constrain how to proceed here)

     

    You now have three people tuned into this thread who can potentially save you from less-obvious but even-more-insidious (because, they're not obvious) problems that potentially lie ahead beyond just function pointers.

    • Like 1
  15. Skimming thru this thread... try this:

     

    From C, call `PostLVUserEvent()`.

     

    Then, in LabVIEW, rather than registering that event reference into a `Register for Events` node, register the event reference with a `Register Event Callback` node.

     

    Here's the game-changer -- `PostLVUserEvent()` in C will behave differently depending on how the associated event reference is registered in LabVIEW.

     

    When a listener is bound to the event using `Register for Events`, `PostLVUserEvent()` is asynchronous and non-blocking, and does not wait for the consumption of the event. It "blocks" only long enough to copy data reliably into the queue shared by its message-reference-counting registrants, then moves along.

     

    On the other hand, `PostLVUserEvent()` will synchronously block until the Callback VI handler has finished executing.

     

    This means, your library can `Post` some pointers/handles into LabVIEW, LabVIEW can fill them in the Callback VI, and downstream logic from the `PostLVUserEvent()` needs no further gymnastics to synchronize/mutex.

    • Like 2
  16. Thanks for the tip, I really appreciate it. For some reasons, since my early days as a LV developer I've had the impression that people wait for the SP1 because of potential bugs when a new version is released. But maybe that is not justified anymore?

     

    Nah, go for it. Think of LV2015 as LabVIEW 8.6.1 SP13.

    • Like 2
  17. configure the path in on the diagram, and to use a separate (inlined) VI to store this path.

     

    Having (exhaustively?) tested combinations, I have found this strategy the least of evils. My heuristics for optimization (perhaps relevant for you, perhaps not): cross-platform distribution and developer design-time woe-minimization. As for performance, the price in run-time microseconds -- a pinch on a modern desktop, a dash on LinuxRT.

     

    The idea of a scripting VI to do this automatically is super-neato in theory, but probably not practical further than advancing your own meta-programming ability, simultaneously tempering and battle-hardening your temptation to write/trust code that modifies code you maintain.  :)

  18. Starting to get up to date with LabVIEW 2015 on a new Windows 10 Parallels VM with default VM settings. Scroll wheel does not work in LabVIEW, but it does work in other applications (including LabVIEW Help).

     

    I'm not sure if this is a Windows 10 problem or Parallels 10.

     

    Any ideas?

     

    *** EDIT: Actually, LabVIEW does respond to the scroll wheel -- but only very fast scrolls to get small/unpredictable scrolling movements.

  19. Great suggestion, topic split.

     

    Thank you; I was reserving continued replies without polish in the spirit of not muddying the water of the original topic, so this helps a more organic and focused conversation.

     

     

     

    Does a Channel Wire fall within your definition of "dataflow"?

     

     

    It's not my definition. You tell me.

     

    Unequivocally, and with a detailed response whose draft seems to have been lost in the thread fork. Stay tuned...

  20. What can an polymorphic VI do that an XNode can't?  What can an Express VI do that an XNode can't?  What can a Generic VI, or VIM do that an XNode can't?

     

    Take less time/knowledge/effort to create and maintain by encapsulating complexity at the meta-language layer instead of dumping it into application repositories. What makes sense for you at what layer? They both have valid cases.  :)

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.