JackDunaway

LabVIEW Software Engineers, here's your opportunity to work with us at SpaceX! I'm hiring for a Sr. Software Engineer on the Ground Software team. We build control and monitoring systems running the Falcon launchpads, mission systems launching astronauts to the International Space Station onboard Dragon, and a fleet of sea-faring recovery vessels bringing spacecraft and rockets back to shore. The screens you see in Mission Control are the UIs we build. Read more about the role and apply here: https://boards.greenhouse.io/spacex/jobs/5480997002?gh_jid=5480997002

Just now seeing this topic, and cross-referencing against some of my own uptime monitoring. Something happened this day which made lavag.org respond faster, but probably affected labviewwiki in the process:

Are all your variables bound to your prepared statement? For example, rather than SELECT foo FROM table1 WHERE col='bar' the statement SELECT foo FROM table1 WHERE col=@value_to_match -- then later binding 'bar' to @value_to_match is going to be more performant (over multiple queries, especially), more precise (for floating-point numbers), and more secure (against SQL Injection Attack, assuming that the original `bar` was inserted at run-time using the likes of the labview `Format to String` function or C `snprintf()`).

Unless something has changed, both the Embedded Web Server and Application Web Server are AppWeb by EmbedThis. That web server in and of itself seems pretty capable and pretty sweet, but the wrapper that LabVIEW puts around it strips and sanitizes virtually all of the configuration file options: https://embedthis.com/appweb/doc/users/configuration.html Also, unless something has changed, setting up `SSLCertificateFile` with a private key signed by a bona fide CA (rather than self-signed) took some jumping-thru-hoops. You can poke around and modify the `ErrorLog` and `Log` specs, but only a couple security tokens are honored by not being sanitized away (https://embedthis.com/appweb/doc/users/security.html and https://embedthis.com/appweb/doc/users/monitor.html) Also -- unless something has changed -- the developer experience between dev-time and deploy-time is vastly-different, the way the web service actually works. At a very high level, the problem was this: the concepts of who calls `StartWebServer()` (maRunWebServer) were hard-coded or something into the way the RTE loads applications, and so therefore was not invoked when you just run your application from source in the IDE. In that case, there was this weird out-of-band deployment via a Project Provider, where it used the application web server rather than the embedded web server. Where of course, managing the configurations of these two disjoint deployments didn't work at all. YMMV, but I don't use it for anything. libappweb always felt pretty awesome, but it was just shackled (oh; and while libappweb continually improved, the bundled version with lv stayed on some old version) On a side note, the AppWeb docs site got a huge facelift -- looks like Semantic UI; looks/feels spectacular.

Exactly. If we're talking TCP here (we are), rate limiting thru backpressure is what you want by setting `SO_RCVBUF` with `setsockopt()` Buried in vi.lib you can find some methods that take a TCP connection reference and expose the underlying file descriptor -- presumably, you could set such options with kernel APIs (I've not tried) -- but in reality using the labview TCP API is kinda jokey if you're concerned about security (or high performance). Granted, best-in-class security and also high-performance are not the presiding requirement for many applications, in which case the TCP API is perfectly fine and convenient. Response to each sentence: Yikes; yikes; fair enough; seems like a reasonable; only a 100% maniac would do this; YIKES NO NO NO. In regards to all instruments/control systems being accessible remotely -- whether intranet, or especially public internet -- one must analyze the risk profile of what happens in the physical word if the controller goes up in smoke, and how easy/likely is that to be triggered remotely? The controller is a finite resource. If demanded, it can spend 100% of its finite power either controlling the process, or 100% of its power fending off an DoS or fuzzing attack. Each can starve the other, and both can starve "well-behaved", authorized remote clients/peers attempting to bring order to a railed-out controller. (Here, "attack" is a word shrouded with improper connotation. It's likely that we are ourselves (and our colleagues) are our own (and colleague's) biggest attackers, just by writing dumb bugs. Networked communication is an *excellent* manner of flushing out bugs we were not even aware we were so good at writing, having become accustomed to the reasonably deterministic and reliable medium of memory and CPU for in-process communication, not always programming against such lossy mediums as the physical jankiness the IP stack must deal with.) All this said; the labview web service is woefully incapable of configuring security and QoS parameters for clients. It's reasonable to use behind VPN on an intranet, but I would strongly recommend against putting it on the internet without sitting behind a production-quality HTTP server acting as a reverse proxy.

This one bit is not quite correct. You're thinking about multiple Event Handler Structures binding to a shared queue created by a single Register For Events Node. That's the scenario that produces undefined behavior. (Not just unpredictable behavior; truly undefined behavior) Multiple Event Handler Structures bound to a "Statically-Registered Event" are OK, but as hooovahh suggests, not usually recommended. --- "Statically-Registered Event" -- contrasted with "Dynamically-Registered Event" -- is just a silly terminology for "magic wires that you don't see that you just 'know' must exist". The original post is an excellent case study for the mental leap required to learn/believe/trust/understand Static Event Registration. For some reason, in LV world it's taught that Static Event Registration is "easier" than the "advanced" topic of Dynamic Event Registration. Maybe if Dynamic Event Registration were taught like this -- it's wires you see, in the code you wrote, that code in front of you on the block diagram; that's what happens and why it happens. And lifetimes of both the event sources and event sinks -- yeah, that's kinda just wires too -- the wires you see in front of you, on the code you wrote. Understand this, and then the concept of "Static Event Registration" might help you by replacing a little syntax with some accepted, implicit convention. Said another way -- Dynamic Event Registration is syntax where the mental model maps to the visual source model, whereas Static Event Registration is just a shortcut that hides implementation details. (Yes, there do exist some notable functional differences between the two, but that's just a shortcoming of both, not a fundamental difference in the model) @Scatterplot -- as an excercise, can you create the same diagram, just with a static reference to `Value 1`, wired into a `Register for Events`, wired into the Event Handler Structure? Post that snippet, then let's use that as an explicit example to the "wireless" model of Static Event Registration.

Envious should be the wrong word here. Such "arcane issues" should have been made more developer-friendly long, long ago -- these topics definitely feel "foreign" coming from labview, but "arcane" -- we're on the far other end of the spectrum from "arcane" as far as C is concerned here -- this knowledge should be some straightforward day-to-day interop. But it's not, unfortunately. That said ... you'll find relatively few questions unanswered, and will rarely be disappointed, appending "RolfK" to your searches on these topics :-) I had started to type a response here, but it got too long -- this is actually one topic with many excellent explanations findable in a search, and to help guide your search, it might be helpful to also look for "data segment" (since this is sometimes where other types of static memory are compiled into the object file). It took me a minute to understand why you had to bring the orange banner into the mix -- but by all means, declaring `dw_LabViewEventRef` as a global variable *might* be justified as a gross simplification for an example program, but this is by no means production-quality. I think this advice holds pretty much absolutely in this domain -- if you find yourself using the orange banner to "solve" problems, it's going to be a long, cold day. But that said -- and to refine what @ShaunR is saying here -- you can't pass the ref into the callback function, since it's the main library who invokes and passes parameters into your callback function. When telling the main library what callback function to use (e.g., I would expect the library has a function with a name along the lines of "register callback function"), does it also give you an opaque data pointer you can define yourself that it will pass to your callback? This is where to initialize the main library with the UserEventRef from labview. That said ... the next topic would be how and when to allocate and deallocate that memory on the heap

This part actually still fascinates me, how labview provides such a clean abstraction over the underlying memory ... ... and you pretty much were able to hit every facet from which my frustrations stem. 1) underpowered interface that 2) never changes that 3) was not dogfooded so 4) it totally blows and therefore so does my developer experience while trying to do something meaningful with the API while 5) NI has their own far more powerful interface, and are therefore unconcerned day-to-day with 1)

Thanks for actually running this and confirming my suspicion; i was just poking thru source code on a 13" macbook air and xcode, and of course couldn't even load the VI or C without compiler error I seem to recall @Darin.K commenting before on the quality of the PRNG in LabVIEW in the past; @ShaunR, perhaps you have demonstrated a better one here!

No prob. First, your statement "nice simple example from NI engineer" threw my BS detector into red-alert mode "space heater" means CPU, polling without any sort of throttle, or more appropriately interrupt-driven procedure, turns into this: https://www.google.com/search?q=%22space+heater%22&tbm=isch Time bug -- publisher waits only long enough to allow "the thing" on the other side of `PostLVUserEvent()` to make a memory copy, but not to reasonably ACK that the event has been processed and the handler is effectively ready for the next event. In other words, publisher is asynchronized from subscriber -- or even bigger problem, subscribers (plural). Space bug -- subscriber is unable to put a cap on its recv queue size in labview, unlike, say, a BSD socket with a cap on recv buffer that has `connect()`-ed to a `bind()`-ed publisher. In that scenario, TCP backpressure from subscriber to publisher is capable of effectively enforcing timeouts and message drops with EAGAIN or ETIMEDOUT on the publisher -- meaning the lack of specifying and limiting space on the subscriber side in labview (in the form of limitign the queue size of an `Register For Events` node) severely limits the publisher to act sanely (since, no backpressure on `PostLVUserEvent()`). You have to jump through totally bogus, undocumented, unsupported, uncharacterized, and unintuitive hoops to hack together a viable solution using `PostLVUserEvent`, `Register Callback VI` node, LabVIEW queues, and and additional User Event to grossly approximate a decent `recv` buffer limit. I do not say this proudly -- I say this defeated and deflated -- but I may have accidentally characterized the only solution in existence to such a fundamental, gaping hole in `extcode.h` -- the ability for C to synchronously call into *LabVIEW* and synchronously wait for a response on the call stack (rather, than the other way around). Summarized: it's easy to create memory leaks when the `send` endpoint cannot detect the `recv` endpoint is not processing quickly enough. Basically, `PostLVUserEvent()` makes wildly dumb assumptions about both space (memory) and time (synchronization between `send` and `recv` endpoints), and perhaps nobody on earth is using it properly, except for serendipity of an application domain where it happens to work, and/or hearty `sleep()` sprinklings. I'm not bitter, just jaded and irritated with `extcode.h`

To be fair, your expectation here is utterly reasonable and desirable. It's the C<->LabVIEW FFI is utterly unreasonable and undesirable, and virtually unchanged since the 1900's.

OK, having downloaded that example; the data posted is allocated on the stack, not on the heap. Major difference. That's why that particular example you cite has no memory leak. But creating a new DSHandle without freeing it -- definitely a memory leak. Now, ideally, `LVPostUserEvent()` would have an additional (boolean) parameter that indicates "take ownership of this data" -- to avoid caller bookkeeping and provide vastly superior performance by not bring the Memory Manager into the mix making a new allocation -- but the LVRT is back in caveman ages, so we are relegated to caveman practices. *edit: And a couple minutes more code review on that example: alignment brittleness, turns CPU into a space heater, assumes receiver is capable of pacing publisher (I sincerely doubt this is true; my hunch is that a slow memory leak would eventually crash LabVIEW, sicne publisher is not throttled in time and recv is not throttled in space; that said, no "substantial" work is done in event handler case, which *might* mean this contrived example itself would not crash, but that certainly doesn't extrapolate to reader-application-domain-du-jour). It's not a horrible example, just misleading and kinda bad. And another race condition on labview side in registration.

I directly contest these statements. Provide an executable test case here that indicates what you say is true. *edit: "what RolfK said"

Good luck here! Report back if/when you run into trouble. BTW, I'd tend to lean toward static linking here; it better matches the risk/reward profile in the circumstances you lay out. In terms of performance, static versus dynamic linking is utterly negligible in virtually all application domains -- it's a mostly a matter of your deployment/distribution/upgrade scenarios required by the application.

It's unavoidable, unfortunately. Consider this race condition -- Let's say your wrapper library uses `PostLVUserEvent()` inside the function callback that the main library calls, in order to send the structure represented by `struct tyMessage` (let's ignore alignment and padding for now; do a little hand-wavy thing knowing it's a surmountable prob). What happens on the LabVIEW side? Let's say this goes into a Register For Events queue and is handled by an event structure. This means, the handling of the event is asynchronous from the callback from the main library. Feel free to pause the tape now and consider the problem here; press play once you think you have an answer ... .... that's right -- race condition, where if you lose the race, the kernel is very likely going to throw an Access Violation (if you're lucky) or you're going to be accessing valid, but undefined memory space. Because, my best guess at how the library works is this: ``` two malloc's to represent the non-fixed width data in struct tyMessage invoke your callback, based on function pointer you had passed in Init() on return of function, perform two free's on those two buffers, having expected the callback to eagerly make its own copy, or otherwise perform all processing before not needing it anymore ``` So the other option is to instead of using Register for Events, use a Register Event Callback instead. Inside, you could `MoveBlock()` into a preallocated LStrHandle, but this the least performant, least maintainable, least reliable, and most moving parts. That callback would then need to work as a Flaky Delegate in some manner to broker that information back to your application. The final option (my recommendation, considering information so far) is to create two LStrHandle's (or some other more sensical structure, based on whatever cType and cData actually represent) within your callback, use `PostLVUserEvent()` to get those back into LabVIEW, then destroy the handles, allowing your callback to return control to the main library. In precisely the same vein as our callback here making its own copies, whatever queueing mechanism within LabVIEW uses to put UHandles into an Event Queue, the LabVIEW Memory Manager ensures that a "copy" is made, such that the original entity calling `PostLVUserEvent()` is able to immediately deallocate all resources it just sent -- conceptually, at least, this maps to the mental model. `DSDisposeHandle()` is woefully lacking in documentation how/whether the Memory Manager reference counts the underlying buffer pointed to by a UPtr, but anecdotally this is how I observe it behaves. Said another way -- even though your callback is responsible for the initial allocation (ref count == 1), `PostLVUserEvent()` then allows the enqueuer to make another claim on that underlying buffer (ref count == 2), then your callback calling `DSDisposeHandle()` will atomically* decrease the count (ref count == 1) without the Memory Manager actually `free()`ing the buffer. Later, in the Event Structure, once that data has been accessed via the left-hand Event Data node and the frame finishes executing, the data goes out of scope, at which point ref count drops to 0 and an actual `free()` (or at least, a conceptual release of that resource) is done. Clear as mud?