Rebar/Rust and G Community

[Ben Leedom]

Cross-posting from the ni.com LabVIEW forum:

Hello! This is a brief announcement that there's a new community here:

https://forums.ni.com/t5/Rust-and-G/gp-p/5381

I started this to discuss a project called Rebar, which is an experiment in combining semantics of the Rust language (https://www.rust-lang.org/) with G syntax, implemented as an addon for LabVIEW NXG. You can read more about it and find a link to install the addon over at the community. Please take a look!

[smithd]

It seems interesting, and very cool that NXG lets you do stuff like this, but I do have some questions and comments.

All of the below is without having installed it. I tried, but it requires NI package manager 19. NIPM 19 is not released as far as I can tell (I downloaded the latest, 18.5.1, from ni.com and when it loads it doesn't seem able to reach the server). More to the point, why does NIPM continuously have this issue where it requires a specific version to unpack the files? I honestly don't think I've ever successfully installed a package with NIPM on the first go <_<

 

To start, I think you need to get more technical on the overview. For example, you say things like local variables and shift registers are "places" and that "places" have lifetimes...what is the lifetime of a local variable or a shift register? The explanation of how in rebar the sets of wires correspond to a single 'place' which...sounds just like a standard labview buffer allocation. My point is that you're making something for the type of person who might say "oh neat a rust+labview mashup!!" but then have a 1000ft explanation of how it works 👻

Quote

Memory performance is easier to control: The language makes potentially expensive data copies explicit, and allows values to be re-used and modified in-place without playing "hide the copy dots" or using special syntax like the In Place Element Structure.

This is just a nitpick, but labview already makes data copies explicit. What it doesn't make explicit is optimizations that remove the need for copies .

Also, they aren't copy dots, they are buffer allocations...

Quote

Values that need safe resource cleanup can be expressed graphically: Because all values have a definite lifetime defined by their owner, any value type can define what happens when it needs to be cleaned up (similar to a destructor in C++, a finalizer in C#, or the Drop trait in Rust). This means that values that represent system or hardware resources or other shared objects--for example, file handles, network connections, device sessions, or queues--that would be implemented by the runtime or by non-graphical plugins in G can be implemented graphically in Rebar.

Its obvious that LabVIEW knows what to do with a visa refnum (and file refnums, and any other refnum I can think of) when the application closes...which I think is a positive for LabVIEW, so I'm not sure how often its actually valuable to define cleanup (especially given it sounds like the cleanup is off diagram). That having been said, I can think of one super exciting use for this which is hardware access via a dll. If this tool lets you say "this pointer sized int is a pointer and when the program exits call this function" thats pretty awesome. And much better than the unreserve callback nonsense in the CLFN.

Quote

Safe references instead of refnums: G refers to objects with lifetimes using refnums, or typed handles to objects managed by the LabVIEW runtime. Refnums have many of the unsafe characteristics of raw pointers: they must be manually freed to avoid leaks, but freeing a refnum in one place leaves any other places holding the same refnum with a dangling reference. Furthermore, the refnum's object's methods must implement their own thread safety, since they can in theory be called from multiple concurrent places in a G application. Rebar references have none of these problems; the compiler prevents you from using or freeing them unsafely, allowing references to be implemented under the hood as raw pointers, without the runtime needing to manage objects.

I'm kind of confused by this and it probably comes from my comments at the top about not really getting whats going on under the hood.

To me the problem with the existing system is that the refnums point at global things. For example, if I allocate a queue inside of A.vi, and I have no terminal which outputs the value of the queue refnum, if A.vi finished executing the compiler SHOULD kill the queue.....except that queues are global. I'd much much much much rather have the compiler figure out the lifetime of the queue for me and kill it when the wire can't go anywhere else. It seems clear that this addon allows for that, which is awesome, but why do we need an addon? I thought the whole point of dataflow was that the compiler knew about the lifetime of data and could make intelligent decisions based on that?

Quote

More modern, higher-level language features: Rebar should be able to provide features that appear in other languages, like anonymous functions and closures or iterators and generators, that would be awkward to implement by-value in G or would require adding new refnums to the language.

I don't know enough about iterators and generators, but why do you think anonymous functions are aided by this? To be honest its not clear to me why anonymous functions aren't possible right now, nor closures.

 

I think a good summary of my comments above is basically this: Hey, thats pretty neat -- why isn't this already part of the language?

 

I wanted to finish up with an application-specific question. One of the big use cases for avoiding copies is image processing, but unfortunately imaq images are not only global, but they are not reference counted like named queues (ie worst of both worlds). So if, for example, you are an image acquisition and you want to share your latest snap with all your closest friends (the logging loop, the network loop, the processing loop, the display loop) its difficult to share the references in such a way that when all the other loops are done, the image is freed. If it were a queue, you could obtain several named references to the same queue and the queue only goes away when all the references are dead. I ended up implementing this myself with a DVR, but it still requires that users call the 'release' function -- ideally, and what I think rebar brings to the table, is an automatic release. Does this rebar system solve that? If so, sweet. You should share it with the imaq group

 

 

Edited February 14, 2019 by smithd

[Ben Leedom]

4 hours ago, smithd said:

All of the below is without having installed it. I tried, but it requires NI package manager 19. NIPM 19 is not released as far as I can tell (I downloaded the latest, 18.5.1, from ni.com and when it loads it doesn't seem able to reach the server). More to the point, why does NIPM continuously have this issue where it requires a specific version to unpack the files? I honestly don't think I've ever successfully installed a package with NIPM on the first go <_<

Apologies for that! I failed to realize that NIPM 19 isn't publically available yet. I will try to spend some time creating a package that is compatible with NIPM 18.5.1 soon.

And yes, I share your frustration with NIPM versioning. I go long enough without having to fiddle with NIPM on my dev machine that when I do I wind up reinstalling it from scratch almost every time.

Quote

To start, I think you need to get more technical on the overview. For example, you say things like local variables and shift registers are "places" and that "places" have lifetimes...what is the lifetime of a local variable or a shift register? The explanation of how in rebar the sets of wires correspond to a single 'place' which...sounds just like a standard labview buffer allocation. My point is that you're making something for the type of person who might say "oh neat a rust+labview mashup!!" but then have a 1000ft explanation of how it works 👻

Fair enough! Over the last week or so I've been trying to get things into a presentable state, but I agree that there is a need for a lot more documentation and explanation. The overview you've been quoting was indeed intended to be a high-level summary of why one might want to use this new thing that looks superficially like the old thing. I hope to spend more time soon adding more in-depth content to the wiki on github. But you're asking the technical questions, so let's get technical.

I had a draft of the overview that listed the specific lifetimes of places, but then removed it. A local variable's lifetime is bounded by that of its containing VI's clone, because it is stored in the VI's dataspace--that is, it persists not just for the entirety of calls to the VI but also across calls. A shift register's lifetime is bounded by the loop where it is defined, unless it is uninitialized, in which case its lifetime is like that of a local variable's--it persists across calls.

I was using the word "place" to avoid using the word "variable," since that has specific and different meaning in G. What I mean is much closer to a local variable or a member variable in a text-based language like C or Java, something that can be defined and then reassigned to. You can reassign values to shift registers or local variables (places) in G by wiring inputs to them, but the language doesn't express directly reassigning to the buffer for a wire (a value); that's the difference.

Quote

This is just a nitpick, but labview already makes data copies explicit. What it doesn't make explicit is optimizations that remove the need for copies .

Also, they aren't copy dots, they are buffer allocations...

Okay, they're not copy dots. "Buffer allocations" is not a precise enough term in my mind, either. VIs allocate data in basically two ways: the dataspace for each VI clone, and heap buffers. Each clone of a VI gets the same size dataspace, so when an object's size can be known at compile time, it generally gets put entirely in the dataspace, and otherwise its dataspace presence is a handle to a heap buffer--the latter includes arrays, strings, paths, value classes, and variants. Substrings and subarrays are handles in the dataspace separate from the main array or string handle but pointing into the same data. As Stephen's post says, LabVIEW does not distinguish between places where it just adds more to the dataspace and where it allocates both in the dataspace and the heap. Moreover, just because LabVIEW shows a buffer being allocated for an array at a particular place, does not mean that that place may not reuse a heap buffer on its second execution that it obtained on its first execution.

Quote

Its obvious that LabVIEW knows what to do with a visa refnum (and file refnums, and any other refnum I can think of) when the application closes...which I think is a positive for LabVIEW, so I'm not sure how often its actually valuable to define cleanup (especially given it sounds like the cleanup is off diagram). That having been said, I can think of one super exciting use for this which is hardware access via a dll. If this tool lets you say "this pointer sized int is a pointer and when the program exits call this function" thats pretty awesome. And much better than the unreserve callback nonsense in the CLFN.

"Hardware access via a DLL" is not altogether different from what a VISA refnum, or any other refnum, does. Refnum types, whether implemented internally in the runtime or with plugins, are basically bundles of pointers to DLL functions, with special functions to use for resource acquisition and cleanup (not unlike the CLFN unreserve callback nonsense).

But yes, part of the point of this is to allow you, the graphical programmer, to define types that wrap pointers and DLL functions, or any other sort of "resource," and have them clean themselves up automatically. And we can do one better than "when the program exits": we can clean up the object at the precise time when we know it is no longer used, which gets to your next question:

Quote

To me the problem with the existing system is that the refnums point at global things. For example, if I allocate a queue inside of A.vi, and I have no terminal which outputs the value of the queue refnum, if A.vi finished executing the compiler SHOULD kill the queue.....except that queues are global. I'd much much much much rather have the compiler figure out the lifetime of the queue for me and kill it when the wire can't go anywhere else. It seems clear that this addon allows for that, which is awesome, but why do we need an addon? I thought the whole point of dataflow was that the compiler knew about the lifetime of data and could make intelligent decisions based on that?

The compiler does try to determine the lifetime and usage of data and make decisions about when to copy it based on that. There are several factors that confound the compiler's analysis and force it to be conservative:

• The compiler tracks signature information for each VI about how its outputs and inputs are in-place to each other. Dynamic dispatch calls represent calling one of many possible signatures, not all of which may be known at compile time (in the case of dynamic loading), so those calls generally have to be conservative with copying data.
• Calling a VI by reference asynchronously also requires being conservative, because you can no longer directly tie when a call begins to when it ends.
• Refnums are particularly bad for determining lifetime, because you can copy the refnums themselves anywhere--uninitialized shift registers, global variables, queues, DVRs, fields of derived classes stored in base class wires, variants--such that it is impossible to determine statically how many different places might all point to the same object. The runtime does not increment a reference count every time it copies a refnum, so it has to be extremely conservative about when an object can be freed--it's pretty much either, "the user just called Close Reference on this" or "the program's going away now." It's not just that some refnum types let you globally access instances by name, it's more that refnums are all the concurrency-unsafe baggage of pointers without the advantage of referring to a literal memory address.

Quote

I don't know enough about iterators and generators, but why do you think anonymous functions are aided by this? To be honest its not clear to me why anonymous functions aren't possible right now, nor closures.

To be clear, I do think anonymous functions are possible in G today. Actually, if you get right down to it, a VI is pretty much what closures are in other languages, in terms of retaining state across calls. I think it's likely that anonymous functions would have to be presented as and implemented as nearly identical to VIs, which means providing configuration options for things like reentrancy.

One of the goals of Rebar is to provide a basic function document that retains absolutely no state across calls, and thus can completely dispense with the notion of reentrancy as a property of a function. That in turn simplifies the notion of defining an anonymous function.

Quote

I wanted to finish up with an application-specific question. One of the big use cases for avoiding copies is image processing, but unfortunately imaq images are not only global, but they are not reference counted like named queues (ie worst of both worlds). So if, for example, you are an image acquisition and you want to share your latest snap with all your closest friends (the logging loop, the network loop, the processing loop, the display loop) its difficult to share the references in such a way that when all the other loops are done, the image is freed. If it were a queue, you could obtain several named references to the same queue and the queue only goes away when all the references are dead. I ended up implementing this myself with a DVR, but it still requires that users call the 'release' function -- ideally, and what I think rebar brings to the table, is an automatic release. Does this rebar system solve that? If so, sweet. You should share it with the imaq group

There are two ways that Rebar might let you safely share an image between multiple places like this. If you want to restrict all the loops to running in lockstep (i.e., each one finishes working with one image before any starts on the next), then you can pass simple references (pointers) to each loop. If that's too restrictive, then the language will provide a basic reference-counted wrapper around any type:

• Cloning the wrapper automatically increments the refcount, and the clones can be sent anywhere;
• When the wrapper goes out of scope, it automatically decrements the refcount;
• When the refcount goes to zero, it destroys the underlying object;
• All of the reference counting is done atomically, so that it's safe to use in parallel.

 

Thanks for your questions! Let me know if you have any others, or if I need to explain anything better.

[drjdpowell]

11 hours ago, smithd said:

If this tool lets you say "this pointer sized int is a pointer and when the program exits call this function" thats pretty awesome. And much better than the unreserve callback nonsense in the CLFN.

You should kudo this then: Means to register a DVR-cleanup callback for use when DVR released when VI goes idle.  This would allow DVRs to wrap all dll pointers, with proper cleanup regardless of how the VI stops.

[drjdpowell]

A note on the discussion about LabVIEW refunms, such as Queues and the like.   These are "owned" by the Top-level VI that created them, and are cleaned up when that Top-level VI goes idles.   This could be the whole App, but async-called VIs are all their own "Top Level", so if your app is made up of many async called VIs ("modules", "actors" or whatever) then refnums have a quite reasonable defined lifetime, the lifetime of the creating module/actor/async-thing.  This behaviour is reliable; I know because I routinely rely on it to trigger automatic shutdown of async running things when their callers stop for any reason.  

[smithd]

2 hours ago, drjdpowell said:

You should kudo this then: Means to register a DVR-cleanup callback for use when DVR released when VI goes idle.  This would allow DVRs to wrap all dll pointers, with proper cleanup regardless of how the VI stops.

Definitely Kudod, but Mercer's responses sound like this is never gonna happen. The abort button thing in particular seems to also apply to rebar, although it's not clear if the nxg abort has the same semantics.

 

I like your point on references, but the counter is also good to bring up -- since everything is top level, reliable sharing is impossible between two things with different lifetimes unless one is always long lived and defined as the owner. This is often fine, but it can bite people who are unaware.

[smithd]

Most of your responses made sense so I'm just going to pick out the few bits that made less:

9 hours ago, Ben Leedom said:

I had a draft of the overview that listed the specific lifetimes of places, but then removed it. A local variable's lifetime is bounded by that of its containing VI's clone, because it is stored in the VI's dataspace--that is, it persists not just for the entirety of calls to the VI but also across calls. A shift register's lifetime is bounded by the loop where it is defined, unless it is uninitialized, in which case its lifetime is like that of a local variable's--it persists across calls.

The local variable lifetime is kind of my point -- its life isn't really limited because the data space for a reentrant VI remains live for the duration of an application. Similarly, I was under the impression the shift register value is also permanent. As an example, I seem to recall a while back running a loop that allocates a giant (>>1mb) array on a shift register but never wire it out of the VI. if you do that, the large array remains part of the VI's data usage. From the perspective of the details you mentioned, yes, that giant array is on the heap somewhere, but all that data is still live until something overwrites it, making that data location permanent as well, even if initialized.

9 hours ago, Ben Leedom said:

The compiler does try to determine the lifetime and usage of data and make decisions about when to copy it based on that. There are several factors that confound the compiler's analysis and force it to be conservative:

• The compiler tracks signature information for each VI about how its outputs and inputs are in-place to each other. Dynamic dispatch calls represent calling one of many possible signatures, not all of which may be known at compile time (in the case of dynamic loading), so those calls generally have to be conservative with copying data.
• Calling a VI by reference asynchronously also requires being conservative, because you can no longer directly tie when a call begins to when it ends.
• Refnums are particularly bad for determining lifetime, because you can copy the refnums themselves anywhere--uninitialized shift registers, global variables, queues, DVRs, fields of derived classes stored in base class wires, variants--such that it is impossible to determine statically how many different places might all point to the same object. The runtime does not increment a reference count every time it copies a refnum, so it has to be extremely conservative about when an object can be freed--it's pretty much either, "the user just called Close Reference on this" or "the program's going away now." It's not just that some refnum types let you globally access instances by name, it's more that refnums are all the concurrency-unsafe baggage of pointers without the advantage of referring to a literal memory address.

 

Good explanation, but for the refnum point I keep coming back around to -- yes, this is something I absolutely would love to see improved and resolved, but it seems like a bandaid over an underlying flaw in how labview does dataflow.

9 hours ago, Ben Leedom said:

One of the goals of Rebar is to provide a basic function document that retains absolutely no state across calls, and thus can completely dispense with the notion of reentrancy as a property of a function. That in turn simplifies the notion of defining an anonymous function.

Now thats something I can definitely get behind. If NXG can have "gvi"s or "good, baggage-free VIs without depending on the design decisions from 3 decades ago that might not have aged as well" I might switch sooner
But in all seriousness, it seems like you've made your own type of VI with compiler rules associated with it. Is there a way to expand that more broadly, making life simpler enough for the compiler (per your points above) that it can figure out the lifetime of objects? IE in order to reach the goal of stateless functions, but without relying on the user to manage lifetimes?

 

 

 

 

[Ben Leedom]

1 hour ago, smithd said:

The local variable lifetime is kind of my point -- its life isn't really limited because the data space for a reentrant VI remains live for the duration of an application. Similarly, I was under the impression the shift register value is also permanent. As an example, I seem to recall a while back running a loop that allocates a giant (>>1mb) array on a shift register but never wire it out of the VI. if you do that, the large array remains part of the VI's data usage. From the perspective of the details you mentioned, yes, that giant array is on the heap somewhere, but all that data is still live until something overwrites it, making that data location permanent as well, even if initialized.

I think it's important to distinguish between what the language semantics are and what the compiled code and runtime actually do. For an initialized shift register on a loop, its value for a particular execution of that loop is only accessible during and immediately after the loop; the language does not let you get that value at any other point, and the value will be overwritten when it is initialized on the next loop execution. (NB: I'm not talking about individual iterations of the loop, but the execution of the loop over all iterations.) The compiled VI does store the shift register value in its dataspace, so yes, that value will be retained in the dataspace between loop executions, but from the language's perspective this doesn't matter because it's inaccessible. The language does not require the value to be retained or its allocation to be reused between loop executions; that's an optimization that the runtime provides.

1 hour ago, smithd said:

Good explanation, but for the refnum point I keep coming back around to -- yes, this is something I absolutely would love to see improved and resolved, but it seems like a bandaid over an underlying flaw in how labview does dataflow.

I'm not sure if we're agreeing or disagreeing here. My claim is that the way LabVIEW does dataflow--meaning wires carry immutable values that can be used in any number of concurrent places downstream--means that refnums are probably the best way of referencing objects that is available to LabVIEW. (I'm not enough of a theoretician to prove that reference systems with better properties aren't possible, but I do trust that if there were a better way, somebody smart around here would have discovered it in the last 30 years or so. However, another hint in this direction is the fact that channel wires have properties that approach what I'm proposing with Rebar, and channel wires are fundamentally different from normal wires.) If you want a reference system with better properties, I believe you must fundamentally change the way LabVIEW does dataflow. And so, to your next point:

1 hour ago, smithd said:

Now thats something I can definitely get behind. If NXG can have "gvi"s or "good, baggage-free VIs without depending on the design decisions from 3 decades ago that might not have aged as well" I might switch sooner
But in all seriousness, it seems like you've made your own type of VI with compiler rules associated with it. Is there a way to expand that more broadly, making life simpler enough for the compiler (per your points above) that it can figure out the lifetime of objects? IE in order to reach the goal of stateless functions, but without relying on the user to manage lifetimes?

Yes, this is a new type of VI, or a new Model of Computation, or a new set of compiler rules, or whatever you want to call it--though I'd prefer you just call it a function, to distinguish it from what a VI is, which is more than a function. Importantly, it is not merely an extension of VIs, because it works differently enough that I do not think it should be mixed with G on the same diagram. Interoperating with VIs is a different story, and I think there is an interesting set of possibilities there.

But I'm not sure I understand your question, "is there way to expand that more broadly?" I think to make life simpler for the compiler, you have to change the language, like I said above--maybe that answers it? Also, rather than "relying on the user to manage lifetimes," I would state it as "providing a set of safe and complete rules for lifetimes and guiding the user into writing code that follows the rules."

[Aristos Queue]

3 hours ago, Ben Leedom said:

If you want a reference system with better properties, I believe you must fundamentally change the way LabVIEW does dataflow.

I agree.

Ben works on making a language safe for references. I work on making a language without references. Our goals are the same: a world where data can be trusted.

[smithd]

4 hours ago, Aristos Queue said:

I agree.

Ben works on making a language safe for references. I work on making a language without references. Our goals are the same: a world where data can be trusted.

 

7 hours ago, Ben Leedom said:

I'm not sure if we're agreeing or disagreeing here. My claim is that the way LabVIEW does dataflow--meaning wires carry immutable values that can be used in any number of concurrent places downstream--means that refnums are probably the best way of referencing objects that is available to LabVIEW. ....

..I think to make life simpler for the compiler, you have to change the language, like I said above--maybe that answers it? Also, rather than "relying on the user to manage lifetimes," I would state it as "providing a set of safe and complete rules for lifetimes and guiding the user into writing code that follows the rules."

Fair enough on all the above -- I think the answer is that we are agreeing for the most part, I'm just being pushy 🧐 . To follow up on the above and on AQ's point, my last comment was more or less attempting to reach this question, but didn't quite get there:

How does this work together with the existing system?

I think most of my questions and comments above were basically revolving around how these concepts apply within the world of a normal VI. To try to illustrate what I'm getting at, I'll go back to my vision application above. If I understand you, it seems like the way it is now, rebar references may exist only within rebar diagrams. But if I have my 5 vision-related loops described above, my logger will definitely be using standard file refnums, acquisition will be using imaq refnums, networking will have tcp refnums, so presumably all these top level loops will be standard, non-rebar VIs. But I want to share my data between them safely and without copies. So I'm questioning what the approach is for melding the two types of code. Is the idea to make more of the refnum types into rebar types, so there is a tcp rebar and a imaq rebar and a file rebar (this is what I was trying to get at with the "baggage-free VI" comment)? Or would the reverse happen? Or am I totally misunderstanding?

Edited February 15, 2019 by smithd

[Ben Leedom]

41 minutes ago, smithd said:

How does this work together with the existing system?

This is a very good question, and while I will try to answer, at the moment I have mostly a hazy vision that I have not fully worked out the details of. My priority is and has been to elaborate a good design for the core of Rebar, then create enough of an implementation of it that you can do some useful things in it and can see a path towards other useful things. How much the design and implementation of Rebar/VI interop gets worked out will depend very much on how much demand there is for it.

A VI calling a Rebar function should look about the same as calling a subVI. A Rebar function's signature will contain information equivalent to the inplaceness information computed by the VI compiler for a VI, so the VI compiler will know when it need to copy the data that it sends to Rebar. Similarly, a Rebar function should be able to call a VI; the Rebar compiler may need to dig out inplaceness information for the VI that is normally invisible to the user. One difference in this direction would be that Rebar should be explicit that it is obtaining a specific clone of the VI and calling that, which it might wrap up into a closure-like object.

So then the question is what kind of data you can pass back and forth between Rebar and VI. The rules here are that Rebar cannot pass any raw values to VI that VI will not guarantee the invariants of. Specifically:

• Rebar can't pass its own references, or anything containing its references to VI, since VI won't do lifetime checking.
• Rebar can't pass values that have destructors, because VI isn't guaranteed to call them when appropriate.

For any types that Rebar cannot pass raw to VI, it must wrap them in refnums. This amounts to having a reference-counted shared object between Rebar and VI, so there are still some Rebar types that wouldn't qualify, but it should be enough to allow the most interesting Rebar-created values to VI and have the runtime maintain their invariants. In this way, you could define a TCP connection type, a file handle type, an IMAQ image type, or whatever you want in Rebar, and provide an API for it back to VI with refnums. This would have the nice result of allowing you to re-implement many parts of the LabVIEW runtime in Rebar.

That's about as far as I've got on interoperability. Obviously the devil is very much in the details here, and often creating an interop system between two different languages is way harder than designing each one in isolation. Like I said above, though, none of this matters much unless Rebar is interesting enough that people want to use the two side-by-side.

[Aristos Queue]

I encourage Ben not to worry too much about the interoperability with G and focus on getting the new language stabilized within itself -- a graphical notation for a safe reference system has value even if it only talks to itself... it can become an ecosystem just as G did. Now, obviously it has *more* value if it can talk to other languages, just as G has gained that ability over time. But that doesn't have to be up-front. Once the Rebar memory management is well-defined, creating the rules for interop should be relatively straightforward. I'd be far more concerned about Rebar cutting a corner to accommodate G than about making the interface seamless. The whole point of building an absolutely provable system is that it is provable -- any shortcuts and the whole attempt loses its value. Interop between Rebar and G will come in time.

[smithd]

1 hour ago, Aristos Queue said:

I encourage Ben not to worry too much about the interoperability with G and focus on getting the new language stabilized within itself...

I'm definitely not expecting anything he said to be set in stone, I was just curious about the concept. And to reinforce one of the bigger use cases that comes to mind for me

1 hour ago, Ben Leedom said:

For any types that Rebar cannot pass raw to VI, it must wrap them in refnums. This amounts to having a reference-counted shared object between Rebar and VI, so there are still some Rebar types that wouldn't qualify, but it should be enough to allow the most interesting Rebar-created values to VI and have the runtime maintain their invariants. In this way, you could define a TCP connection type, a file handle type, an IMAQ image type, or whatever you want in Rebar, and provide an API for it back to VI with refnums. This would have the nice result of allowing you to re-implement many parts of the LabVIEW runtime in Rebar.

That sounds like a pretty cool concept. I don't really have any more specific thoughts, it just sounds like a nice way forward.

 

I do have one more somewhat related...I guess question, that comes to mind. So I think for me the exciting part is all of the reference lifetime-related stuff, but your first bullet point is improved memory management for large data sets. Most large data sets are such because they are for analysis, and most of the analysis functions (both for images as well as for waveforms) are C DLLs. My question, if you want to call it that, is does that confound things at all? For example I know some types passed to the CLFN require full data copies anyway (ie the substring, per that buffer allocations thread I linked to earlier).

[Ben Leedom]

Rebar will take yet another cue from Rust in the C interop case. Rust has the notion of an unsafe context, which can be either an entire function or a block within a function, and within which you can do certain actions that would be not be permitted otherwise, like using raw pointers or calling any kind of external code. The language basically says, "At this point you assert that you know what you're doing; I will still enforce some rules, but it's up to you to ensure that this C DLL isn't going to violate data safety." (You can read more about unsafe Rust here.) Once interop with arbitrary C DLLs becomes important, I'd expect to add a similar kind of unsafe context--probably a special structure that marks off the unsafe area from the rest of the surrounding code--plus the ability to use calls targeting NXG's Shared Library Interface (SLI) document. That way, you can pass pointers into C functions without performing extra data copies, as long as you pinky-swear that you know what you're doing. 😀

[Ben Leedom]

@smithd, I updated the nipkg attached to the v0.1.0 release (at https://github.com/ni/rebar/releases/tag/v0.1.0-alpha2) so that it should install with NIPM 18.5.1. Please try it out if you get a chance and let me know if it doesn't install for you.