Feedback request - allow libraries to protect references they create

[Yair]

I want to post the following idea to the IE, but before I do, I would like some feedback.

There are times when we might want to expose a reference from inside a library (e.g. a queue or an event which will provide some data to someone), but doing so allows the non-member VI to close the reference:

In some cases this can be avoided (e.g. in the queue case you can output an object and a wrapper around the dequeue primitive, so that the caller doesn't have the queue reference and can't destroy it), but this can't be always avoided (e.g. the event reference).

For such cases I would like to propose that libraries gain a new option:

When this option is checked, any function which closes a reference (close reference, close file, release queue, etc.) will check if the reference was created in a member VI and in that case will only close the reference if it is called from a member VI. If it's called from another VI, it will not close the reference and will instead return an error. This will allow libraries to guarantee that references that they created remain valid as long as they don't close them explicitly.

A couple of comments:

1. The example shows a class, but this should probably apply to all classes. At least XControls can probably gain from this.
   
2. Named references will behave like other references, so if you open two references to the same notifier, one inside the library and one outside it, the one opened outside it can be destroyed by anyone (although I'm not sure how the Force Destroy input should behave in such a scenario. Does anyone have thoughts on this?).
   

What do people think of this?

Edited November 17, 2011 by Yair

[asbo]

In order for the function to decide that the reference was created within the class, it would to check every single member VI to see if had an input of this reference type. What if it makes sense for the class to accept an externally opened reference and keep it in its class data for a time?

I think that the simpler (albeit more tedious) solution would be to never expose an elementary reference type; you'd have to always wrap it in a cluster or class.

It's a cool idea, but I think it would be a lot of work to implement under the hood and still allow developers unusual flexibility like the above.

[Yair]

In order for the function to decide that the reference was created within the class, it would to check every single member VI to see if had an input of this reference type

No, it just needs to log which VI creates each reference (something which may well already happen today, since LV knows which hierarchy owns the reference).

I think that the simpler (albeit more tedious) solution would be to never expose an elementary reference type; you'd have to always wrap it in a cluster or class.

But what if you want the library to provide an event people can register for? You can't expose that as anything other than an event reference.

As for the external reference pushed into the library, it wasn't created by the library and therefore would not be protected by this option.

[asbo]

No, it just needs to log which VI creates each reference (something which may well already happen today, since LV knows which hierarchy owns the reference).

I guess the way I envisioned it, the IDE should break the wire at development time, not just no-op at run-time. Only the latter would be too subtle, I think.

But what if you want the library to provide an event people can register for? You can't expose that as anything other than an event reference.

This is a perfectly good use case I think didn't think of because I don't do this

As for the external reference pushed into the library, it wasn't created by the library and therefore would not be protected by this option.

This was intended to be the caveat to my "scan every input VI" method. If LV isn't already tracking which VI opened the reference (and I can't really find a compelling reason that it would have needed to before this feature) then I don't see an ideal solution for resolving where the reference was opened.

[Yair]

Breaking the wire at edit time was my original idea, but it's not feasible, as the problem is not with the wire, but with the reference, and the reference can be transported using other means (locals, globals, etc.). That leaves a run-time check.

Anyone else?

[mje]

I can't say the issue has ever caused me any problems, but from a security stand point it's always caught me a little off guard*. You can always keep the write method of a class/library private, but when you need to expose a refnum to external code, the very act of making it readable exposes it to the potential of being closed without you knowing. Many times you can abstract out the refnum such that it is protected, but...

But what if you want the library to provide an event people can register for? You can't expose that as anything other than an event reference.

Exactly this. The problem is with things like user events. You have no other option than to expose the raw refnums.

If LV isn't already tracking which VI opened the reference (and I can't really find a compelling reason that it would have needed to before this feature) then I don't see an ideal solution for resolving where the reference was opened.

Indeed, it might not be easily done, but I expect somewhere LabVIEW is aware of where refnums are created, because you can always get bitten by refnums going invalid if the hierarchy that created the refnum returns, for example.

*Granted exposing a refnum which could be closed is not that fundamentally different from passing around pointers in C++, where calling code could maliciously perform a delete operation on a pointer to a location it never created. Then again, just because other languages do something, is not a reason for LabVIEW to do the same.

Part of me feels that if I lock down the ability of external code to write which refnum is being used, I feel I ought to be able to control who can also close that refnum, but ultimately I think enforcing such a mechanism is not the job of the language. I think it's really just a matter of contract, you only delete what you create. I'm not sure controlling the scope of matching create/delete operations is even possible, let alone a good idea.

But I've been back and forth on the issue many times over the years. Right now, I'm back in the camp of it's working as intended, though I definitely won't be surprised to see a solid consensus to the contrary.

[Aristos Queue]

To me, this sounds like a problem for User Events and User Events only. All other refnums can be wrapped in classes.

Why would I oppose building something more common for references generally? First of all, you only put a single checkbox in the Library Properties, but just because I have one reference that I want to restrict does not imply I want to restrict all references. Second, you're asking for restricting destroying the reference, but with queues, it is not uncommon to want to restrict either enqueue or dequeue -- any given subset of functionality might be the subset you wish to restrict (I even imagine one case where the *only* thing the outside can do is destroy the queue, which stops the internal producer/consumer loop). Third, you might want to share functionality to some people but not to others.

For all these reasons, I think building a class that exposes the functionality that you want is the more desirable route. Asking for some scripting tools to create those wrappers faster would be reasonable.

The restrictions for DVRs of class types do not fall in the above categories because those are restrictions that apply to the *entire type* of refnum. A type-wide restriction is different than a per-instance restriciton. If you wanted to prevent anyone from ever destroying a "User Event of Subtype X", that would be more viable as a request than the per instance restriction.

As for the User Event, asking for a new node that, given a User Event, outputs a "Registration-Only User Event" gives you something that could still be connected to a Register Event node. With that small adjustment, you could build a refnum class that can register, send, etc.

but I expect somewhere LabVIEW is aware of where refnums are created, because you can always get bitten by refnums going invalid if the hierarchy that created the refnum returns, for example.

The *top-level* VI is tracked, but not the specific subVI, and this would need to be on the specific subVI. It would require a new tracking system.

but ultimately I think enforcing such a mechanism is not the job of the language

It is the language's job when the restriction is applied to the whole type (like having a private destructor in C++). It's not the language's job when we're talking specific instances of a type.

[Yair]

You bring up a good point which I haven't considered originally, which is that closing the reference is not the only way to abuse it. Other VIs can do all kinds of things with it once they have it and there is no way of creating a single API to define all the restrictions you would want to place.

You're probably right that user events are the only API really affected by this (DVRs could also be, since they also use a structure, but it should probably always be possible to create a member VI which will do the deref-reref actions and have the external VI call that).

I'll consider it some more, but if I'll post anything, it will probably be a request for the register-only event reference. Of course, for something like that you need to ask how the reference is destroyed (e.g. maybe it can only be destroyed when the "real" reference is closed).

[mje]

As for the User Event, asking for a new node that, given a User Event, outputs a "Registration-Only User Event" gives you something that could still be connected to a Register Event node.

That is a great idea. Other functionality, such as only allowing signalling to a user event can be encapsulated, so there should be no need to further extend this beyond reg-only refnums.

[GregR]

Wrapping the register for event in a safe class is not an issue. You wrap it in a class and create a method that wraps the register for events node. The only issue with preventing access to the user event refnum is that the event handler frame has access to the refnum. Seems like the feature there should really be the ability to specify at event creation that the handler should not have access to the refnum. This avoids the Registration-Only user event and means your wrapper is free to expose whatever subset of user event functionality is appropriate for your use case (register-only, generate-only, register and generate).

The DVR is a harder problem. The only way to effectively wrap that currently would be to make your safe class have a method that takes a strict VI ref and calls it inside the structure. This would be more acceptable if LabVIEW had some sort of closure/anonymous VI. Giving this the DVR syntax would require a class to define border node behavior for the inplace element structure.

[Yair]

Wrapping the register for event in a safe class is not an issue. You wrap it in a class and create a method that wraps the register for events node. The only issue with preventing access to the user event refnum is that the event handler frame has access to the refnum. Seems like the feature there should really be the ability to specify at event creation that the handler should not have access to the refnum. This avoids the Registration-Only user event and means your wrapper is free to expose whatever subset of user event functionality is appropriate for your use case (register-only, generate-only, register and generate).

The DVR is a harder problem. The only way to effectively wrap that currently would be to make your safe class have a method that takes a strict VI ref and calls it inside the structure. This would be more acceptable if LabVIEW had some sort of closure/anonymous VI. Giving this the DVR syntax would require a class to define border node behavior for the inplace element structure.

That's interesting, because I see it in exactly the opposite way - the event registration node HAS to be associated with the specific event structure (because the event registration refnum is strictly typed and holds all the events and because splitting the registration refnum to more than one structure creates undefined and buggy behavior), so you have to provide the event reference (or create a member VI for every single type of event structure which will ever be used by someone who uses the event, something which is obviously impossible). Since you have to do that, locking the event reference makes sense and solves the basic issue.

For the DVR, it should be theoretically simple to create a member VI for any type of interaction you want to allow with the DVR, such as this VI which adds:

There is one additional option - rather than restricting this to registration-only events, we could introduce a new Set Reference Permissions node, which will allow you to set specific permissions for a reference:

This would cause subsequent nodes to return errors if they do not have those permissions.

Of course, such a change is considerably more extensive and I don't see it happening, because we do have another solution (one which is also arguably better).

Edited November 20, 2011 by Yair

[drjdpowell]

so you have to provide the event reference (or create a member VI for every single type of event structure which will ever be used by someone who uses the event, something which is obviously impossible).

I think GregR means that you provide a function that produces an event registration for just the encapsulated User Event, and that registration is clustered with another registration for any other events before being passed to the event structure. Thus you only need one member VI. However, this doesn't keep the User Event encapsulated, because it can be accessed in the event structure frame itself. See below:

-- James

BTW, I learned only recently that one can use a cluster of multiple event registrations for handling by the same event structure; does anyone know if there are any performance differences with using multiple registrations instead of a single one?

[Yair]

Ah, yes, I keep forgetting about that trick. In any case, as you both said, the event is still exposed and I don't think a better case could be made for not exposing the reference than for limiting it (and it has some other issues).

Now that the discussion has been turned around to this, I remembered that there was already an idea along these lines in the IE. A quick search brought up this, which hardly anyone (including me) voted for. What does that mean? I don't know, but it's another thing to think about.

[Aristos Queue]

WHOA. You can bundle event registrations?!!?!?!? WHY HAS NO ONE EVER SHOWN ME THAT TRICK BEFORE?!?!

[drjdpowell]

That was my reaction a month or two ago.

[asbo]

WHOA. You can bundle event registrations?!!?!?!? WHY HAS NO ONE EVER SHOWN ME THAT TRICK BEFORE?!?!

Consequences will never be the same!

[Norm Kirchner]

WARNING! With regards to placing an event registration node inside of a sub-vi or saving the reference in a class.

An event registration reference can be used once and only once. Also I wouldn't put too much untested faith in the idea that each time a sub-vi runs w/an event reg node in it, that the executions of that node will be wholly unrelated.

Although not the vector of this thread I felt it critical to mention at this juncture. I'm not saying don't do it, just test it thoroughly.

And for my 2cents on the topic, since most destroy typed ref methods spit out a data type of that type, the destroy method on a non scope allowed vi on a class should cause a broken arrow.

[Aristos Queue]

> Also I wouldn't put too much untested faith in the idea that each time a sub-vi runs w/an event reg node in it, that the executions of that node will be wholly unrelated.

Though I agree with Norm's admonitions on this topic generally, if you make the subVI that generates the event refnum be inlined (LV 2011 feature on the VI Properties dialog >> Execution page), I'd be very confident that it is just like having the event registration node on the caller diagram.

[drjdpowell]

Hi Norm, I'm not sure I follow. Can you (or AQ) explain more about possible issues with putting an event registration in a subVI?

[Yair]

Hi Norm, I'm not sure I follow. Can you (or AQ) explain more about possible issues with putting an event registration in a subVI?

The output of the Register for Events node (R4EN) has to go to a single event structure. If you connect that reference to more than one event structure, you're going to get undefined (=weird) behavior, such as events being processed by multiple event structures or by none at all, without any predictability.

What Norm was saying is that you can't necessarily trust that if you put the R4EN in a subVI that you will get a separate reference each time the VI runs. Maybe you would, but without testing it thoroughly you can't be sure. This is made much worse if you take the reference and save it, because then you increase the risk of using it with more than one event structure, a danger which doesn't exist if the R4EN is wired directly to the structure.

If you want more details, you can look at this old thread - http://forums.ni.com/t5/LabVIEW/Register-for-events-causes-freezes-if-used-in-more-than-one/m-p/497567#M237840

[drjdpowell]

If you connect that reference to more than one event structure, you're going to get undefined (=weird) behavior, such as events being processed by multiple event structures or by none at all, without any predictability.

I learned that through painful experience. But I don't understand the relation to subVI's.

What Norm was saying is that you can't necessarily trust that if you put the R4EN in a subVI that you will get a separate reference each time the VI runs. Maybe you would, but without testing it thoroughly you can't be sure. This is made much worse if you take the reference and save it, because then you increase the risk of using it with more than one event structure, a danger which doesn't exist if the R4EN is wired directly to the structure.

I understand that if the subVI returns a previously created and saved reference, then it will be the same each time. But if the subVI creates a new event registration, then it should be different each time. And both these are independent of the subVI being "inlined".

Here is an example. Since I did this two days ago, and it is the first time I've put an event reg in a subVI, it is an important case for me!

Is there something I should be worried about with this code? It's my understanding that this subVI should always create a new event registration for each call.

-- James

Edited December 5, 2011 by drjdpowell

[Yair]

Is there something I should be worried about with this code? It's my understanding that this subVI should always create a new event registration for each call.

I certainly didn't run into specific problems with this and I'm guessing Norm hasn't either. The point was that not all references behave the same and it's not a given that the R4EN returns a new reference if it's called in the same subVI without a reference input. It probably does, but unless it's documented, it's something which is suspect until demonstrated otherwise. There are certainly examples of nodes which don't return a new reference (or at least one example - generate occurrence always returns the same reference).

[drjdpowell]

I found this conversation which illustrates that event registrations seem to behave strangely and not like other refnum objects as I understand them. Something is going on under the LabVIEW hood that might be affected be a subVI, and perhaps I would be better to restrict event registrations to the same block diagram as their corresponding event structure