Malleable Buffer (seeing what VIMs can do)

[drjdpowell]

As an experiment in seeing what VIMs can do, here is an all-VIM implementation of a "circular buffer", based on code from a 2D DBL array circular buffer I had previously used.  In LabVIEW 2018.

Features:

• Works on any scalar type, giving a 1D array buffer, or any 1D array, giving a 2D array buffer (could be extended to a 3D buffer of 2D arrays)
• Has VIMs that accept either a by-Value Buffer, or a DVR of a Buffer

Package contains simple examples.  Comments welcome.  

 

jdp_science_lib_malleable_buffer-0.1.0.5.vip

[ShaunR]

The examples were missing when I installed the VIP package so it took me a bit of time to figure out why wires were broken (see bottom row of VIs). The rows above are what happens when the correct data type is applied to the "Add To Buffer"

Is it possiblee to to make the "Add To Buffer" select the correct instance depending on the "Create Buffer" data type?

[drjdpowell]

Last thing I did was move the examples into the standard example location.  You should be able to find them either from the show examples button in VIPM, or in the Example Finder.  I had hoped the menu would update to the new location but I guess not.

[drjdpowell]

So far I've noticed that it takes a lot of time to figure out why a VIM is broken.  Any input being wrong makes everything wrong, with only an unhelpful error message.   It's the biggest problem with VIMs that have input's that must have matching datatypes.  I'd like to be able to specify somehow that the "Buffer" input's should always be accepted, even if the added data is unspecified, but I'm not sure how to do that (though I've just had an idea I'll try later).

[ShaunR]

5 hours ago, drjdpowell said:

Last thing I did was move the examples into the standard example location.  You should be able to find them either from the show examples button in VIPM, or in the Example Finder.  I had hoped the menu would update to the new location but I guess not.

I can't find them in the examples finder (tried a few different words). The button on VIPM shows the directory with them, though.

FWIW. I hate the example finder. You neeed to know the right words for what you're searching for and it's a lot of clicks and typing to get there (ignoring the excrutiating pain of actually creating the Example Finder special files). It's much preferable to have them in the same pallet as the VIs; as you had originally. They need to take that requirement off of the tools network checklist.

[drjdpowell]

One can also just "Browse" the directory structure (which is what I've learned to do as then you know you can see everything):

But the easiest with VIPM packages is to use the "Show Examples" button in VIPM:

Edited November 13, 2019 by drjdpowell

[drjdpowell]

22 hours ago, ShaunR said:

Is it possiblee to to make the "Add To Buffer" select the correct instance depending on the "Create Buffer" data type?

I think I can fix this by making the last case of the Type-Specialisation Structure be a case that passes the Buffer input to Buffer Output.  I make sure this last case is broken, so it is only used for type propagation when all cases are broken.

[ShaunR]

If you were to use a DVR (or the user supplied DVR) and put the reference onto a single element queue inside the ViM. Wouldn't you get all the type checking and polymorphism for free across all the VIs without all the type cases?

[drjdpowell]

Sorry, I don't understand what you're meaning.  

[Aristos Queue]

On 11/13/2019 at 6:16 AM, drjdpowell said:

So far I've noticed that it takes a lot of time to figure out why a VIM is broken.  Any input being wrong makes everything wrong, with only an unhelpful error message.   It's the biggest problem with VIMs that have input's that must have matching datatypes.  I'd like to be able to specify somehow that the "Buffer" input's should always be accepted, even if the added data is unspecified, but I'm not sure how to do that (though I've just had an idea I'll try later).

If you have suggestions for how to report those errors, I'm all ears. I've been in hours of design meetings trying to come up with some algorithm for reversing type prop to figure out which inputs contributed to a given break *and* figuring out how to get the errors invalidated when the VIM gets edited. The current "break them all" is the best we've got at this time. There are some simple cases where the VIM's FP term connects directly to a particular node and the node refuses that input, but the further downstream, the harder it gets. And the most syntactically correct solution is "any wire that connects to the node is at fault", but that is essentially equivalent to the current solution as soon as you put one Type Specialization structure or error Case structure down (those tend to encompass the whole diagram, so all the FP terms wire into them).

Opening the instance VI and looking at the diagram to see what broke is the best I've got. I would like to make the diagram visible without having to convert the instance to a real VI, but if there's a better option, please let me (and the rest of R&D) know.

[Aristos Queue]

11 hours ago, drjdpowell said:

I think I can fix this by making the last case of the Type-Specialisation Structure be a case that passes the Buffer input to Buffer Output.  I make sure this last case is broken, so it is only used for type propagation when all cases are broken

That should work. That is how the Assert Type VIs work.

[ShaunR]

7 hours ago, drjdpowell said:

Sorry, I don't understand what you're meaning.  

Never mind. It seems that you cannot have DVRs inside VIMs any more.

[TomOrr0W]

15 hours ago, drjdpowell said:

I think I can fix this by making the last case of the Type-Specialisation Structure be a case that passes the Buffer input to Buffer Output.  I make sure this last case is broken, so it is only used for type propagation when all cases are broken.

If you do this, you still end up with a broken buffer wire (you just also get a broken data wire if you choose to incorrectly wire in a scalar).  Note that the type of the output wire now matches the input wire now instead of being a DVR.  I am using LV2019 64-bit in case 2018 32-bit has a different implementation.

If you then convert the malleable vi to an instance vi or copy the type specialization node into the top-level vi, both wires unbreak.  Also note that nothing changes if you also do the same change (adding a third case with a passthrough on top and broken data wire on bottom) on the Add to Buffer (By Value).vim called in the Type Specialization [0] and [1] cases.

[Aristos Queue]

22 hours ago, ShaunR said:

Never mind. It seems that you cannot have DVRs inside VIMs any more.

DVRs in VIMs works just fine (just double checked). It has worked since the feature released, still works in 2019. 

I have no idea what you’re talking about. 

[ShaunR]

 

2 hours ago, Aristos Queue said:

DVRs in VIMs works just fine (just double checked). It has worked since the feature released, still works in 2019. 

I have no idea what you’re talking about. 

VIMs have to be inlined now therefore:

[drjdpowell]

DVRs can be used, but not created or destroyed, inside an inline VI.  Not clear to me why that is.  Thus this library can handle buffers inside DVRs, but the User has to explicitly create a by-value buffer and put it in a DVR themselves.

Edited November 16, 2019 by drjdpowell

[drjdpowell]

Question for AQ: I'm trying to use "Unbundle by name" to access the elements of the cluster (the buffer array and the index integers).   This works, but if the User accidently connects a cluster without the right-named elements, the unbundles can get scrambled, and not properly reset to the right names when a proper Buffer cluster is attached.   The only solution then is to delete that VIM instance and re-drop it from the pallets.  I can't find a solution and may have to switch to an unnamed unbundle, using the positions in the cluster.  That is less than ideal, and won't work in NXG (you know my negative opinions on cluster changes made in NXG).

Edited November 16, 2019 by drjdpowell

[Aristos Queue]

On 11/16/2019 at 3:50 AM, drjdpowell said:

Question for AQ: I'm trying to use "Unbundle by name" to access the elements of the cluster (the buffer array and the index integers).   This works, but if the User accidently connects a cluster without the right-named elements, the unbundles can get scrambled, and not properly reset to the right names when a proper Buffer cluster is attached.   

Huh. I never considered that case. Everything is behaving as designed... but that’s clearly not desirable.

So far, no node has custom type prop behavior when inside a VIM other than the adaptive class method calls. You’re suggesting (I think rightly) that "(un)bundle by name" nodes need to adapt different inside VIMs than in VIs.

I’d have to teach the bundle by name nodes to have deeper history than they currently have so they always compute from original configuration instead of latest configuration. Or... would it be better to just freeze the named bundle nodes inside VIMs so they always look for the exact name? There’s some risk with either solution to break existing VIMs, but the current model seems mostly useless. 

What option sounds best? One of the two options is waaaaay easier to implement, and I’m trying not to let that sway me. 🙂

Edited November 18, 2019 by Aristos Queue

[Aristos Queue]

17 hours ago, drjdpowell said:

DVRs can be used, but not created or destroyed, inside an inline VI.  Not clear to me why that is.  

I have no idea why that limitation exists. I’ll ask on Monday. 

[drjdpowell]

5 hours ago, Aristos Queue said:

What option sounds best?

I'm favouring just freeze the name, as that is simplest for the User to understand, given that it is difficult to diagnose, let alone fix, any problem if the name adaption goes wrong.

[Aristos Queue]

14 hours ago, drjdpowell said:

I'm favouring just freeze the name, as that is simplest for the User to understand, given that it is difficult to diagnose, let alone fix, any problem if the name adaption goes wrong.

I’m pretty sure that’s the easiest to implement, so I’m liking your opinion. (Trying to be non-biased, I do think that’s the better option from usability. If I pick a name in a template, it’s because I expect the provided thing to use that name. It is consistent with the class method behavior, but for cluster fields.)

[drjdpowell]

On 11/15/2019 at 2:29 AM, TomOrr0W said:

On 11/14/2019 at 10:08 AM, drjdpowell said:

I think I can fix this by making the last case of the Type-Specialisation Structure be a case that passes the Buffer input to Buffer Output.  I make sure this last case is broken, so it is only used for type propagation when all cases are broken.

If you do this, you still end up with a broken buffer wire (you just also get a broken data wire if you choose to incorrectly wire in a scalar).  Note that the type of the output wire now matches the input wire now instead of being a DVR.

I've now getting the buffer wire to be unbroken when the Data input is unwired, by making the default Data an "undefined type" (a class call "Undefined Type").  I added a case that is unbroken when Data is this "undefined" type, which it is when the User has not wired that input.  The error message the User gets in this case is just the one for a required input not being wired, which is much less confusing than a broken "Buffer" wire and an additional error  message about type mismatch.

 

AQ, what do you think about an actual "undefined type type in LabVIEW and/or NXG?  One thing that could be useful for NXG is a undefined type in a cluster constant, which defines teh name of the item without the type, and allows the type to adapt to that supplied by the first Bundle-by-Name.  This would avoid some of my concerns about clusters in NXG.

[drjdpowell]

BTW, does anyone know of any other "malleable cluster" libraries or work using VIMs?

[Aristos Queue]

1 hour ago, drjdpowell said:

I've now getting the buffer wire to be unbroken when the Data input is unwired, by making the default Data an "undefined type" (a class call "Undefined Type").  I added a case that is unbroken when Data is this "undefined" type, which it is when the User has not wired that input.  The error message the User gets in this case is just the one for a required input not being wired, which is much less confusing than a broken "Buffer" wire and an additional error  message about type mismatch.

 

AQ, what do you think about an actual "undefined type type in LabVIEW and/or NXG?  One thing that could be useful for NXG is a undefined type in a cluster constant, which defines teh name of the item without the type, and allows the type to adapt to that supplied by the first Bundle-by-Name.  This would avoid some of my concerns about clusters in NXG.

We have an undefined type. Create an output tunnel from Your Favorite Structure node and don't wire the inside. You'll get a void wire, which is distinct from a broken wire. Void is also the type of a wire output from a subVI call node when you delete the terminal from the subVI's connector pane. Void wires don't have a control/indicator, and most poly nodes won't adapt to void (deliberately), so the VIM instances cannot adapt to void type.

Void is probably not what you're looking for.

Something like empty cluster is what you're looking for, but VIMs deliberately cannot have that type in their conpane, and it contradicts a key principle of VIMs: the conpane shows the user the intended usage, and by having one required working base case, we avoid many of the ambiguous interpretation issues of C++ templates. The code has to be semantically valid for at least one set of types. Having a terminal whose type is "undefined" doesn't tell the user the kind of data that they should be wiring to that terminal.

I see your problem. I don't see a solution at the moment. An undefined type seems fundamentally flawed to me.

 

[drjdpowell]

Maybe we need a "placeholder" type, or "Your Type Here".  Actually, NI seems to be using an "anything" cluster of zero elements in primitives like "Flatten To String" and "Flatten to JSON".  This has never struck me as being flawed.