Calling arbitrary code straight from the diagram

[dadreamer]

I want to remind once again that all this information is just to have fun playing with LabVIEW and not intended for real projects use. I believe you all understand that. 🙂

Not that a big opening here and even not an opening for some ones, but I found this interesting enough to make a thread. As you may already know, when some library is being called using CLF Node, LabVIEW enters ExtFuncWrapper first to do some guard checks to prevent itself from a silent crash and output some error message to the user instead. I've always considered that wrapper boring enough to study, thus never looked inside. But when once again I faced that I can't call some function through CLFN and have to write my own wrapper library, I asked myself why we cannot call some code by its pointer as in almost any well-known text language?.. Therefore I decided to know how ExtFuncWrapper calls the function. It turned out that ExtFuncWrapper receives the function's pointer (along with the parameters struct pointer) and calls that pointer later as it is (i.e., not doing any manipulations with it). So we can use it to call any function and even any code chunk directly from the diagram! After further research I found ExtFuncWrapper not very convenient to use, because to bypass some checks the parameters struct should be prepared accordingly before the call. But there are many ExtFunc... wrappers in labview.exe and ExtFuncCBWrapper is much easier to use. It has the following prototype:

int32_t ExtFuncCBWrapper(uintptr_t CodeChunk, int32_t UseTLS, void *CodeParams);

Here CodeChunk is our func / code pointer, UseTLS is 0 as we don't use LabVIEW's Thread Local Storage and CodeParams is our parameters struct. When called ExtFuncCBWrapper runs that CodeChunk, passing CodeParams to it, so we can freely use it later to do what we want. Nuff said, here are the samples.

This one increments a Numeric.

ExtFuncCBWrapper-Increment.vi

As you can see, I'm passing a Numeric value as CodeParams pointer into ExtFuncCBWrapper and in the assembly I have to pull that pointer out to deal with my parameters. I'm not that excellent in asm codes, so I used one of many online x86 compilers-decompilers out there. It's even simplier in 64-bit IDE as the first parameter is already written into RCX. Okay, here goes a more advanced example - it calculates a sum of two input Numerics.

ExtFuncCBWrapper-SumOfTwo.vi

Here I'm passing a cluster of three parameters as CodeParams pointer (two Numerics and the resulting Sum) and in the asm I'm grabbing the first parameter, adding it to the second one and writing the result into the third one. Pretty simple operations. Now let's do some really wild asm on the diagram! 😉 The latter example calls MessageBox function from user32.dll.

ExtFuncCBWrapper-MsgBox.vi

This is what Rolf calls a diagram voodoo. 😃 I have to provide 4 parameters to MessageBox, 2 of which are string pointers. Thus I'm getting these pointers and writing them into my params cluster (along with the panel handle and the dialog type). When ExtFuncCBWrapper is called, in the asm code I have to use the prologue and epilogue pieces to prevent the stack corruption as I'm pushing 4 parameters later onto the stack to provide them to MessageBox. After the call I'm writing the result into the function return parameter. In 64-bit IDE the prologue/epilogue is somewhat simplier.

Maybe you already noticed that I'm doing VirtualProtect before calling ExtFuncCBWrapper. This is done to pass through Windows Data Execution Prevention (DEP) protection. I'm setting execute and read/write access for the memory page with my code, otherwise LabVIEW refuses to run my code and throws an exception. Surprisingly it is thrown only in 64-bit IDE, but in 32-bit LV I can just wire the U8 array to ExtFuncCBWrapper, not going through that DSNewPtr-MoveBlock-VirtualProtect-DSDisposePtr chain. I did not start to figure out such a behaviour.

Well, to be honest, I doubt that someone will find all these samples really useful for his/her tasks, because these are very low-level operations and it's much easier to use a common CLFN or a helper DLL. They are here just to show that the things described are definitely doable from an ordinary diagram, and that doesn't require writing any libraries. With a good asm skills it's even possible to realize callback functions or call some exotic functions (like C++ class methods). Some things might be improved also, e.g. embedding a generic assembly compiler to have a possibility to write the codes instead of raw bytes on the diagram. Ideally it's even possible to implement an Inline Assembly Node. Unfortunately, I have neither the time nor much desire to do it myself.

Edited September 26, 2020 by dadreamer

[ShaunR]

This is sweet.

I think we can use this information to create LabVIEW callbacks for C/C++ functions like we do with .NET.

[Taylorh140]

No this is definitely cool. It’s cool to know that there is a way to keep the data in the same scope. Now all you need is a (c compiler/assembler) written in labview along with prebuild actions and the world would be your oyster. 

[Rolf Kalbermatter]

11 hours ago, Taylorh140 said:

No this is definitely cool. It’s cool to know that there is a way to keep the data in the same scope. Now all you need is a (c compiler/assembler) written in labview along with prebuild actions and the world would be your oyster. 

That's a bit ambitious! 😀

I would rather think something in the sense of the Python ctypes package to allow arbitrary function calls to DLLs including callbacks and such. We just need to find a method that does the opposite for this: calling a VI as C function pointer. 😀

[ShaunR]

4 hours ago, Rolf Kalbermatter said:

We just need to find a method that does the opposite for this: calling a VI as C function pointer. 😀

hmmmm. 

Edited September 28, 2020 by ShaunR

[Rolf Kalbermatter]

14 hours ago, ShaunR said:

hmmmm. 

No not really. I mean something quite different.

Given a VI create a sort of function wrapper around it that works as a C function pointer. For that we would need something like

MgErr CallVIFunc(VIDSRef viRef, int32 numInParams,  VIParams *inParams, int32 numOutParams, VIParams *outParams);

with both parameters something like an array of

typedef struct
{
  LStrHandle controlName;
  int16 *typedesc;
  void *data;
} VIParams;

That way one could do a C function wrapper in assembly code that then converts its C parameters into LabVIEW parameters and then calls the VI as function.

These are not actual functions that exist but just something I came up with. I'm sure something similar actually exists!

Edited September 29, 2020 by Rolf Kalbermatter

[dadreamer]

On 9/29/2020 at 12:54 AM, Rolf Kalbermatter said:

These are not actual functions that exist but just something I came up with. I'm sure something similar actually exists!

All I could find about this is just these two internal functions:

• CallVIFromDll
• NCGRunVirtualInstrument

For the first one I was able to find .NET prototype only:

CallVIFromDll.Invoke(Int32 epIndex, IntPtr lvClient, IntPtr entryPointDataSpace)

I'm kind of unsure, how it could be used for the mentioned task. It looks like it doesn't accept the VI parameters. And what do these arguments mean exactly?..

As to the second one, it doesn't accept the VI parameters as well and must be called in UI Thread only. The prototype is as follows:

int32_t NCGRunVirtualInstrument(uint32_t VIRef);

I did a limited testing and it appears to work. The VI is launched with the parameters on its FP and no panel is shown. We could prepare the parameters before the call with Control Value.Set method. Not very flexible solution, as I think.

 

I saw your post from 2016, where you said that you have found some functions suitable for the task. Do you remember the details?

[ShaunR]

7 hours ago, dadreamer said:

All I could find about this is just these two internal functions:

• CallVIFromDll
• NCGRunVirtualInstrument

For the first one I was able to find .NET prototype only:

CallVIFromDll.Invoke(Int32 epIndex, IntPtr lvClient, IntPtr entryPointDataSpace)

I'm kind of unsure, how it could be used for the mentioned task. It looks like it doesn't accept the VI parameters. And what do these arguments mean exactly?..

As to the second one, it doesn't accept the VI parameters as well and must be called in UI Thread only. The prototype is as follows:

int32_t NCGRunVirtualInstrument(uint32_t VIRef);

I did a limited testing and it appears to work. The VI is launched with the parameters on its FP and no panel is shown. We could prepare the parameters before the call with Control Value.Set method. Not very flexible solution, as I think.

 

I saw your post from 2016, where you said that you have found some functions suitable for the task. Do you remember the details?

What about VIRefPrepNativeCall and VIRefFinishNativeCall? They sound interesting but maybe a red herring.

[dadreamer]

16 hours ago, ShaunR said:

What about VIRefPrepNativeCall and VIRefFinishNativeCall? They sound interesting but maybe a red herring.

I'm afraid, we can't use them, because they don't actually run a VI, but "prepare" its state for a run. I guess it's used only for LabVIEW Adapters to be called later from TestStand. VIRefPrepNativeCall requires a VI to be in reserved state, otherwise it returns 1027 error. If we mark the VI as reserved with StatVIRefReserve, then it all goes OK, but the target VI is not executed. Something like this:

int32_t StatVIRefReserve(uintptr_t viDS, uint32_t *pVIRef, int32_t unknown, int32_t setUnset);

int32_t VIRefPrepNativeCall(uint32_t viRef, uintptr_t *pVIDS);

int32_t VIRefFinishNativeCall(uint32_t viRef);

void  StatVIRefRelease(uint32_t viRef);

There must be something between VIRefPrepNativeCall and VIRefFinishNativeCall like NCGRunVirtualInstrument, but with the ability to pass the parameters. Of course, we could use WriteDCOTransferData before the call to set our parameters, but the BD / assembly code becomes kinda cumbersome then.

[Rolf Kalbermatter]

I think CallInstrument() is more promising although the documenttion I found seems to indicate that it is an old function that is superseded by something called C Interface in LabVIEW. But I haven't found any information about that new interface.

/* Legacy C function access to call a VI. Newer code should consider upgrading to use the C Interface to LabVIEW. */

/*  Flags to influence window behavior when calling a VI synchronously via CallInstrument* functions.

	The following flags offer a refinement to how the former 'modal' input to
	CallInstrument* functions works. For compatibility, a value of TRUE still
	maps to a standard modal VI. Injecting the kCI_AppModalWindow flag will allow
	the VI to stack above any Dlg*-based (C-based LV) windows that may be open as well.
	Use kCI_AppModalWindow with caution! Dlg*-based dialogs run at root loop, and VIs
	that run as app modal windows might be subject to deadlocking the UI. */
const int32 kCI_DefaultWindow	= 0L;		///< in CallInstrument*, display VI's window using VI's default window styles
const int32 kCI_ModalWindow		= 1L<<0;	///< in CallInstrument*, display VI's window as modal
const int32 kCI_AppModalWindow	= 1L<<1;	///< in CallInstrument*, display VI's window as 'application modal'

/* Legacy C function access to call a VI. Newer code should consider upgrading to use the C Interface to LabVIEW. */
/*
	@param viPath fully qualified path to the VI
	@param windowFlags flags that influence how the VIs window will be shown
	@param nInputs number of input parameters to send to the VI
	@param nOutputs number of output parameters to read from the VI
	@return error code describing whether the VI call succeeded

	The actual parameters follow nOutputs and are specified by a combination of
	parameter Name (PStr), type (int16*) and data pointer.

	@example
		CallInstrument(vi, kCI_ModalWindow, 2, 1,
		               "\07Param 1", int16_Type_descriptor_in1, &p1,
		               "\07Param 2", int16_Type_descriptor_in2, &p2,
		               "\06Result", int16_Type_descriptor_res, &res);


	@note This function does not allow the caller to specify a LabVIEW context in which to load
	      and run the VI. Use a newer API (such as the C Interface to LV) to do so.

	@note Valid values for windowFlags are:
	        kCI_DefaultWindow  (0L)
	        kCI_ModalWindow    (1L<<0)
	        kCI_AppModalWindow (1L<<1)
*/
TH_SINGLE_UI EXTERNC MgErr _FUNCC CallInstrument(Path viPath, int32 windowFlags, int32 nInputs, int32 nOutputs, ...);

 

[dadreamer]

Nice catch, Rolf! It works and I am able to pass the input/output parameters now.

RunVI.vi

SubVI.vi

But it appears that CallInstrument wants to be ran in UI Thread only, else LabVIEW goes big crash. It calls the VI synchronously, waiting until it finishes. That makes me think that this function is not the best idea to use as a callback, because when the user will be interacting with some GUI elements or the program will be running some property/invoke nodes, the callback VI will be waiting UI Thread to become idle, thus we could experience the delay between the events from our callback (or even loss of earlier ones?). It would be much better to run the VI in any thread somehow, but CallInstrument doesn't support that. That's why I decided not to adapt the asm samples for that function for now. Maybe I would be lucky enough to find some other options or overcome the threading issues somehow. Or end on PostLVUserEvent until some better ideas come to mind. 🙂

On 10/3/2020 at 8:52 PM, Rolf Kalbermatter said:

But I haven't found any information about that new interface.

It's likely has to do with these two functions: GetCIntVIServerFuncs and GetCInterfaceFunctionTable. They return some tables, filled with the function pointers. There are InitLVClient / InitLVClient2, UninitLVClient / UninitLVClientNoDelay, WaitLVClientReady, WaitLVShuttingDown and a whole bunch of unnamed functions (safety precautions?). It'll take a serious effort to study how those work.

Edited October 5, 2020 by dadreamer
added some thoughts

[Rolf Kalbermatter]

55 minutes ago, dadreamer said:

Nice catch, Rolf! It works and I am able to pass the input/output parameters now.

RunVI.vi 10.95 kB · 0 downloads

SubVI.vi 12.43 kB · 0 downloads

But it appears that CallInstrument wants to be ran in UI Thread only, else LabVIEW goes big crash. It calls the VI synchronously, waiting until it finishes. That makes me think that this function is not the best idea to use as a callback, because when the user will be interacting with some GUI elements or the program will be running some property/invoke nodes, the callback VI will be waiting UI Thread to become idle, thus we could experience the delay between the events from our callback (or even loss of earlier ones?). It would be much better to run the VI in any thread somehow, but CallInstrument doesn't support that. That's why I decided not to adapt the asm samples for that function for now. Maybe I would be lucky enough to find some other options or overcome the threading issues somehow. Or end on PostLVUserEvent until some better ideas come to mind. 🙂

I'm aware of the limitation with this function. It not only wants to run in the UI thread but also doesn't allow to specify the context it should be running under, so will likely always use the main application context (which could be fine really, as each project has its own context). And that with the missing context is logical since this function existed before LabVIEW 8.0 which introduced (at least publically) application contexts.

I think the mentioned C interface has to do with the function GetCInterfaceFunctionTable(), exported since around LabVIEW 8.0, but that is a big black hole. From what I could see, it returns a pointer to a function pointer table containing all kinds of functions. Some of them used to be exported as external functions in LabVIEW too. But without a header file declaring this structure and the actual functions it exports, it is totally hopeless to think one could use it. As most of the functions aren't really exported from LabVIEW in other ways they also didnt retain the function name, like all those other functions in the official export table. But even with a name it would be very tedious to find out what parameters a function takes and how to call it especially if it needs to be called in conjunction with other functions in there.

Edited October 5, 2020 by Rolf Kalbermatter

[Rolf Kalbermatter]

2 hours ago, dadreamer said:

There are InitLVClient / InitLVClient2, UninitLVClient / UninitLVClientNoDelay, WaitLVClientReady, WaitLVShuttingDown and a whole bunch of unnamed functions (safety precautions?). It'll take a serious effort to study how those work.

Not really safety precautions. Most C(++) compilers will strip by default all symbols from linked non-debug code, unless these symbols are needed for certain puposes like function export tables. While these functions also return a pointer to some sort of export table, they are not official export tables, just as virtual tables in C++ aren't really export tables. The name is unneeded as far as the compiler is concerned, so they get all stripped. This has certain anti reverse engineering reasons as someone distributing a release version isn't usually interested in letting its users reverse engineer their software  (just check your license agreement you entered into when installing LabVIEW 😀) but the main reason is really that these symbols simply blow up the executable image size for no useful reason and it's an easy thing to do by the linker. The functions with retained symbol names in there are usually functions that are also exported in the official export table.

GetCIntVIServerFuncs() existed before LabVIEW 8.0 and was/is mostly related to functions needed by various VI Server interfaces. The first version of VI server was a very small set of exported functions that got called by a CIN. This was then changed into the fully diagram accessible VI server interface as we know it now around LabVIEW 5. Sometimes the LabVIEW compiler needs to create callbacks into the LabVIEW kernel. Initially this was done through direct calls of exported functions but that was changed for several reasons to call a special internal export interface. Hiding things was likely more a byproduct than the main reason for this, making this interface more uniform among the different platforms was likely more important.

The C Interface supposedly took this idea further and here hiding LabVIEW internas might have been part of the decision. But because this privately exported function table is highly inflexible and can only be amended to in subsequent LabVIEW versions but never modified without creating serious trouble with version compatiblity, I think it's not something they would want to use for everything. The advantage is that you do not really need to use dynamic link functionality for this except for the one function to get at this interface, so there is one simple function that will use platform specific dynamic loading and everything else is simply a function pointer call into a table at a specific offset.

A COM like interface would be much more flexible in terms of version compatibility and NI uses that in some parts even across platforms despite that real COM isn't supported on non-Windows platforms, but it is also a lot more complicated to create and use even when you program everything in C++.

Edited October 5, 2020 by Rolf Kalbermatter