I think CallInstrument() is more promising although the documenttion I found seems to indicate that it is an old function that is superseded by something called C Interface in LabVIEW. But I haven't found any information about that new interface.
/* Legacy C function access to call a VI. Newer code should consider upgrading to use the C Interface to LabVIEW. */
/* Flags to influence window behavior when calling a VI synchronously via CallInstrument* functions.
The following flags offer a refinement to how the former 'modal' input to
CallInstrument* functions works. For compatibility, a value of TRUE still
maps to a standard modal VI. Injecting the kCI_AppModalWindow flag will allow
the VI to stack above any Dlg*-based (C-based LV) windows that may be open as well.
Use kCI_AppModalWindow with caution! Dlg*-based dialogs run at root loop, and VIs
that run as app modal windows might be subject to deadlocking the UI. */
const int32 kCI_DefaultWindow = 0L; ///< in CallInstrument*, display VI's window using VI's default window styles
const int32 kCI_ModalWindow = 1L<<0; ///< in CallInstrument*, display VI's window as modal
const int32 kCI_AppModalWindow = 1L<<1; ///< in CallInstrument*, display VI's window as 'application modal'
/* Legacy C function access to call a VI. Newer code should consider upgrading to use the C Interface to LabVIEW. */
/*
@param viPath fully qualified path to the VI
@param windowFlags flags that influence how the VIs window will be shown
@param nInputs number of input parameters to send to the VI
@param nOutputs number of output parameters to read from the VI
@return error code describing whether the VI call succeeded
The actual parameters follow nOutputs and are specified by a combination of
parameter Name (PStr), type (int16*) and data pointer.
@example
CallInstrument(vi, kCI_ModalWindow, 2, 1,
"\07Param 1", int16_Type_descriptor_in1, &p1,
"\07Param 2", int16_Type_descriptor_in2, &p2,
"\06Result", int16_Type_descriptor_res, &res);
@note This function does not allow the caller to specify a LabVIEW context in which to load
and run the VI. Use a newer API (such as the C Interface to LV) to do so.
@note Valid values for windowFlags are:
kCI_DefaultWindow (0L)
kCI_ModalWindow (1L<<0)
kCI_AppModalWindow (1L<<1)
*/
TH_SINGLE_UI EXTERNC MgErr _FUNCC CallInstrument(Path viPath, int32 windowFlags, int32 nInputs, int32 nOutputs, ...);