Can Queues be accessed through CIN?

[Taylorh140]

I was looking through the Labview.lib (its just an archive) functions and found:
    Line 1162: ./objects/labview_lib/win32U/i386/msvc71/release/QueuePreview.i386.obj    _QueuePreview
    Line 1211: ./objects/labview_lib/win32U/i386/msvc71/release/QueueTimeoutRead.i386.obj    _QueueTimeoutRead
    Line 1231: ./objects/labview_lib/win32U/i386/msvc71/release/QueueLossyEnqueue.i386.obj    _QueueLossyEnqueue
    Line 1238: ./objects/labview_lib/win32U/i386/msvc71/release/QueueStatus.i386.obj    _QueueStatus
    Line 1286: ./objects/labview_lib/win32U/i386/msvc71/release/QueueDequeue.i386.obj    _QueueDequeue
    Line 1348: ./objects/labview_lib/win32U/i386/msvc71/release/QueueObtain.i386.obj    _QueueObtain
    Line 1691: ./objects/labview_lib/win32U/i386/msvc71/release/QueueEnqueue.i386.obj    _QueueEnqueue

I assume that means i can interact with Labview Queues in application space, which would be a nice way to transfer information and call back. 

But I cant find any documentation on how to use these eg. documentation or prototypes. If someone can point me in the right direction that would be great!

 

[dadreamer]

Queues, Notifiers, DVRs and similar stuff, even when seems to be exposed from labview.exe in some form, is totally undocumented. Of course, you could try to RE those functions and if you're lucky enough, you could use few, maybe. But it will take a significant effort of you and won't become worth it at all. To synchronize your library with LabVIEW, you'd better try OS-native API (like Events, Mutexes, Semaphores or WaitableTimers on Windows) or some documented things like PostLVUserEvent or Occur of Occurrence API. To be honest, there are more Occurrence functions revealed, but they're undocumented as well, so use them at your own risk.

What about CINs, I do recall that former Queues/Notifiers implementations were made on CINs entirely. I never had a chance to study their code, and not that I really wanted to. I suppose, they're not functional in modern LV versions anymore as they got replaced with better internal analogues.

Edited June 7, 2021 by dadreamer

[Taylorh140]

Odd that Occurrences are documented but Queues are not?
I had a feeling that it might be undocumented. LV User Events have nearly the same properties, so that is nice (however these seem more tedious). I might try playing with these functions if I can figure out the prototypes but if not there is just too much to guess.

I'm not so familiar how the OS-native api's play with Labview. Other than using TCP/IP.

Thanks @dadreamer!

[Rolf Kalbermatter]

4 hours ago, Taylorh140 said:

Odd that Occurrences are documented but Queues are not?
I had a feeling that it might be undocumented. LV User Events have nearly the same properties, so that is nice (however these seem more tedious). I might try playing with these functions if I can figure out the prototypes but if not there is just too much to guess.

I'm not so familiar how the OS-native api's play with Labview. Other than using TCP/IP.

Thanks @dadreamer!

No, it's not odd! Occurrences exist in LabVIEW since at least version 2.5. And back then NI sort of made a slip by distributing the full internal extcode.h file that exposed pretty much every single function that LabVIEW exported and could be accessed from a CIN (the only way to call external code back then). They fixed that in subsequent releases of 2.5.x (which was a pre release version of LabVIEW for Windows 3.0).

Much of what was declared in that header was fairly useless anyways, either because it only was usable with other parts of LabVIEW that were not accessible from external code, or because the functionality was to complex that it could be inferred from just the header alone. NI never officially documented the Occurrence functions but someone with access to those OOOOOOLD headers can simply take them from there, which this poster probably did.

There is one caveat though: While those functions probably remained the same, the documentation posted is most likely from those 2.5 headers and might be not entirely accurate anymore with the current prototype of the functions as LabVIEW exports them nowadays.

The Queues, Semaphores and Notifiers that came out with LabVIEW 4 or 5, were indeed CIN based. The CIN implemented the entire functionality using internally Occurrences for the signal handling to allow low cost CPU waits. Around LabVIEW 6 or 7 the full CIN code for the Queues and Notifiers was revamped and fully moved into the LabVIEW kernel itself and integrated as built in nodes. Semaphores and Rendezvous were reimplemented in LabVIEW code using internally Queues.

Since there are no headers floating around on the internet from LabVIEW 7 or later dates, the declarations for the Queues and Notifiers are nowhere to be found although with enough time at hand and ignorance of the LabVIEW license terms, one could in fact reverse engineer them. The problem with that is that you can never quite be sure that you got it all right even if it doesn't crash at the moment. That makes this a pretty useless exercise for real use of these APIs and for just some hobby usage, the effort is both too high and requires way to specialistic knowledge. Only crazy nerds disassemble code nowadays and only super crazy nerds do that with an executable of the size of LabVIEW. 😀

Edited June 7, 2021 by Rolf Kalbermatter

[Rolf Kalbermatter]

5 hours ago, Taylorh140 said:

I was looking through the Labview.lib (its just an archive) functions and found:
    Line 1162: ./objects/labview_lib/win32U/i386/msvc71/release/QueuePreview.i386.obj    _QueuePreview
    Line 1211: ./objects/labview_lib/win32U/i386/msvc71/release/QueueTimeoutRead.i386.obj    _QueueTimeoutRead
    Line 1231: ./objects/labview_lib/win32U/i386/msvc71/release/QueueLossyEnqueue.i386.obj    _QueueLossyEnqueue
    Line 1238: ./objects/labview_lib/win32U/i386/msvc71/release/QueueStatus.i386.obj    _QueueStatus
    Line 1286: ./objects/labview_lib/win32U/i386/msvc71/release/QueueDequeue.i386.obj    _QueueDequeue
    Line 1348: ./objects/labview_lib/win32U/i386/msvc71/release/QueueObtain.i386.obj    _QueueObtain
    Line 1691: ./objects/labview_lib/win32U/i386/msvc71/release/QueueEnqueue.i386.obj    _QueueEnqueue

I assume that means i can interact with Labview Queues in application space, which would be a nice way to transfer information and call back. 

But I cant find any documentation on how to use these eg. documentation or prototypes. If someone can point me in the right direction that would be great!

It means you COULD theoretically interact with Queues from your own C code. In reality the function name alone is pretty useless. And there is NO publically available documentation for these functions. Theoretically if you are an important enough account for NI (requires definitely 7 digits or more yearly sales in US$ for NI) are willing to sign over your mother, wife and children in an NDA document in case you breach it, you may be able to get the headers for these APIs.

In practice the only people with access to that information do work in the LabVIEW development team and would likely get into some serious problems if they gave that to someone else.

If you really need it, there is a workaround though that comes with much less financial and legal trouble but can get a bit of a maintenance problem if you intend to use it in multiple LabVIEW versions and platforms:

Create a LabVIEW library that wraps the Queue functions you are interested in into some VIs, create a DLL from those VIs and export them as function, then call those functions from this DLL from within your C code.

Edited June 7, 2021 by Rolf Kalbermatter

[dadreamer]

7 hours ago, Rolf Kalbermatter said:

NI never officially documented the Occurrence functions but someone with access to those OOOOOOLD headers can simply take them from there, which this poster probably did.

If you meant me, then no, I even didn't use your conversations with Jim Kring on OpenG subject. Seriously, what's the joy of just rewriting the prototypes?.. I have studied those on my own, even though I have LV 2.5 distro for a while and do know, that some Occurrence functions are exposed there (in MANAGER.H, to be more precise). Moreover, those headers don't contain the entire interface. This is all, that is presented:

/*
	Occurrence routines
*/
typedef Private *OccurHdlr;
#define kNonOccurrence 0L
#define kMaxInterval 0x7FFFFFFFL
extern uInt32 gNextTimedOccurInterval;

typedef void (*OHdlrProcPtr)(int32);
Occurrence	AllocOccur(void);
int32		DeallocOccur(Occurrence o);
OccurHdlr	AllocOccurHdlr(Occurrence o, OHdlrProcPtr p, int32 param);
int32		DeallocOccurHdlr(OccurHdlr oh);
int32		Occur(Occurrence o);
void		OccurAtTime(Occurrence o, uInt32 t);
int32		OnOccurrence(OccurHdlr oh, boolean noPrevious);
int32		CancelOnOccur(OccurHdlr oh);
boolean		ChkOccurrences(void);
boolean		ChkTimerOccurrences(void);

The headers lack OnOccurrenceWithTimeout and FireOccurHdlr and some others (likely, they seem to be non-existent in those early versions). Having said that, I admit that Occurrence API is not that complicated and is easily reversible for more or less experienced LV and asm programmers.

[Rolf Kalbermatter]

I understand and admire your reverse engineering effort 😀.

My answer was partly directed to the OPs claim that the Occurrence API was documented. It's not (at least officially although the accidental leak from early LabVIEW days could be counted as semi official documentation).

You're right that those functions you mention as lacking from those headers didn't even exist back in those days. They were added in the meantime in later versions. The additional 50MB of code in LabVIEW.exe aren't just useless garbage. 😀 (And only a fraction of what LabVIEW gained in weight over those 30 years since all the manager core is now located in external DLLs).

That also points out another danger of using such APIs. They are not fixed unless officially documented. While NI generally didn't just go and change existing functions for the fun of it, they have only gone to extreme lengths to avoid doing that for functions that are officially documented. Any other function is considered fair game to be changed in a later LabVIEW version if the change is technically necessary and not doing so would require a big extra effort.

This is also the main reason they haven't done any documentation of new functions (with very few exceptions such as PostLVUserEvent() ) since the initial days. Once officially documented, that API has to be considered cast in stone and any change to its prototype or even semantical behaviour is something that is basically considered impossible unless absolutely unavoidable for some reason.

Edited June 8, 2021 by Rolf Kalbermatter

[Taylorh140]

Quote

Create a LabVIEW library that wraps the Queue functions you are interested in into some VIs, create a DLL from those VIs and export them as function, then call those functions from this DLL from within your C code.

I thought about this. I was initially turn off because code called from a dll supposedly runs in a different application space. I was concerned I would not be able to access queues by name through the divide. But I guess that plays well to me now knowing enough about the different run (domains) available to LabVIEW. I guess i wouldn't know where to look up the details for these.

The Actor framework seems to use these.

DLL calls can use these depending on the settings.

-Is it just a head-free version of LabVIEW but sharing application memory? is this where the AZ and DS memory differences come in?  more to learn here clearly.

I got the PostLVUserEvent to work nicely.. I am working in Rust to see how the two play together, which is a new experience for me. It also gives some nice promises about how the safe code will work... but so far everything to do with this interface is currently unsafe :). This also means that most of the goodness of the language doesn't apply, but i can wrap up all the unsafe things in nice calls and get a good interface. 

 

Here is a sample of how the interface can look in rust. 

#[no_mangle]
pub extern "C" fn AddONE(ptr:LvUHandle){
	let h=LvArray1DW::<f64>::from(ptr); //Wrap pointer and get type as a 1D array of doubles.
	let g=h.to_array_mut(); // Access mutable slice
	
	for i in 0..g.len(){
		g[i]=g[i]+1.0; // Add one to each element
	}
}

Note: I am using LabVIEW Handles for array in windows (but the pdf describing the interface says that windows array handles are not supported .. why?) 
Note Response: I guess the documentation says the Windows API does not support LabVIEW handles, like user32.dll not the LV interface to the custom DLL.

 

Edited June 8, 2021 by Taylorh140
Answered one of my own questions.

[Rolf Kalbermatter]

40 minutes ago, Taylorh140 said:

I thought about this. I was initially turn off because code called from a dll supposedly runs in a different application space. I was concerned I would not be able to access queues by name through the divide. But I guess that plays well to me now knowing enough about the different run (domains) available to LabVIEW. I guess i wouldn't know where to look up the details for these.

The Actor framework seems to use these.

DLL calls can use these depending on the settings.

-Is it just a head-free version of LabVIEW but sharing application memory? is this where the AZ and DS memory differences come in?  more to learn here clearly.

I got the PostLVUserEvent to work nicely.. I am working in Rust to see how the two play together, which is a new experience for me. It also gives some nice promises about how the safe code will work... but so far everything to do with this interface is currently unsafe :). This also means that most of the goodness of the language doesn't apply, but i can wrap up all the unsafe things in nice calls and get a good interface. 

 

Here is a sample of how the interface can look in rust. 

#[no_mangle]
pub extern "C" fn AddONE(ptr:LvUHandle){
	let h=LvArray1DW::<f64>::from(ptr); //Wrap pointer and get type as a 1D array of doubles.
	let g=h.to_array_mut(); // Access mutable slice
	
	for i in 0..g.len(){
		g[i]=g[i]+1.0; // Add one to each element
	}
}

Note: I am using LabVIEW Handles for array in windows (but the pdf describing the interface says that windows array handles are not supported .. why?

No! Your guesswork got you in many ways wrong. AZ and DS memory spaces is an old Mac OS Classic programming distinction. AZ memory could be automatically relocated by the OS/kernel unless it was explicitly locked by the user space application. DS memory stays always at a fixed memory location. It also meant that when you try to access AZ memory and didn't lock it first with AZLock() you could badly crash if the OS decided that it needed that space and moved the memory (possibly into a cache file) during the time you tried to access that memory block.

With modern virtualized memory manager hardware support in modern OSes such as Windows NT and MacOS X, this distinction got superfluous. In user space memory nowadays always behaves as appearing at fixed virtual memory address location. Where it is actually stored in real memory (or a disk cache file) is all handled transparently by the OS virtual memory manager supported by a powerful hardware memory management unit directly integrated in the CPU. As soon as NI sacked support for MacOS Classic in LabVIEW, they also removed consequently the AZ memory manager completely. In order to support old legacy C code and CINs using the AZ memory manager functions explicitly, the exports still exist but simply are linked to the corresponding DS counterparts where existing, and for those that have no DS counterpart like the Lock and Unlock functions they simply call an empty function that does nothing.

The actor framework as far as I know does not use explicit C code for anything but simply builds on the existing LabVIEW OOP technology so I'm not quite sure what you refer to here. The old LVOOP package used a technique similar to what was used in the old CINs for the queues, semaphores, notifiers and rendevous functions, to store the "class" data for a specific object instance in a sort registry to implement some sort of class support before the native LabVIEW OOP was introduced, but that wasn't really using any build in queues or similar functionality internally (as that build in functionality didn't fully exist at that time either).

As to using a LabVIEW created DLL running in a seperate application instance, this is actually more complicated than you might guess. That is one aspect that I find makes the use of LabVIEW created DLLs for use in LabVIEW itself extra maintenance intense. If the DLL was created in the same LabVIEW version (or compiled to be executable in a newer runtime version than build since LabVIEW 2017), LabVIEW will load the DLL into the current application instance. If the versions don't match and the option to execute the DLL in a newer runtime version wasn't enabled when building that DLL, LabVIEW will startup the according LabVIEW runtime system, load the DLL into it and setup interprocess marshalling to execute every call to this DLL through it. Needless to say that this has some implications. Marshalling calls across process boundaries costs time and resources so it is less performant than when it is all happening in process. And as you noted, the application instance separation will prevent access of named resources such as queues between the two systems. And this possible marshalling is so transparent to a normal user that he may never ever guess why his queue he created through the DLL call doesn't share data with another queue he "obtained" in the native LabVIEW code. Logically they are totally different entities but the fact if they are or not may depend on subtle differences of what LabVIEW version was used to create your DLL and what LabVIEW version you call it from.

As to handles I'm not quite sure what PDF you refer to. The whole handle concept originates from the Mac OS Classic days. And MacOS had handles that could be directly created by external code through calls to the MacOS Classic Toolbox. LabVIEW had special primitives that could refer to such handles so that you did not need to copy them always between the external handle and a LabVIEW handle. That support was however very limited. You basically had a Peek and Poke function only with a handle input and an offset in addition to the value. This functionality never made sense on non Mac OS Classic platforms although I believe the primitives still existed there but where hidden. No need to confuse the user with an obscure feature that was totally useless on the platform in question. Even on Mac OS it was hardly ever used except maybe for a few hacky NI interfaces themselves.

Almost all of this information has mostly just archeological value nowadays. I'm explaining it here to save you from going down some seemingly interesting path that has absolutely no merits nowadays.

Edited June 8, 2021 by Rolf Kalbermatter

[Taylorh140]

Wow, Thanks! It's good to be wrong.

It does sound though I was right to be weary of calling DLL's meant to call LabVIEW functions, especially when LabVIEW versions change.

But it sound like LabVIEW calls its own exposed functions (lvrt.dll i assume) when dealing with Queues. I wonder if building a dll for calling the queues and then debugging these would give me the prototype. (following the data to the builtin's) #unrecommended but a quick way to do devious things. (most likely learning that it is a bad idea and leaving it alone ha ha).

[Rolf Kalbermatter]

26 minutes ago, Taylorh140 said:

Wow, Thanks! It's good to be wrong.

It does sound though I was right to be weary of calling DLL's meant to call LabVIEW functions, especially when LabVIEW versions change.

But it sound like LabVIEW calls its own exposed functions (lvrt.dll i assume) when dealing with Queues. I wonder if building a dll for calling the queues and then debugging these would give me the prototype. (following the data to the builtin's) #unrecommended but a quick way to do devious things. (most likely learning that it is a bad idea and leaving it alone ha ha).

Well, ultimately everything LabVIEW does is written in C(++). Some (a very small part of it) is exported to be accessible from external code. Most goes a very different and more direct way to calling the actual C code functions. Functions don't need to be exported from LabVIEW in order to be available for build in nodes to be called. That can all happen much more directly than through a (platform depending) export table.

Edited June 8, 2021 by Rolf Kalbermatter