C declarations of callbacks from VICodePtrsRec

[Mefistotelis]

Do we have declarations of these callbacks? Ie. here's one:

void __cdecl _InitCodePtrsProc(struct VICodePtrsRec **viCodePtrs);

 

Just to explain what I'm talking about:

* VI files and other RSRC files, can contain compiled code stored in VICD resource

* VICD consists of the actual assembly, and list of 'patches'; these patches combine function of imports table and relocation table

* The actual assembly always has a InitCodePtrsProc() function, which sole purpose is to fill an array of code pointers which it receives as parameter

* The code pointers are just references to other functions which exist in the assembly; there are up to 30 such callbacks

* executing a VI file means just calling these callbacks, in proper order

 

It's not that hard to figure out the parameters of these callbacks, I'm just wondering if we already have that information.

 

Here's example init function for a very simple VI:

void __cdecl _InitCodePtrsProc(struct VICodePtrsRec **viCodePtrs)
{
  struct VICodePtrsRec *ptr;

  _VINormalInitCProcsHelper(viCodePtrs); // Imported routine, pointed in list of patches
  ptr = *viCodePtrs;
  *((_DWORD *)ptr + 5) = _InitProc;
  *((_DWORD *)ptr + 25) = _CodeDebugProc;
  *((_DWORD *)ptr + 26) = _CodeErrHandlingProc;
  *((_DWORD *)ptr + 30) = _RunProc;
  *((_DWORD *)ptr + 28) = _InitCodePtrsProc;
}

 

Edited June 29, 2020 by Mefistotelis

[Rolf Kalbermatter]

I first thought it may have to do with the legacy CINs but it doesn't look like that although it may be a similar idea.

On second thought it actually looks like it could be the actual patch interface for the compiled object code of the VIs. It seems the actual code dispatch table that is generated during compiling of a VI. As such I doubt it is very helpful for anything unless you want to modify the generated code somehow after the fact.

 

[Mefistotelis]

52 minutes ago, Rolf Kalbermatter said:

It seems the actual code dispatch table that is generated during compiling of a VI.

Yes, that's the table. Surprisingly, it's not just written directly as an array, but instead there's an init function which fills it.

Sounds a bit unnecessary. And introduces possible security issues.

52 minutes ago, Rolf Kalbermatter said:

As such I doubt it is very helpful for anything unless you want to modify the generated code somehow after the fact.

I hope the function is verified before execution, and executed later than when the VI is opened. Otherwise the ease of modification might be an issue.

Besides modifying it, you may just want to look at it, to either debug an issue within LV, or just check how the code works.

With the declarations and all structs defined, you could also turn the VI code into COFF; that would make it easier to disasseble, or write a wrapper which executes it.

That's the same code you can look at in "Heap Peek", but there you get symbols and labels instead of specific offsets.

 

CINs - I don't know much about these ATM, didn't looked into LabWindows at all. Interesting hint. Thanks.

Edited June 30, 2020 by Mefistotelis

[dadreamer]

14 hours ago, Mefistotelis said:

It's not that hard to figure out the parameters of these callbacks, I'm just wondering if we already have that information.

You might try loading macOS LV version into debugger, because it has more debug symbols unstripped unlike Windows and Linux versions. As I recall I was able to read out the rest of the parameters and their types just by browsing the code in IDA.

2 hours ago, Mefistotelis said:

CINs - I don't know much about these ATM, didn't looked into LabWindows at all.

It's mostly about old LV versions before LV 2009. Check LVSB and PLAT resource sections (and maybe LIsb for external subroutines), if you're going to study how CINs work. There are Rolf's articles also, that could help you to put all the pieces together: https://forums.ni.com/t5/LabVIEW/What-happened-to-CINs-And-how-else-can-another-language-work/m-p/2726539#M807177

[Rolf Kalbermatter]

6 hours ago, Mefistotelis said:

CINs - I don't know much about these ATM, didn't looked into LabWindows at all. Interesting hint. Thanks.

CINs have nothing to do with LabWindows CVI, aside of the fact that there was a possibility to create them in LabWindows CVI.

They were the way of embedding external code in a LabVIEW VI, before LabVIEW got the Call library Node. They were based on the idea of code resources that every program on a Macintosch consisted of before Apple moved to Mac OS X. Basically any file on a Mac consisted of a data fork that contained whatever the developer decided to be the data and a resource fork that was the model after which the LabVIEW resource format was modelled. For the most part the LabVIEW resource format is almost a verbatim copy of the old Macintosh resource format.

A Macintosh executable usually consisted of an almost empty data fork and a resource fork where all the compiled executable code objects where just one of many Apple defined resource types together with icons, images, (localized) string tables and custom resource types that could be anything else a developer could dream up. Usually with the exception for very well known resource types these files also contained resource descriptions (a sort type descriptor like what LabVIEW uses for its type system) as an extra resource type for all its used resource types.

The idea of CINs was interesting but cumbersome to maintain as you had to use the special lvsbutil executable to put the CIN code resource into the VI file. And in my opinion they stopped short of creating a good system by only allowing one CIN code resource per CIN name. If they had instead allowed for multiple CINs to exist for a specifc name, one for each supported platform (m68k, mppc, mx86, sparc, wx86, wx64, vxwk, arm, etc) one could have created a VI that truely runs on every possible platform by putting all the necessary code resources in there. As it was, if you put a Mac 68k code resource into the VI it would be broken on a Mac PPC or on Windows system and if you put a Windows code resource in it it would be broken on the Mac.

Also once the Call Library Node supported most possible datatypes, CINs basically lost every appeal unless you wanted to hide the actual code resource by obfuscating it inside the VI itself. And that was hardly an interesting feature, but was bought with lots of trouble from having to create seperate C code resources for every single CIN (shared libraries can contain hundreds of functions all neatly combined in one file) and also a maintenance nightmare if you wanted to support multiple platforms.

As to the articles mentioned in the link from dadreamer, I resurrected them from the wayback engine a few years ago and you can find them on https://blog.kalbermatter.nl

Edited June 30, 2020 by Rolf Kalbermatter

[Mefistotelis]

8 hours ago, dadreamer said:

You might try loading macOS LV version into debugger, because it has more debug symbols unstripped unlike Windows and Linux versions

I just did. Wow. Now I know I was missing a lot by ignoring Apple. At least from this perspective.

Not a single function name is missing, and all names are decorated with types.

[Mefistotelis]

On 6/30/2020 at 1:40 PM, Rolf Kalbermatter said:

each supported platform (m68k, mppc, mx86, sparc, wx86, wx64, vxwk, arm, etc)

Do we have a full list of these architectures somewhere? I've seen:

'i386', 'wx64', 'ux86', 'ux64', 'm386', 'mx64', 'PWNT', 'axwn', 'axlx', 'axdu', 'ARM ' (those are all little endians)

I'm not exactly sure what each of these means.

Are different OSes counted as separate architectures?

 

[dadreamer]

AFAIK i386 is for Windows 32-bit and wx64 is for Windows 64-bit. Following this logic u should be for Linux and m for macOS, but I'm not 100% sure. I'm just saying, what I saw when browsing through a number of CINs with their PLAT sections.

[Rolf Kalbermatter]

10 hours ago, Mefistotelis said:

Do we have a full list of these architectures somewhere? I've seen:

'i386', 'wx64', 'ux86', 'ux64', 'm386', 'mx64', 'PWNT', 'axwn', 'axlx', 'axdu', 'ARM ' (those are all little endians)

I'm not exactly sure what each of these means.

Are different OSes counted as separate architectures?

 

I just made them up! I believe NI used the 'i386' as a FourCC identifier for the Win32 CINs.

From my notes:

i386      Windows 32-bit

POWR      PowerPC Mac

PWNT      PowerPC on Windows

POWU      PowerPC on Unix

M68K       Motorola  680xx Mac

sprc          Sparc on Solaris

PA<s><s>    PA Risc on HP Unix

ux86          Intel x86 on Unix (Linux and Solaris)

axwn         Alpha on Windows NT

axln          Alpha on Linux

As should be obvious some of these platforms never saw the light of an official release and all the 64-bit variants as well as the vxworks versions never were created at all, as CINs were long considered obsolete before VxWorks was released  in 8.2 and the first 64-bit version of LabVIEW was released with LabVIEW 2009 for Windows. There even was some work done for a MIPS code generator at some point.

And yes the problem about adding multiple CIN resources for different architectures was that it relied on the 'plat" resource inside the VI. So you only could add one CIN resource per CIN name into a VI, rather than multiple ones.

All platforms except the i386 and Alpha, used to be Big Endian. Later ARM came as an additional Little Endian target to the table. Currently only the VxWorks target is still a supported Big Endian platform in LabVIEW.

Edited July 2, 2020 by Rolf Kalbermatter

[Mefistotelis]

Thank you!

That added some important info to my list:

https://github.com/mefistotelis/pylabview/blob/master/LVcode.py#L224

A little below the arch list, there's getVICodeProcName() function where I added mangled names for the dispatch table items which I got from LV for MacOS.

Pylabview will now, upon extraction of a VI file, create a MAP file with offsets and names of known symbols within the executable bytecode.

 

[Rolf Kalbermatter]

4 hours ago, Mefistotelis said:

Thank you!

That added some important info to my list:

https://github.com/mefistotelis/pylabview/blob/master/LVcode.py#L224

A little below the arch list, there's getVICodeProcName() function where I added mangled names for the dispatch table items which I got from LV for MacOS.

Pylabview will now, upon extraction of a VI file, create a MAP file with offsets and names of known symbols within the executable bytecode.

 

One correction. the i386 is really always a 32 bit code resource. LabVIEW for Windows 3.1 which was a 16-bit operating system was itself fully 32-bit using the Watcom 32-bit DOS extender. LabVIEW was compiled using the Watcom C compiler which was the only compiler that could create 32-bit object code to run under Windows 16-bit, by using the DOS extender. Every operating system call was translated from the LabVIEW 32-bit execution environment into the required 16-bit environment and back after the call through the use of the DOS extender.

But the LabVIEW generated code and the CINs were fully 32-bit compiled code. While the CINs were in the Watcom REX object file format, and LabVIEW for Windows 32-bit later by default used the standard Windows COFF object format for the CINs resources, it could even under Windows 32-bit still load and use the Watcom generated CINs in REX object file format. The main difference was simply that a REX object file format had a different header than a COFF object file format but the actual compiled object code in there was in both cases simply i386 32-bit object code.

Also LabVIEW 2021 or more likely 2022 is very likely going to have an  'mARM' platform too. 😃

Edited July 2, 2020 by Rolf Kalbermatter

[Neil Pate]

@Rolf Kalbermatter when is your kickstarter launching? You know the one where you write all this useless but interesting crap down in a nice coffee table book.

I would back this project...

[Mefistotelis]

Thanks @Rolf Kalbermatter, did proper update. Watcom actually had Windows extender, separate to the DOS extender.

Also, interesting prediction. Personally, I think Apple overestimates scalability of their phone silicon. But we'll see.

 

[Rolf Kalbermatter]

2 hours ago, Mefistotelis said:

Thanks @Rolf Kalbermatter, did proper update. Watcom actually had Windows extender, separate to the DOS extender.

Also, interesting prediction. Personally, I think Apple overestimates scalability of their phone silicon. But we'll see.

 

Sure it was a Windows extender (Win386) but basically built fully on the DOS/4GW  extender they licensed from Rational Systems.  It all was based on the DPMI specifiation as basically all DOS Extenders were.

Windows 3.x was after all nothing more than a graphical shell on top of DOS. You still needed a valid DOS license to install Windows on top of it.

Edited July 2, 2020 by Rolf Kalbermatter