VI options in form of 51 integers

[Mefistotelis]

VI file defines types used in FP and BD; but besides that, there are a few other type definitions, for more internal use. One is an array of 51 integers, with initialized values stored within VIs DFDS block.

These values are internally accessible in LabVIEW by call of DSInitPtr().

Are these values defined in documentation somewhere? Do we know what each value means? There are some hints here and there, but my list is at the moment full of holes:

Quote

2    DSHiliteTableTypeDesc
4    unkn InvariantDataPtr
5    DSProbeTableTypeDesc
7    FPdcoInfoTblIndex
8    FPdsoInfoTblTypeDesc
12    nConnectorPorts
15    nExtraDCOInfoEntries
16    DCOInfoRelated2 InvariantDataPtr
17    TMI ?

25    unkn index
27    unkn InvariantDataPtr

49    NormalVIResumeProc
50    NormalVIRetryProc

Example dump of the values:

      <RepeatedBlock>
        <I32>2</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>2</I32>
        <I32>806</I32>
        <I32>11</I32>
        <I32>2</I32>
        <I32>824</I32>
        <I32>14</I32>
        <I32>0</I32>
        <I32>-1</I32>
        <I32>-1</I32>
        <I32>2</I32>
        <I32>700</I32>
        <I32>2</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>15</I32>
        <I32>1</I32>
        <I32>976</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>1</I32>
        <I32>0</I32>
        <I32>980</I32>
        <I32>1</I32>
        <I32>984</I32>
        <I32>2</I32>
        <I32>3</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>-1</I32>
        <I32>-1</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>-1</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>0</I32>
        <I32>-1</I32>
        <I32>3</I32>
        <I32>4</I32>
        <I32>5</I32>
        </RepeatedBlock>

All the "internal" types, stored before the types used in heaps, are called "DCO Info Entries". I'd be interested in info about any of these.

Edited May 3, 2020 by Mefistotelis

[Mefistotelis]

I now know what each value means, but am a bit baffled by the Invariant Offsets.

There's quite a few of them:

    hiliteTableOffset    = 1,
    probeTableOffset    = 4,
    fpdcoTableOfst    = 7,
    clumpQEAllocOffset    = 10,
    viParamTableOffset    = 13,
    extraDCOInfoOffset    = 16,
    localInputConnIdxOffset    = 19,
    nonLocalInputConnIdxOffset = 22,
    condIndIdxOffset    = 24,
    outputConnIdxOffset    = 27,
    inputConnIdxOffset    = 29,
    syncDisplayIdxOffset = 33,
    enpdTdOffsetsDso    = 37,
    stepIntoNodeIdxTableOffset = 43,

Currently I'm assuming last offset indicates end of the filled space and then going backwards to match other ones. This works for all the files I have, but I'm pretty sure it is not a generic way.

How should the offsets be computed? Does the Data Space contain all entries from TM80? or all entries in the entire VCTP? I can see that entries filled by DFDS are not enough. Or maybe there's just a constant-size header before DFDS content which elevated the offsets?

 

[Mefistotelis]

I figured out how the offsets work.

 

I seriously hope the lack of answers is not due to me getting close to some corpse in the closet.

 

[Rolf Kalbermatter]

On 5/20/2020 at 7:16 PM, Mefistotelis said:

I seriously hope the lack of answers is not due to me getting close to some corpse in the closet.

That's easy to answer. You lost everybody including me already quite some time ago. The only ones who could answer your questions are people with access to the LabVIEW source code and they can't and won't answer here. The rest have never gone that far into LabVIEW interna and likely have much much more important things to do.

[Mefistotelis]

12 hours ago, Rolf Kalbermatter said:

You lost everybody including me already quite some time ago. The only ones who could answer your questions are people with access to the LabVIEW source code and they can't and won't answer here. The rest have never gone that far into LabVIEW interna and likely have much much more important things to do.

Oh, I see. From the knowledge I've seen shared, I assumed devs are here.

So to explain my road to here:

Many parts of the VI file revolve around definitions of Data Types, called TypeDesc or TD in code. TypeDesc defines kind of each type (can be Cluster, RepeatedBlock, various types of ints and floats, etc.) and any additional properties of the type (ie. Array has dimensions, and RepeatedBlock has number of repeats). The TypeDescs in LV>7 are stored in VCTP (VI Consolidated Types) block, then subset of the types creates Data Space - this subset is defined and given flags (tmFlags) by TM80 block.

In older versions of LV, there is DSTM (Data Space Type Map) block instead of TM80, and TypeDescs are defined directly there instead of pointing to VCTP. So it looks like types were consolidated in LV8 - before, each type was defined in the place where it was needed; after - all types were moved to VCTP, and other places just point to specific TypeId in consolidated list.

Now for the Data Space - part of it is filled by values stored is DFDS (Default Fill of Data Space), the rest is just initialized by default values, ie. zeros. But in DFDS, the values are serialized (LV calls this Flattened), so they don't exactly match the look they have in the final Data Space created when the VI is being loaded. Actually, the structure of the Data Space when it's loaded is such that offset of data for each TypeDesc is invariant - so if there is any data of dynamic size (meaning it can change size without TypeDesc change), the Data Space stores it as a pointer.

And this is how we come to the Offsets I was asking about - these are Invariant Offsets (because they don't move as long as TypeDescs doesn't change) pointing to where the items are in the unflattened Data Space.

To visualize all that, this is how the Data Space looks in Heap Peek; marked values are tmFlags:

Edited May 23, 2020 by Mefistotelis

[Neil Pate]

5 minutes ago, Mefistotelis said:

Oh, I see. From the knowledge I've seen shared, I assumed devs are here.

None of the NI devs come here, this is LAVA after all.  (/ducks)

[Rolf Kalbermatter]

3 hours ago, Mefistotelis said:

Oh, I see. From the knowledge I've seen shared, I assumed devs are here.

Sure a few are here who even respond to various posts. But to respond to this type of topic could easily be construed as violating non disclosure agreements you nowaday have to sign anywhere when starting a job and as such could be an immediate reason for terminating their job and even liability claims. They know better than to risk such things.

Besides this type of archeological digging may be fun to do in your free time, but it leads basically nowhere in terms of productivity. It's up to you what you do with your free time, and if this gives you a fuzzy feeling somehow, then I suppose it is not a bad thing. But don't expect that there are many more out there who feel the same.

[Mefistotelis]

2 hours ago, Rolf Kalbermatter said:

Besides this type of archeological digging may be fun to do in your free time, but it leads basically nowhere in terms of productivity. It's up to you what you do with your free time, and if this gives you a fuzzy feeling somehow, then I suppose it is not a bad thing. But don't expect that there are many more out there who feel the same.

Are you completely sure about narrowing what is 'productive' in this way? That excludes a lot.

I have specific goals in my research, though this is side project of a side project now.

Knowing exactly how LV works should be one of a main goals of healthy community around the project. Unless there is no real community, and everyone is here only due to business ties.

[Rolf Kalbermatter]

I see it similar to how a car works. I know how to operate it and the traffic rules and similar but I really do not plan on learning how to take it apart and put it back together again. Some people do, but if you try that with a modern car you are pretty quickly limited by the sheer complexity of the whole thing.

[pawhan11]

I work 5 years as labview 'developer' and when someone from other technologies asks me how this compiles and really works my answer is i ve no idea.   When on the  other side open technologies give much deeper ride and control. It might be due to my limited understanding of LV but detailed information is not there on official NI forums. I guess it is up to discussion if You need low level understanding how LV or other technology works. In my case as sw engineer I always want to know how things work, and the longer i work as engineer i realize i ve no idea how 90% of things work   

[Rolf Kalbermatter]

1 hour ago, pawhan11 said:

I work 5 years as labview 'developer' and when someone from other technologies asks me how this compiles and really works my answer is i ve no idea.   When on the  other side open technologies give much deeper ride and control. It might be due to my limited understanding of LV but detailed information is not there on official NI forums. I guess it is up to discussion if You need low level understanding how LV or other technology works. In my case as sw engineer I always want to know how things work, and the longer i work as engineer i realize i ve no idea how 90% of things work   

Are you really wanting to tell me that you understand how a C compiler works? Because that would be impressive!

[Mefistotelis]

2 hours ago, Rolf Kalbermatter said:

Are you really wanting to tell me that you understand how a C compiler works? Because that would be impressive!

Isn't that common knowledge in our field? When I attended Technical University, that was one of mandatory classes.

 

In case of LabVIEW, the compiler is LLVM. Meaning anyone who looked into clang also looked into that.

The amount of developers who looked into CLang or GCC isn't as small as you suggest. This is actually quite common, if a job requires that.

I used LLVM as well at some point, ie. I worked on optimizations performed on Intermediate Language tree. And it so happens that in "Heap Peek", you can look at that IL for any VI.

 

[Rolf Kalbermatter]

34 minutes ago, Mefistotelis said:

Isn't that common knowledge in our field? When I attended Technical University, that was one of mandatory classes.

In case of LabVIEW, the compiler is LLVM. Meaning anyone who looked into clang also looked into that.

The amount of developers who looked into CLang or GCC isn't as small as you suggest. This is actually quite common, if a job requires that.

I used LLVM as well at some point, ie. I worked on optimizations performed on Intermediate Language tree. And it so happens that in "Heap Peek", you can look at that IL for any VI

So you understand the gcc source code? Wow! Just wow!