dadreamer

2 hours ago, ensegre said:

Could someone attach the snippet saved as VI for LV<=2019??

Here's the version back-saved for LV 2011: HeapPeek_WinMonitor.vi

Heap Peek and Window Monitor are both available on Linux and macOS. And if one wishes to recreate these brown nodes manually, then (s)he should put SuperSecretPrivateSpecialStuff=True into LabVIEW.ini and restart LabVIEW; when done, a number of hidden private nodes appears.

Okay, now that is simplified.

To extract VIs out of .lvlibp (no matter, if FFF-enabled or not):

1. Open extract_lvlibp_vis.vi (attached -> extract_lvlibp_vis.vi ), set path to your PPL (.lvlibp) and the destination folder and run the VI. The files should be extracted and placed to the chosen folder.
2. Extract the inner resource blocks from a single VI with readRSRC.py -x -i "YourVI.vi" command. YourVI.xml and a bunch of various resource sections should appear.
3. Open YourVI.xml and change the library name extension from .lvlibp to .lvlib (between <LIBN></LIBN> tags). Remember the .lvlib name. Save and close the .xml.
4. Pack the .xml and all the resources into a single VI with readRSRC.py -c -m "YourVI.xml" command. You should get YourVI.vi recreated. Do not open it yet.
5. In LabVIEW create new empty library and save it into the same folder, where YourVI.vi resides. Use that name, you obtained on the step 3.
6. Now drag YourVI.vi from the folder into the library's root. LabVIEW should reload the VI from the new path and report.
7. * Untie YourVI.vi from the library w/ RMB -> Remove from Library menu option, save both files w/ File -> Save option. Now you may close YourVI.vi and open again to make sure, it's not broken.
8. Optionally repeat the steps 2-7 for any other VIs from that .lvlib.

To extract VIs out of .exe (for non-FFF EXE's you finish on the 4th step):

1. Open YourApp.ini and add two lines to its end according to this article.
2. Launch YourApp.exe and leave it in this state.
3. Open extract_exe_vis.vi (attached -> extract_exe_vis.vi ), set path to the destination folder and run the VI. The files should be extracted and placed to the chosen folder. Make sure, you're using the same port number, that you've set on the step 1. If 3364 doesn't work for some reason, try another number (3363 etc.).
4. For EXE's without FFF option: Now you may open your VIs and make sure, they are fine. For EXE's with FFF: go to step 5.
5. Load a single VI into flarn's Resource Editor. ("Load" button).
6. Find CLIv resource section and change its Type to LIvi.
7. Find CLId resource section and change its Type to LIds.
8. Save the VI with "Save" button. Don't do any other actions here like fixing checksums or whatever. Now you may open YourVI.vi to make sure, it's not broken.
9. Optionally repeat the steps 5-8 for any other VIs from that .exe.

* N.B.: In projects with more or less complex hierarchies it might be a good idea not to untie your VIs from the library, because it's very easy to mess with the correct order of such an unlinking. When done incorrectly, your VIs could become broken and to fix them you have to recreate the original VI(s) from the .lvlibp. Also to note, such an untying doesn't work neatly for VIs of different levels of dependence, i.e., a VI has some SubVI(s) and they all are contained into that .lvlibp; on unlinking them you'll likely get those SubVI(s) fine and the main VI broken. But if you really want to detach each and every VI from the library, you first make sure, that your VI is working fine inside that library, then unpack the VI with readRSRC.py, in .xml remove lines like <String>Untitled Library 1.lvlib</String> in LIvi and LIds sections, then pack again to VI, move dependent SubVI(s) out of the library and open up the main VI. It may work or may not, you test it out by yourself.

So far I have left those instructions as they are, because I don't know atm, how to eliminate some steps. I believe, there should be a LabVIEW-native way to unlink the VI from the library, thus we could exclude steps 11-14. It would be easier then to automate this algorithm to process multiple VIs. Unfortunately Disconnect From Library method doesn't work on locked libraries and Open VI Reference or even Open.VI Without Refees don't work on broken VIs. There are Linker.Read Info From File and Linker.Write Info To File private nodes also, but I still have not figure out, whether they could be used for the task (any ideas?).

That's why I decided to switch to LEIF formatted lib's and exe's. I found that we can bypass LEIF loader RE-ing using Save.To Buffer private node (always felt uncomfortable studying that loader, because it's rather complex). The node returns convenient RSRC with LVINLBVW block instead of LEIF, so we can easily save that into the file to deal later. I suppose, LEIF data is always translated into common resources by LabVIEW and they're in memory until the VI's are unloaded.

Here's how a single VI may be extracted out of .lvlibp:

extract_lvlibp_vi.vi

And here's how the same may be done with the EXE:

extract_exe_vi.vi

To open VI inside EXE I used technique, described in this article: Passing Data Between LabVIEW Executable Reference VI and a VI

Upd: see next message for the instructions.

On 6/12/2020 at 1:08 AM, Mefistotelis said:

assuming I will continue to support it long enough

I appreciate if you find time to take a look at LEIF binaries some day, as I think you're more experienced in VI contents and internals than me. 🙂 By the way I tried readRSRC.py with LabVIEW 2020 built executable without FFF option and it worked like a charm. I can't do the same with PPL's, because that Fast File Format is forced onto them hardly and I've found no way to switch it off.

9 hours ago, Dataflow_G said:

There was an R&D project in 2009 called LabVIEW Notebook Pioneer.

There was even a (very) short thread on LAVA:

By the way I always wanted to try, but I was a bit late to it, so never seen any files to download and play around freely. Does anyone still have it on their hands? Or does it require a special licensing and thereby unavailable?

I have updated this message, as after trial-and-error takes I've got success finally!

As you can see, the VI is in runnable state and behaves without any errors. My steps to reproduce:

1. Unpacked lvlibp with 7-Zip unarchiver (as I'm on Windows currently) and pulled out "2" file (LIBPLBVW resource).

2. Extracted the inner resource blocks with readRSRC.py -x -i "2" command, got 2.xml, 2_DATA0.bin to 2_DATA4.bin and 2_LVzp.bin files.

3. Unpacked 2_LVzp.bin with an unarchiver, got NI_Embedded_Library.xml and Untitled 1.vi files.

4. Extracted the inner resource blocks from Untitled 1.vi with readRSRC.py -x -i "Untitled 1.vi" command, got Untitled 1.xml and a bunch of various resource sections.

5. Renamed 2_DATA0.bin (from step 2) as Untitled 1_VCTP.bin and placed it near Untitled 1.xml.

6. Added the following text to the end of Untitled 1.xml (before the last </RSRC> tag):

  <VCTP>
    <!-- VI Consolidated Types -->
    <Section Index="0" Format="bin" File="Untitled 1_VCTP.bin" />
  </VCTP>

7. Packed the .xml and all the resources into a single VI with readRSRC.py -c -m "Untitled 1.xml" command, got Untitled 1.vi.

8. Extracted the inner resource blocks from Untitled 1.vi (again!) with readRSRC.py -x -i "Untitled 1.vi" command, got Untitled 1.xml and a bunch of various resource sections (is done to get VCTP processed correctly).

9. Opened Untitled 1.xml and removed everything from <LIBN> to </LIBN> (including the tags as well) changed the library name extension from .lvlibp to .lvlib (between <LIBN></LIBN> tags). To wit, this line

<Library>Untitled Library 1.lvlibp</Library>

should be turned into this

<Library>Untitled Library 1.lvlib</Library>

10. Packed the .xml and all the resources into a single VI with readRSRC.py -c -m "Untitled 1.xml" command, got Untitled 1.vi.

11. Renamed NI_Embedded_Library.xml (from step 3) to Untitled Library 1.lvlib.

12. Opened Untitled Library 1.lvlib in a text editor and altered the lib URL to point to the new location. To wit, this line

<Item Name="Untitled 1.vi" Type="VI" URL="Untitled 1.vi"/>

should be turned into this

<Item Name="Untitled 1.vi" Type="VI" URL="../Untitled 1.vi"/>

13. Opened the VI (LabVIEW did the library reload from the new path and reported), opened .lvlib.

14. In Project Explorer untied the VI from the library w/ RMB -> Remove from Library menu option, saved the both files w/ File -> Save option.

Now the VI runs fine, so it might be a time to simplify this tutorial a little 🙃 If your tools could support lack of VCTP and recreate it automagically, that would be great for sure.

3 hours ago, Mefistotelis said:

The only real issue is lack of VCTP section. The rest of removing the VI from library is trivial, if you check how the file looks before and after compiling the library.

I've got nearly the same result, using flarn's Resource Editor, so fixing LVSR and BDPW blocks is pretty easy. Untying from the library is no problem too. Tomorrow I'm gonna try your tools to see, how they'll go for me, and try to restore VCTP as well according to your info (manually maybe, but hope it could be automated one day). On success I could take a look at LEIF packed libraries to figure out, how to do the same.

3 hours ago, Mefistotelis said:

'tampered with password' message

I was kind of surprised here, because both EXEs and PPLs don't have block diagrams, unless you explicitly set 'debugging' option in the build properties to get the BDs saved. Thus that 'tampered' message is absolutely unnecessary in compiled app or library, but likely there's no additional check in lvrt.dll for such a case, when the checksums don't match.

Just to clarify, gButtonsHidden field is not only those settings, that you could see in VI Properties -> Window Appearance -> Customize dialog and could manipulate with scripting properties like Tool Bar:Show Run Button etc. There are some additional VI flags also to show or hide the toolbar buttons, they're called HiddenButtonsFlags in Heap Peek (buttHid for LV2013 and earlier versions). You may investigate them on this sample VI. As you can see, the VI has Pause button hidden completely and you cannot show it with VI Properties entries, even if you set "Allow debugging" checkbox. Heap Peek displays, that HiddenButtonsFlags equals to 0x348.

Honestly I don't know, why NI made that VI to have Pause button hidden, but I know, that LabVIEW stores the buttons' display state in HiddenButtonsFlags e.g. when switching to Subroutine Priority or vice versa. Sadly I have not found a way to alter all these flags from the diagram (using scripting or some other tools), so if one wants to modify the flags, (s)he has to manipulate the memory. In Heap Peeek's VI Fields there's the VI pointer in form vi=[some value]. Let's go there with any debugger of our choice.

Now go to the address at the "blue" offset (I'm not saying concrete numbers, because they could change between platforms, bitnesses, LV versions, weather or who knows, what else; I was testing on LV2019 32-bit on Windows). And here we should have our HiddenButtonsFlags equal to 0x348.

(The mem block is actually 2-bytes long and I'm too lazy to remake a screenshot).

Now if we set that memory block to zeros, we'd have all the toolbar buttons shown (the debugging should be set earlier in the VI Properties).

Such a task could even be done easier with some utility, which is able to search integers in memory and filter them in stages (ArtMoney or similar).

Mefistotelis

Hi! Did you try to do the similar operation with packed library (*.lvlibp)? I mean building it from a common .lvlib with any VIs inside, unpacking it into the initial VIs and rebuilding again. I just tried and it doesn't go well. For that I especially installed LV 2014 to have the library's resources in convenient LIBPLBVW and not in modern LEIF Fast File Format. This is what I did:

1. Made a .lvlibp build with one .lvlib, having one super-simple test VI inside.

2. Unpacked that .lvlibp with 7-Zip.

3. Grabbed \.rsrc\0\RCDATA\2 file and fed it into my own extractor (written a long ago) and finally got un-CRC-ed .zip archive.

4. Unpacked .zip and got NI_Embedded_Library.xml and my test VI.

Pretty the same steps, that I did many times to get the original VIs, CTLs etc. from a common EXE app. But that unpacked VI didn't want to load and reported, that it cannot find the owning packed library. Okay, then I created a .lvlib from NI_Embedded_Library.xml simply renaming it and fixing the URL path, it was referring to. Besides of that I also corrected the path in LIBN resource to point to that new .lvlib instead of the original .lvlibp. So after these manipulations the VI started to ask for some absent resource to load. So, to get it working I had to:

- add VCTP block from the original VI to the unpacked VI;

- replace LVSR and BDPW blocks with those from the original VI (to avoid 'tampered with password' message);

- replace LIvi block with that from the original VI;

- remove LIBN (to get the VI untied from any .lvlib(p)) (optional step, maybe).

I wonder, if there's some easier way to get the VI(s) workable, because for some third party .lvlibp it would be impossible to do, not having the original sources (to read out VCTP at least). Do you have any insights or thoughts on it?

Maybe it would be an option for you to run the installer in silent mode (and/or) with the reboot suppression: Customize and Automate Installation of a Single NI Installer This article might be of some use also: Running a Batch File Automatically After an Installation to Access Installed Files And here's the similar thread, that points to these two articles mentioned.

I also think, there's an option to not use a single LabVIEW installer, but use a bunch of msi installers by pulling them from that .exe and putting all the files into one final Inno setup .exe. You could try to run each msi with one of those flags to suppress the reboot message:

msiexec /qn /i Application.msi REBOOT=ReallySuppress

msiexec /qn /i Application.msi REBOOT=R

msiexec /package Application.msi /norestart

On 6/1/2020 at 9:38 PM, hooovahh said:

Most of the time I care to find a window's HWND I use a private function which lets me do what I want.

I used that FP.NativeWindow property for years and it never failed for me. I was using it on Windows only though. Maybe I could test it on other platforms as well, but I never needed that.

Oh, and here's one more way to invoke Heap Peek and Window Monitor (absolutely forgot about it).

Another way to view Heap Peek window - NCGLaunchHeapPeek internal function. Doesn't need any parameters and returns nothing. But must be invoked in UI Thread only, else LV goes crash. Starting from LV 2015.

N.B.: Both methods do not work in RTE (compiled app).

1 hour ago, ShaunR said:

It doesn't seem to have a scripting counterpart. Is that correct, or have I just missed it?

I browsed through all the properties and methods of Call Library object and didn't find such scripting analogue. Seems like VI Server doesn't expose that specific property.

1 hour ago, ShaunR said:

Is the setting sticky, or does distributed source code require the INI setting too?

Yes, like any other option, I mentioned above. Tried to open my DbgPrintf sample VI from another machine and all the parameters are in place. If you don't have extFuncGenNoWrapperEnabled=True and extFuncShowMagic=True in your .ini, then the option is "invisible" in the menu or on BD, but it's there. 🙂

13 hours ago, ShaunR said:

Interesting. How does this feature compare with disabling the error checking on the Error Checking tab?

Even when Error Checking is set to Disabled, LabVIEW still enters ExtFuncWrapper to do some basic checks before the target function is called. A few internal functions, such as _clearfp and _controlfp, are being called also. Thereby disabling "Generate Wrapper" option should make CLFN a little faster, than disabling Error Checking. You can take it like you're calling a built-in yellow node (not taking into account the called function's own speed, of course). I did not do concrete benchmarking to compare these two options. If there's an interest, I could check this out.

On 5/22/2020 at 9:11 PM, Rolf Kalbermatter said:

Before that LabVIEW required you to wire inputs no matter what

I checked that in LabVIEW 3.1.1 and it doesn't require the Return input terminal to be connected to some constant. I also tried LabVIEW 3.0 and 2.5 but I don't see CLF Node there, only CIN, so can't check there.

Got the same in LabVIEW 4.0 (in fact it's a demo version, but DLLs are callable).

I still think, there's some sort of a bug, because in LabVIEW 7.1 and 8.0 when you created that unnecessary constant on the left, connected it to CLFN and changed the Return type back to void, the VI's Run arrow becomes broken. The same applies to the wire from the constant to CLFN. But in LabVIEW 2009 and higher the Run arrow is fine and LabVIEW allows to run the VI despite oddity with the CLFN (look at the image above).

This is the screenshot from LV 8.0:

Might be an omission when porting to the new managers interface?..

1 hour ago, X___ said:

Can't use it with 64-bit LV, AFAIK...

I successfully used oglib_lvzip-4.1.0-b3 from this post, enhanced for 64-bit LV by Rolf. It always worked good for me.

Here are some "secret" INI keys related to CLFNs:

• extFuncGenNoWrapperEnabled=True - the most interesting one here; adds "Generate Wrapper" option to CLFN context menu (on by default). When this option is unchecked (off), LabVIEW doesn't generate the wrapper for DLL call, i.e. ExtFuncWrapper function is not called and the user's function is inlined into the VI code and called. This feature could slightly improve performance of external code communications, saving approx. 5 to 10 ms on each call. But you must be absolutely sure, that your CLFN is set correctly as no error checking is done in this case and any small mistake in the parameters leads to LV crash. There might be an indirect use of this feature, e.g. manual fine-tuning/debugging of some problematic function or a function with unclear parameters etc. When the option is enabled and extFuncShowMagic is set, CLFN is painted red.
• extFuncBreakOnGenNoWrapper=True - if enabled, disallows to run VI with CLFN's "Generate Wrapper" option unchecked. The Run arrow is broken and the error is stated as "Call Library Nodes with 'gen no wrapper' are not supported.". The error is detailed as "This Call Library Node is set to 'gen no wrapper'.  This setting skips certain steps when calling DLLs in order to be faster, and can be dangerous.".
• extFuncShowMagic=True - adds some coloring to CLFNs, when "Generate Wrapper" and "Type Only Argument" options are used.
• extFuncExtendedOptionsEnabled=True - adds "Allow Sub-Arrays" option to CLFN context menu (off by default). I assume, it should have worked this way, but I have not found any signs that this feature is working at all. If I set "Constant" option, then a sub-array is passed into a function, else the whole array is copied and passed, no matter if "Allow Sub-Arrays" is set or not. Maybe I missed smth or it's a relic option from pre-LV2009 era...
• extFuncArgTypeOnlyEnabled=True - adds "Type Only Argument" option to CLFN context menu, when clicking on a node's terminal (off by default). When this option is checked, LabVIEW passes 0 (aka NULL) instead of a real value of the parameter, no matter if passed by value or by reference. When the option is enabled and extFuncShowMagic is set, the terminal is painted bluegreen (cyan). Any thoughts, what was it used for? Here's its scripting counterpart.
• allowPreallocsInDll=True - adds "Allow Preallocated Data" option to CLFN context menu, when clicking on a node's terminal (off by default). Did not study this one well, but it seems to not do anything or I'm wrong? Here's its scripting counterpart.
• callLibNodeMayHaveLVClassParams=True - if enabled, allows to run VI with LV class wire(s) passed into CLFN. By default the Run arrow is broken, when class wire is connected to CLFN. Interesting, but what can we do with it in the external code?
• AllowCLFNTakeSlice=True - on by default, even if absent in the .ini; appears to behave the way, that "Allow Sub-Arrays" option should do, when enabled. Let's say, some array slice (i.e., a sub-array) is passed into CLFN and the input parameter is configured as "Constant" -> "Array Data Pointer". When that .ini setting is on, no intermediate copy of the array is made, thus the called library receives a pointer to some location within the initial array. When the setting is off (AllowCLFNTakeSlice=False), then the sub-array is always copied and fed into the library, no matter if slice or not.

This is the pic with some options activated:

DbgPrintf.vi

I also noticed that there are some private flag bits for CLFNs, e.g. extFuncVarArgs, extFuncParamsFromTL, extFuncNIValidated, extFuncTopLevelOnlyInUI.

I believe, some bits should be available with scripting also. Var args feature looks interesting, but it's likely an abandoned thing, isn't it? AFAIK CLFNs never supported that in any manner.

Finally some sort of off-topic, but somewhat related.

Did you know that it's possible to create an input constant for CLFN's return terminal?

Just set Return output as non-void and do a RMB click on the left terminal. The Run arrow is not broken and the VI runs fine. Ancient bug or intended for something? I've never seen that value being passed into external code in any ways.

On 5/13/2020 at 10:47 PM, avogadro839 said:

Does anyone know how to create a string constant with the context menu option of "Set String Length?"

I wouldn't recommend using this internal feature as it appears to be somewhat unstable. Doing a quick test I crashed LabVIEW a few times. But if you really want, you may call "Prealloc Length" (aka "PreallocLen") private property on a String control / indicator and after that "Set String Length" context menu entry is available with RMB click.

BTW, that thread about fixed/bounded arrays also belongs to here:

String_Len.vi

As your fork's Issues section is disabled, I'm going to ask you here. Have you had a chance to investigate the Fast File Format option, which was introduced starting from LV 2015? It's activated by default for LVLibp and can be turned on for DLL/EXE also in the build properties. If enabled, it produces LEIF resource instead of convenient LVzp, so the whole technique for extracting the resources from EXE's seems to be useless. As some dev's wrote on NI forums, LEIF is more similar to a statically linked PE or ELF file (but neither of those is used actually). There's no any insights on LEIF loader or the binaries format, so I wonder if you dealt with it in some ways.

I might be wrong, but I've never seen a LabVIEW built-in and exported function like nextafter or nexttowards in extcode.h or inside any Managers. But there's a WinAPI implementation according to MSDN: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/nextafter-functions?redirectedfrom=MSDN&view=vs-2019

So you might try to use those func's from msvcrt.dll by the means of CLFNs.

Untitled 1.vi

Hope, it helps somehow. Oh, and by the way, for Linux you could use libm.so and libSystem.dylib for macOS to call the mentioned functions, if you're not on Windows.

I have just tested on LabVIEW 2019 and this works fine. Here are the VI and the Snippet, back-saved to LV 2014.

SHA256.vi

8 hours ago, Rolf Kalbermatter said:

That makes no sense. But I"m not going to tell you you can't do that. 😀

Okay, maybe this will make a sense?.. We want to write something into LV control/indicator (say, numeric array, doesn't matter) directly. So, convenient DS... functions are not enough for us. We need some well-established storage in VI Data Space, that is associated with our control. *HdlPtr behaves like that storage, because it's actually a memory address in VI Data Space, where we could read/write from/to and all the changes are immediately applied to our control. If you'd use the traditional method with MoveBlock and DS..., you get a handle equal to NULL. But you want to write to the control, how would you do that then? You need to tell LabVIEW, that you've created a fresh handle and want to have it applied to your control.

When I've been experimenting w/ all this, my perspective was that the pointer can be passed anywhere you want, e.g. into external code, and you can do anything with it there. LabVIEW reacts to your manipulations and updates the control as you wish. In the examples everything is done on BD, but it all could be easily translated into C/C++ or another language of your choice. I might write a sample DLL to illustrate this better, but why to bother, if it may be done from the diagram?.. It's only LV-related disadvantage, that many CLFNs look bulky, in C the corresponding calls occupy few lines of code.

9 hours ago, Rolf Kalbermatter said:

The control value is completely seperated from the data value in the wire. Assuming that they are the same is utterly useless.

I fact, this is what I always kept in mind, when creating this thread. 🙂 There's even some short but neat NI presentation - Improving the Performance of your LabVIEW Applications.

9 hours ago, Rolf Kalbermatter said:

it also can and will change between versions. The only thing you can say is that in a particular version without any change to the diagram you SHOULD get the same result but anything else is simply not safe to assume.

I assume, that something could change in the memory managements, but not so drastically. DCOs/DDOs concept existed from the very first LV versions (I saw it in 2.5 already), so the basic rules to deal with these objects should stay the same. If the changes would be such significant, then you'd get a partial or even full lack of backward compatibility w/ the previous versions. Remember, modern LV still loads and runs all the versions back to 7.1. And I've personally never seen, that a generic control's pointer changes during the execution, or even that I'm unable to get it with the way described. Never liked to be wrong (who likes?), so has checked my samples on many LV versions back to 2009. Anyway, it's all not for the real projects, so even if something changes, who cares?.. 😀 As to me, I very seldomly use hacks in my work, I could count 3 or maybe 4 private property nodes for the whole term of my LV programming. The hacks are just a matter of interest, no more.

9 hours ago, Rolf Kalbermatter said:

Trying to use this for a (possibly perceived) performance gain is utterly asking for a lot of pain in the long run. So don't do it!

Actually I'm stopping at this point, because I feel like it's a waste of time, if speak about CINs, as no any man on Earth uses CINs these days. As to CLFNs, I told the essence of what I know about the subject, so it's done. From the very LV programming days I had a question, if we could get a classic pointer to a wire or a control. Well, now ten years later, I've got the answer. It's not such a classic and not such a practical though.

Quote

My point is, that for the retrieval of the pointer that a handle is, the two SPrintf() calls are utterly Rube Goldberg. They simply return the pointer that a handle is. If you passed the array as Array Handle, Pass Handle by value to the first MoveBlock() function you would achieve the same!

I don't need a handle (i.e., TD1 **TD1Hdl) here. I want a pointer to a handle (i.e., TD1Hdl *HdlPtr). That's why I do that cumbersome SPrintf stuff. When the VI is started for the very first time, the array's handle is NULL. Later I resize the array or do some other things with it, so the handle becomes a real value. It also can change between the runs or on a new resize. Of course, I could do some logic and use a shift register to store the actual handle, but as to me it's better to do some complicated task once, rather than do it again and again in a loop. Moreover, I use the similar method not only on arrays, but also on controls, so I need a pointer to the control as well. MoveBlock dereferences the pointer and I actually get nothing to deal with later.

Quote

Yes I mean the data parameter. In your diagram you pass the U64 value of the control/indicator to it and declare the parameter as pointer sized integer (passed by reference?). But it should be the data type of the control passed by reference (so U64 passed by reference). For more complex values like clusters and arrays it is probably best to configure this parameter simply as Adapt to Type (pass handles by reference).

This is my omission. I didn't even notice until you said. Numerous copy-paste operations do their evil things, when using on complex nodes like CLFNs. Fixed the VIs in my first post. Thanks. 😸

Quote

CINS always passed data by reference.

When I had a very limited test of the above VIs on LabVIEW 8.5, I noticed, that LabVIEW gives away fine (non-copied) pointer for array and the whole stuff works, but for numeric control it is no longer working. With CINs the situation is even worse. Regardless of the way how the parameters are passed, LabVIEW makes their copies and passes them instead of the originals. But under some circumstances no copies are created, e.g. when I pass the wire from Random Number VI into CIN directly instead of passing it from the control's terminal. Maybe I could try to find out what’s the matter, if possible.

Quote

That's all nice and pretty until you place an Array Resize node in the array wire to increase the array size. Et voila, the internal pointer most likely (not necessarily always) will change as LabVIEW will have to reallocate a new memory area and copy the old content into the new area and deallocate the original memory. So while this is no real news to people familiar with the LabVIEW memory manager C function interface, it is at best a brittle approach to rely on.

When you write a C DLL that receives handles from LabVIEW, the handle is only guaranteed to exist for the duration of the function call and after you return control to the LabVIEW diagram, LabVIEW reserves the right to resize, delete, or reuse that array handle for other things as it sees fit.

Yeah, I'm aware of it and seen such behaviour during my experiments. In this case ArrayMemInfo node should be called again on a "new" wire to retrieve fresh data pointer. Or ArrayMemInfo might be replaced by those SPrintf nodes to obtain the pointer to array's handle instead. That pointer is also stable while VI is executing and the wire exists, and it's updated by LV with new array's handle, when the array is resized or whatever. Nevertheless I strongly doubt that someone would use all those codes for something except "funny" time wasting in LV, even for simple data display, that's why I did not describe absolutely all the nuances. There are many articles on the subject, e.g. "Using External Code in LabVIEW", so I assume, the experimenter understands the deal.

Quote

Your array_test.vi is a very Rube Goldberg solution for something that can be solved with a simple Resize Array node. What you basically do is to format the handle value (which is a pointer to the actual memory pointer) into text, then convert that text back into a pointer (handle) and then resize it with NumericArrayResize() and finally copy in the data into that resized handle. It's equivalent to doing a Resize Array on the original array and then a copy into that resized array, although in LabVIEW you wouldn't resize the handle for this but simply create a new one with that data, most easily by branching the wire but if you really want to, you could also use an autoindexing loop to make it a little bit more Rube Goldberg.

Agree, it looks a little odd, but I didn't want to introduce additional conditional checks, but in the same time I'd like to show how simple operations like read, write and resize of the array work together. They are combined into one Event frame and do their job there. For that I especially studied the source code of NumericArrayResize and made sure, that when the input array's dimSize and the requested new size totalNewSize do match, then NumericArrayResize does nothing and returns. So, no need to check that in LV as it's already checked internally.

As for that twin SPrintf formatting, I didn't find a reliable way in pure LV to pull out the pointer except this one. Maths in extcode.h aren't adapted for 64-bit integers, so I cannot use Max, Abs or something in 64-bit IDE. There exists a trick with _byteswap_uint64 function from msvcr100.dll, but I feel it's not okay to tie to this specific MSVCR version and some additional operations are needed to restore the pointer's byte order back to normal.

Anyway, it's all a "proof-of-concept" thing, so may be changed/played around as desired.

Quote

I also kind of question the datatype of the third parameter for your ReadDCOTransferData and WriteDCOTransferData. As you set it, you treat it as a pointer (most likely passed by reference) but it should probably be the datatype of the control (passed by reference) (in this specific case the easiest would be then to simply configure it as Adapt to Type).

Which parameter do you mean exactly?

int32_t ReadDCOTransferData(uintptr_t viDS, int32_t dcoIdx, uintptr_t *data, int32_t dType, uintptr_t unused, uintptr_t unused);

int32_t WriteDCOTransferData(uintptr_t viDS, int32_t dcoIdx, uintptr_t *data, int32_t dType, uintptr_t unused, uintptr_t unused, int32_t unused);

If you mean data parameter, it is passed as Pointer to Value, because it's a buffer, which the read/write operations are performed on. If you mean dType, it's integer, because it sets the DCO type in VI Data Space (possible values are 0, 1 and 2 as per my BD). I checked all the arguments many times, they seem to fit fine. Last 2 or 3 parameters aren't actually used in the functions, but passed anyways. Maybe it's some relict of earlier LV versions, I did not have a chance to test yet.

Also I tried to reproduce the similar pointer reception on CINs according to this excerpt:

But LV makes a copy for the input data, no matter how hard I try. Did it work as described at all?

Long time didn't see those hackish threads about LV internals on LAVA, so here's something. As always it's strictly experimental and "just for fun", so don't run the things below on your work systems, otherwise you'll easily get LV crashed, your files deleted and your disk reformatted. You're warned.

As some of you maybe know, starting from LV 2017 there exists hidden ArrayMemInfo node, available w/ the scripting. It provides some info on the input array including its data pointer.

ArrayMemInfo.vi

My first discovery was that this pointer is stable and doesn't change until VI is stopped. So its common use case could be the memory copy operations for some third-party DLLs. It's obvious and it's kinda boring usage. Playing with it after some time I discovered that I'm able to not only read but also write to that pointer, able to get its handle, resize the array etc. - read on to see the examples. I could conclude that this way it's possible to manipulate Transfer Buffer or DCO's Transfer Data or whatever it's called. From now on I started to question myself, where does ArrayMemInfo node take that pointer from and could we do the same without this instrument. And here was another big surprise for me - yes, we can! LV familiar CLFNs easily do that, when you configure the input parameter as:

Type = Adapt to Type

Constant = Yes (checked)

Data format = Pointers to Handles

Actually you could set Type to Array and set Data type and Array format to the actual data type of your array, it's not that important. What is important, Constant checkbox should be checked. In this case LV not only passes the native data types, but also does not create the input data copy, providing the pointer/handle to the Transfer Buffer's data instead! I was very surprised, because it was there from LV 2009, but I didn't pay attention to it. I also did not find it to be described somewhere in the official documents.

Knowing that I have replaced ArrayMemInfo with CLFNs in my basic samples.

Here's how we could do read/write of a common numeric indicator:

numeric_test.vi

And this is the same for a common numeric array:

array_test.vi

As you can see, there's a major disadvantage of the whole technique - one should pass the wire to the indicator in a loop to get the indicator updated in time. Nevertheless there exists a pair of VIs, that do the similar and do not require any wires to be connected to a control/indicator. I'm saying about Get Control Values by Index and Set Control Values by Index, first introduced in LV 2013. So I dug these VIs deeper and came to these two internal functions, which are used by the foresaid VIs and ArrayMemInfo behind the scenes.

ReadDCOTransferData:

ReadDCOTransferData.vi

WriteDCOTransferData:

WriteDCOTransferData.vi

As their top-level wrappers, these functions don't require any wires and care about low-level operations, i.e. no need to resize the array w/ NumericArrayResize, manually set its dimensions and actual data w/ MoveBlock etc. Although you don't get absolute control over a control or indicator (e.g., its pointer or handle), you definitely don't need it as all the work is done by ReadDCOTransferData/WriteDCOTransferData. Finally we came to the conclusion that the best has already been implemented. But... if you really want... there's a way to get Operate Data pointer for a generic control/indicator.

numeric_test_ncg.vi

array_test_ncg.vi

However I'd like to notice, that this technique contains a lot of pitfalls besides shown low-level manipulations. Front Panel object doesn't get updated at all, for example, so it's required to redraw it manually w/ Invalidate method (or click on the object or hide-show the window). Local variables of the object don't get updated too, even when the object is invalidated. I'm sure, there are other things not done, which usually LV does on normal read/write. So, these diagrams should be perceived as the concept only.

In the end I'd say that I hardly imagine, where I could really use any of these non-standard r/w methods. Surely in LV I will continue to use native Get Control Values by Index and Set Control Values by Index VIs, because they do everything, what I want. There might be one use-case though, when communicating with some external code. It would be easier to write debug (or any other) data from a custom DLL into LV's indicator directly, not using PostLVUserEvent or DbgPrintf. Extcode.h doesn't expose many APIs to the user, so calling ReadDCOTransferData/WriteDCOTransferData might be preferred inside the library. But I'm not going to check/test it now, because I haven't written DLLs for LabVIEW for a while.