MoveFileWithProgress problem from kernel32.dll

[Houmoller]

Hi.

I have a small program that help me move som specific files to specific folders. 

The filesize vary from 400MB to arround 10GB, they are transfered from a local disk to a netværk disk (mounted as z: drive)

 

Currently I am using the MoveFileA function from the kernel32.dll, and it works fine. It requires two arguments, path from and path to, easy peasy. 

 

My problem/request is that I would like to have som sort of progress bar when moving large files. So I would like to monitor the size of the copied data. 

For this I think I should use the function MoveFileWithProgressA also from the kernel32.dll.

 

from microsoft: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365242(v=vs.85).aspx

I get:

BOOL WINAPI MoveFileWithProgress(  _In_	  LPCTSTR lpExistingFileName,  _In_opt_  LPCTSTR lpNewFileName,  _In_opt_  LPPROGRESS_ROUTINE lpProgressRoutine,  _In_opt_  LPVOID lpData,  _In_	  DWORD dwFlags);

where we again have  path from and to, 

then (arg3+4) I should give a pointer to a CopyProgressRoutine callback function - and some arguments for this function.

 

My problem is, that I dont know what that means, and further more, I dont know how to get a file size number or something from this.

 

I hope my problem/request is clear, I would like to have som help in understanding the CopyProgressRoutine callback function, and how to use it.

 

(Windows 7 64bit)

 

Regards

Jørgen Houmøller

 

 

[ShaunR]

Hi.

I have a small program that help me move som specific files to specific folders. 

The filesize vary from 400MB to arround 10GB, they are transfered from a local disk to a netværk disk (mounted as z: drive)

 

Currently I am using the MoveFileA function from the kernel32.dll, and it works fine. It requires two arguments, path from and path to, easy peasy. 

 

My problem/request is that I would like to have som sort of progress bar when moving large files. So I would like to monitor the size of the copied data. 

For this I think I should use the function MoveFileWithProgressA also from the kernel32.dll.

 

from microsoft: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365242(v=vs.85).aspx

I get:

BOOL WINAPI MoveFileWithProgress(  _In_	  LPCTSTR lpExistingFileName,  _In_opt_  LPCTSTR lpNewFileName,  _In_opt_  LPPROGRESS_ROUTINE lpProgressRoutine,  _In_opt_  LPVOID lpData,  _In_	  DWORD dwFlags);

where we again have  path from and to, 

then (arg3+4) I should give a pointer to a CopyProgressRoutine callback function - and some arguments for this function.

 

My problem is, that I dont know what that means, and further more, I dont know how to get a file size number or something from this.

 

I hope my problem/request is clear, I would like to have som help in understanding the CopyProgressRoutine callback function, and how to use it.

 

(Windows 7 64bit)

 

Regards

Jørgen Houmøller

 

Labview has no way to create callbacks that can be called from external code (the exception being some .NET functions). You need to create a dll wrapper that supplies the callback function and proxy it via a LV prototype (e.g. an event using PostLVUserEvent) which can then be used to get the callback data.

 

If there is an equivalent .NET function, you maybe able to use the callback primitive in the .NET pallet to interface to it. 

Edited February 26, 2014 by ShaunR

[Phillip Brooks]

Labview has no way to create callbacks that can be called from external code (the exception being some .NET functions). You need to create a dll wrapper that supplies the callback function and proxy it via a LV prototype (e.g. an event using PostLVUserEvent) which can then be used to get the callback data.

 

If there is an equivalent .NET function, you maybe able to use the callback primitive in the .NET pallet to interface to it. 

 

found on stackoverflow: ( http://stackoverflow.com/questions/187768/can-i-show-file-copy-progress-using-fileinfo-copyto-in-net )

 

 

FileInfo.CopyTo is basically a wrapper around the Win32 API call "CopyFile" in the kernel32.dll. This method does not support progress callback.

 

However, the CopyFileEx method does, and you can write your own .NET wrapper around it in a few minutes, like it is described here: 

 

http://www.pinvoke.net/default.aspx/kernel32.CopyFileEx

[ShaunR]

found on stackoverflow: ( http://stackoverflow.com/questions/187768/can-i-show-file-copy-progress-using-fileinfo-copyto-in-net )

 

So that's a no then-there is no equivalent (I also couldn't find one listed). You have to write a wrapper whether your choice is DLL or .NET assembly.

[SDietrich]

I agree, a wrapper is necessary. But you can write that wrapper with LabVIEW! Here is how it works in general:

• Write a VI to be called back.
• Create a build specification as a DLL, containing that VI, and build it.
• Load the DLL from your application via kernel32.dll/LoadLibraryA().
• Get the address of your callback VI within the DLL via kernel32.dll/GetProcAddress().
• Pass that address to the function wanting a callback.
• when finished, unload the DLL via kernel32.dll/FreeLibrary().

Please note, the loaded DLL runs in a separate application instance. So you can not communicate via queues or events. You can use IPC mechanisms like UDP, but actually the DLL is loaded in the same process, so other methods may be available, too.

[Rolf Kalbermatter]

I agree, a wrapper is necessary. But you can write that wrapper with LabVIEW! Here is how it works in general:

• Write a VI to be called back.
• Create a build specification as a DLL, containing that VI, and build it.
• Load the DLL from your application via kernel32.dll/LoadLibraryA().
• Get the address of your callback VI within the DLL via kernel32.dll/GetProcAddress().
• Pass that address to the function wanting a callback.
• when finished, unload the DLL via kernel32.dll/FreeLibrary().

Please note, the loaded DLL runs in a separate application instance. So you can not communicate via queues or events. You can use IPC mechanisms like UDP, but actually the DLL is loaded in the same process, so other methods may be available, too.

 

While that is generally true it is IMHO a pain in the ass with no real advantage other than not requiring you do write a little C code and run it though a C compiler. Of course for someone who has no C knowledge, this option is all that is available to them, save from hiring a C programmer, but it is a bad choice for a lot of reasons.

 

First, you need to know quite a bit about C programming anyhow to be able to make this work reliable.

Second, the DLL will for all practical purposes not only run in a separate application instance but many times in a separate process. When you upgrade your LabVIEW code to a new LabVIEW version, the DLL needs to either be recompiled too every time or it will run in a separate LabVIEW runtime process that has to be the same version as in what the DLL was created. So unless you upgrade your DLL too, you will have to remember to install the runtime version for your DLL and the one for your application. Consider an app using more than one such callback functionality and you easily end up having to install several LabVIEW runtime versions after some progressive development of your app.

 

And moving platforms (eg, Windows 32 bit to 64 bit) will most likely have every other user of your callback solution stumped, since the LabVIEW created DLL is somewhat unintuitive for most casual LabVIEW users (and illogical for more advanced programmers).

[SDietrich]

RolfK, that is an interesting point of view. I agree, the only advantage is to stay within LabVIEW the whole time and every developer may decide, how big of an advantage that is for him. For me it is a big one because otherwise I would have to document and maintain a complete second toolchain.

 

However, I think the following is not true:

Second, the DLL will for all practical purposes not only run in a separate application instance but many times in a separate process.

 

"LoadLibrary() loads the DLL into the calling process", which is our process. That is also the process, into which the original third-party DLL is loaded, so there are no other processes involved. The whole concept of callbacks would not work if there were multiple processes, because one process can not call a subroutine within another process.

 

Unfortunately I also doubt the statement about a separate "LabVIEW runtime process", because I never heard of such a thing and found no reference to it, neither on NI nor on the internet. As I understand it, a runtime-engine is a library that is used by a process (our LabVIEW executable) not a process itself (e.g. a server process).

 

The interesting question remains: What happens, when a process uses two different LabVIEW-runtime-engines?

This obviously would be the case, if I compile the EXE in LV2013 and the DLL in LV2012. But it would also be the case if a C program uses two LabVIEW-DLLs compile with different versions of LabVIEW. And because the latter surely must work, I expect the former also to. Nonetheless I will certainly test that in the next days and post my findings here.

 

When moving platforms one has to recompile no matter what method is used. After all, a C wrapper is platform dependent and last but not least, the original third party DLL is, too.

 

One interesting thought to close with:

Why do we even have to create a wrapper-DLL? A nice feature would be to have LabVIEW "export" certain VIs in an EXE, too. This way one could just use the GetProcAddress() without loading a DLL at all. Maybe this way the callback VI could even be run within the same application instance?

 

[ShaunR]

The interesting question remains: What happens, when a process uses two different LabVIEW-runtime-engines?

 

 

National Instruments also recommends that if you have more than one LabVIEW-built DLL in any program, that you build them in the same version of LabVIEW. If you use different versions, passing information between the DLLs is more difficult because the only common memory space is the calling language memory space. In contrast, if you build them in the same version, all DLLs use the same instance of the LabVIEW Run-Time Engine, and share a common memory space.

 

If all DLLs use the same version of the LabVIEW Run-Time Engine and you load the DLLs dynamically, National Instruments recommends you dynamically load the LabVIEW Run-Time Engine before you load any of the LabVIEW-built DLLs by loading the lvrt.dll file. Loading the LabVIEW Run-Time Engine before you load the DLLs reduces overhead by preventing the Run-Time Engine from loading and unloading when the DLLs are loaded and unloaded from memory.

 

Integrating LabVIEW code

[Rolf Kalbermatter]

RolfK, that is an interesting point of view. I agree, the only advantage is to stay within LabVIEW the whole time and every developer may decide, how big of an advantage that is for him. For me it is a big one because otherwise I would have to document and maintain a complete second toolchain.

 

However, I think the following is not true:

 

"LoadLibrary() loads the DLL into the calling process", which is our process. That is also the process, into which the original third-party DLL is loaded, so there are no other processes involved. The whole concept of callbacks would not work if there were multiple processes, because one process can not call a subroutine within another process.

 

Unfortunately I also doubt the statement about a separate "LabVIEW runtime process", because I never heard of such a thing and found no reference to it, neither on NI nor on the internet. As I understand it, a runtime-engine is a library that is used by a process (our LabVIEW executable) not a process itself (e.g. a server process).

 

The interesting question remains: What happens, when a process uses two different LabVIEW-runtime-engines?

This obviously would be the case, if I compile the EXE in LV2013 and the DLL in LV2012. But it would also be the case if a C program uses two LabVIEW-DLLs compile with different versions of LabVIEW. And because the latter surely must work, I expect the former also to. Nonetheless I will certainly test that in the next days and post my findings here.

 

When moving platforms one has to recompile no matter what method is used. After all, a C wrapper is platform dependent and last but not least, the original third party DLL is, too.

 

One interesting thought to close with:

Why do we even have to create a wrapper-DLL? A nice feature would be to have LabVIEW "export" certain VIs in an EXE, too. This way one could just use the GetProcAddress() without loading a DLL at all. Maybe this way the callback VI could even be run within the same application instance?

 

Well there is a chance that it is not really an out of process invocation. But a LabVIEW DLL consists of 3 things.

 

1) a stub loader

2) a C compiled wrapper for each function

3) the compiled LabVIEW VI (and subVIs) for each function

 

The stub loader is responsible to locate the right LabVIEW runtime and load it but does skip that if it determines that the current calling process already contains a compatible LabVIEW runtime of the same version. This can be also the runtime environment of the LabVIEW IDE. This not only speeds up the initialization of the DLL but also allows more efficient passing of function parameters since otherwise all parameters need to be copied from the calling environment into the DLL environment, as Shaun has quoted already.

 

So while the LabVIEW DLL is loaded into the process, it is not entirely sure if the LabVIEW runtime is also loaded into the same process if the stub loader determines that the current LabVIEW runtime kernel can not execute the VIs in the DLL because of version differences. While possible it certainly poses some difficulties in terms of heap management but out of process invocation would also cause its own kind of challenges.

[SDietrich]

So while the LabVIEW DLL is loaded into the process, it is not entirely sure if the LabVIEW runtime is also loaded into the same process if the stub loader determines that the current LabVIEW runtime kernel can not execute the VIs in the DLL because of version differences.

 

And that is exactly what I wanted to find out: What does the stub loader do if it determines there is already a LV-runtime loaded, but with a different version? Does it crash or abort?

The answer is as expected: It simply loads the correct runtime version and they coexist side by side.

 

I attached a little program to show this. The EXE is compiled with LV2009 and it uses a DLL that is compiled with LV2013. When the program starts two front panels are shown, one from the startup VI in the executable and one from the called VI within the DLL. Use the "Help->About" in each of the front panels to verify.

 

Of course if the runtime (or IDE) for 2009 and 2013 is not installed it won't work.

Test2009with2013dll.zip