Copy File from network.

[Chris_S]

I am currently using the File Copy.VI trying to copy a folder from a network location. It is a remote server location and the copy can take upwards of 3 minutes for a 200k file.

I changed the program to use System EXEC.vi and execute XCOPY, which is significantly faster.

Does anyone have another way to accomplish this? Possibly a way to do this that will have some type of progress indication?

Any help is appreciated. Thanks

Chris

[asbo]

File Copy.VI? Do you mean the Copy node from the Advanced File Functions palette?

If you want to have a progress indicator, you'll like have to roll your own copy routine.

[Chris_S]

File Copy.VI? Do you mean the Copy node from the Advanced File Functions palette? Yes

If you want to have a progress indicator, you'll like have to roll your own copy routine. Correct. I’m looking for help on how to make an alternate copy routine.

-Red Text- My responses

Chris

[asbo]

Well, at its core, copying a file is just opening two different files, then read a chunk from one and writing it to the other. The chunk size can have a huge effect on speed, but I don't have any particular advice for network transfers. Start on the order of bytes and do some benchmarking to see what works best in your situation. Or you could get really ambitious and code it to be adaptive You should only need Open/Read/Write/Close nodes and those are all in the file palette.

To implement a progress indicator (easily) you could use a functional global, storing in (bytes written)/(size of source file) and read it back where ever you'd like.

[jdunham]

I am currently using the File Copy.VI trying to copy a folder from a network location. It is a remote server location and the copy can take upwards of 3 minutes for a 200k file.

I changed the program to use System EXEC.vi and execute XCOPY, which is significantly faster.

Does anyone have another way to accomplish this? Possibly a way to do this that will have some type of progress indication?

It was suggested that you roll your own, but you will lose any metadata (modification time, permissions, creator, etc.) which may be attached to the file. It also seems like a waste of your time to rewrite components of your OS. What's wrong with System Exec.vi and invoke XCOPY if it's working for you?

If you really need to write something, i think there is a network queue library somewhere around here. That sounds like the easiest way to send a file across a network, as a queue of strings. The first element in thEdite queue could be the number of chunks to expect, and maybe any other metadata you want to send, like file timestamp (though then you would have to find some API function to fix the timestamp to match the other system.

Edited May 25, 2010 by jdunham

[Tim_S]

-Red Text- My responses

Chris

I agree with asbo... Your best bet would be to open a source and create a target file as binary and to read/write out chunks, updating a progress bar every N-th itteration of your loop. There is a JKI progress bar (I believe it's in the Code Repository) that may help you out with that portion.

What's wrong with System Exec.vi and invoke XCOPY if it's working for you?

It is a remote server location and the copy can take upwards of 3 minutes for a 200k file.

Three minutes with the user wondering if the program is locked. My experience is users do not have much patience and have a tendency to kill programs that they think aren't responding.

[Ton Plomp]

If you are going to make your own copy VI, the following should be the fastest:

1. Read original file size
2. Create target file
3. Set target file size to original file size
4. Read chunks in multiples of Sectors (1024?)
5. write these chunks to the target file

(this is only neccesary if you want a queue).

As far as I'm concerned there is no copy.vi, there is only a copy primitive (or am I absolutly mistaken)

Ton

[Chris_S]

If you are going to make your own copy VI, the following should be the fastest:

1. Read original file size
2. Create target file
3. Set target file size to original file size
4. Read chunks in multiples of Sectors (1024?)
5. write these chunks to the target file

(this is only neccesary if you want a queue).

As far as I'm concerned there is no copy.vi, there is only a copy primitive (or am I absolutly mistaken)

Ton

Is this how the Copy node from the Advanced File Functions palette does it, only without a progress update?

[jdunham]

Is this how the Copy node from the Advanced File Functions palette does it, only without a progress update?

I would be shocked if the Copy primitive used something other than a direct OS/WinAPI call. I still think rewriting OS functions is a crazy thing to do, but I understand the need to keep your users from pulling the plug. When the file is getting transferred does anything appear on the other end? To wit, you may be able to:

1. read the remote file size (the source file)

2. Use the Copy File primitive to start the transfer to a new target file

3. In parallel, start monitoring the target file size, displaying a progress bar for the percentage transferred.

4. Terminate the progress bar popup when the target is at 100% size.

The great part is that if your code has bugs, it shouldn't affect the data, just the user experience.

[asbo]

I've seen other applications do this, but never tried myself: per this stackoverflow posting you might be able to invoke the standard Windows copy dialog ... if you're willing to use .NET.

[Rolf Kalbermatter]

Is this how the Copy node from the Advanced File Functions palette does it, only without a progress update?

No the Copy function calls more or less directly the Windows API CopyFile() from kernel32.dll. So it is likely that the network provider for your shared directory has some problems with the underlaying network transport mechanisme. I would think it points towards some badly configured networking parameter which might be hard to track down as Windows tends to try to hide all kinds of things from the user nowadays, supposedly to not confuse the average user, for me sometimes rather frustrating as one needs to get all kinds of extra tools and utilities to get at the cause of such problems.