Jump to content

Slow FTP transfer to LV RT


Recommended Posts

Hello,

since this is my first post here I start with a welcome to everybody!

Now for my problem: In our project for LV RT we use a LV framework that requires me to build the project as source distribution and save it to the PXI controller where it is loaded on next boot-up. So essentially I have now a folder containing ~2500 VIs totalling to ~50 MByte and I want to load it to a 8133 controller via FTP. The problem is that the transfer starts at acceptable speed (>150 KByte/s) but becomes incredibly slow after the first ~1000 files are transferred (< 5KByte/s). I have tried up to now with three different FTP clients (Filezilla, WinSCP and the FTP client in LV MAX) and from two different machines. It seems to be connected to the number of files in the target directory. I managed to upload the source distribution by manually distributing it to four target directories but now again moving the contents of the four directories to one on the server with WinSCP takes ages. Even if this would work doing this manually is really no solution as it is error-prone and way too slow.

Am I missing something here or could the number of files in the target directory really be such a performance problem. Any recommendations?

Best regards

flintstone

Link to comment

Am I missing something here or could the number of files in the target directory really be such a performance problem. Any recommendations?

That's likely the problem. Filesystems in general do not deal well with huge numbers of small files. Also, there will be a substantial amount of overhead transferring that number of files over ftp since each file is a separate transaction.

Link to comment

Hi Flinstone,

I have seen real time (although on VxWorks, not PXI) struggle with large directories before, if you can break it down this is probably best.

The only other thing I could suggest is maybe trying the file transfer through the web interface. If it was protocol related then this uses a different technique (something over HTTP) but if it is the number of files you will probably see the same thing again.

Cheers,

James

On trying to find a recommmended number I found this: http://digital.ni.com/public.nsf/allkb/C9B0A1443BF1C3398625760A004DB976?OpenDocument seems 'significant' means a few hundred files

Link to comment

Thank you James for this link, this seems to be exactly the problem I am facing. During FTP upload it looks like the list command starts to time out at some point. So I guess I'll have to start thinking about reducing the number of files, e.g. by using libraries.

Regards

Flintstone

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.