(php cli) FTP PUT fails on large file

Beginners - Learning PHP
#1

I have written a CLI app in php. Apache is NOT involved in any way.

The app runs on Linux ubuntu 13.10 64bit. It will FTP a bunch of files, one at a time to another Linux ubuntu server.

php gives me a return code saying the FTP PUT failed on the large file. I realize that you want to look at the source code but I ask you to ignore it for a moment.

Why no source? Read my comments first:

  • I have tried many sample programs off the net. They all give the same results
  • If I FTP to a FTP server in the same building, the 900meg file transfers 100% ok (less than 1minute to transfer)
  • If I FTP to a FTP server that is on the Internet, the 900meg file takes 45 minutes to transfer and PHP gives a failure return code.
  • Even though PHP says the file transfer failed, I can gunzip the file and it appears to be fully transferred.
  • We are both behind a NAT firewall and I am using PASV (as suggested in EVERY script that I have read)
  • PHP 5.5.3-1ubuntu2.2 (cli) (built: Feb 28 2014 20:06:05)

I have some high level comments:

  • Reading the source will not help because I think it is not an error in my script.
  • Are there known bugs in FTP_PUT with PHP 5.5.3 ?
  • If I move the server to the same subnet, it works. Over the slow internet, it falsely reports an error. Can this be due to the length of time that has elapsed? The max time limits of php seem to only apply to web apps, not CLI apps.

I think I am missing something really basic here.

FYI: It will FTP 4 files: 5megs, 11megs, 400megs, 900megs. All transfer ok based on file compares but the return code on the 900meg files says failed.

Any high level comments?

#2

I have been able to determine that it is a firewall issue. The control port is timing out. If I force a firewall rule the FTP transfers successfully with the proper status.

Are there any keep-alive settings for FTP or CURL? I did not see any but I thought I would ask.

#3

Do you need to use FTP? I’d rather use ssh/scp, or just vpn over and mount a shared folder. FTP is just considered to be old and insecure…

You can find curl timeout settings here
http://www.php.net/manual/en/function.curl-setopt.php

CURLOPT_CONNECTTIMEOUT_MS etc

#4

FTP is convenient because everything supports it, even those cheap WD NAS boxes. If FTP is a no-go, I would simply use rsync.

The CURL_OPT settings seem to be connect or data timeout related. The service always connects and sends data ok. With the 2nd IP stream (the reversed flow/control stream) being idle for a long period, the firewall closes off the connection. We do not want a timeout, we want the reverse of a timeout AKA keepalive. With ssh, I could configure a NOOP (no operation keepalive) but I do not see it with FTP.

Sponsor our Newsletter | Privacy Policy | Terms of Service