Thanks for programming 101. I'll keep your advice when my goal will be optimizing my current work, which is not currently the case. I do not simply want something to work here. I am fully capable of finding workarounds whenever I need/want them. I'll leave the 'I do not care how it works as long as it works' motto to business-related goals ;o)
I need to understand the PHP/Nginx communication. And having searched for it on the Web showed me a lot of unsatisfaying/dirty workarounds, no real solution/explanation. If anyone could enlighten me on those Nginx timeouts, I'd be more than glad! --- *B. R.* On Sun, May 26, 2013 at 10:44 PM, Steve Holdoway <st...@greengecko.co.nz>wrote: > OK, I leave you to it. > > However, asynchronously spawning subprocesses *will* allow you to > parallelise the process. I'd call it design, rather than a workaround, > but there you go (: > > Steve > On Sun, 2013-05-26 at 22:38 -0400, B.R. wrote: > > One way or another, even if an external script is called, PHP will > > need to wait for the scripts completion, making the parallelization > > impossible or at least useless (since, to wait for a return code of an > > external script is still blocking). > > > > > > I am not trying to find a workaround, I need to know how the > > fastcgi_reload_timeout works (if I understood it properly), if I > > properly disabled PHP buffering for my example case and how eventually > > to control those timeouts. > > > > I'd like to address the central problem here, not closing my eyes on > > it. > > > > > > --- > > B. R. > > > > > > On Sun, May 26, 2013 at 10:24 PM, Steve Holdoway > > <st...@greengecko.co.nz> wrote: > > Surely, you're still serialising the transfer with a loop? > > > > On Sun, 2013-05-26 at 22:11 -0400, B.R. wrote: > > > Thanks for your answer. > > > > > > I didn't go into specifics because my problem doesn't rely > > at the > > > application-level logic. > > > > > > What you describe is what my script does already. > > > > > > > > > However in this particular case I have 16 files weighting > > each a few > > > MB which need to be transfered back at once. > > > > > > > > > PHP allocates 30s for each loop turn (far enough to copy the > > file + > > > echo some output message about successes/failed completion). > > > > > > Nginx cuts the execution avec fastcgi_read_timeout time even > > with my > > > efforts to cut down any buffering on PHP side (thus forcing > > the output > > > to be sent to Nginx to reinitialize the timeout counter). > > > > > > That Nginx action is the center of my attention right now. > > How can I > > > get read of it in a scalable fashion (ie no > > fastcgi_read_time = > > > 9999999) ? > > > --- > > > B. R. > > > > > > > > > > > > > > > On Sun, May 26, 2013 at 9:46 PM, Steve Holdoway > > > <st...@greengecko.co.nz> wrote: > > > Write a script that lists the remote files, then > > checks for > > > the > > > existence of the file locally, and copy it if it > > doesn't > > > exist? That way > > > no internal loop is used - use a different exit code > > to note > > > whether > > > there was one copied, or there were none ready. > > > > > > That way you scale for a single file transfer. > > There's nothing > > > to be > > > gained from looping internally - well > > performance-wise that > > > is. > > > > > > Steve > > > > > > On Sun, 2013-05-26 at 21:31 -0400, B.R. wrote: > > > > No ideas? > > > > > > > > --- > > > > B. R. > > > > > > > > > > > > On Sat, May 25, 2013 at 1:01 PM, B.R. > > > <reallfqq-ng...@yahoo.fr> wrote: > > > > Hello, > > > > > > > > > > > > I am trying to understand how > > fastcgi_read_timout > > > works in > > > > Nginx. > > > > > > > > > > > > Here is what I wanna do: > > > > > > > > I list files (few MB each) on a distant > > place which > > > I copy one > > > > by one (loop) on the local disk through > > PHP. > > > > > > > > I do not know the amount of files I need > > to copy, > > > thus I do > > > > not know the total amount of time I need > > for the > > > script to > > > > finish its execution. What I know is that > > I can > > > ensure is a > > > > processing time limit per file. > > > > > > > > I would like my script not to be > > forcefully > > > interrupted by > > > > either sides (PHP or Nginx) before > > completion. > > > > > > > > > > > > > > > > What I did so far: > > > > > > > > - PHP has a 'max_execution_time' of 30s > > (default?). > > > In the > > > > loop copying files, I use the > > set_time_limit() > > > procedure to > > > > reinitialize the limit before each file > > copy, hence > > > each file > > > > processing has 30s to go: way enough! > > > > > > > > > > > > - The problem seems to lie on the Nginx > > side, with > > > the > > > > 'fastcgi_read_timeout' configuration > > entry. > > > > > > > > I can't ensure what maximum time I need, > > and I would > > > like not > > > > to use way-off values such as 2 weeks or 1 > > year > > > there. ;o) > > > > > > > > What I understood from the documentation > > is that the > > > timeout > > > > is reinitialized after a successful read: > > am I > > > right? > > > > > > > > > > > > The challenge is now to cut any buffering > > occurring > > > on the PHP > > > > side and let Nginx manage it (since the > > buffering > > > will occur > > > > after content is being read from the > > backend). Here > > > is what I > > > > did: > > > > > > > > * PHP's zlib.output_compression is > > deactivated by > > > default in > > > > PHP > > > > > > > > * I deactivated PHP's output_buffering > > (default is > > > 4096 bytes) > > > > > > > > * I am using the PHP flush() procedure at > > the end of > > > each > > > > iteration of the copying loop, after a > > message is > > > written to > > > > the output > > > > > > > > > > > > > > > > Current state: > > > > > > > > * The script seems to still be cut after > > the > > > expiration of the > > > > 'fastcgi_read_timout' limit (confirmed by > > the error > > > log entry > > > > 'upstream timed out (110: Connection timed > > out) > > > while reading > > > > upstream') > > > > > > > > * The PHP loop is entered several times > > since > > > multiple files > > > > have been copied > > > > > > > > * The output sent to the browser is cut > > before any > > > output from > > > > the loop appears > > > > > > > > > > > > It seems that there is still some unwanted > > buffering > > > on the > > > > PHP side. > > > > > > > > I also note that the PHP's flush() > > procedure doesn't > > > seem to > > > > work since the output in the browser > > doesn't contain > > > any > > > > message written after eahc file copy. > > > > > > > > > > > > Am I misunderstanding something about > > Nginx here > > > (especially > > > > about the 'fastcgi_read_timeout' > > directive)? > > > > > > > > Have you any intel/piece of advice on hte > > matter? > > > > > > > > Thanks, > > > > > > > > --- > > > > B. R. > > > > > > > > > > > > > > > _______________________________________________ > > > > nginx mailing list > > > > nginx@nginx.org > > > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > > > -- > > > Steve Holdoway BSc(Hons) MNZCS > > <st...@greengecko.co.nz> > > > http://www.greengecko.co.nz > > > MSN: st...@greengecko.co.nz > > > Skype: sholdowa > > > > > > _______________________________________________ > > > nginx mailing list > > > nginx@nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > > > > > > _______________________________________________ > > > nginx mailing list > > > nginx@nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > -- > > Steve Holdoway BSc(Hons) MNZCS <st...@greengecko.co.nz> > > http://www.greengecko.co.nz > > MSN: st...@greengecko.co.nz > > Skype: sholdowa > > > > _______________________________________________ > > nginx mailing list > > nginx@nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > > > > > _______________________________________________ > > nginx mailing list > > nginx@nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx > > -- > Steve Holdoway BSc(Hons) MNZCS <st...@greengecko.co.nz> > http://www.greengecko.co.nz > MSN: st...@greengecko.co.nz > Skype: sholdowa > > _______________________________________________ > nginx mailing list > nginx@nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx >
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx