I've tried using the --scan-all-first option on a mirror script. I can reproduce with this single command on both 32bit and 64bit Cygwin:
--8<---------------cut here---------------start------------->8--- set mirror:parallel-directories on mirror -e -c --scan-all-first --delete-first --depth-first --use-cache \ -X '*' -i '^(x86|x86_64|noarch)/' -X md5.sum -x \ '/(Cygwin64|cygwin32(-[^/]*)?)/' \ --script=cygwin.lftp \ http://ftp-stud.hs-esslingen.de/pub/Mirrors/sources.redhat.com/cygwin/ \ /mnt/mirror/cygwin --8<---------------cut here---------------end--------------->8--- While it seems to start out massively faster (which is what I hoped for) than without this option, in short time a stack corruption occurs: --8<---------------cut here---------------start------------->8--- assertion "stack_ptr>0" failed: file "/mnt/share/packages/lftp.x86_64/src/lftp-4.6.1/src/SMTask.cc", line 156, function: static void SMTask::Leave(SMTask*) --8<---------------cut here---------------end--------------->8--- Running under gdb didn't give a better clue since all five threads are reported as having a corrupt stack. I've increased the stack and heap sizes to no avail. Running the same command on Linux works without an error, looking at the echo and the network utilization it would seem that the crash on Cygwin happens when the first reconnect delay kicks in. Switching mirror:parallel-directories off keeps lftp from crashing, but also makes the operation much slower. Based on that observation I've limited net:connection-limit instead to 256 by binary search. That produces a short peak of high network traffic (mostly upstream!) at the beginning, then drops to almost zero as the limit is reached. Independently of whether lftp crashes, hitting a server with so many connections at once strikes me as an odd thing to do, so the default for these connection parameters might need to be tuned a bit. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada _______________________________________________ lftp-devel mailing list lftp-devel@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp-devel
