Ok, I've been testing the patch you sent me. It seems to solve the
problem when opening files. But, unfortunately I'm getting confused with
some strange behavior in one of my tests:
I've just compiled the "sftp_nonblock.c" example, and tried it with an
80 MB text file in a remote Linux server. I've commented the line where
the output is printed in the stdout to avoid performance penalties. The
problem is that using the compression flag takes more time that without
the using the compression flag! I've tried several times, and all the
tests returned the same results: about 120 seconds with the compression
flag, and about 90 seconds without it. I've tried with buffers of 1K,
16K and 32K.
The compression is enabled at the OpenSSH server side, if I try the
transfer with an SFTP client, like the sftp command, then it takes about
10 or 20 seconds.
Ezequiel
El 01/09/2010 12:23 a.m., Ezequiel Ruiz escribió:
It seams you find the problem, I'll try your patch and give you some
feedback.
Thanks a lot for your research, nice job!
Regards,
Ezequiel
El 31/08/2010 08:03 p.m., TJ Saunders escribió:
[libssh2] 2.676521 Failure Event: -29 - compress/decompression failure
[libssh2] 2.676537 Failure Event: -1 - transport read
[libssh2] 2.676547 Failure Event: -21 - Read part of packet
[libssh2] 2.676558 Failure Event: -1 - Timeout waiting for status message
And there, I think, is a sign of the issue:
[libssh2] 2.676521 Failure Event: -29 - compress/decompression failure
The question is, though: why did the decompression fail? And why does it
fail at this point, when reading file data (as opposed to earlier, when
decompressing the rest of the messages on that subsystem/channel)?
I'll keep poking at this to see if I can find any additional clues...
I patched the src/comp.c file (see first attached patch) to try to get
more information; in my particular use case, I saw:
[libssh2] 3.316516 Transport: unhandled zlib error -5
According to the zlib.h header, a return value of -5 is Z_BUF_ERROR, which
indicates (in the case of calling the inflate() zlib function) that there
isn't enough space in the output buffer for the decompressor to make any
progress.
That being the case, I tried increasing the size of the output buffers
used for zlib decompression (see second attached patch), and things looked
much better. I'm not sure of just what the best increase in output buffer
sizes/growth factors should be, though.
Cheers,
TJ
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We are ashamed of everything that is real about us; ashamed of
ourselves, of our relatives, of our incomes, of our accents, of
our opinions, of our experience, just as we are ashamed of our
naked skins.
-George Bernard Shaw
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
libssh2-develhttp://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
