Hello! I have finally found what's going on with upload over fish, which doesn't work reliably with large files (100k and more).
The problem is that "dd" (GNU fileutils-4.1) doesn't reread a block if read() returns less bytes than requested. Unfortunately, this happens when dd gets data from ssh (OpenSSH 3.4, linux 2.4.19-rc1, glibc 2.2.4). I believe that at ~100k dd just gets all the data that ssh has available, and interprets the end of data as a "partial block". The solution would be to use blocks 1 byte long - read() is guaranteed to return at least one byte it it's successful. But this approach would use too many resources on the remote machine, since dd actually would call read() and write() for every byte. Using a bigger obs (output block size) helps, but not much, since we still have one read() per byte. I don't know of any widespread and portable alternative to dd that would read a certain number of bytes and then quit (unlike e.g. gzip, that will search for the next file), that would reread partial blocks (unlike dd), and that would not require non-trivial software to decode it's output (od would need sed or awk capable of writing binary zeroes). Maybe I'm missing a very simple solution. The largest block that works for me is 256 bytes long. That's the patch I'm applying now, and it fixes fish upload for me, but better ideas are welcome. There is no guarantee that dd won't get partial reads with 256 bytes blocks on other operating systems or with outher versions of libc. =================================== --- ChangeLog +++ ChangeLog @@ -2,2 +2,5 @@ + * fish.c (file_store): Reduce input block size to 256 bytes, + otherwise dd gets partial blocks. + * ftpfs.c (login_server) [HSC_PROXY]: Fix crash. --- fish.c +++ fish.c @@ -486,22 +486,22 @@ file_store(vfs *me, vfs_s_fh *fh, char * print_vfs_message(_("fish: store %s: sending command..."), name ); /* * FIXME: Limit size to unsigned long for now. - * Files longer than 4096 * ULONG_MAX are not supported. + * Files longer than 256 * ULONG_MAX are not supported. */ if (command (me, super, WAIT_REPLY, "#STOR %lu /%s\n" "> \"/%s\"\n" "echo '### 001'\n" "(\n" - "dd bs=4096 count=%lu\n" + "dd ibs=256 obs=4096 count=%lu\n" "dd bs=%lu count=1\n" ") 2>/dev/null | (\n" "cat > \"/%s\"\n" "cat > /dev/null\n" "); echo '### 200'\n", (unsigned long) s.st_size, name, name, - (unsigned long) (s.st_size >> 12), - ((unsigned long) s.st_size) & (4096 - 1), name) + (unsigned long) (s.st_size >> 8), + ((unsigned long) s.st_size) & (256 - 1), name) != PRELIM) ERRNOR(E_REMOTE, -1); =================================== -- Regards, Pavel Roskin _______________________________________________ Mc-devel mailing list [EMAIL PROTECTED] http://mail.gnome.org/mailman/listinfo/mc-devel