Re: [Samba] Samba over PVFS: Corrupted Data
On Tue, Dec 20, 2005 at 08:08:24AM -0800, Jeremy Allison wrote: Ok, what you need is to create files locally on the windows client of 512 bytes, 1024 bytes etc. and then just copy them onto the Samba drive. Check what the smallest size is that the first file becomes corrupted. That will show the simplest problem that reproduces the issue. At *that* point then send in logs etc. Without an *exact* method of reproducing this with detailed information on what is going wrong (ie. the last 20 bytes of a 512 byte file get corrupted, but a 256 byte file is fine - here is the comparitive copy) it will be a massive investment of time to track this down, that we currently don't have. You're going to have to do more work on this I'm afraid. OK, I made a bunch of files consisting of the string 123456789abcdef\n where \n is the UNIX newline (ASCII 0xA) and performed the binary search. Files 64KB (=65536 bytes) and higher are corrupted and files under 64KB are fine. In the attached tarball area the results of my tests for two files, one 64KB in length and one 65535B (64K-1B) in length. In more detail: pvfs-detailed-test/log.andy-ibm.64k pvfs-detailed-test/log.andy-ibm.65535 - samba logs with the log level = 10 pvfs-detailed-test/64k-orig.txt pvfs-detailed-test/65535-orig.txt - original files pvfs-detailed-test/64k-share.txt pvfs-detailed-test/65535-share.txt - files as put on the share by our Windows client. The 64k file from the share starts to differ from the original file at offset 0xF000. The file from the share is filled with 0 bytes. pvfs-detailed-test/ethereal-64k pvfs-detailed-test/ethereal-65535 - whole-packet ethereal traces. pvfs-detailed-test/smb.conf - the Samba configuration for this test. The client was accessing the Cluster share. I did these tests with just-released Samba 3.0.21a on a Mandrake 2005LE (10.2) system. Sorry for the delay in getting this extra information -- holidays and such. If there is any more information you need, please let me know! --Justin -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Thu, 2006-01-05 at 14:48 -0500, Justin Mazzola Paluska wrote: On Tue, Dec 20, 2005 at 08:08:24AM -0800, Jeremy Allison wrote: Ok, what you need is to create files locally on the windows client of 512 bytes, 1024 bytes etc. and then just copy them onto the Samba drive. Check what the smallest size is that the first file becomes corrupted. That will show the simplest problem that reproduces the issue. At *that* point then send in logs etc. Without an *exact* method of reproducing this with detailed information on what is going wrong (ie. the last 20 bytes of a 512 byte file get corrupted, but a 256 byte file is fine - here is the comparitive copy) it will be a massive investment of time to track this down, that we currently don't have. You're going to have to do more work on this I'm afraid. OK, I made a bunch of files consisting of the string 123456789abcdef\n where \n is the UNIX newline (ASCII 0xA) and performed the binary search. Files 64KB (=65536 bytes) and higher are corrupted and files under 64KB are fine. Have you looked into tools such as fsx and fstest to determine this isn't a filesystem failure? http://www.freebsd.org/cgi/cvsweb.cgi/src/tools/regression/fsx/ http://samba.org/ftp/unpacked/junkcode/fstest.c Andrew Bartlett -- Andrew Bartletthttp://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Student Network Administrator, Hawker College http://hawkerc.net signature.asc Description: This is a digitally signed message part -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Tue, Dec 20, 2005 at 10:46:18AM -0500, Justin Mazzola Paluska wrote: I did a more careful study of the files and the corrupted version seems corrupt throughout -- including an odd string of NULs at the end of the file. The files are the same size, so perhaps something is padding the file to be the right length? Ok, what you need is to create files locally on the windows client of 512 bytes, 1024 bytes etc. and then just copy them onto the Samba drive. Check what the smallest size is that the first file becomes corrupted. That will show the simplest problem that reproduces the issue. At *that* point then send in logs etc. Without an *exact* method of reproducing this with detailed information on what is going wrong (ie. the last 20 bytes of a 512 byte file get corrupted, but a 256 byte file is fine - here is the comparitive copy) it will be a massive investment of time to track this down, that we currently don't have. You're going to have to do more work on this I'm afraid. Jeremy. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Mon, Dec 19, 2005 at 08:01:27AM -0800, Jeremy Allison wrote: 1. Corruption of PDF files. Copying PDF files from my Windows desktop to the Samba share results in corruption. It looks like only the beginning of the file is getting corrupted. For example, the file normally starts with: Get an ethereal trace and debug level 10 of this and then do an The samba log, ethereal trace, the original PDF and the version we get via samba are attached in a tarball. md5 checksum of a truncated copy of the start of the file - say 512 bytes then 1024 bytes then 2048 and binary chop until you find the sizer that's being corrupted. I did a more careful study of the files and the corrupted version seems corrupt throughout -- including an odd string of NULs at the end of the file. The files are the same size, so perhaps something is padding the file to be the right length? Samba works well with many network filesystems so it's doubtful this is a Samba bug (although anything is possible :-). Indeed, Samba's great on most everything we've thrown at it! Sincerely, --Justin -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Sat, Dec 17, 2005 at 09:46:08AM -0800, Andrew Bartlett wrote: The Samba 3.0.13 would be the first thing I would fix. Samba 3.0.21 is about to be released, and with oplock rewrites and other things since 3.0.13, it should provide a better basis for distributed filesystem work. My guess is that the lack of posix locking is causing Word to fail, as it uses a lot of locks. I upgraded to 3.0.21rc2 last night and still have the same problems, though I kept much better records of what's going on, so maybe that helps. 1. Corruption of PDF files. Copying PDF files from my Windows desktop to the Samba share results in corruption. It looks like only the beginning of the file is getting corrupted. For example, the file normally starts with: %PDF-1.4 5 0 obj /Length 3232 /Filter /FlateDecode But after the copy from the windows desktop, the same file starts with: 678 0 729 562 716 0 0 0 0 0 0 0 0 0 0 0 0 511 460 460 511 460 307 460 511 307 0 460 256 818 562 511 0 460 422 409 332 537 460 0 0 486 ] endobj 25 0 obj /Length1 1997 /Length2 14184 /Length3 532 /Length 15276 /Filter /FlateDecode The binary data that follows is different too. 2. We can copy Word files to and from the share without any problems. However, Word refuses to save to the share. It reports the error The save failed due to out of memory or disk space. The share has terabytes of space available (as reported by du), so this shouldn't be a problem. 3. We also use Avid -- a video editing program -- that complains Assertion Failed: ReadSize 0, file/coresw/core/filesys/diskrtnsWIN.c,line 444 and then locks up. We've run some other tests: 1. using the same configuration, but writing to an ext3 or xfs partition works normally with no problems. 2. Writing to and from the PVFS partition from within Linux also works with no problems. All of the above lead me to believe that there's something about PVFS that samba doesn't like. We have logs for these tests, at log level 3 and at log level 10. They're quite big, so I can put them on a website instead of posting them to the list if they would be helpful. Sincerely, --Justin -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Mon, Dec 19, 2005 at 10:11:10AM -0500, Justin Mazzola Paluska wrote: On Sat, Dec 17, 2005 at 09:46:08AM -0800, Andrew Bartlett wrote: The Samba 3.0.13 would be the first thing I would fix. Samba 3.0.21 is about to be released, and with oplock rewrites and other things since 3.0.13, it should provide a better basis for distributed filesystem work. My guess is that the lack of posix locking is causing Word to fail, as it uses a lot of locks. I upgraded to 3.0.21rc2 last night and still have the same problems, though I kept much better records of what's going on, so maybe that helps. 1. Corruption of PDF files. Copying PDF files from my Windows desktop to the Samba share results in corruption. It looks like only the beginning of the file is getting corrupted. For example, the file normally starts with: Get an ethereal trace and debug level 10 of this and then do an md5 checksum of a truncated copy of the start of the file - say 512 bytes then 1024 bytes then 2048 and binary chop until you find the sizer that's being corrupted. Samba works well with many network filesystems so it's doubtful this is a Samba bug (although anything is possible :-). Jeremy. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Samba over PVFS: Corrupted Data
On Fri, 2005-12-16 at 20:56 -0500, Justin Mazzola Paluska wrote: Hello, I'm trying to export Samba shares that access a PVFS2 (http://www.pvfs.org/pvfs2)-mounted partition. PVFS2 is a parallel, distributed file system for Linux clusters. PVFS2 gets mounted like any other partition and it offers non-POSIX file semantics similar to NFS. We can use standard shell commands (mv, ls, cp, etc.) to read and write files on the PVFS2 file system without any problems. On Samba (3.0.13) we've had quite a few problems: The Samba 3.0.13 would be the first thing I would fix. Samba 3.0.21 is about to be released, and with oplock rewrites and other things since 3.0.13, it should provide a better basis for distributed filesystem work. My guess is that the lack of posix locking is causing Word to fail, as it uses a lot of locks. Andrew Bartlett -- Andrew Bartletthttp://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Student Network Administrator, Hawker College http://hawkerc.net signature.asc Description: This is a digitally signed message part -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
[Samba] Samba over PVFS: Corrupted Data
Hello, I'm trying to export Samba shares that access a PVFS2 (http://www.pvfs.org/pvfs2)-mounted partition. PVFS2 is a parallel, distributed file system for Linux clusters. PVFS2 gets mounted like any other partition and it offers non-POSIX file semantics similar to NFS. We can use standard shell commands (mv, ls, cp, etc.) to read and write files on the PVFS2 file system without any problems. On Samba (3.0.13) we've had quite a few problems: 1. When we first made a Samba share, Windows (XP) explorer could see files, but would not let us manipulate them. For example, if we tried copying a file on the share, Windows would complain about a bad file handle. The Samba logs indicated that send_file_readX was causing a problem, so we put use sendfile = no in the share's configuration. We can now read from the share and save files using Notepad. 2. However, if we try copying files to the share with explorer or cmd, such as a pdf file, they get corrupted. The files are consistently corrupted in the same way. Similarly, Word refuses to write to the share. Unfortunately, for these problems, the samba logs say nothing -- I just see that the Windows computer connected to Samba server and the logs stop there. Has anyone had luck sharing PVFS volumes, or if not, are there any special settings I need for file systems with NFS-like semantics? I'm using a 2.6.14 Linux kernel on Mandrake 10.2. Samba is the aforementioned version 3.0.13. Thanks, --Justin -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
[Samba] Samba over PVFS: Corrupted Data
Hello, I'm trying to export Samba shares that access a PVFS2 (http://www.pvfs.org/pvfs2)-mounted partition. PVFS2 is a parallel, distributed file system for Linux clusters. PVFS2 gets mounted like any other partition and it offers non-POSIX file semantics similar to NFS. We can use standard shell commands (mv, ls, cp, etc.) to read and write files on the PVFS2 file system without any problems. On Samba (3.0.13) we've had quite a few problems: 1. When we first made a Samba share, Windows (XP) explorer could see files, but would not let us manipulate them. For example, if we tried copying a file on the share, Windows would complain about a bad file handle. The Samba logs indicated that send_file_readX was causing a problem, so we put use sendfile = no in the share's configuration. We can now read from the share and save files using Notepad. 2. However, if we try copying files to the share with explorer or cmd, such as a pdf file, they get corrupted. The files are consistently corrupted in the same way. Similarly, Word refuses to write to the share. Unfortunately, for these problems, the samba logs say nothing -- I just see that the Windows computer connected to Samba server and the logs stop there. Has anyone had luck sharing PVFS volumes, or if not, are there any special settings I need for file systems with NFS-like semantics? I'm using a 2.6.14 Linux kernel on Mandrake 10.2. Samba is the aforementioned version 3.0.13. Thanks, --Justin -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/listinfo/samba
