Re: CIFS and data integrity ; Sorry for the noise
On Tue, 31 Jul 2012 01:15:28 + (UTC) Mark Fletcher mark2...@gmail.com wrote: ... compatibility. I have heard of people wresting their NAS out of the grasp of the OS it comes with and installing Linux on it, presumably by mucking about with firmware etc, but I have never attempted to do any such thing as I would probably screw it up :-) So it is running whatever it was running when it left the factory. IIUC, NAS hacking is usually done to units that ship with linux, just a locked down / semi-proprietary version. The hacking enables the unit to run some more or less standard distribution. Celejar -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120731220103.0073eedd.cele...@gmail.com
Re: CIFS and data integrity ; Sorry for the noise
On 20120730_122543, Paul E Condon wrote: On 20120730_065640, Mark Fletcher wrote: Joe joe at jretrading.com writes: On Mon, 30 Jul 2012 01:50:14 +0900 Mark Fletcher mark27q1 at gmail.com wrote: It looks like what got stored on the NAS is not exactly what was originally on the host. This is a huge problem for me as it means I can't rely on backups dumped on that device. Is there something wrong with the way I am mounting the NAS that is leading to this? Probably. I'd guess it is a matter of permissions. If you create the archive elsewhere, copy it to the NAS, copy it back again, presumably there is no difficulty. I also use a Buffalo NAS, but my backups are created on my server, then copied. It is possible that if the compression and expansion is done on the NAS, that a temp file involved may not have the correct permissions to write, or more likely, amend. But is your backup not running under cron as root? I put the exact commands I was executing in my original post. There's no job involved, I am typing these commands at the command prompt. I'll bring a job into it once it works reliably. If you read my original post, you can see I create the archive on the host's local disk, test it to make sure it is good, and then copy it to the NAS in a separate operation. I use the cp command to do the copy. I'm inclined to rule out a bug in the cp command, which leaves something about the way the data is being transferred to my NAS. Hence my concern about whether my mount command (again, see details in my original post) was correct. And yes, to answer someone else's question, this is reproducible, reliably, every time. The copy on the NAS is always the same length as the copy on the host's local disk, but diff says they are two different binary files and the one on the NAS cannot be extracted. A quick way to by-pass the permissions issue is to log in as super-user root, and type your commands and tests as root. As I understand it, root is unstoppable. That is why it is so dangerous to use it in day-to-day mucking about. A moments inattention and real damaged is done. But... as a test, and you are testing, use root. Having posted this, which I thought was reasonable, I went and looked at the archives to see what OP (Mark Fletcher) had written. It turns out that all of his investigation was done using commands typed in as root. For me, this thread is a real puzzle. And very scary. Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based open/free software? What vintage? (best guess). IMHO, something bad is happening as we rush into the future. Layers of software can cover bugs in basic functionality. Complexity beyond human comprehension. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120730200934.gb22...@big.lan.gnu
Re: CIFS and data integrity ; Sorry for the noise
On 30/07/12 04:09 PM, Paul E Condon wrote: On 20120730_122543, Paul E Condon wrote: On 20120730_065640, Mark Fletcher wrote: Joejoeat jretrading.com writes: On Mon, 30 Jul 2012 01:50:14 +0900 Mark Fletchermark27q1at gmail.com wrote: It looks like what got stored on the NAS is not exactly what was originally on the host. This is a huge problem for me as it means I can't rely on backups dumped on that device. Is there something wrong with the way I am mounting the NAS that is leading to this? Probably. I'd guess it is a matter of permissions. If you create the archive elsewhere, copy it to the NAS, copy it back again, presumably there is no difficulty. I also use a Buffalo NAS, but my backups are created on my server, then copied. It is possible that if the compression and expansion is done on the NAS, that a temp file involved may not have the correct permissions to write, or more likely, amend. But is your backup not running under cron as root? I put the exact commands I was executing in my original post. There's no job involved, I am typing these commands at the command prompt. I'll bring a job into it once it works reliably. If you read my original post, you can see I create the archive on the host's local disk, test it to make sure it is good, and then copy it to the NAS in a separate operation. I use the cp command to do the copy. I'm inclined to rule out a bug in the cp command, which leaves something about the way the data is being transferred to my NAS. Hence my concern about whether my mount command (again, see details in my original post) was correct. And yes, to answer someone else's question, this is reproducible, reliably, every time. The copy on the NAS is always the same length as the copy on the host's local disk, but diff says they are two different binary files and the one on the NAS cannot be extracted. A quick way to by-pass the permissions issue is to log in as super-user root, and type your commands and tests as root. As I understand it, root is unstoppable. That is why it is so dangerous to use it in day-to-day mucking about. A moments inattention and real damaged is done. But... as a test, and you are testing, use root. Having posted this, which I thought was reasonable, I went and looked at the archives to see what OP (Mark Fletcher) had written. It turns out that all of his investigation was done using commands typed in as root. For me, this thread is a real puzzle. And very scary. Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based open/free software? What vintage? (best guess). IMHO, something bad is happening as we rush into the future. Layers of software can cover bugs in basic functionality. Complexity beyond human comprehension. May I suggest that it could be nothing more than a defective network card or cable? It may also be the result of some other hardware error. That's why I suggested doing a file cmp on the files and using rsync. I find exactly the same problem when backing up to rewritable optical media. You can't always trust the writes to work properly. Always to a verify and correct if the files are at all important. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5016eb67.70...@rogers.com
Re: CIFS and data integrity ; Sorry for the noise
Paul E Condon pecondon at mesanetworks.net writes: Having posted this, which I thought was reasonable, I went and looked at the archives to see what OP (Mark Fletcher) had written. It turns out that all of his investigation was done using commands typed in as root. For me, this thread is a real puzzle. And very scary. Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based open/free software? What vintage? (best guess). IMHO, something bad is happening as we rush into the future. Layers of software can cover bugs in basic functionality. Complexity beyond human comprehension. Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live in Japan) at Christmas 2010. I don't know what OS it will be running but doubt it will be Linux since Buffalo will be aiming to maximise Windows compatibility. I have heard of people wresting their NAS out of the grasp of the OS it comes with and installing Linux on it, presumably by mucking about with firmware etc, but I have never attempted to do any such thing as I would probably screw it up :-) So it is running whatever it was running when it left the factory. I'm simultaneously glad that someone else is as unnerved by this happening as I am, and increasingly nervous that the issue isn't inspiring oh that old chestnut reaction in list readers... :-) Mark -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20120731t030407-...@post.gmane.org
Re: CIFS and data integrity ; Sorry for the noise
On Tue, 31 Jul 2012, Mark Fletcher wrote: Paul E Condon pecondon at mesanetworks.net writes: Having posted this, which I thought was reasonable, I went and looked at the archives to see what OP (Mark Fletcher) had written. It turns out that all of his investigation was done using commands typed in as root. For me, this thread is a real puzzle. And very scary. Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based open/free software? What vintage? (best guess). IMHO, something bad is happening as we rush into the future. Layers of software can cover bugs in basic functionality. Complexity beyond human comprehension. Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live in Japan) at Christmas 2010. I don't know what OS it will be running but doubt it will be Linux since Buffalo will be aiming to maximise Windows compatibility. I have heard of people wresting their NAS out of the grasp of the OS it comes with and installing Linux on it, presumably by mucking about with firmware etc, but I have never attempted to do any such thing as I would probably screw it up :-) So it is running whatever it was running when it left the factory. I'm simultaneously glad that someone else is as unnerved by this happening as I am, and increasingly nervous that the issue isn't inspiring oh that old chestnut reaction in list readers... :-) Well... there is an awlful lot of CIFS and NFS-related fixes in the kernel stable queue. Check that. Also make sure it is not your NIC driver or memory (or the NAS' memory) that went bad... -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120731012712.gb25...@khazad-dum.debian.net
Re: CIFS and data integrity ; Sorry for the noise
Henrique de Moraes Holschuh hmh at debian.org writes: Well... there is an awlful lot of CIFS and NFS-related fixes in the kernel stable queue. Check that. Also make sure it is not your NIC driver or memory (or the NAS' memory) that went bad... I wondered about this too -- and the defective cable / NIC idea posted by Gary earlier. But I would have expected that network card / driver / memory issues would result in a failed transfer that would either end in tears (error messages) or would cause part of the transfer to be retried and thus take longer but eventually end with correct data having been transferred. What's giving me the heebie-jeebies (if you'll pardon the technial term) about this is that every software and hardware component seems to think that everything went swimmingly as far as I can tell, but the data on the target is nonetheless not correct. Mark -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20120731t044059-...@post.gmane.org
Re: CIFS and data integrity ; Sorry for the noise
On 31/07/12 13:15, Mark Fletcher wrote: Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live in Japan) at Christmas 2010. I don't know what OS it will be running but doubt it will be Linux since Buffalo will be aiming to maximise Windows compatibility. http://opensource.buffalo.jp/gpl_storage.html I think it quite possibly is :-) Oldish though - 2.4.20 by the looks of it. Richard -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5017541a.2020...@walnut.gen.nz
