Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files on different ext3 file

2007-09-18 Thread Stephen Samuel
You may want to excercise your I/O subsystem. Given that you probably don't want to stomp on a live filesystem, you might want to create a file of a couple of gigabytes and turn it into a pseudo-device with 'lofs(1)'. EG: # make a 15GB test file dd if=/dev/zero of=the_testfile bs=1M count=15000

Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files on dif

2007-09-17 Thread Maurice Volaski
In using drbd 8.0.5 recently, I have come across at least two instances where a bit on disk apparently flipped spontaneously in the ext3 metadata on volumes running on top of drbd. Also, I have been seeing regular corruption of a mysql database, which runs on top of drbd, and when I reported

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files on

2007-09-17 Thread Maurice Volaski
On Sep 17, 2007 13:31 -0400, Maurice Volaski wrote: In using drbd 8.0.5 recently, I have come across at least two instances where a bit on disk apparently flipped spontaneously in the ext3 metadata on volumes running on top of drbd. Also, I have been seeing regular corruption of a mysql

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files on

2007-09-17 Thread Jeremy Cole
Hi Maurice, If you're running into corruption both in ext3 metadata and in MySQL data, it is certainly not he fault of MySQL as you're likely aware. There are absolutely many places where corruption could occur between MySQL and the physical bits on disk. The corruption you're seeing does

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files on

2007-09-17 Thread Jeremy Cole
Hi Maurice, Do you mean a Serially-Attached SCSI aka SAS controller, I assume? Is this a custom build machine or a vendor integrated one? Regards, Jeremy Maurice Volaski wrote: On Sep 17, 2007 13:31 -0400, Maurice Volaski wrote: In using drbd 8.0.5 recently, I have come across at least

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files

2007-09-17 Thread Maurice Volaski
Hi Maurice, If you're running into corruption both in ext3 metadata and in MySQL data, it is certainly not he fault of MySQL as you're likely aware. I am hoping they are not related. The problems with MySQL surfaced almost immediately after upgrading to 5.0.x. [details deleted] You can

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large files

2007-09-17 Thread Jeremy Cole
Hi Maurice, If you're running into corruption both in ext3 metadata and in MySQL data, it is certainly not he fault of MySQL as you're likely aware. I am hoping they are not related. The problems with MySQL surfaced almost immediately after upgrading to 5.0.x. It's possible that they are

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large file

2007-09-17 Thread Maurice Volaski
I guess I will watch it closely for now and if it trips up again failover to the drbd peer and see what happens there. I suppose I could even deattach the local disks and have it run using the peer over the wire. That should eliminate the local I/O subsystem. It's kind of scary there is no

Re: Could drbd randomly flip bits? Was: Database page corruption on disk occurring during mysqldump on a fresh database and Was: Spontaneous development of supremely large file

2007-09-17 Thread Maurice Volaski
I failed over the server and ran a short backup and there were no didn't compare errors where on the first server, they are there pretty reliably. I guess this confirms some hardware on the first server is flipping bits. Essentially, users could have any number of munged files (most files are

Re: Database page corruption on disk occurring during mysqldump on a fresh database

2007-09-13 Thread Maurice Volaski
It certainly seems that 5.0.44 and 5.0.45 are unstable. I have logged this as bug http://bugs.mysql.com/bug.php?id=31008 A 64-bit Gentoo Linux box had just been upgraded from MySQL 4.1 to 5.0.44 fresh (by dumping in 4.1 and restoring in 5.0.44) and almost immediately after that, during which

Re: Database page corruption on disk occurring during mysqldump on a fresh database

2007-09-10 Thread Maurice Volaski
Thank you for your replies. I attempted to restore again and most oddly, mysql complained that it couldn't restore to a particular table because it wasn't in the database, which, of course, it had to be because the restore itself had just recreated it. So I blew away the entire mysql directory

Re: Database page corruption on disk occurring during mysqldump on a fresh database

2007-09-05 Thread Krishna Chandra Prajapati
Hi This might be happening due to two reasons; 1 The system date might not be correct. 2. Some things wrong with log postion (Incorrect log position) Regards, Krishna Chandra Prajapati On 8/31/07, Maurice Volaski [EMAIL PROTECTED] wrote: A 64-bit Gentoo Linux box had just been upgraded from

Re: Database page corruption on disk occurring during mysqldump on a fresh database

2007-09-05 Thread Alex Arul Lurthu
The checksum errors might be due to various reasons. We had similar issue where we restored the database multiple times, replaced the ram sticks nothing helped. Finally we drilled down the issue to the chassis. Recommend testing the restore on a different machine to rule out any hardware issue.

Database page corruption on disk occurring during mysqldump on a fresh database

2007-08-31 Thread Maurice Volaski
A 64-bit Gentoo Linux box had just been upgraded from MySQL 4.1 to 5.0.44 fresh (by dumping in 4.1 and restoring in 5.0.44) and almost immediately after that, during which time the database was not used, a crash occurred during a scripted mysqldump. So I restored and days later, it happened