Re: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
Hi! On Oct 06, Christopher L. Everett wrote: I've also found a problem with myisamchk --sort-keys: I recently fixed one bug in myisamchk --sort-keys where fulltext indexes are present. Try 4.1.7 release. Regards, Sergei -- __ ___ ___ __ / |/ /_ __/ __/ __ \/ / Sergei Golubchik [EMAIL PROTECTED] / /|_/ / // /\ \/ /_/ / /__ MySQL AB, Senior Software Developer /_/ /_/\_, /___/\___\_\___/ Osnabrueck, Germany ___/ www.mysql.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
Hi! On Oct 06, Christopher L. Everett wrote: I have an application where I create a faily large table (835MB) with a fulltext index. One of our development workstations and our production server will run the script to load the table, but afterwards we have a pervasive corruption, with out of range index index pointer errors. Oddly, my development workstation doesn't have those problems. My box and the ones having the problems have the following differences: - my box runs ReiserFS, the problem boxes run XFS - my box has a nice SCSI HD subsystem, the problem boxes do IDE. All three boxes run Linux 2.6.x kernels, and my workstation and production server share the same mobo. Come to think of it, I saw similar corruption issues under 2.4.x series kernels and MySQL v4.0.x, it just wasn't the show stopper it is now. Could you try to repeat the problem with the smaller dataset ? Create a repeatable test case for us ? Also, on all three boxes, altering the table to drop an index and create a new one requires a myisamchk -rq run afterwards when a fulltext index either exists or gets added or dropped, which I'd also call a bug. Sorry, I don't understand. Could you elaborate ? Regards, Sergei -- __ ___ ___ __ / |/ /_ __/ __/ __ \/ / Sergei Golubchik [EMAIL PROTECTED] / /|_/ / // /\ \/ /_/ / /__ MySQL AB, Senior Software Developer /_/ /_/\_, /___/\___\_\___/ Osnabrueck, Germany ___/ www.mysql.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
I have an application where I create a faily large table (835MB) with a fulltext index. One of our development workstations and our production server will run the script to load the table, but afterwards we have a pervasive corruption, with out of range index index pointer errors. Oddly, my development workstation doesn't have those problems. My box and the ones having the problems have the following differences: - my box runs ReiserFS, the problem boxes run XFS - my box has a nice SCSI HD subsystem, the problem boxes do IDE. All three boxes run Linux 2.6.x kernels, and my workstation and production server share the same mobo. Come to think of it, I saw similar corruption issues under 2.4.x series kernels and MySQL v4.0.x, it just wasn't the show stopper it is now. Also, on all three boxes, altering the table to drop an index and create a new one requires a myisamchk -rq run afterwards when a fulltext index either exists or gets added or dropped, which I'd also call a bug. -- Christopher L. Everett Chief Technology Officer www.medbanner.com MedBanner, Inc. www.physemp.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
I've also found a problem with myisamchk --sort-keys: christopher:/var/lib/mysql/dmoz# myisamchk -rq xurls - check record delete-chain - recovering (with sort) MyISAM-table 'xurls' Data records: 1981904 - Fixing index 1 - Fixing index 2 - Fixing index 3 Data records: 4332227 christopher:/var/lib/mysql/dmoz# myisamchk -ce xurls Checking MyISAM file: xurls Data records: 4332227 Deleted blocks: 0 - check file-size - check record delete-chain - check key delete-chain - check index reference - check data record references index: 1 - check data record references index: 2 - check data record references index: 3 - check records and index references christopher:/var/lib/mysql/dmoz# myisamchk -aS xurls - Sorting index for MyISAM-table 'xurls' christopher:/var/lib/mysql/dmoz# myisamchk -ce xurls Checking MyISAM file: xurls Data records: 4332227 Deleted blocks: 0 - check file-size - check record delete-chain - check key delete-chain - check index reference - check data record references index: 1 - check data record references index: 2 - check data record references index: 3 myisamchk: error: Found key at page 208005120 that points to record outside datafile - check records and index references 1307000^C Also, I gt a segfault trying to use myisamchk --sort-records. Christopher L. Everett wrote: I have an application where I create a faily large table (835MB) with a fulltext index. One of our development workstations and our production server will run the script to load the table, but afterwards we have a pervasive corruption, with out of range index index pointer errors. Oddly, my development workstation doesn't have those problems. My box and the ones having the problems have the following differences: - my box runs ReiserFS, the problem boxes run XFS - my box has a nice SCSI HD subsystem, the problem boxes do IDE. All three boxes run Linux 2.6.x kernels, and my workstation and production server share the same mobo. Come to think of it, I saw similar corruption issues under 2.4.x series kernels and MySQL v4.0.x, it just wasn't the show stopper it is now. Also, on all three boxes, altering the table to drop an index and create a new one requires a myisamchk -rq run afterwards when a fulltext index either exists or gets added or dropped, which I'd also call a bug. -- Christopher L. Everett Chief Technology Officer www.medbanner.com MedBanner, Inc. www.physemp.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
-Original Message- From: Christopher L. Everett [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 1:47 AM To: Mysql List Subject: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes I have an application where I create a faily large table (835MB) with a fulltext index. One of our development workstations and our production server will run the script to load the table, but afterwards we have a pervasive corruption, with out of range index index pointer errors. Oddly, my development workstation doesn't have those problems. My box and the ones having the problems have the following differences: - my box runs ReiserFS, the problem boxes run XFS - my box has a nice SCSI HD subsystem, the problem boxes do IDE. All three boxes run Linux 2.6.x kernels, and my workstation and production server share the same mobo. Come to think of it, I saw similar corruption issues under 2.4.x series kernels and MySQL v4.0.x, it just wasn't the show stopper it is now. Also, on all three boxes, altering the table to drop an index and create a new one requires a myisamchk -rq run afterwards when a fulltext index either exists or gets added or dropped, which I'd also call a bug. The problems you're describing are similar to what I've run into when there have been hardware related problems. One system had a problem with ram. Memory tests would test and report ram as ok, but everything started working when I replaced the ram. I think it was just brand incompatibility or something odd, because the ram never gave any problems in another system. One system had hard drive media slowly failing and this wasn't obvious until we ran several full scan chkdsks. The funniest situation was where enough dust had collected in the CPU fan to cause slight over heating, which resulted in oddball errors. In each of these cases, everything would work fine until the system would start processing larger amounts of data. Small amounts of corruption began to show up that seemed to build on itself. This may or may not relate to what you're dealing with, but maybe it will help =) -Ed -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes
Ed Lazor wrote: -Original Message- From: Christopher L. Everett [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 1:47 AM To: Mysql List Subject: Repeated corruption with MySQL 4.1.x using FULLTEXT indexes I have an application where I create a faily large table (835MB) with a fulltext index. One of our development workstations and our production server will run the script to load the table, but afterwards we have a pervasive corruption, with out of range index index pointer errors. Oddly, my development workstation doesn't have those problems. My box and the ones having the problems have the following differences: - my box runs ReiserFS, the problem boxes run XFS - my box has a nice SCSI HD subsystem, the problem boxes do IDE. All three boxes run Linux 2.6.x kernels, and my workstation and production server share the same mobo. Come to think of it, I saw similar corruption issues under 2.4.x series kernels and MySQL v4.0.x, it just wasn't the show stopper it is now. Also, on all three boxes, altering the table to drop an index and create a new one requires a myisamchk -rq run afterwards when a fulltext index either exists or gets added or dropped, which I'd also call a bug. The problems you're describing are similar to what I've run into when there have been hardware related problems. One system had a problem with ram. Memory tests would test and report ram as ok, but everything started working when I replaced the ram. I think it was just brand incompatibility or something odd, because the ram never gave any problems in another system. I can generate the problem on much smaller data sets, in the mid tens of thousands of records rather than the millions of records. I'll do a memtest86 run on the development boxes overnight, but as I did that just after I installed linux on them and used the linux badram patch to exclude iffy sections of RAM, I don't think thats a problem. One system had hard drive media slowly failing and this wasn't obvious until we ran several full scan chkdsks. 3 hard drives all of different brand, model size, and the problem happening in the same place on both? Not likely. The funniest situation was where enough dust had collected in the CPU fan to cause slight over heating, which resulted in oddball errors. This isn't a problem on my box. I have a 1.5 pound copper heatsink with a 90mm heat sensitive fan and a fan+heatsink for the hard drive, and I saw myisamchk consistently generate the same error in the same place over and over. The sensors report my CPU running in the 45 degree centigrade range on my box pretty consistently. In each of these cases, everything would work fine until the system would start processing larger amounts of data. Small amounts of corruption began to show up that seemed to build on itself. This may or may not relate to what you're dealing with, but maybe it will help =) I'll look, but I don't think that's the problem. I'm going to see how small of a data set will cause this problem and file a bug report. -- Christopher L. Everett Chief Technology Officer www.medbanner.com MedBanner, Inc. www.physemp.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
