Re: Non-linear degradation in bulk loads?

2005-10-18 Thread Heikki Tuuri
] <mailto:[EMAIL PROTECTED]>> Vastaanottaja: "Heikki Tuuri" <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>; <mailto:mysql@lists.mysql.com>> Lähetetty: Tuesday, October 18, 2005 4:39 AM Aihe: RE: Non-linear degradation in bulk loads? Side question: If I use a 64-bit

RE: Non-linear degradation in bulk loads?

2005-10-17 Thread Jon Frisby
the sole application on the machine? -JF > -Original Message- > From: Heikki Tuuri [mailto:[EMAIL PROTECTED] > Sent: Monday, October 17, 2005 10:55 AM > To: mysql@lists.mysql.com > Subject: Re: Non-linear degradation in bulk loads? > > Jon, > > I am not 100 % s

RE: Non-linear degradation in bulk loads?

2005-10-17 Thread Jon Frisby
-Original Message- > From: Heikki Tuuri [mailto:[EMAIL PROTECTED] > Sent: Monday, October 17, 2005 10:52 AM > To: mysql@lists.mysql.com > Subject: Re: Non-linear degradation in bulk loads? > > Jon, > > I do not know. Why not install a 64-bit Linux in your compute

Re: Non-linear degradation in bulk loads?

2005-10-17 Thread Heikki Tuuri
programs that occupy a few GB of memory. Regards, Heikki Oracle/Innobase - Alkuperäinen viesti - Lähettäjä: "Jon Frisby" <[EMAIL PROTECTED]> Vastaanottaja: "Heikki Tuuri" <[EMAIL PROTECTED]>; Lähetetty: Monday, October 17, 2005 8:49 PM Aihe: RE: Non-

RE: Non-linear degradation in bulk loads?

2005-10-17 Thread Jon Frisby
com > Subject: Re: Non-linear degradation in bulk loads? > > Jon, > > your hardware/OS combination is quite new and unusual. It > might indeed be an OS problem. We observed from a 64-bit RHEL > 4 that when the file cache of the OS grew bigger than 4 GB, > then the file I/

Re: Non-linear degradation in bulk loads?

2005-10-17 Thread Heikki Tuuri
r 17, 2005 8:46 PM Aihe: RE: Non-linear degradation in bulk loads? Actually, I believe we're running 32-bit, with bigmem... Does similar behavior occur in such a scenario? -JF -Original Message- From: Heikki Tuuri [mailto:[EMAIL PROTECTED] Sent: Sunday, October 16, 2005 2:23 AM To: m

RE: Non-linear degradation in bulk loads?

2005-10-17 Thread Jon Frisby
" > > Regards, > > Heikki > > > - Original Message - > From: ""Jon Frisby"" <[EMAIL PROTECTED]> > Newsgroups: mailing.database.myodbc > Sent: Friday, October 14, 2005 10:39 PM > Subject: RE: Non-linear degradation in bulk

RE: Non-linear degradation in bulk loads?

2005-10-17 Thread Jon Frisby
About 6GB... Is there any way to forcibly limit this? -JF > -Original Message- > From: Heikki Tuuri [mailto:[EMAIL PROTECTED] > Sent: Sunday, October 16, 2005 2:23 AM > To: mysql@lists.mysql.com > Subject: Re: Non-linear degradation in bulk loads? > > Jon,

Re: Non-linear degradation in bulk loads?

2005-10-16 Thread Heikki Tuuri
setting)). -JF -Original Message- From: Heikki Tuuri [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 12, 2005 8:15 AM To: mysql@lists.mysql.com Subject: Re: Non-linear degradation in bulk loads? =20 Jon, =20 hmm... maybe one of the indexes inevitably is in a random order. =20 Please p

RE: Non-linear degradation in bulk loads?

2005-10-14 Thread Jon Frisby
L PROTECTED] > Sent: Wednesday, October 12, 2005 8:15 AM > To: mysql@lists.mysql.com > Subject: Re: Non-linear degradation in bulk loads? > > Jon, > > hmm... maybe one of the indexes inevitably is in a random order. > > Please post a typical > > SHOW INNODB

Re: Non-linear degradation in bulk loads?

2005-10-12 Thread Heikki Tuuri
AIL PROTECTED]> Vastaanottaja: "Heikki Tuuri" <[EMAIL PROTECTED]>; Lähetetty: Wednesday, October 12, 2005 3:08 AM Aihe: RE: Non-linear degradation in bulk loads? Two solutions: 1) sort the rows to be inserted on the key 'email' before inserting. 2) Or: http://dev.mysql.co

RE: Non-linear degradation in bulk loads?

2005-10-11 Thread Jon Frisby
> Two solutions: 1) sort the rows to be inserted on the key > 'email' before inserting. > > 2) Or: > > http://dev.mysql.com/doc/mysql/en/innodb-tuning.html > " > If you have UNIQUE constraints on secondary keys, starting from MySQL > 3.23.52 and 4.0.3, you can speed up table imports by > tempor

Re: Non-linear degradation in bulk loads?

2005-10-11 Thread mos
sday, October 11, 2005 4:12 AM Subject: Non-linear degradation in bulk loads? Everyone, We're trying to do some bulk data loads on several different tables (on several different machines, using several different techniques) and seeing dramatically worse-than-linear performance. We've tr

Re: Non-linear degradation in bulk loads?

2005-10-11 Thread Heikki Tuuri
Jon, my guess is that the inserts to the UNIQUE secondary index cause the workload to be seriously disk-bound. Two solutions: 1) sort the rows to be inserted on the key 'email' before inserting. 2) Or: http://dev.mysql.com/doc/mysql/en/innodb-tuning.html " If you have UNIQUE constraints on

Re: Non-linear degradation in bulk loads?

2005-10-11 Thread Devananda
Jon Frisby wrote: Everyone, We're trying to do some bulk data loads on several different tables (on several different machines, using several different techniques) and seeing dramatically worse-than-linear performance. We've tried the bulk-INSERT syntax, and the LOAD DATA INFILE syntax. We've d

Re: Non-linear degradation in bulk loads?

2005-10-10 Thread C.R. Vegelin
ject: Non-linear degradation in bulk loads? Everyone, We're trying to do some bulk data loads on several different tables (on several different machines, using several different techniques) and seeing dramatically worse-than-linear performance. We've tried the bulk-INSERT syntax, and t

RE: Non-linear degradation in bulk loads?

2005-10-10 Thread Jon Frisby
ginal Message- > From: Manoj [mailto:[EMAIL PROTECTED] > Sent: Monday, October 10, 2005 8:51 PM > To: Jon Frisby > Cc: mysql@lists.mysql.com > Subject: Re: Non-linear degradation in bulk loads? > > Not sure but given that you suffer from non-linear > degradation in performan

Re: Non-linear degradation in bulk loads?

2005-10-10 Thread Manoj
Not sure but given that you suffer from non-linear degradation in performance;my guess is you might be extending your ibdata file every too frequently during the batch load process. Check the ibdata_data_file_path variable in my.cnf for more details. Cheers Manoj On 10/11/05, Jon Frisby <[EMAIL P

Non-linear degradation in bulk loads?

2005-10-10 Thread Jon Frisby
Everyone, We're trying to do some bulk data loads on several different tables (on several different machines, using several different techniques) and seeing dramatically worse-than-linear performance. We've tried the bulk-INSERT syntax, and the LOAD DATA INFILE syntax. We've done ALTER TABLE ...