Hello,
Is there a way to force the queue to flush? That would be very nice. In
delphi and interbase I would use transactions. I would start a transaction
and process it every 1000 records. This was pretty efficient and the number
of records depended on the amount of ram.
Mark
p.s. your english is great. "we" is correct.
----- Original Message -----
From: "Z Julian" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Thursday, September 13, 2001 9:49 PM
Subject: Re: high speed insertion required
> Hi Mark:
> "insert delayed" helps little.
> I think there are two reasons.
> 1. One reason "insert delayed" speeding up insertion is that the client
will return immediately after it issues a "insert delayed" query. I think
all those inserts must be stored in a delayed queue and will be executed
whenever it is possible or necessary. But since the delayed queue can't hold
too many inserts and my program inserts data at a steady and high speed and
will run for a long period of time (couple of days or maybe weeks), the
queue will be full inevitably. And after the queue is full, I think no
inserts can be delayed any longer. At that time, the insertion speed of my
program will low down to the normal speed as insertion without "delayed"
option chosen.
> 2. Another reason "insert delayed" speeding up insertion is that inserts
from many clients are bundled together and written in one block. But on my
server there are no other clients. So the insertion of my program can't be
accelerated in this way neither.
>
> >Disable or remove indexes. This sometimes speeds it up tremendously
> >depending on your indexes.
>
> Well, index... I do have some indexes in the tables, but I can't give them
up because those tables will be accessed at any time during the insertion.
>
> >Is the hash table in the same server/database?
>
> The hash tables are in the same database.
>
> Well, I think maybe I should face the reality, maybe my program has
already tried its best, and so have we(us? hehe, poor english :).
> Any way, thanks for your help.
>
> >From: "Mark Rissmann" <[EMAIL PROTECTED]>
> >To: "Z Julian" <[EMAIL PROTECTED]>
> >Subject: Re: high speed insertion required
> >Date: Thu, 13 Sep 2001 11:14:26 -0700
> >
> >I use Delphi and am having a similar problem. But here are some things
that
> >can be done (that I haven't tried).
> >
> >INSERT DELAY -- this is supposed to help because my SQL will recognize a
> >group of inserts into a table and queue them. This should be faster.
You'll
> >have to experiment on how long you allow the queue to build.
> >
> >Disable or remove indexes. This sometimes speeds it up tremendously
> >depending on your indexes.
> >
> >Is the hash table in the same server/database?
> >
> >Please post your findings to the listserver. Let me know if this works or
if
> >you have any more questions.
> >
> >Mark Rissmann
> >-Drifting Sands LLC
> >
> >----- Original Message -----
> >From: "Z Julian" <[EMAIL PROTECTED]>
> >To: <[EMAIL PROTECTED]>
> >Sent: Thursday, September 13, 2001 1:24 AM
> >Subject: Re: high speed insertion required
> >
> >
> > > Hi:
> > > Thanks for your reply. :)
> > > I write that program in C. I get data from a mysql heap table which is
> >always full of records. That's why I use queries like "insert into table1
> >select from table2" to insert data. Obviously table2 is the heap table I
> >mentioned above.
> > > I use that program to do some log jobs. Someone tells me to use
"insert
> >HIGH_PRIORIRY" to speed up the insertion, but I think that helps little
> >since there are no other database clients running on my machine except
the
> >one which fills the heap table for my program.
> > > Do you have any suggestion? Please mail. Thank you and have a nice
day! :)
> > >
> > > >From: "Mark Rissmann" <[EMAIL PROTECTED]>
> > > >To: [EMAIL PROTECTED]
> > > >Subject: Re: high speed insertion required
> > > >Date: Thu, 13 Sep 2001 01:02:31 -0700
> > > >
> > > >Hello,
> > > >
> > > >What development tool are you programming in? Where are you getting
these
> >records from?
> > > >
> > > >Have you received any good tips?
> > > >
> > > >Mark Rissmann
> > > >-Drifting Sands LLC
> > > >
> > > >
> > > >
> > > >"I do not fear computers. I fear the lack of them." - Isaac Asimov
> > >
> > >
>
>
---------------------------------------------------------------------
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail <[EMAIL PROTECTED]>
To unsubscribe, e-mail <[EMAIL PROTECTED]>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
