import size

2002-12-05 Thread Sarah Killcoyne 
I'm attempting to import 36 files of varying size from about 500kb to 80mb.
I've found that using the local infile import query on files that are
several mb in size takes up to a min per mb and much longer if it get's over
100mb.  The system we're using has 500mb RAM and a 1600mHz processor.  Is
there a faster way to do these bulk imports?  This is supposed to be faster
than the insert function especially on large numbers of imports.  Can anyone
tell me anything about this to speed this up perhaps?

Thank you,
Sarah


-
Before posting, please check:
   http://www.mysql.com/manual.php   (the manual)
   http://lists.mysql.com/   (the list archive)

To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

Re: import size

2002-12-05 Thread Steve Yates 
On Thu, 5 Dec 2002 10:16:16 -0700, Sarah Killcoyne wrote:

I've found that using the local infile import query on files that are
several mb in size takes up to a min per mb 

I've posted a couple times recently on a Delphi project I wrote
to upload data from a client's Access database (3 tables) to the web on
demand.  It is about 4 MB in size and though it does take time it takes
way less than 4 minutes.  I did find out that using a query made a huge
difference over using a table (i.e. editing the MySQL table directly),
and using the REPLACE...VALUES syntax with multiple value entries made
a huge difference too.  I ended up iterating through the table like
this (pseudocode):

while (still data left)
  i = 1
  querystr = 'replace delayed into(cols) values '
  while (still data left and i  20)
querystr = querystr + (data)
i = i + 1
  end
  execute query
end

DELAYED made very little difference in the upload speed, but
seemed to allow use of the web database while the upload was happening.
 Be careful about the size of the data though...one of the three tables
had many columns including a text field and I found that using i  20
would result in a string that was too long.

 - Steve Yates
 - Between two evils, always pick the one you haven't tried.

~ Taglines by Taglinator - www.srtware.com ~


-
Before posting, please check:
   http://www.mysql.com/manual.php   (the manual)
   http://lists.mysql.com/   (the list archive)

To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php