Are you doing single insert statements, multiple insert statements or
LOAD DATA INFILE statements?
On 12/4/05, Chenzhou Cui [EMAIL PROTECTED] wrote:
Dear MySQL fans,
I have a 260 GB huge file with 1045175762 rows. Two weeks ago, I wrote a
Java program to read the huge plain text file into
After 4 days and a half, my program finished last night. The 1045175762
rows costed the Java program 109 hours to load. The speed is about 9.6
million per hour.
I didn't use multiple insert statements and LOAD DATA INFILE, but only
insert into statement. The Java program reads one line from
Chenzhou Cui wrote:
I didn't use multiple insert statements and LOAD DATA INFILE, but
only insert into statement. The Java program reads one line from the
source file, and then execute an insert statement.
I can't comment on the speed of 5.0.x vs 4.1.x, but I can suggest that
you optimize
Also, if the file looks anything like a CSV file, I recommend using
LOAD DATA INFILE
http://dev.mysql.com/doc/refman/4.1/en/load-data.html
You'll probably load that data in half to a quarter of the time.
On 11/30/05, Daniel Kasak [EMAIL PROTECTED] wrote:
Chenzhou Cui wrote:
I didn't use
The file is in fix column width format. If it is in CSV format, I don't
need a program to read it.
The first few lines are as following:
---
usnob_read: no zone specified, read from South Pole
#USNOB (whole)
#USNO-B1.0 Tycho-2RA (J2000) Dec
Chenzhou Cui wrote:
The file is in fix column width format. If it is in CSV format, I
don't need a program to read it.
'load data infile' can import from a fixed-width format. Check the
documentation. I've set up a number of these imports, and they're a
little messy to set up ( you have to
Dear MySQL fans,
I have a 260 GB huge file with 1045175762 rows. Two weeks ago, I wrote a
Java program to read the huge plain text file into MySQL 4.1.12. 300
million of rows could be loaded in one day.
Last Thursday, I updated the MySQL to 5.0.16 and then run the same
program. Only 225
