Re: loading a subset of input file using mysqlimport?

2005-11-28 Thread sheeri kritzer
As far as I can tell, there's no way to do what you specify.  If you
want less pre-processing, you can use the IGNORE n LINES command of
LOAD DATA INFILE:

http://dev.mysql.com/doc/refman/5.0/en/load-data.html

The pre-processing involved with that is:
1) Decide how many parts you want to split your file into (say, x)
2) Figure out how many rows are in each part (y rows * x parts = 10
million rows)
3) copy the ascii file x-1 times
4) edit *1* of the files to only contain the first y rows
5) use LOAD DATA INFILE x times -- the first time you'll use LOAD DATA
INFILE from the file you edited in step 4, the other x-1 times you'll
use LOAD DATA INFILE IGNORE n LINES, where n is y for the 2nd batch,
2y for the 3rd batch, 3y for the 4th batch, 4y for the 5th batch, etc.

If you really are worried about the pre-processing, make a perl
script, or download one like the one at
http://www.hotscripts.com/Detailed/28161.html (note, I claim no
liability, I just found that script doing a web search).


-Sheeri

On 11/22/05, Jacek Becla [EMAIL PROTECTED] wrote:
 Hi,

 Is there a way to load a section of an input file into mysql
 (MyISAM table) using mysqlimport or LOAD DATA INTO?
 The input data is in relatively large ascii files (10 million
 rows per file), and I'd like to break the load into smaller
 pieces rather than load whole file at once. Of course I could
 pre-process each file and split it, but I'd like to avoid this
 extra I/O.

 Thanks,
 Jacek

 --
 MySQL General Mailing List
 For list archives: http://lists.mysql.com/mysql
 To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]



loading a subset of input file using mysqlimport?

2005-11-22 Thread Jacek Becla

Hi,

Is there a way to load a section of an input file into mysql
(MyISAM table) using mysqlimport or LOAD DATA INTO?
The input data is in relatively large ascii files (10 million
rows per file), and I'd like to break the load into smaller
pieces rather than load whole file at once. Of course I could
pre-process each file and split it, but I'd like to avoid this
extra I/O.

Thanks,
Jacek

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]