Hi David, I think you can try pipes(hope u r using linux/unix). this is a method we used to export/sqlload for oracle. craete a pipe file, say abc.txt(mkfile -p abc.txt). start the the dumpfile reading in bg(should be run first). start the dump-file generation of C++. with this setup we could save a lot of time and space.
i have not tried this for mysql. this is only my suggestion. goog luck. Eldo. On Fri, 24 Sep 2004 15:28:43 -0500 (CDT), David Mehringer <[EMAIL PROTECTED]> wrote: > Hello, > I have a C++ application which spits out data continuously which I need to > load into a database. The data rate is roughly 50,000 rows * 50 bytes/row > per second. I use LOAD DATA INFILE (the quickest way I can find > to load data into the db) to load these data into MyISAM tables which is > accomplished by continously running another C++ application which uses > the MySQL C API (which ultimately runs LOAD DATA INFILE). In order to do > this, I have modified the first app to write the data to ASCII files as > required by LOAD DATA INFILE. The problem is that there is a bottleneck > in organzing the data (which I do by appending to C++ strings) and then > dumping the strings to the flat files, so I'm looking for ways to increase > the performance in this area. Writing binary data would be faster. Does > MySQL support something similar to LOAD DATA INFILE for loading data from > binary files? Is there a faster way of getting the data into the db > without having to dump it to flat files? I've tried INSERTs, writing code > which essentially bridges the two apps, but of course INSERTs are much > slower than LOAD DATA INFILE. The data types themselves are simple: ints, > smallints, text, doubles, and floats. > Thanks for any help. > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] > > -- Thanks & Regards, Eldo Skaria -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
