Hi,
I want to insert massive amounts of data in near real-time into a MySQL database. The data is in a structured binary format, and I have code that parses the data into logical data structures.
Right now I can load into a MySQL database via ODBC, but I need to improve throughput. How can I bulk insert the data more efficiently?
I would prefer not to use LOAD FILE because it involves converting the binary format to text, which will have a much larger footprint. To illustrate the pain of text files in my application, a single binary file might contain up to a million rows! And I might load hundreds, if not thousands of these files in a day!
How can I load data directly into the database at a lower more direct level than LOAD FILE? What are my options to integrate my data parser with MySQL?
I'm curious why you're concerned about the efficiency of LOAD FILE when you're using ODBC, an API known to add a fair amount of overhead.
Have you actually tried LOAD FILE and found it empirically to be slower than ODBC?
thanks, casey
-- Paul DuBois, Senior Technical Writer Madison, Wisconsin, USA MySQL AB, www.mysql.com
Are you MySQL certified? http://www.mysql.com/certification/
-- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
