DBOutputFormat will very likely put significantly more load on your MySQL server vs. LOAD DATA INFILE. DBOutputFormat will trounce your MySQL server with at least one connection per reducer. This may be OK if you have a small number of reducers and a small amount of output data. LOAD DATA INFILE is a much safer approach, in that it won't potentially axe your MySQL database.
Alex On Tue, May 19, 2009 at 2:58 PM, dealmaker <vin...@gmail.com> wrote: > > Does DBOutputFormat have similar performance as Load Data Infile? > Thanks. > > > TimRobertson100 wrote: > > > > So you are using a java program to execute a "load data infile" > > command on mysql through JDBC? > > If so I *think* you would have to copy it onto the mysql machine from > > HDFS first, or the machine running the command and then try a 'load > > data local infile'. > > > > Or pehaps use the > > > http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/db/DBOutputFormat.html > > > > Cheers > > > > Tim > > > > On Tue, May 19, 2009 at 9:54 PM, dealmaker <vin...@gmail.com> wrote: > >> > >> Hi, > >> I am want to load data in mysql using a hadoop file similar to > >> following: > >> LOAD DATA INFILE 'test.txt' INTO TABLE test > >> FIELDS TERMINATED BY ',' LINES STARTING BY 'xxx'; > >> > >> But how do I load the hdfs file into the mysql comand above? Do I start > >> the > >> file name with hdfd://test.txt? I am using java. > >> Thanks. > >> > >> -- > >> View this message in context: > >> > http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23623256.html > >> Sent from the Hadoop core-user mailing list archive at Nabble.com. > >> > >> > > > > > > -- > View this message in context: > http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23625140.html > Sent from the Hadoop core-user mailing list archive at Nabble.com. > >
