DBOutputFormat with one reducer

Hi, We have MapReduce program which writes data to mysql database using DBOutputFormat. Our program has one reducer. I understand that all the inserts happen during the close() operation of the reducer. Is it gauranteed that this operation is atomic ? i.e; what happens if the writes fail in

RE: Moving Secondary NameNode

Node" daemon process from "master" on to "slave" machine ? Thanks, Giridhar. From: Giridhar Addepalli [mailto:giridhar.addepa...@komli.com] Sent: Friday, May 13, 2011 2:43 PM To: mapreduce-user@hadoop.apache.org Subject: Moving Secondary NameNode Hi, As o

Moving Secondary NameNode

Hi, As of now, primary namenode and secondary namenode are running on the same machine in our configuration. As both are RAM heavy processes, we want to move secondary namenode to another machine in the cluster. What does this move take? Please refer me to some article which discusse

generating crc for files on hdfs

HI, How to generate crc for files on hdfs? I copied files from hdfs to remote machine, I want to verify integrity of files ( using copyToLocal command , I tried using -crc option too , but it looks like crc files does not exist on hdfs ) How should I proceed ? Thanks, Giridhar.

is configure( ) method of mapred api same as setup( ) method of mapreduce api

Hi, setup( ), method present in mapred api, is called once at the start of each map/reduce task. Is it the same with configure( ) method present in mapreduce api ? Thanks, Giridhar.

does Oozie work with new mapreduce api ?

Hi, I am using hadoop-0.20.2 version of hadoop. Want to use Oozie for managing workflows. All of my mapreduce programs use 'mapreduce' api instead of deprecated 'mapred' api. Downloaded oozie-2.2.1+78 , I see examples in here are using 'mapred' api. Is there any version of Oozie which suppor

trying to write output to a file whose name i can have control over

Hi, I am using hadoop 0.20.2 Maperduce framework by default writes output to part-r- etc. I want to write to a file with different name. I am trying to override "getDefaultWorkFile" method in TextOutputFormat class. I am getting following error : java.lang.IllegalArgumentExceptio

RE: Problem with DBOutputFormat

PM, Giridhar Addepalli wrote: > Hi, > > > > I am trying to write output to MYSQL DB, > > I am getting following error > > > > java.io.IOException > >     at > org.apache.hadoop.mapreduce.lib.db.DBOutputFormat.getRecordWriter(DBOut

Problem with DBOutputFormat

Hi, I am trying to write output to MYSQL DB, I am getting following error java.io.IOException at org.apache.hadoop.mapreduce.lib.db.DBOutputFormat.getRecordWriter(DBOutp utFormat.java:180) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.j