I used stdout for debugging while writing codes in hadoop MR programs and it worked fine ... Can you please tell me wch part of the code u found confusing so that i can explain it a bit clearly ...
On Thu, Jul 23, 2009 at 9:06 PM, Jean-Daniel Cryans <[email protected]>wrote: > What you wrote is a bit confusing to me, sorry. > > The usual way to debug MR jobs is to define a logger and post with > either info or debug level, not sysout like you did. I'm not even sure > where the standard output is logged when using a local job. Also since > this is local you won't see anything in your host:50030 web UI. So use > apache common logging and you should see your output. > > J-D > > On Thu, Jul 23, 2009 at 11:13 AM, bharath > vissapragada<[email protected]> wrote: > > Thanks for ur reply J-D ... Im pasting some part of the code ... > > > > Im doing it frm the command line .. Iam pasting some part of the code > here > > .... > > > > public void mapp(ImmutableBytesWritable row, RowResult value, > > OutputCollector<Text, Text> output, Reporter reporter) throws IOException > { > > System.out.println(row); > > } > > > > public JobConf createSubmittableJob(String[] args) throws IOException { > > JobConf c = new JobConf(getConf(), > MR_DS_Scan_Case1.class); > > c.set("col.name", args[1]); > > c.set("operator.name",args[2]); > > c.set("val.name",args[3]); > > IdentityTableMap.initJob(args[0], args[1], > this.getClass(), > > c); > > c.setOutputFormat(NullOutputFormat.class); > > return c > > } > > > > As u can see ... im just printing the value of row in the map .. i can't > see > > in the terminal ..... > > I only wan't the map phase ... so i didn't write any reduce phase .. is > my > > jobConf correct?? > > > > Also as i have already asked how to check the job logs and web interface > > like "localhost:<port>/jobTracker.jsp"... since im running in local mode > ... > > > > On Thu, Jul 23, 2009 at 6:32 PM, Jean-Daniel Cryans <[email protected] > >wrote: > > > >> What output do you need exactly? I see that you have 8 output records > >> in your reduce task so if you take a look in your output folder or > >> table (I don't know which sink you used) you should see them. > >> > >> Also did you run your MR inside Eclipse or in command line? > >> > >> Thx, > >> > >> J-D > >> > >> On Thu, Jul 23, 2009 at 8:30 AM, bharath > >> vissapragada<[email protected]> wrote: > >> > This is the output i go t.. seems everything is fine ..but no output!! > >> > > >> > 09/07/23 23:25:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with > >> > processName=JobTracker, sessionId= > >> > 09/07/23 23:25:36 WARN mapred.JobClient: No job jar file set. User > >> classes > >> > may not be found. See JobConf(Class) or JobConf#setJar(String). > >> > 09/07/23 23:25:36 INFO mapred.TableInputFormatBase: split: > >> > 0->localhost.localdomain:, > >> > 09/07/23 23:25:37 INFO mapred.JobClient: Running job: job_local_0001 > >> > 09/07/23 23:25:37 INFO mapred.TableInputFormatBase: split: > >> > 0->localhost.localdomain:, > >> > 09/07/23 23:25:37 INFO mapred.MapTask: numReduceTasks: 1 > >> > 09/07/23 23:25:37 INFO mapred.MapTask: io.sort.mb = 100 > >> > 09/07/23 23:25:37 INFO mapred.MapTask: data buffer = 79691776/99614720 > >> > 09/07/23 23:25:37 INFO mapred.MapTask: record buffer = 262144/327680 > >> > 09/07/23 23:25:37 INFO mapred.MapTask: Starting flush of map output > >> > 09/07/23 23:25:37 INFO mapred.MapTask: Finished spill 0 > >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: > >> Task:attempt_local_0001_m_000000_0 > >> > is done. And is in the process of commiting > >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task > >> > 'attempt_local_0001_m_000000_0' done. > >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > >> > 09/07/23 23:25:37 INFO mapred.Merger: Merging 1 sorted segments > >> > 09/07/23 23:25:37 INFO mapred.Merger: Down to the last merge-pass, > with 1 > >> > segments left of total size: 333 bytes > >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: > >> Task:attempt_local_0001_r_000000_0 > >> > is done. And is in the process of commiting > >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: reduce > reduce > >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task > >> > 'attempt_local_0001_r_000000_0' done. > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Job complete: job_local_0001 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Counters: 11 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: File Systems > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Local bytes read=38949 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Local bytes written=78378 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map-Reduce Framework > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce input groups=8 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Combine output records=0 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map input records=8 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce output records=8 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map output bytes=315 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map input bytes=0 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Combine input records=0 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map output records=8 > >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce input records=8 > >> > > >> > > >> > On Thu, Jul 23, 2009 at 12:17 PM, bharath vissapragada < > >> > [email protected]> wrote: > >> > > >> >> since i haven;t started the cluster .. i can even see the details in > >> >> "localhost:<port>/jobTracker.jsp" .. i didn't even add anything to > >> >> hadoop/conf/hadoop-site.xml > >> >> > >> >> > >> >> On Thu, Jul 23, 2009 at 12:16 PM, bharath vissapragada < > >> >> [email protected]> wrote: > >> >> > >> >>> Hi all , > >> >>> > >> >>> I wanted to run HBase in standalone mode to check my Hbase MR > programs > >> ... > >> >>> I have dl a built version of hbase-0.20. and i have hadoop 0.19.3 > >> >>> > >> >>> "I have set JAVA_HOME in both of them" .. then i started hbase and > >> >>> inserted some tables using JAVA API .. Now i have written some MR > >> programs > >> >>> onHBase and when i run them on Hbase it runs perfectly without any > >> errors > >> >>> and all the Map -reduce statistics are displayed correctly but i > get > >> no > >> >>> output . > >> >>> > >> >>> I have one doubt now .. how do HBase recognize hadoop in stand alone > >> >>> mode(i haven;t started my hadoop even) .. Even simple print > statements > >> donot > >> >>> work .. no output is displayed on the screen ... I doubt my config > .... > >> >>> > >> >>> Do i need to add some config to run them ... Please reply ... > >> >>> > >> >> > >> >> > >> > > >> > > >
