On Thu, Jul 23, 2009 at 8:54 AM, bharath vissapragada < [email protected]> wrote:
> I am really thankful to you J-D for replying me inspite of ur busy > schedule. > I am still in a learning stage and there are no good guides on HBase other > than Its own one .. So please spare me and I really appreciate ur help . > > Now i got ur point that there is no need of hadoop while running Hbase MR > programs .... But iam confused abt the config . I have only set the > JAVA_HOME in the "hbase-env.sh" and other than that i didn't do anything .. > so i wonder if my conf was wrong or some error in that simple code ... > because stdout worked for me while writing mapreduce programs ... > You seem to be asking basic questions about how hadoop works. If you want an digestible overview on hadoop and all of its pieces and you are not getting satisfaction from the hadoop wiki or from its documentation, you might do better with one of the Hadoop books: http://wiki.apache.org/hadoop/Books. Just a suggestion, St.Ack > > Thanks once again! > > On Thu, Jul 23, 2009 at 9:14 PM, Jean-Daniel Cryans <[email protected] > >wrote: > > > The code itself is very simple, I was referring to your own > > description of your situation. You say you use standalone HBase yet > > you talk about Hadoop configuration. You also talk about the > > JobTracker web UI which is in no use since you run local jobs directly > > on HBase. > > > > J-D > > > > On Thu, Jul 23, 2009 at 11:41 AM, bharath > > vissapragada<[email protected]> wrote: > > > I used stdout for debugging while writing codes in hadoop MR programs > and > > it > > > worked fine ... > > > Can you please tell me wch part of the code u found confusing so that i > > can > > > explain it a bit clearly ... > > > > > > > > > On Thu, Jul 23, 2009 at 9:06 PM, Jean-Daniel Cryans < > [email protected] > > >wrote: > > > > > >> What you wrote is a bit confusing to me, sorry. > > >> > > >> The usual way to debug MR jobs is to define a logger and post with > > >> either info or debug level, not sysout like you did. I'm not even sure > > >> where the standard output is logged when using a local job. Also since > > >> this is local you won't see anything in your host:50030 web UI. So use > > >> apache common logging and you should see your output. > > >> > > >> J-D > > >> > > >> On Thu, Jul 23, 2009 at 11:13 AM, bharath > > >> vissapragada<[email protected]> wrote: > > >> > Thanks for ur reply J-D ... Im pasting some part of the code ... > > >> > > > >> > Im doing it frm the command line .. Iam pasting some part of the > code > > >> here > > >> > .... > > >> > > > >> > public void mapp(ImmutableBytesWritable row, RowResult value, > > >> > OutputCollector<Text, Text> output, Reporter reporter) throws > > IOException > > >> { > > >> > System.out.println(row); > > >> > } > > >> > > > >> > public JobConf createSubmittableJob(String[] args) throws > IOException > > { > > >> > JobConf c = new JobConf(getConf(), > > >> MR_DS_Scan_Case1.class); > > >> > c.set("col.name", args[1]); > > >> > c.set("operator.name",args[2]); > > >> > c.set("val.name",args[3]); > > >> > IdentityTableMap.initJob(args[0], args[1], > > >> this.getClass(), > > >> > c); > > >> > c.setOutputFormat(NullOutputFormat.class); > > >> > return c > > >> > } > > >> > > > >> > As u can see ... im just printing the value of row in the map .. i > > can't > > >> see > > >> > in the terminal ..... > > >> > I only wan't the map phase ... so i didn't write any reduce phase .. > > is > > >> my > > >> > jobConf correct?? > > >> > > > >> > Also as i have already asked how to check the job logs and web > > interface > > >> > like "localhost:<port>/jobTracker.jsp"... since im running in local > > mode > > >> ... > > >> > > > >> > On Thu, Jul 23, 2009 at 6:32 PM, Jean-Daniel Cryans < > > [email protected] > > >> >wrote: > > >> > > > >> >> What output do you need exactly? I see that you have 8 output > records > > >> >> in your reduce task so if you take a look in your output folder or > > >> >> table (I don't know which sink you used) you should see them. > > >> >> > > >> >> Also did you run your MR inside Eclipse or in command line? > > >> >> > > >> >> Thx, > > >> >> > > >> >> J-D > > >> >> > > >> >> On Thu, Jul 23, 2009 at 8:30 AM, bharath > > >> >> vissapragada<[email protected]> wrote: > > >> >> > This is the output i go t.. seems everything is fine ..but no > > output!! > > >> >> > > > >> >> > 09/07/23 23:25:36 INFO jvm.JvmMetrics: Initializing JVM Metrics > > with > > >> >> > processName=JobTracker, sessionId= > > >> >> > 09/07/23 23:25:36 WARN mapred.JobClient: No job jar file set. > User > > >> >> classes > > >> >> > may not be found. See JobConf(Class) or JobConf#setJar(String). > > >> >> > 09/07/23 23:25:36 INFO mapred.TableInputFormatBase: split: > > >> >> > 0->localhost.localdomain:, > > >> >> > 09/07/23 23:25:37 INFO mapred.JobClient: Running job: > > job_local_0001 > > >> >> > 09/07/23 23:25:37 INFO mapred.TableInputFormatBase: split: > > >> >> > 0->localhost.localdomain:, > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: numReduceTasks: 1 > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: io.sort.mb = 100 > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: data buffer = > > 79691776/99614720 > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: record buffer = > > 262144/327680 > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: Starting flush of map > output > > >> >> > 09/07/23 23:25:37 INFO mapred.MapTask: Finished spill 0 > > >> >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: > > >> >> Task:attempt_local_0001_m_000000_0 > > >> >> > is done. And is in the process of commiting > > >> >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > > >> >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task > > >> >> > 'attempt_local_0001_m_000000_0' done. > > >> >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > > >> >> > 09/07/23 23:25:37 INFO mapred.Merger: Merging 1 sorted segments > > >> >> > 09/07/23 23:25:37 INFO mapred.Merger: Down to the last > merge-pass, > > >> with 1 > > >> >> > segments left of total size: 333 bytes > > >> >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: > > >> >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: > > >> >> Task:attempt_local_0001_r_000000_0 > > >> >> > is done. And is in the process of commiting > > >> >> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: reduce > reduce > > >> >> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task > > >> >> > 'attempt_local_0001_r_000000_0' done. > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Job complete: > > job_local_0001 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Counters: 11 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: File Systems > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Local bytes > read=38949 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Local bytes > > written=78378 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map-Reduce Framework > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce input > groups=8 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Combine output > > records=0 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map input records=8 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce output > > records=8 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map output bytes=315 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map input bytes=0 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Combine input > > records=0 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Map output records=8 > > >> >> > 09/07/23 23:25:38 INFO mapred.JobClient: Reduce input > records=8 > > >> >> > > > >> >> > > > >> >> > On Thu, Jul 23, 2009 at 12:17 PM, bharath vissapragada < > > >> >> > [email protected]> wrote: > > >> >> > > > >> >> >> since i haven;t started the cluster .. i can even see the > details > > in > > >> >> >> "localhost:<port>/jobTracker.jsp" .. i didn't even add anything > > to > > >> >> >> hadoop/conf/hadoop-site.xml > > >> >> >> > > >> >> >> > > >> >> >> On Thu, Jul 23, 2009 at 12:16 PM, bharath vissapragada < > > >> >> >> [email protected]> wrote: > > >> >> >> > > >> >> >>> Hi all , > > >> >> >>> > > >> >> >>> I wanted to run HBase in standalone mode to check my Hbase MR > > >> programs > > >> >> ... > > >> >> >>> I have dl a built version of hbase-0.20. and i have hadoop > 0.19.3 > > >> >> >>> > > >> >> >>> "I have set JAVA_HOME in both of them" .. then i started hbase > > and > > >> >> >>> inserted some tables using JAVA API .. Now i have written some > MR > > >> >> programs > > >> >> >>> onHBase and when i run them on Hbase it runs perfectly without > > any > > >> >> errors > > >> >> >>> and all the Map -reduce statistics are displayed correctly but > i > > >> get > > >> >> no > > >> >> >>> output . > > >> >> >>> > > >> >> >>> I have one doubt now .. how do HBase recognize hadoop in stand > > alone > > >> >> >>> mode(i haven;t started my hadoop even) .. Even simple print > > >> statements > > >> >> donot > > >> >> >>> work .. no output is displayed on the screen ... I doubt my > > config > > >> .... > > >> >> >>> > > >> >> >>> Do i need to add some config to run them ... Please reply ... > > >> >> >>> > > >> >> >> > > >> >> >> > > >> >> > > > >> >> > > >> > > > >> > > > > > >
