Re: passing arguments to hadoop job

2013-01-21 Thread Satbeer Lamba
Do you mean to say you want to count the words from 20 th line onwards? OR You want to start counting from 200? For example if HELLO appears once it's count is 201. Please clarify On Jan 22, 2013 5:22 AM, jamal sasha jamalsha...@gmail.com wrote: Hi, Lets say I have the standard

Re: passing arguments to hadoop job

2013-01-21 Thread jamal sasha
The second one. If the word hello appears once, its count is 201. :) On Mon, Jan 21, 2013 at 5:40 PM, Satbeer Lamba satbeer.la...@gmail.comwrote: Do you mean to say you want to count the words from 20 th line onwards? OR You want to start counting from 200? For example if

Re: passing arguments to hadoop job

2013-01-21 Thread Satbeer Lamba
Please be sure that you are getting the value of baseSum in reducer by adding a logger or syso. Also consider removing static in declaration of baseSum as it would add counts of previous keys. On Jan 22, 2013 7:17 AM, jamal sasha jamalsha...@gmail.com wrote: The second one. If the word hello

Re: passing arguments to hadoop job

2013-01-21 Thread jamal sasha
Hi, Thanks for taking out time. Will this not work : conf.setInt(basecount,20); ?? I am not sure how to add loger or syso (new to both java and hadoop :( ) On Mon, Jan 21, 2013 at 5:55 PM, Satbeer Lamba satbeer.la...@gmail.comwrote: Please be sure that you are getting the value of

Re: passing arguments to hadoop job

2013-01-21 Thread Satbeer Lamba
Just add System.out.println (baseCount); To check if expected value is getting displayed. If not then the value is not set propely in jobconf. Also you don't need to remove static. Sorry about that, it doesn't change the expected output, though it's not required. On Jan 22, 2013 7:34 AM, jamal

Re: passing arguments to hadoop job

2013-01-21 Thread Hemanth Yamijala
Hi, Please note that you are referring to a very old version of Hadoop. the current stable release is Hadoop 1.x. The API has changed in 1.x. Take a look at the wordcount example here: http://hadoop.apache.org/docs/r1.0.4/mapred_tutorial.html#Example%3A+WordCount+v2.0 But, in principle your

Re: passing arguments to hadoop job

2013-01-21 Thread jamal sasha
Hi, The driver code is actually the same as of java word count old example: copying from site public static void main(String[] args) throws Exception { JobConf conf = new JobConf(WordCount.class); conf.setJobName(wordcount); conf.setOutputKeyClass(Text.class);

Re: passing arguments to hadoop job

2013-01-21 Thread Mohammad Tariq
Hello Jamal, When you set something using conf.set(extparam, value), you need to read it by using context.getConfiguration().get(extparam) in your mapper or reducer. Also, no need to declare it as a global variable. One more thing, try to use the new API. HTH Warm Regards, Tariq

Re: passing arguments to hadoop job

2013-01-21 Thread Hemanth Yamijala
OK. The easiest way I can think of for debugging this is to add a System.out.println in your Reduce.configure code. The output will come in the logs specific to your reduce tasks. You can access these logs from the web ui of the jobtracker. Navigate to your job page from the Jobtracker UI reduce