Re: getting Configuration object in mapper
On Dec 5, 2008, at 12:32 PM, Craig Macdonald wrote: I have a related question - I have a class which is both mapper and reducer. How can I tell in configure() if the current task is map or a reduce task? Parse the taskid? Get the taskid, then use http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/TaskAttemptID.html#forName(java.lang.String) followed by http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/TaskAttemptID.html#isMap() Arun
Re: getting Configuration object in mapper
check : mapred.task.is.map Craig Macdonald wrote: I have a related question - I have a class which is both mapper and reducer. How can I tell in configure() if the current task is map or a reduce task? Parse the taskid? C Owen O'Malley wrote: On Dec 4, 2008, at 9:19 PM, abhinit wrote: I have set some variable using the JobConf object. jobConf.set("Operator", operator) etc. How can I get an instance of Configuration object/ JobConf object inside a map method so that I can retrieve these variables. In your Mapper class, implement a method like: public void configure(JobConf job) { ... } This will be called when the object is created with the job conf. -- Owen
Re: getting Configuration object in mapper
I have a related question - I have a class which is both mapper and reducer. How can I tell in configure() if the current task is map or a reduce task? Parse the taskid? C Owen O'Malley wrote: On Dec 4, 2008, at 9:19 PM, abhinit wrote: I have set some variable using the JobConf object. jobConf.set("Operator", operator) etc. How can I get an instance of Configuration object/ JobConf object inside a map method so that I can retrieve these variables. In your Mapper class, implement a method like: public void configure(JobConf job) { ... } This will be called when the object is created with the job conf. -- Owen
Re: getting Configuration object in mapper
On Dec 4, 2008, at 9:19 PM, abhinit wrote: I have set some variable using the JobConf object. jobConf.set("Operator", operator) etc. How can I get an instance of Configuration object/ JobConf object inside a map method so that I can retrieve these variables. In your Mapper class, implement a method like: public void configure(JobConf job) { ... } This will be called when the object is created with the job conf. -- Owen