Re: new mapreduce API and NLineInputFormat

2011-01-14 Thread Attila Csordas 
Hi,

what other jars should be added to the build path from 0.21.0
besides hadoop-common-0.21.0.jar in order to make 0.21.0 NLineInputFormat
work in 0.20.2 as suggested below?

Generally can somebody provide me a working example code?

Thanks,
Attila


On Wed, Nov 10, 2010 at 5:06 AM, Harsh J qwertyman...@gmail.com wrote:

 Hi,

 On Tue, Nov 9, 2010 at 8:42 PM, Henning Blohm henning.bl...@zfabrik.de
 wrote:
 
  However in 0.20.2 you cannot call
 
  job.setInputFormatClass(NLineInputFormat.class);
 
  as NLineInputFormat does not extend the right InputFormat interface
  (in
  contrast to the 0.21 version).
 

 0.20.2 does not have it. You can pull the implementation from 0.21.0
 and use it from within your packages if you require it, though. There
 should be no problems in doing it.

 Here's the file from the 0.21 branch:

 http://svn.apache.org/viewvc/hadoop/mapreduce/tags/release-0.21.0/src/java/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.java?view=co

 --
 Harsh J
 www.harshj.com

Re: new mapreduce API and NLineInputFormat

2011-01-14 Thread Edward Capriolo 
On Fri, Jan 14, 2011 at 5:05 PM, Attila Csordas attilacsor...@gmail.com wrote:
 Hi,

 what other jars should be added to the build path from 0.21.0
 besides hadoop-common-0.21.0.jar in order to make 0.21.0 NLineInputFormat
 work in 0.20.2 as suggested below?

 Generally can somebody provide me a working example code?

 Thanks,
 Attila


 On Wed, Nov 10, 2010 at 5:06 AM, Harsh J qwertyman...@gmail.com wrote:

 Hi,

 On Tue, Nov 9, 2010 at 8:42 PM, Henning Blohm henning.bl...@zfabrik.de
 wrote:
 
  However in 0.20.2 you cannot call
 
  job.setInputFormatClass(NLineInputFormat.class);
 
  as NLineInputFormat does not extend the right InputFormat interface
  (in
  contrast to the 0.21 version).
 

 0.20.2 does not have it. You can pull the implementation from 0.21.0
 and use it from within your packages if you require it, though. There
 should be no problems in doing it.

 Here's the file from the 0.21 branch:

 http://svn.apache.org/viewvc/hadoop/mapreduce/tags/release-0.21.0/src/java/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.java?view=co

 --
 Harsh J
 www.harshj.com



Should should not add 0.21 jars to the 0.20 classpath. It will likely
cause a conflict. Just put NLineInputFormat.java in your own project
and try to get it to compile.

Edward

new mapreduce API and NLineInputFormat

2010-11-09 Thread Henning Blohm 
I am using the new mapreduce.* API in my jobs on hadoop 0.20.2. - I
actually 
have some utilities around job scheduling and such.

Now I would like to use NLineInputFormat to parallelize some data import
task into a number of mappers.

However in 0.20.2 you cannot call

job.setInputFormatClass(NLineInputFormat.class);

as NLineInputFormat does not extend the right InputFormat interface
(in 
contrast to the 0.21 version).

Is there any way to make this work on 0.20.2? 

Thanks,
  Henning

Re: new mapreduce API and NLineInputFormat

2010-11-09 Thread Harsh J 
Hi,

On Tue, Nov 9, 2010 at 8:42 PM, Henning Blohm henning.bl...@zfabrik.de wrote:

 However in 0.20.2 you cannot call

 job.setInputFormatClass(NLineInputFormat.class);

 as NLineInputFormat does not extend the right InputFormat interface
 (in
 contrast to the 0.21 version).


0.20.2 does not have it. You can pull the implementation from 0.21.0
and use it from within your packages if you require it, though. There
should be no problems in doing it.

Here's the file from the 0.21 branch:
http://svn.apache.org/viewvc/hadoop/mapreduce/tags/release-0.21.0/src/java/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.java?view=co

-- 
Harsh J
www.harshj.com

Re: new mapreduce API and NLineInputFormat

2010-11-09 Thread Henning Blohm 
Right, that's what I will do.

Thanks,
Henning

On Wed, 2010-11-10 at 10:36 +0530, Harsh J wrote:

 Hi,
 
 On Tue, Nov 9, 2010 at 8:42 PM, Henning Blohm henning.bl...@zfabrik.de 
 wrote:
 
  However in 0.20.2 you cannot call
 
  job.setInputFormatClass(NLineInputFormat.class);
 
  as NLineInputFormat does not extend the right InputFormat interface
  (in
  contrast to the 0.21 version).
 
 
 0.20.2 does not have it. You can pull the implementation from 0.21.0
 and use it from within your packages if you require it, though. There
 should be no problems in doing it.
 
 Here's the file from the 0.21 branch:
 http://svn.apache.org/viewvc/hadoop/mapreduce/tags/release-0.21.0/src/java/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.java?view=co