On Wednesday 28 September 2011 12:25 PM, Arsen Zahray wrote:
Hey! Thank you for replying!
Please, confirm that I understand you correctly:
1. Use a class, which extends mapper
class MyMapper extends Mapper<LongWritable, Text, IntWritable, MyPage> {
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
//implement all logic here
}
}
2. Leave MultithreadMapper class empty:
public class MyMultithreadMapper extends
MultithreadedMapper<LongWritable, Text, IntWritable, MyPage> {
}
3. In main set
MultithreadedMapper.setMapperClass(job, MyMapper.class);
Should I set
job.setMapperClass(MyMapper.class);
or
job.setMapperClass(MyMultithreadMapper.class);
?
Arsen
On Wed, Sep 28, 2011 at 9:38 AM, Kamesh <kames...@imaginea.com
<mailto:kames...@imaginea.com>> wrote:
On Wednesday 28 September 2011 11:33 AM, Arsen Zahray wrote:
MultithreadMapper extends MultithreadedMapper<LongWritable, Text,
IntWritable, MyPage> {
ConcurrentLinkedQueue<MyScraper> scrapers = new
ConcurrentLinkedQueue<MyScraper>();
public static final int nThreads = 5;
public MyMultithreadMapper() {
for (int i = 0; i < nT
implement the map logic in a separate Mapper class and in the main
method set the following property
MultithreadedMapper.setMapperClass(job, MyMapper.class);
--
/Thanks&Regards,/
/Bh.V.S.Kamesh/
set the following property
job.setMapperClass(MyMultithreadMapper.class);
--
/Thanks&Regards,/
/Bh.V.S.Kamesh/
