Availability of Job traces

Hi guys ! Apart from generating the job traces from RUMEN , can i get logs or job traces of varied sizes from some organizations. How can i make sure that the rumen generates only say 25 jobs,50 jobs or so ? Thanks, Arun

Re: number of running reducers exceed max number of mappers on cluster

For how to control Pig's parallelism: http://pig.apache.org/docs/r0.9.1/perf.html#parallel For your 'running' task count bug: https://issues.apache.org/jira/browse/MAPREDUCE-486 (Which would mean that only 5 may be 'running', while the rest may have already completed and were awaiting finaliza

How to use mapper counters in reducer

Hi, I m defining custom counters in mapper that I want to access in reducer in new API. Does anyone know how to do this ? Thanks, JJ Sent from my iPhone

Problem building native library

Hi all, I am trying to build hadoop on a Intel(R) Xeon(R) CPU X5675. It seems that there is a problem with the build.xml, it does not accept any target ant -Dcompile.native=true Linux-x86_64-64 the same with : ant -Dcompile.native=true Linux-i386-32 am I doing wrong? Bests,

Re: Is there a way to re-use the output of job which was killed?

Hi Harsh, thanks for the answer. This is the same approach I was thinking of. But, let me try to give you some more details about the problem and my proposal for its solution. The problem I'm trying to solve can be defined as "restarting the job which was killed without re-processing data". As I k

Re: Can I set mappers to use overlapping record ranges from a sequence file?

I have never done this, but I think it should be possible. You will likely not get much data locality doing this, but you should be able create your own input format and have it write out entries in the split file that indicate the ranges you wanted. I may be wrong but I thought that the Input

Re: Renaming WritableComparable and existing SequenceFiles

Solved! Another tool reading the same files expects various childs of WritableComparable and Writable, this of course won't throw a class cast exception. That same tool writes the using the new class. In essence, i already had a converter tool. Thanks On Wednesday 30 November 2011 16:44:49 Ma

Renaming WritableComparable and existing SequenceFiles

Hi, I've built some SequenceFiles using a custom WritableComparable. I also decided to reorganize package structure and ended up renaming the whole thing. Since the key and value classes are embedded in the file i cannot read them because it either doesn't exist or cannot be cast in case the sa

how to access a mapper counter in reducer

I have created a counter in mapper to count something, I wanna get the counter's value in reducer phase, the code segment is as follow: public class MM extends Mapper { static enum TEST{ pt } @Override public void map(LongWritable key, Text values, Context context) throws IOExceptio