discp problem: global counters are inaccurate
The following distcp command has been working fine until today. Does anyone have any idea on what does global counters are inaccurate mean? ~/projects/pig] hadoop distcp s3n://$ACCESS_KEY_ID:$secret_access_key...@$bucket_amf/ hdfs://hadoop01.dc1.foo.com:9000/user/bar/data/amfdata With failures, global counters are inaccurate; consider running with -i Copy failed: java.lang.NullPointerException at org.apache.hadoop.tools.DistCp.makeRelative(DistCp.java:801) at org.apache.hadoop.tools.DistCp.setup(DistCp.java:979) at org.apache.hadoop.tools.DistCp.copy(DistCp.java:614) at org.apache.hadoop.tools.DistCp.run(DistCp.java:764) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.tools.DistCp.main(DistCp.java:784)
Re: Problem with Counters
Thank you all so much.
That works. I made a stupid mistake with the naming of a local variable. so
the error. :(
On Thu, Feb 5, 2009 at 9:49 AM, Tom White t...@cloudera.com wrote:
Try moving the enum to inside the top level class (as you already did)
and then use getCounter() passing the enum value:
public class MyJob {
static enum MyCounter{ct_key1};
// Mapper and Reducer defined here
public static void main(String[] args) throws IOException {
// ...
RunningJob running =JobClient.runJob(conf);
Counters ct = running.getCounters();
long res = ct.getCounter(MyCounter.ct_key1);
// ...
}
}
BTW org.apache.hadoop.mapred.Task$Counter is a built-in MapReduce
counter, so that won't help you retrieve your custom counter.
Cheers,
Tom
On Thu, Feb 5, 2009 at 2:22 PM, Rasit OZDAS rasitoz...@gmail.com wrote:
Sharath,
You're using reporter.incrCounter(enumVal, intVal); to increment
counter,
I think method to get should also be similar.
Try to use findCounter(enumVal).getCounter() or getCounter(enumVal).
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
In fact I put the enum in my Reduce method as the following link (from
Yahoo) says so:
http://public.yahoo.com/gogate/hadoop-tutorial/html/module5.html#metrics
---Look at the section under Reporting Custom Metrics.
2009/2/5 some speed speed.s...@gmail.com
Thanks Rasit.
I did as you said.
1) Put the static enum MyCounter{ct_key1} just above main()
2) Changed result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 1,
Reduce.MyCounter).getCounter();
Still is doesnt seem to help. It throws a null pointer exception.Its
not
able to find the Counter.
Thanks,
Sharath
On Thu, Feb 5, 2009 at 8:04 AM, Rasit OZDAS rasitoz...@gmail.com
wrote:
Forgot to say, value 0 means that the requested counter does not
exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but
it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result
seems
to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com
wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com
wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed
speed.s...@gmail.com
wrote:
Hi,
Can someone help me with the usage of counters please? I am
incrementing
a
counter in Reduce method but I am unable to collect the counter
value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
Problem with Counters
Hi,
Can someone help me with the usage of counters please? I am incrementing a
counter in Reduce method but I am unable to collect the counter value after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
Re: Problem with Counters
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com wrote:
Hi,
Can someone help me with the usage of counters please? I am incrementing a
counter in Reduce method but I am unable to collect the counter value after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
Re: Problem with Counters
I Tried the following...It gets compiled but the value of result seems to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com wrote:
Hi,
Can someone help me with the usage of counters please? I am incrementing
a
counter in Reduce method but I am unable to collect the counter value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
Re: Problem with Counters
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com wrote:
Hi,
Can someone help me with the usage of counters please? I am incrementing
a
counter in Reduce method but I am unable to collect the counter value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
Re: Problem with Counters
Forgot to say, value 0 means that the requested counter does not exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com wrote:
Hi,
Can someone help me with the usage of counters please? I am incrementing
a
counter in Reduce method but I am unable to collect the counter value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
Re: Problem with Counters
Thanks Rasit.
I did as you said.
1) Put the static enum MyCounter{ct_key1} just above main()
2) Changed result = ct.findCounter(org.apache.hadoop.mapred.Task$Counter,
1, Reduce.MyCounter).getCounter();
Still is doesnt seem to help. It throws a null pointer exception.Its not
able to find the Counter.
Thanks,
Sharath
On Thu, Feb 5, 2009 at 8:04 AM, Rasit OZDAS rasitoz...@gmail.com wrote:
Forgot to say, value 0 means that the requested counter does not exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems
to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com
wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com
wrote:
Hi,
Can someone help me with the usage of counters please? I am
incrementing
a
counter in Reduce method but I am unable to collect the counter
value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
Re: Problem with Counters
In fact I put the enum in my Reduce method as the following link (from
Yahoo) says so:
http://public.yahoo.com/gogate/hadoop-tutorial/html/module5.html#metrics
---Look at the section under Reporting Custom Metrics.
2009/2/5 some speed speed.s...@gmail.com
Thanks Rasit.
I did as you said.
1) Put the static enum MyCounter{ct_key1} just above main()
2) Changed result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 1,
Reduce.MyCounter).getCounter();
Still is doesnt seem to help. It throws a null pointer exception.Its not
able to find the Counter.
Thanks,
Sharath
On Thu, Feb 5, 2009 at 8:04 AM, Rasit OZDAS rasitoz...@gmail.com wrote:
Forgot to say, value 0 means that the requested counter does not exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems
to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com
wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com
wrote:
Hi,
Can someone help me with the usage of counters please? I am
incrementing
a
counter in Reduce method but I am unable to collect the counter
value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
Re: Problem with Counters
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com wrote:
Hi,
Can someone help me with the usage of counters please? I am incrementing
a
counter in Reduce method but I am unable to collect the counter value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
Re: Problem with Counters
Sharath,
You're using reporter.incrCounter(enumVal, intVal); to increment counter,
I think method to get should also be similar.
Try to use findCounter(enumVal).getCounter() or getCounter(enumVal).
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
In fact I put the enum in my Reduce method as the following link (from
Yahoo) says so:
http://public.yahoo.com/gogate/hadoop-tutorial/html/module5.html#metrics
---Look at the section under Reporting Custom Metrics.
2009/2/5 some speed speed.s...@gmail.com
Thanks Rasit.
I did as you said.
1) Put the static enum MyCounter{ct_key1} just above main()
2) Changed result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 1,
Reduce.MyCounter).getCounter();
Still is doesnt seem to help. It throws a null pointer exception.Its not
able to find the Counter.
Thanks,
Sharath
On Thu, Feb 5, 2009 at 8:04 AM, Rasit OZDAS rasitoz...@gmail.com wrote:
Forgot to say, value 0 means that the requested counter does not exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems
to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com
wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com
wrote:
Hi,
Can someone help me with the usage of counters please? I am
incrementing
a
counter in Reduce method but I am unable to collect the counter
value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
Re: Problem with Counters
Try moving the enum to inside the top level class (as you already did)
and then use getCounter() passing the enum value:
public class MyJob {
static enum MyCounter{ct_key1};
// Mapper and Reducer defined here
public static void main(String[] args) throws IOException {
// ...
RunningJob running =JobClient.runJob(conf);
Counters ct = running.getCounters();
long res = ct.getCounter(MyCounter.ct_key1);
// ...
}
}
BTW org.apache.hadoop.mapred.Task$Counter is a built-in MapReduce
counter, so that won't help you retrieve your custom counter.
Cheers,
Tom
On Thu, Feb 5, 2009 at 2:22 PM, Rasit OZDAS rasitoz...@gmail.com wrote:
Sharath,
You're using reporter.incrCounter(enumVal, intVal); to increment counter,
I think method to get should also be similar.
Try to use findCounter(enumVal).getCounter() or getCounter(enumVal).
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
In fact I put the enum in my Reduce method as the following link (from
Yahoo) says so:
http://public.yahoo.com/gogate/hadoop-tutorial/html/module5.html#metrics
---Look at the section under Reporting Custom Metrics.
2009/2/5 some speed speed.s...@gmail.com
Thanks Rasit.
I did as you said.
1) Put the static enum MyCounter{ct_key1} just above main()
2) Changed result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 1,
Reduce.MyCounter).getCounter();
Still is doesnt seem to help. It throws a null pointer exception.Its not
able to find the Counter.
Thanks,
Sharath
On Thu, Feb 5, 2009 at 8:04 AM, Rasit OZDAS rasitoz...@gmail.com wrote:
Forgot to say, value 0 means that the requested counter does not exist.
2009/2/5 Rasit OZDAS rasitoz...@gmail.com:
Sharath,
I think the static enum definition should be out of Reduce class.
Hadoop probably tries to find it elsewhere with MyCounter, but it's
actually Reduce.MyCounter in your example.
Hope this helps,
Rasit
2009/2/5 some speed speed.s...@gmail.com:
I Tried the following...It gets compiled but the value of result seems
to be
0 always.
RunningJob running = JobClient.runJob(conf);
Counters ct = new Counters();
ct = running.getCounters();
long result =
ct.findCounter(org.apache.hadoop.mapred.Task$Counter, 0,
*MyCounter*).getCounter();
//even tried MyCounter.Key1
Does anyone know whay that is happening?
Thanks,
Sharath
On Thu, Feb 5, 2009 at 5:59 AM, some speed speed.s...@gmail.com
wrote:
Hi Tom,
I get the error :
Cannot find Symbol* **MyCounter.ct_key1 *
On Thu, Feb 5, 2009 at 5:51 AM, Tom White t...@cloudera.com wrote:
Hi Sharath,
The code you posted looks right to me. Counters#getCounter() will
return the counter's value. What error are you getting?
Tom
On Thu, Feb 5, 2009 at 10:09 AM, some speed speed.s...@gmail.com
wrote:
Hi,
Can someone help me with the usage of counters please? I am
incrementing
a
counter in Reduce method but I am unable to collect the counter
value
after
the job is completed.
Its something like this:
public static class Reduce extends MapReduceBase implements
ReducerText,
FloatWritable, Text, FloatWritable
{
static enum MyCounter{ct_key1};
public void reduce(..) throws IOException
{
reporter.incrCounter(MyCounter.ct_key1, 1);
output.collect(..);
}
}
-main method
{
RunningJob running = null;
running=JobClient.runJob(conf);
Counters ct = running.getCounters();
/* How do I Collect the ct_key1 value ***/
long res = ct.getCounter(MyCounter.ct_key1);
}
Thanks,
Sharath
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
--
M. Raşit ÖZDAŞ
