On older versions of hadoop, you cannot set the counters to a higher
value. That was only introduced later. I had this issue on CDH3 (~1.5
years ago) and my solution was to disable all counters for the giraph
job, to make it work. If you use a more modern version of hadoop, it
should be possible to increase the limit though.
- André
2013/9/9 Avery Ching <ach...@apache.org>:
> If you are running out of counters, you can turn off the superstep counters
>
> /** Use superstep counters? (boolean) */
> BooleanConfOption USE_SUPERSTEP_COUNTERS =
> new BooleanConfOption("giraph.useSuperstepCounters", true,
> "Use superstep counters? (boolean)");
>
>
> On 9/9/13 6:43 AM, Claudio Martella wrote:
>
> No, I used a different counters limit on that hadoop version. Setting
> mapreduce.job.counters.limit to a higher number and restarting JT and TT
> worked for me. Maybe 64000 might be too high? Try setting it to 512. Does
> not look like the case, but who knows.
>
>
> On Mon, Sep 9, 2013 at 2:57 PM, Christian Krause <m...@ckrause.org> wrote:
>>
>> Sorry, it still doesn't work (I ran into a different problem before I
>> reached the limit).
>>
>> I am using Hadoop 0.20.203.0. Is the limit of 120 counters maybe
>> hardcoded?
>>
>> Cheers
>> Christian
>>
>> Am 09.09.2013 08:29 schrieb "Christian Krause" <m...@ckrause.org>:
>>
>>> I changed the property name to mapred.job.counters.limit and restarted it
>>> again. Now it works.
>>>
>>> Thanks,
>>> Christian
>>>
>>>
>>> 2013/9/7 Claudio Martella <claudio.marte...@gmail.com>
>>>>
>>>> did you restart TT and JT?
>>>>
>>>>
>>>> On Sat, Sep 7, 2013 at 7:09 AM, Christian Krause <m...@ckrause.org> wrote:
>>>>>
>>>>> Hi,
>>>>> I've increased the counter limit in mapred-site.xml, but I still get
>>>>> the error: Exceeded counter limits - Counters=121 Limit=120. Groups=6
>>>>> Limit=50.
>>>>>
>>>>> This is my config:
>>>>>
>>>>> cat conf/mapred-site.xml
>>>>> <?xml version="1.0"?>
>>>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>>>>>
>>>>> <!-- Put site-specific property overrides in this file. -->
>>>>>
>>>>> <configuration>
>>>>> ...
>>>>> <property>
>>>>> <name>mapreduce.job.counters.limit</name>
>>>>> <value>64000</value>
>>>>> </property>
>>>>> <property>
>>>>> <name>mapred.task.timeout</name>
>>>>> <value>2400000</value>
>>>>> </property>
>>>>> ...
>>>>> </configuration>
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Cheers,
>>>>> Christian
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claudio Martella
>>>> claudio.marte...@gmail.com
>>>
>>>
>
>
>
> --
> Claudio Martella
> claudio.marte...@gmail.com
>
>
