you can set it in your giraph-site.xml, but it should work on the command
line.
On Tue, Sep 10, 2013 at 1:44 PM, Christian Krause <m...@ckrause.org> wrote:
> I still see the number of counters increasing in the job tracker :(. Can I
> also set it in my giraph-site.xml or directly in my MasterCompute class?
>
> Cheers,
> Christian
>
>
> 2013/9/10 Claudio Martella <claudio.marte...@gmail.com>
>
>> one the command line, you can use the -D option after the GiraphRunner
>> class before the GiraphRunner specific parameters, e.g. -D giraph.
>> useSuperstepCounters=false
>>
>>
>> On Tue, Sep 10, 2013 at 1:15 PM, Christian Krause <m...@ckrause.org> wrote:
>>
>>> Thanks a lot. One last question: where do I set options like
>>> USE_SUPERSTEP_COUNTERS?
>>>
>>> Christian
>>>
>>>
>>> 2013/9/9 André Kelpe <efeshundert...@googlemail.com>
>>>
>>>> On older versions of hadoop, you cannot set the counters to a higher
>>>> value. That was only introduced later. I had this issue on CDH3 (~1.5
>>>> years ago) and my solution was to disable all counters for the giraph
>>>> job, to make it work. If you use a more modern version of hadoop, it
>>>> should be possible to increase the limit though.
>>>>
>>>> - André
>>>>
>>>> 2013/9/9 Avery Ching <ach...@apache.org>:
>>>> > If you are running out of counters, you can turn off the superstep
>>>> counters
>>>> >
>>>> > /** Use superstep counters? (boolean) */
>>>> > BooleanConfOption USE_SUPERSTEP_COUNTERS =
>>>> > new BooleanConfOption("giraph.useSuperstepCounters", true,
>>>> > "Use superstep counters? (boolean)");
>>>> >
>>>> >
>>>> > On 9/9/13 6:43 AM, Claudio Martella wrote:
>>>> >
>>>> > No, I used a different counters limit on that hadoop version. Setting
>>>> > mapreduce.job.counters.limit to a higher number and restarting JT and
>>>> TT
>>>> > worked for me. Maybe 64000 might be too high? Try setting it to 512.
>>>> Does
>>>> > not look like the case, but who knows.
>>>> >
>>>> >
>>>> > On Mon, Sep 9, 2013 at 2:57 PM, Christian Krause <m...@ckrause.org>
>>>> wrote:
>>>> >>
>>>> >> Sorry, it still doesn't work (I ran into a different problem before I
>>>> >> reached the limit).
>>>> >>
>>>> >> I am using Hadoop 0.20.203.0. Is the limit of 120 counters maybe
>>>> >> hardcoded?
>>>> >>
>>>> >> Cheers
>>>> >> Christian
>>>> >>
>>>> >> Am 09.09.2013 08:29 schrieb "Christian Krause" <m...@ckrause.org>:
>>>> >>
>>>> >>> I changed the property name to mapred.job.counters.limit and
>>>> restarted it
>>>> >>> again. Now it works.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> Christian
>>>> >>>
>>>> >>>
>>>> >>> 2013/9/7 Claudio Martella <claudio.marte...@gmail.com>
>>>> >>>>
>>>> >>>> did you restart TT and JT?
>>>> >>>>
>>>> >>>>
>>>> >>>> On Sat, Sep 7, 2013 at 7:09 AM, Christian Krause <m...@ckrause.org>
>>>> wrote:
>>>> >>>>>
>>>> >>>>> Hi,
>>>> >>>>> I've increased the counter limit in mapred-site.xml, but I still
>>>> get
>>>> >>>>> the error: Exceeded counter limits - Counters=121 Limit=120.
>>>> Groups=6
>>>> >>>>> Limit=50.
>>>> >>>>>
>>>> >>>>> This is my config:
>>>> >>>>>
>>>> >>>>> cat conf/mapred-site.xml
>>>> >>>>> <?xml version="1.0"?>
>>>> >>>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>>>> >>>>>
>>>> >>>>> <!-- Put site-specific property overrides in this file. -->
>>>> >>>>>
>>>> >>>>> <configuration>
>>>> >>>>> ...
>>>> >>>>> <property>
>>>> >>>>> <name>mapreduce.job.counters.limit</name>
>>>> >>>>> <value>64000</value>
>>>> >>>>> </property>
>>>> >>>>> <property>
>>>> >>>>> <name>mapred.task.timeout</name>
>>>> >>>>> <value>2400000</value>
>>>> >>>>> </property>
>>>> >>>>> ...
>>>> >>>>> </configuration>
>>>> >>>>>
>>>> >>>>> Any ideas?
>>>> >>>>>
>>>> >>>>> Cheers,
>>>> >>>>> Christian
>>>> >>>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Claudio Martella
>>>> >>>> claudio.marte...@gmail.com
>>>> >>>
>>>> >>>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Claudio Martella
>>>> > claudio.marte...@gmail.com
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>>
>> --
>> Claudio Martella
>> claudio.marte...@gmail.com
>>
>
>
--
Claudio Martella
claudio.marte...@gmail.com
