one the command line, you can use the -D option after the GiraphRunner
class before the GiraphRunner specific parameters, e.g. -D giraph.
useSuperstepCounters=false
On Tue, Sep 10, 2013 at 1:15 PM, Christian Krause <m...@ckrause.org> wrote:
> Thanks a lot. One last question: where do I set options like
> USE_SUPERSTEP_COUNTERS?
>
> Christian
>
>
> 2013/9/9 André Kelpe <efeshundert...@googlemail.com>
>
>> On older versions of hadoop, you cannot set the counters to a higher
>> value. That was only introduced later. I had this issue on CDH3 (~1.5
>> years ago) and my solution was to disable all counters for the giraph
>> job, to make it work. If you use a more modern version of hadoop, it
>> should be possible to increase the limit though.
>>
>> - André
>>
>> 2013/9/9 Avery Ching <ach...@apache.org>:
>> > If you are running out of counters, you can turn off the superstep
>> counters
>> >
>> > /** Use superstep counters? (boolean) */
>> > BooleanConfOption USE_SUPERSTEP_COUNTERS =
>> > new BooleanConfOption("giraph.useSuperstepCounters", true,
>> > "Use superstep counters? (boolean)");
>> >
>> >
>> > On 9/9/13 6:43 AM, Claudio Martella wrote:
>> >
>> > No, I used a different counters limit on that hadoop version. Setting
>> > mapreduce.job.counters.limit to a higher number and restarting JT and TT
>> > worked for me. Maybe 64000 might be too high? Try setting it to 512.
>> Does
>> > not look like the case, but who knows.
>> >
>> >
>> > On Mon, Sep 9, 2013 at 2:57 PM, Christian Krause <m...@ckrause.org>
>> wrote:
>> >>
>> >> Sorry, it still doesn't work (I ran into a different problem before I
>> >> reached the limit).
>> >>
>> >> I am using Hadoop 0.20.203.0. Is the limit of 120 counters maybe
>> >> hardcoded?
>> >>
>> >> Cheers
>> >> Christian
>> >>
>> >> Am 09.09.2013 08:29 schrieb "Christian Krause" <m...@ckrause.org>:
>> >>
>> >>> I changed the property name to mapred.job.counters.limit and
>> restarted it
>> >>> again. Now it works.
>> >>>
>> >>> Thanks,
>> >>> Christian
>> >>>
>> >>>
>> >>> 2013/9/7 Claudio Martella <claudio.marte...@gmail.com>
>> >>>>
>> >>>> did you restart TT and JT?
>> >>>>
>> >>>>
>> >>>> On Sat, Sep 7, 2013 at 7:09 AM, Christian Krause <m...@ckrause.org>
>> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>> I've increased the counter limit in mapred-site.xml, but I still get
>> >>>>> the error: Exceeded counter limits - Counters=121 Limit=120.
>> Groups=6
>> >>>>> Limit=50.
>> >>>>>
>> >>>>> This is my config:
>> >>>>>
>> >>>>> cat conf/mapred-site.xml
>> >>>>> <?xml version="1.0"?>
>> >>>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> >>>>>
>> >>>>> <!-- Put site-specific property overrides in this file. -->
>> >>>>>
>> >>>>> <configuration>
>> >>>>> ...
>> >>>>> <property>
>> >>>>> <name>mapreduce.job.counters.limit</name>
>> >>>>> <value>64000</value>
>> >>>>> </property>
>> >>>>> <property>
>> >>>>> <name>mapred.task.timeout</name>
>> >>>>> <value>2400000</value>
>> >>>>> </property>
>> >>>>> ...
>> >>>>> </configuration>
>> >>>>>
>> >>>>> Any ideas?
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Christian
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Claudio Martella
>> >>>> claudio.marte...@gmail.com
>> >>>
>> >>>
>> >
>> >
>> >
>> > --
>> > Claudio Martella
>> > claudio.marte...@gmail.com
>> >
>> >
>>
>
>
--
Claudio Martella
claudio.marte...@gmail.com
