at the JT would prevent speculative
> execution no matter what the client says, but I could be mistaken.
>
> Sandy
>
> > -Original Message-
> > From: anil gupta [mailto:anilgupt...@gmail.com]
> > Sent: Friday, March 30, 2012 14:53
> > To: user@hbase.a
com]
> Sent: Friday, March 30, 2012 14:53
> To: user@hbase.apache.org
> Subject: HBase bulk loader doing speculative execution when it set to false in
> mapred-site.xml
>
> Hi All,
>
> I am using cdh3u2. I ran HBase bulk loading with property
> "mapred.reduce.tasks.spec
+common-u...@hadoop.apache.org
Hi Harsh,
Thanks for the information.
Is there any way to differentiate between a client side property and
server-side property?or a Document which enlists whether a property is
server or client-side? Many times i have to speculate over this and try out
test runs.
Hash
I think you have a good point here - It is a good practice that the
utilities given from HBase also adapt HBase own recommendations.
For example the RowCounter
(org.apache.hadoop.hbase.mapreduce.RowCounter.java) utility is neither
setting the speculative execution to 'false', nor the scan cach
Anil,
You can also disable speculative execution on a per-job basis. See
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setMapSpeculativeExecution(boolean)
(Which is why it is called a client-sided property - it applies
per-job).
If HBase strongly recommends
Hi Doug,
Yes, that's why i had set that property as false in my mapred-site.xml.
But, to my surprise i didnt know that setting that property would be
useless for Hadoop jobs unless the mapred-site.xml is in classpath. The
idea of client side property is a little confusing to me at present since
th
Speculative execution is on by default in Hadoop. One of the Performance
recommendations in the Hbase RefGuide is to turn it off.
On 3/30/12 6:12 PM, "Jean-Daniel Cryans" wrote:
>Well that's not an HBase configuration, that's Hadoop. I'm not sure if
>this is listed anywhere, maybe in the b
Well that's not an HBase configuration, that's Hadoop. I'm not sure if
this is listed anywhere, maybe in the book.
BTW usually HBase has a "client" somewhere in the same to indicate
it's client side.
J-D
On Fri, Mar 30, 2012 at 3:08 PM, anil gupta wrote:
> Thanks for the quick reply, Jean. Is t
Thanks for the quick reply, Jean. Is there any link where i can find the
name of all client-side configuration for HBase?
~Anil
On Fri, Mar 30, 2012 at 3:01 PM, Jean-Daniel Cryans wrote:
> This is a client-side configuration so if your mapred-site.xml is
> _not_ on your classpath when you start
This is a client-side configuration so if your mapred-site.xml is
_not_ on your classpath when you start the bulk load, it's not going
to pick it up. So either have that file on your classpath, or put it
in whatever other configuration file you have.
J-D
On Fri, Mar 30, 2012 at 2:52 PM, anil gupt
Hi All,
I am using cdh3u2. I ran HBase bulk loading with property
"mapred.reduce.tasks.speculative.execution" set to "false" in
mapred-site.xml. Still, i can see 6 killed task in Bulk Loading job and
after short analysis i realized that these jobs are killed because another
worker node completed t
11 matches
Mail list logo