RE: Confusing Yarn RPC Configuration

>>> I believe it is the same issue for node manage connection This would be probably related to below issues https://issues.apache.org/jira/i#browse/YARN-3944 https://issues.apache.org/jira/i#browse/YARN-3238 Thanks & Regards Rohith Sharma K S From: Jeff Zhang [mailto:zjf...@gmail.com] Sent: 18

Re: Confusing Yarn RPC Configuration

Thanks, looks like it is resolved in 2.7 On Wed, Aug 19, 2015 at 3:03 PM, Rohith Sharma K S < rohithsharm...@huawei.com> wrote: > >>> I believe it is the same issue for node manage connection > > This would be probably related to below issues > > https://issues.apache.org/jira/i#browse/YARN-3944

How to auto relaunch a YARN Application Master on a failure?

Is this possible? If yes, can someone get back to me as to how?

How to auto relaunch a YARN Application Master on a failure?

Is this possible? If yes, can someone get back to me as to how?

Custom comparator when using Kryo serializer for MapReduce serialization

Hi all, (I'm using Hadoop 1.2.1) I'm using Kryo (with chill ) as my serializer (instead of the Writable interface). However, I'm having trouble with the comparator: on one hand, since none of my objects are Writable, I can

RE: How to auto relaunch a YARN Application Master on a failure?

It is possible.. You can set the number of attempts to be launched in case of AM failures. yarn.resourcemanager.am.max-attempts. Default is 2, you can increase it. This is at global level. Per application level, you need to send in ApplicationSubmissionContext# setMaxAppAttempts Thanks & Regard

Re: Custom comparator when using Kryo serializer for MapReduce serialization

In a general sense, if you can guarantee that your objects serialize in lexicographical order, then you should be able to do a comparator on the raw bytes themselves without any interpretation. On Wed, Aug 19, 2015 at 5:21 AM, Yaron Gonen wrote: > Hi all, > (I'm using Hadoop 1.2.1) > I'm using K

App Master takes ~30min to re-schedule task attempts.

Hello all, I'm running Apache2.6.0. I'm trying to remove a node from a Hadoop Cluster and the add it back. The taskattempts on the node which was removed are rescheduled only after 30min. During this 30min period looks like the App Master is trying to connect( check the log below ) the same node

App Master takes ~30min to re-schedule task attempts.

Hello all, I'm running Apache2.6.0. I'm trying to remove a node from a Hadoop Cluster and the add it back. The taskattempts on the node which was removed are rescheduled only after 30min. During this 30min period looks like the App Master is trying to connect( check the log below ) the same node

Fwd: App Master takes ~30min to re-schedule task attempts.

Hello all, I'm running Apache2.6.0. I'm trying to remove a node from a Hadoop Cluster and the add it back. The taskattempts on the node which was removed are rescheduled only after 30min. During this 30min period looks like the App Master is trying to connect( check the log below ) the same node

Specifying replication factor & block size during distcp

Hi, I am running a Distcp programmatically from Hadoop cluster to another - using Hadoop 2.7 and distcp v2. I would like to set a custom block size and replication factor for my files. How can I achieve that ? Thanks ! Varun

Re: Specifying replication factor & block size during distcp

I looked at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java but didn't find such switch. FYI On Wed, Aug 19, 2015 at 12:20 PM, Varun Sharma wrote: > Hi, > > I am running a Distcp programmatically from Hadoop cluster to another - > using Hadoop 2.7 and di

Re: Specifying replication factor & block size during distcp

Not sure if this feature is available. A workaround would be to update replication factor and block size at the HDFS level and reverting the changes after the distcp is complete. This is good for a one time copy. :-) On Wed, Aug 19, 2015 at 12:52 PM, Ted Yu wrote: > I looked at > hadoop-tools/h

RE: UserGroupInformation and login with password

I don't see how to do this in our architecture. Our clients may be across VPNs and completely inaccessible to the KDC. Our server basically functions as the Hadoop client, even though it is a long-running service on the OS host. When a user logs in, it performs these steps: - Spawn a new process

Re: App Master takes ~30min to re-schedule task attempts.

Change mapreduce.reduce.shuffle.connect.timeout, mapreduce.reduce.shuffle.read.timeout. By default they are 18. On 8/20/15, manoj wrote: > Hello all, > > I'm running Apache2.6.0. > I'm trying to remove a node from a Hadoop Cluster and the add it back. > The taskattempts on the node which was

Re: App Master takes ~30min to re-schedule task attempts.

Change mapreduce.reduce.shuffle.connect.timeout, mapreduce.reduce.shuffle.read.timeout. By default they are 18. On 8/20/15, manoj wrote: > Hello all, > > I'm running Apache2.6.0. > I'm trying to remove a node from a Hadoop Cluster and the add it back. > The taskattempts on the node which was r