Jayush Luniya created AMBARI-14312:
--------------------------------------
Summary: Express Upgrade: Failure during restart of all Node
Managers during EU from 2.1 to 2.3.4 with timeout error
Key: AMBARI-14312
URL: https://issues.apache.org/jira/browse/AMBARI-14312
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.2.0
Reporter: Jayush Luniya
Assignee: Jayush Luniya
Priority: Blocker
Fix For: 2.2.0
*Steps:*
Setup HDP 2.1 cluster with Ambari 2.2.0
Modify the relevant DB tables
Start Express Upgrade to 2.3.4
Result:
EU reported failures during restart of all NameNodes
*stderr*
{code}
Python script has been killed due to timeout after waiting 1200 secs
{code}
*stdout*
{code}
2015-12-09 10:58:32,783 -
File['/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid'] {'action':
['delete'], 'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'"}
2015-12-09 10:58:32,844 - Execute['ulimit -c unlimited; export
HADOOP_LIBEXEC_DIR=/usr/hdp/2.3.4.0-3426/hadoop/libexec &&
/usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh --config
/usr/hdp/2.3.4.0-3426/hadoop/conf start nodemanager'] {'not_if':
"ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'user': 'yarn'}
2015-12-09 10:58:34,031 - Execute['ambari-sudo.sh su yarn -l -s /bin/bash -c
'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] {'not_if':
"ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat
/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'tries': 5,
'try_sleep': 1}
2015-12-09 10:58:34,081 - Skipping Execute['ambari-sudo.sh su yarn -l -s
/bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p
`cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] due to not_if
2015-12-09 10:58:34,081 - Executing NodeManager Stack Upgrade post-restart
2015-12-09 10:58:34,083 - NodeManager executing "yarn node -list
-states=RUNNING" to verify the node has rejoined the cluster...
2015-12-09 10:58:34,083 - call['yarn node -list -states=RUNNING'] {'user':
'yarn'}
{code}
Attachments: stdout and screenshots
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
