Dmytro Sen created AMBARI-9992:
----------------------------------
Summary: Could not start Storm after Ambari only upgrade from
1.5.1 to 2.0.0
Key: AMBARI-9992
URL: https://issues.apache.org/jira/browse/AMBARI-9992
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.0.0
Environment: ambari-server version: ambari-server-2.0.0-123
ambari-server --hash: e6c791affb7583f1fcb5c428b1816d2d4874540d
HDP Stack: 2.1
Ambari DB: :PostgreSQL
Oozie/Hive DB: MySQL/MySQL
Security:no
OS: SUSE Linux Enterprise Server 11
Reporter: Dmytro Sen
Assignee: Dmytro Sen
Fix For: 2.0.0
STR:
1)Deploy old version with all services
2)Make ambari only upgrade to 2.0.0
Expected result:
All services are started
Actual result
Could not start Storm after Ambari only upgrade from 1.5.1 to 2.0.0
{code}
{
"href" : "http://172.18.145.49:8080/api/v1/clusters/cl1/requests/9/tasks/287";,
"Tasks" : {
"attempt_cnt" : 1,
"cluster_name" : "cl1",
"command" : "START",
"command_detail" : "STORM_REST_API START",
"end_time" : 1425897713322,
"error_log" : "/var/lib/ambari-agent/data/errors-287.txt",
"exit_code" : 1,
"host_name" : "amb-upg151-sles113postgres1425889596-10.cs1cloud.internal",
"id" : 287,
"output_log" : "/var/lib/ambari-agent/data/output-287.txt",
"request_id" : 9,
"role" : "STORM_REST_API",
"stage_id" : 0,
"start_time" : 1425897642009,
"status" : "FAILED",
"stderr" : "2015-03-09 10:41:53,130 - Error while executing command
'start':\nTraceback (most recent call last):\n File
\"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
line 214, in execute\n method(env)\n File
\"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py\",
line 60, in start\n service(\"rest_api\", action=\"start\")\n File
\"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1.2.1/package/scripts/service.py\",
line 72, in service\n path=params.storm_bin_dir\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line
148, in __init__\n self.env.run()\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 152, in run\n self.run_action(resource, action)\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 118, in run_action\n provider_action()\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\",
line 274, in action_run\n raise ex\nFail: Execution of
'/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1.",
"stdout" : "2015-03-09 10:40:48,803 -
u\"Directory['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts/']\"
{'recursive': True}\n2015-03-09 10:40:48,977 -
u\"File['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip']\"
{'content':
DownloadSource('http://amb-upg151-sles113postgres1425889596-4.cs1cloud.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip')}\n2015-03-09
10:40:49,067 - Not downloading the file from
http://amb-upg151-sles113postgres1425889596-4.cs1cloud.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip,
because /var/lib/ambari-agent/data/tmp/UnlimitedJCEPolicyJDK7.zip already
exists\n2015-03-09 10:40:49,227 - u\"Group['hadoop']\" {'ignore_failures':
False}\n2015-03-09 10:40:49,228 - Modifying group hadoop\n2015-03-09
10:40:49,278 - u\"Group['nobody']\" {'ignore_failures': False}\n2015-03-09
10:40:49,279 - Modifying group nobody\n2015-03-09 10:40:49,325 -
u\"Group['users']\" {'ignore_failures': False}\n2015-03-09 10:40:49,326 -
Modifying group users\n2015-03-09 10:40:49,374 - u\"User['hive']\" {'gid':
'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}\n2015-03-09
10:40:49,374 - Modifying user hive\n2015-03-09 10:40:49,566 -
u\"User['oozie']\" {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'users']}\n2015-03-09 10:40:49,568 - Modifying user oozie\n2015-03-09
10:40:49,737 - u\"User['nobody']\" {'gid': 'hadoop', 'ignore_failures': False,
'groups': [u'nobody']}\n2015-03-09 10:40:49,739 - Modifying user
nobody\n2015-03-09 10:40:49,911 - u\"User['ambari-qa']\" {'gid': 'hadoop',
'ignore_failures': False, 'groups': [u'users']}\n2015-03-09 10:40:49,912 -
Modifying user ambari-qa\n2015-03-09 10:40:50,084 - u\"User['hdfs']\" {'gid':
'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}\n2015-03-09
10:40:50,085 - Modifying user hdfs\n2015-03-09 10:40:50,239 -
u\"User['storm']\" {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}\n2015-03-09 10:40:50,240 - Modifying user storm\n2015-03-09
10:40:50,313 - u\"User['mapred']\" {'gid': 'hadoop', 'ignore_failures': False,
'groups': [u'hadoop']}\n2015-03-09 10:40:50,315 - Modifying user
mapred\n2015-03-09 10:40:50,382 - u\"User['hbase']\" {'gid': 'hadoop',
'ignore_failures': False, 'groups': [u'hadoop']}\n2015-03-09 10:40:50,383 -
Modifying user hbase\n2015-03-09 10:40:50,453 - u\"User['tez']\" {'gid':
'hadoop', 'ignore_failures': False, 'groups': [u'users']}\n2015-03-09
10:40:50,455 - Modifying user tez\n2015-03-09 10:40:50,623 -
u\"User['zookeeper']\" {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}\n2015-03-09 10:40:50,625 - Modifying user zookeeper\n2015-03-09
10:40:50,858 - u\"User['false']\" {'gid': 'hadoop', 'ignore_failures': False,
'groups': [u'hadoop']}\n2015-03-09 10:40:50,859 - Modifying user
false\n2015-03-09 10:40:50,971 - u\"User['falcon']\" {'gid': 'hadoop',
'ignore_failures': False, 'groups': [u'hadoop']}\n2015-03-09 10:40:50,972 -
Modifying user falcon\n2015-03-09 10:40:51,029 - u\"User['sqoop']\" {'gid':
'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}\n2015-03-09
10:40:51,030 - Modifying user sqoop\n2015-03-09 10:40:51,178 -
u\"User['yarn']\" {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}\n2015-03-09 10:40:51,179 - Modifying user yarn\n2015-03-09
10:40:51,292 - u\"User['hcat']\" {'gid': 'hadoop', 'ignore_failures': False,
'groups': [u'hadoop']}\n2015-03-09 10:40:51,293 - Modifying user
hcat\n2015-03-09 10:40:51,450 -
u\"File['/var/lib/ambari-agent/data/tmp/changeUid.sh']\" {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2015-03-09 10:40:51,733 -
u\"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']\"
{'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}\n2015-03-09
10:40:51,778 - Skipping u\"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh
ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']\"
due to not_if\n2015-03-09 10:40:51,778 -
u\"Directory['/grid/0/hadoop/hbase']\" {'owner': 'hbase', 'recursive': True,
'mode': 0775, 'cd_access': 'a'}\n2015-03-09 10:40:52,310 -
u\"File['/var/lib/ambari-agent/data/tmp/changeUid.sh']\" {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2015-03-09 10:40:52,606 -
u\"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/grid/0/hadoop/hbase']\"
{'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}\n2015-03-09
10:40:52,651 - Skipping u\"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh
hbase
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/grid/0/hadoop/hbase']\"
due to not_if\n2015-03-09 10:40:52,652 - u\"Group['hdfs']\" {'ignore_failures':
False}\n2015-03-09 10:40:52,653 - Modifying group hdfs\n2015-03-09 10:40:52,700
- u\"User['hdfs']\" {'ignore_failures': False, 'groups': [u'hadoop', 'hdfs',
u'hdfs']}\n2015-03-09 10:40:52,702 - Modifying user hdfs\n2015-03-09
10:40:52,888 - u\"Directory['/etc/hadoop']\" {'mode': 0755}\n2015-03-09
10:40:53,038 - u\"Directory['/etc/hadoop/conf.empty']\" {'owner': 'hdfs',
'group': 'hadoop', 'recursive': True}\n2015-03-09 10:40:53,200 -
u\"Link['/etc/hadoop/conf']\" {'not_if': 'ls /etc/hadoop/conf', 'to':
'/etc/hadoop/conf.empty'}\n2015-03-09 10:40:53,248 - Skipping
u\"Link['/etc/hadoop/conf']\" due to not_if\n2015-03-09 10:40:53,275 -
u\"File['/etc/hadoop/conf/hadoop-env.sh']\" {'content': InlineTemplate(...),
'owner': 'hdfs', 'group': 'hadoop'}\n2015-03-09 10:40:53,540 -
u\"Execute['('setenforce', '0')']\" {'sudo': True, 'only_if': 'test -f
/selinux/enforce'}\n2015-03-09 10:40:53,606 - Skipping
u\"Execute['('setenforce', '0')']\" due to only_if\n2015-03-09 10:40:53,607 -
u\"Directory['/var/opt/teradata/log/hadoop']\" {'owner': 'root', 'mode': 0775,
'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}\n2015-03-09
10:40:54,226 - u\"Directory['/var/run/hadoop']\" {'owner': 'root', 'group':
'root', 'recursive': True, 'cd_access': 'a'}\n2015-03-09 10:40:54,662 -
u\"Directory['/tmp/hadoop-hdfs']\" {'owner': 'hdfs', 'recursive': True,
'cd_access': 'a'}\n2015-03-09 10:40:55,001 -
u\"File['/etc/hadoop/conf/commons-logging.properties']\" {'content':
Template('commons-logging.properties.j2'), 'owner': 'hdfs'}\n2015-03-09
10:40:55,252 - u\"File['/etc/hadoop/conf/health_check']\" {'content':
Template('health_check-v2.j2'), 'owner': 'hdfs'}\n2015-03-09 10:40:55,499 -
u\"File['/etc/hadoop/conf/log4j.properties']\" {'content': '...', 'owner':
'hdfs', 'group': 'hadoop', 'mode': 0644}\n2015-03-09 10:40:55,754 -
u\"File['/etc/hadoop/conf/hadoop-metrics2.properties']\" {'content':
Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}\n2015-03-09
10:40:56,002 - u\"File['/etc/hadoop/conf/task-log4j.properties']\" {'content':
StaticFile('task-log4j.properties'), 'mode': 0755}\n2015-03-09 10:40:56,297 -
u\"File['/etc/hadoop/conf/configuration.xsl']\" {'owner': 'hdfs', 'group':
'hadoop'}\n2015-03-09 10:40:56,745 - u\"Directory['/var/log/storm']\" {'owner':
'storm', 'group': 'hadoop', 'recursive': True, 'mode': 0775}\n2015-03-09
10:40:56,917 - u\"Directory['/var/run/storm']\" {'owner': 'storm', 'group':
'hadoop', 'recursive': True, 'cd_access': 'a'}\n2015-03-09 10:40:57,345 -
u\"Directory['/grid/0/hadoop/storm']\" {'owner': 'storm', 'group': 'hadoop',
'recursive': True, 'cd_access': 'a'}\n2015-03-09 10:40:57,874 -
u\"Directory['/etc/storm/conf']\" {'group': 'hadoop', 'recursive': True,
'cd_access': 'a'}\n2015-03-09 10:40:58,031 - Changing group for /etc/storm/conf
from 0 to hadoop\n2015-03-09 10:40:58,361 -
u\"File['/etc/storm/conf/config.yaml']\" {'owner': 'storm', 'content':
Template('config.yaml.j2'), 'group': 'hadoop'}\n2015-03-09 10:40:58,625 -
u\"File['/etc/storm/conf/storm.yaml']\" {'owner': 'storm', 'content':
Template('storm.yaml.j2'), 'group': 'hadoop'}\n2015-03-09 10:40:58,884 -
u\"File['/etc/storm/conf/storm-env.sh']\" {'content': InlineTemplate(...),
'owner': 'storm'}\n2015-03-09 10:40:59,141 - u\"Execute['source
/etc/storm/conf/storm-env.sh ; export PATH=$PATH:$JAVA_HOME/bin ; java -jar
/usr/lib/storm/contrib/storm-rest/`ls /usr/lib/storm/contrib/storm-rest | grep
-wE storm-rest-[0-9.-]+\\\\.jar` server /etc/storm/conf/config.yaml >
/var/log/storm/restapi.log 2>&1']\" {'wait_for_finish': False, 'path':
['/usr/bin'], 'user': 'storm', 'not_if': 'ls /var/run/storm/restapi.pid
>/dev/null 2>&1 && ps -p `cat /var/run/storm/restapi.pid` >/dev/null
2>&1'}\n2015-03-09 10:40:59,193 - u\"Execute['/usr/jdk64/jdk1.7.0_45/bin/jps -l
| grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid']\" {'logoutput': True, 'path': ['/usr/bin'],
'tries': 6, 'user': 'storm', 'try_sleep': 10}\n2015-03-09 10:40:59,691 -
Retrying after 10 seconds. Reason: Execution of '/usr/jdk64/jdk1.7.0_45/bin/jps
-l | grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1. \n2015-03-09 10:41:10,149 - Retrying
after 10 seconds. Reason: Execution of '/usr/jdk64/jdk1.7.0_45/bin/jps -l |
grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1. \n2015-03-09 10:41:20,599 - Retrying
after 10 seconds. Reason: Execution of '/usr/jdk64/jdk1.7.0_45/bin/jps -l |
grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1. \n2015-03-09 10:41:32,211 - Retrying
after 10 seconds. Reason: Execution of '/usr/jdk64/jdk1.7.0_45/bin/jps -l |
grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1. \n2015-03-09 10:41:42,671 - Retrying
after 10 seconds. Reason: Execution of '/usr/jdk64/jdk1.7.0_45/bin/jps -l |
grep /usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1. \n2015-03-09 10:41:53,130 - Error while
executing command 'start':\nTraceback (most recent call last):\n File
\"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
line 214, in execute\n method(env)\n File
\"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py\",
line 60, in start\n service(\"rest_api\", action=\"start\")\n File
\"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1.2.1/package/scripts/service.py\",
line 72, in service\n path=params.storm_bin_dir\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line
148, in __init__\n self.env.run()\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 152, in run\n self.run_action(resource, action)\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 118, in run_action\n provider_action()\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\",
line 274, in action_run\n raise ex\nFail: Execution of
'/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ &&
/usr/jdk64/jdk1.7.0_45/bin/jps -l | grep
/usr/lib/storm/contrib/storm-rest/storm-rest-.*\\.jar$ | awk {'print $1'} >
/var/run/storm/restapi.pid' returned 1.",
"structured_out" : { }
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
