[ https://issues.apache.org/jira/browse/AMBARI-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hurley updated AMBARI-22522: ------------------------------------- Status: Patch Available (was: Open) > Livy server fails to start during downgrade due to absence of 'conf' directory > ------------------------------------------------------------------------------ > > Key: AMBARI-22522 > URL: https://issues.apache.org/jira/browse/AMBARI-22522 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Affects Versions: 2.6.1 > Reporter: Jonathan Hurley > Assignee: Jonathan Hurley > Priority: Blocker > Fix For: 2.6.1 > > Attachments: AMBARI-22522.patch > > > *STR* > # Deployed cluster with Ambari version: 2.4.3.0-30 and HDP version: > 2.5.5.0-157 > # Upgrade Ambari to 2.6.1.0-43 > # Start EU to 2.6.4.0-36 and downgrade at final step > *Result* > Observed during downgrade that Livy server start failed with below error: > {code} > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", > line 141, in <module> > LivyServer().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 367, in execute > method(env) > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 970, in restart > self.start(env, upgrade_type=upgrade_type) > File > "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", > line 61, in start > self.configure(env) > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 120, in locking_configure > original_configure(obj, *args, **kw) > File > "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", > line 51, in configure > setup_livy(env, 'server', upgrade_type=upgrade_type, action = 'config') > File > "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/setup_livy.py", > line 56, in setup_livy > mode=0644, > File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", > line 166, in __init__ > self.env.run() > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 160, in run > self.run_action(resource, action) > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 124, in run_action > provider_action() > File > "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", > line 120, in action_create > raise Fail("Applying %s failed, parent directory %s doesn't exist" % > (self.resource, dirname)) > resource_management.core.exceptions.Fail: Applying > File['/usr/hdp/current/livy-server/conf/livy-env.sh'] failed, parent > directory /usr/hdp/current/livy-server/conf doesn't exist > {code} > On livy node, observed the following: > {code} > root@ctr-e134-1499953498516-324773-01-000007:/usr/hdp/current/livy-server# ls > -la /usr/hdp/current/livy-server/ > total 40 > drwxr-xr-x 8 root root 4096 Nov 20 23:49 . > drwxr-xr-x 42 root root 4096 Nov 21 00:05 .. > drwxr-xr-x 2 root root 4096 Nov 20 23:43 bin > lrwxrwxrwx 1 root root 14 Apr 21 2017 conf -> /etc/livy/conf > drwxr-xr-x 3 root root 4096 Nov 20 23:43 etc > drwxr-xr-x 2 root root 12288 Nov 20 23:43 jars > drwxr-xr-x 2 livy livy 4096 Nov 20 23:49 logs > drwxr-xr-x 2 root root 4096 Nov 20 23:43 repl-jars > drwxr-xr-x 2 root root 4096 Nov 20 23:43 rsc-jars > =============== > root@ctr-e134-1499953498516-324773-01-000007:~# ls -la /etc/livy/ > total 16 > drwxr-xr-x 4 root root 4096 Nov 21 04:32 . > drwxr-xr-x 120 root root 4096 Nov 21 03:26 .. > drwxr-xr-x 3 root root 4096 Nov 21 03:30 2.6.4.0-36 > lrwxrwxrwx 1 root root 21 Nov 21 04:32 conf -> /usr/hdp/current/livy > drwxr-xr-x 2 root root 4096 Nov 20 23:49 conf.backup > {code} > Looks like there is no directory '/etc/livy/2.5.5.0-157/0' > From downgrade wizard, found that 'Set Version on All Hosts' Upgrade group > printed the following for livy: > {code} > 2017-11-21 04:32:08,022 - Link['/etc/livy/conf'] {'to': > '/usr/hdp/current/livy'} > 2017-11-21 04:32:08,022 - Link['/etc/livy/conf'] replacing old symlink to > /grid/0/hdp/current/livy > 2017-11-21 04:32:08,022 - Warning: linking to nonexistent location > /usr/hdp/current/livy > 2017-11-21 04:32:08,022 - Creating symbolic Link['/etc/livy/conf'] to > /usr/hdp/current/livy > {code} > The cause of this appears to be that livy doesn't create a conf directory as > expected: > {code} > Skipping the conf-select tool on livy since /etc/livy/conf does not exist. > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)