[jira] [Commented] (AMBARI-17898) Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
[ https://issues.apache.org/jira/browse/AMBARI-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136350#comment-16136350 ] Qin Liu commented on AMBARI-17898: -- Hi Eric, Hmmm, The patch passed all the unit test on July 5. Let me check. > Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor > - > > Key: AMBARI-17898 > URL: https://issues.apache.org/jira/browse/AMBARI-17898 > Project: Ambari > Issue Type: Improvement > Components: ambari-metrics >Affects Versions: 2.2.0 >Reporter: Qin Liu >Assignee: Qin Liu > Fix For: trunk > > Attachments: AMBARI-17898-trunk_2.patch, AMBARI-17898_trunk_june.patch > > > This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to > authenticate". > In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari > Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But > Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does > not support Kerberos HTTP SPNEGO authentication. > /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out: > 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: > http://metrics-collector:6188/ws/v1/timeline/metrics > 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to > server. HTTP Error 401: Authentication required > 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-17898) Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
[ https://issues.apache.org/jira/browse/AMBARI-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136342#comment-16136342 ] Qin Liu commented on AMBARI-17898: -- Hi Eric, Thank you so much! > Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor > - > > Key: AMBARI-17898 > URL: https://issues.apache.org/jira/browse/AMBARI-17898 > Project: Ambari > Issue Type: Improvement > Components: ambari-metrics >Affects Versions: 2.2.0 >Reporter: Qin Liu >Assignee: Qin Liu > Fix For: trunk > > Attachments: AMBARI-17898-trunk_2.patch, AMBARI-17898_trunk_june.patch > > > This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to > authenticate". > In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari > Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But > Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does > not support Kerberos HTTP SPNEGO authentication. > /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out: > 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: > http://metrics-collector:6188/ws/v1/timeline/metrics > 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to > server. HTTP Error 401: Authentication required > 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (AMBARI-17898) Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
[
https://issues.apache.org/jira/browse/AMBARI-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang reopened AMBARI-17898:
[~qinliu] Ambari Agent seems to have trouble shutdown after applying this
patch, and you check?
{code}
==
FAIL: test_wait_for_process_death (TestProcessUtils.TestProcessUtils)
--
Traceback (most recent call last):
File
"/home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-common/src/test/python/mock/mock.py",
line 1199, in patched
return func(*args, **keywargs)
File
"/home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py",
line 202, in test_wait_for_process_death
{code}
> Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
> -
>
> Key: AMBARI-17898
> URL: https://issues.apache.org/jira/browse/AMBARI-17898
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-metrics
>Affects Versions: 2.2.0
>Reporter: Qin Liu
>Assignee: Qin Liu
> Fix For: trunk
>
> Attachments: AMBARI-17898-trunk_2.patch, AMBARI-17898_trunk_june.patch
>
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to
> authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari
> Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But
> Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does
> not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server:
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-17898) Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
[ https://issues.apache.org/jira/browse/AMBARI-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136278#comment-16136278 ] Hudson commented on AMBARI-17898: - FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7918 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7918/]) AMBARI-17898. Add Kerberos HTTP SPNEGO authentication support to Ambari (eyang: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d260c89263ee0471514b4cfb8ce938134eceea12]) * (edit) ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py * (add) ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py * (edit) ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini * (edit) ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py * (edit) ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 * (edit) ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml * (edit) ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py * (edit) ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py * (edit) ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py * (edit) ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json * (add) ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py > Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor > - > > Key: AMBARI-17898 > URL: https://issues.apache.org/jira/browse/AMBARI-17898 > Project: Ambari > Issue Type: Improvement > Components: ambari-metrics >Affects Versions: 2.2.0 >Reporter: Qin Liu >Assignee: Qin Liu > Fix For: trunk > > Attachments: AMBARI-17898-trunk_2.patch, AMBARI-17898_trunk_june.patch > > > This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to > authenticate". > In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari > Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But > Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does > not support Kerberos HTTP SPNEGO authentication. > /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out: > 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: > http://metrics-collector:6188/ws/v1/timeline/metrics > 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to > server. HTTP Error 401: Authentication required > 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21755) Unexpected Package Names in CentOS Cause Installation Failures
[
https://issues.apache.org/jira/browse/AMBARI-21755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136243#comment-16136243
]
Hudson commented on AMBARI-21755:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #41 (See
[https://builds.apache.org/job/Ambari-branch-2.6/41/])
AMBARI-21755 Unexpected Package Names in CentOS Cause Installation (hapylestat:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d2ca468cc1f09d69ca4af504076a20602e48fd27])
* (edit) ambari-server/src/test/python/custom_actions/TestInstallPackages.py
* (edit)
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
> Unexpected Package Names in CentOS Cause Installation Failures
> --
>
> Key: AMBARI-21755
> URL: https://issues.apache.org/jira/browse/AMBARI-21755
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.6.0
> Environment: An installation on CentOS 7 failed because of the
> following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
>Reporter: Dmytro Grinenko
>Assignee: Dmytro Grinenko
>Priority: Critical
> Fix For: 2.6.0
>
>
> An installation on CentOS 7 failed because of the following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21755) Unexpected Package Names in CentOS Cause Installation Failures
[
https://issues.apache.org/jira/browse/AMBARI-21755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136239#comment-16136239
]
Hudson commented on AMBARI-21755:
-
SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7917 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/7917/])
AMBARI-21755 Unexpected Package Names in CentOS Cause Installation (hapylestat:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=6de8094a45c536340b56d73d4b0116c68349096c])
* (edit)
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
* (edit) ambari-server/src/test/python/custom_actions/TestInstallPackages.py
> Unexpected Package Names in CentOS Cause Installation Failures
> --
>
> Key: AMBARI-21755
> URL: https://issues.apache.org/jira/browse/AMBARI-21755
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.6.0
> Environment: An installation on CentOS 7 failed because of the
> following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
>Reporter: Dmytro Grinenko
>Assignee: Dmytro Grinenko
>Priority: Critical
> Fix For: 2.6.0
>
>
> An installation on CentOS 7 failed because of the following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-17898) Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor
[ https://issues.apache.org/jira/browse/AMBARI-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated AMBARI-17898: --- Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this, thanks Qin Liu. > Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor > - > > Key: AMBARI-17898 > URL: https://issues.apache.org/jira/browse/AMBARI-17898 > Project: Ambari > Issue Type: Improvement > Components: ambari-metrics >Affects Versions: 2.2.0 >Reporter: Qin Liu >Assignee: Qin Liu > Fix For: trunk > > Attachments: AMBARI-17898-trunk_2.patch, AMBARI-17898_trunk_june.patch > > > This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to > authenticate". > In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari > Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But > Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does > not support Kerberos HTTP SPNEGO authentication. > /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out: > 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: > http://metrics-collector:6188/ws/v1/timeline/metrics > 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to > server. HTTP Error 401: Authentication required > 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AMBARI-21755) Unexpected Package Names in CentOS Cause Installation Failures
[
https://issues.apache.org/jira/browse/AMBARI-21755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmytro Grinenko resolved AMBARI-21755.
--
Resolution: Fixed
Committed:
To https://git-wip-us.apache.org/repos/asf/ambari.git
d84cda89a9..d2ca468cc1 branch-2.6 -> branch-2.6
d5904b9a95..6de8094a45 trunk -> trunk
> Unexpected Package Names in CentOS Cause Installation Failures
> --
>
> Key: AMBARI-21755
> URL: https://issues.apache.org/jira/browse/AMBARI-21755
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.6.0
> Environment: An installation on CentOS 7 failed because of the
> following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
>Reporter: Dmytro Grinenko
>Assignee: Dmytro Grinenko
>Priority: Critical
> Fix For: 2.6.0
>
>
> An installation on CentOS 7 failed because of the following exception:
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install kafka_3_1_0_0_49_centos' returned 1.
> Error: Nothing to do
> {code}
> The problem is with this query:
> {code}
> 2017-08-15 18:17:04,086 - checked_call['rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
> 2017-08-15 18:17:04,162 - checked_call returned (0, '3.1.0.0-54.centos', '')
> {code}
> {code}
> [root@ctr-e134-1499953498516-100220-01-06 ~]# rpm -q --queryformat
> '%{version}-%{release}' hdf-select | sed -e 's/\.el[0-9]//g'
> 3.1.0.0-54.centos
> {code}
> On a good system, this looks like:
> {code}
> [root@c6403 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
> 2.5.0.0-1237.el6
> {code}
> It seems as though we're expecting the package to end with "el?" and that's
> not always the case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doroszlai, Attila updated AMBARI-21768:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to
[branch-2.5|https://git1-us-west.apache.org/repos/asf?p=ambari.git&a=commit&h=d1df4d9aa658975a82b5cb5e16be893d36268c87].
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136170#comment-16136170
]
Hudson commented on AMBARI-21770:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #1825 (See
[https://builds.apache.org/job/Ambari-branch-2.5/1825/])
AMBARI-21770 - Spark1 Shuffle Property Is Removed Incorrectly on a Stack
(jhurley:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=24debeed14654249055077ad5483dc45d127a074])
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
> Attachments: AMBARI-21770.patch
>
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21776) Move druid version to druid 0.10.1
slim bouguerra created AMBARI-21776: --- Summary: Move druid version to druid 0.10.1 Key: AMBARI-21776 URL: https://issues.apache.org/jira/browse/AMBARI-21776 Project: Ambari Issue Type: Bug Reporter: slim bouguerra Assignee: slim bouguerra Fix For: 2.6.0 Move druid version to 0.10.1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21775) [Druid] Wrong logrotate configuration for generating an excess of cron email messages
slim bouguerra created AMBARI-21775:
---
Summary: [Druid] Wrong logrotate configuration for generating an
excess of cron email messages
Key: AMBARI-21775
URL: https://issues.apache.org/jira/browse/AMBARI-21775
Project: Ambari
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: slim bouguerra
Fix For: 2.6.0
Druid related configuration is generating the following corn email
{code}
/etc/cron.daily/logrotate:
error: druid:12 unknown unit 'B'
error: found error in /var/log/druid/*.log , skipping
{code}
This is due to wrong logrotate config
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (AMBARI-21775) [Druid] Wrong logrotate configuration for generating an excess of cron email messages
[
https://issues.apache.org/jira/browse/AMBARI-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra reassigned AMBARI-21775:
---
Assignee: slim bouguerra
> [Druid] Wrong logrotate configuration for generating an excess of cron email
> messages
> -
>
> Key: AMBARI-21775
> URL: https://issues.apache.org/jira/browse/AMBARI-21775
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
> Fix For: 2.6.0
>
>
> Druid related configuration is generating the following corn email
> {code}
> /etc/cron.daily/logrotate:
> error: druid:12 unknown unit 'B'
> error: found error in /var/log/druid/*.log , skipping
> {code}
> This is due to wrong logrotate config
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136033#comment-16136033 ] Hudson commented on AMBARI-21771: - FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #40 (See [https://builds.apache.org/job/Ambari-branch-2.6/40/]) AMBARI-21771 : Maven cleanup of ambari-metrics and ambari-logsearch (avijayan: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=4f52f6d409590a6d4355aa2f8702624726bf9735]) * (edit) ambari-metrics/ambari-metrics-assembly/pom.xml * (edit) ambari-logsearch/ambari-logsearch-assembly/pom.xml > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136034#comment-16136034
]
Hudson commented on AMBARI-21770:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #40 (See
[https://builds.apache.org/job/Ambari-branch-2.6/40/])
AMBARI-21770 - Spark1 Shuffle Property Is Removed Incorrectly on a Stack
(jhurley:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d84cda89a940fa61701972d9ff011010dc0be08f])
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
> Attachments: AMBARI-21770.patch
>
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136016#comment-16136016
]
Hudson commented on AMBARI-21769:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #1824 (See
[https://builds.apache.org/job/Ambari-branch-2.5/1824/])
AMBARI-21769. zeppelin principal and livy.superusers property do not (rlevas:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0ae0494dcc264dd2bbb9cb84bad4b13fdfa2a9ef])
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json
* (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
* (edit) ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21765) Missing 'hdfs_scheme' variable in params.py file for Apache Knox
[ https://issues.apache.org/jira/browse/AMBARI-21765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136015#comment-16136015 ] Hudson commented on AMBARI-21765: - FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #1824 (See [https://builds.apache.org/job/Ambari-branch-2.5/1824/]) AMBARI-21765. Missing hdfs_scheme variable in params.py file for Apache (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ca6ce335d7bd3d50e5892a3169d90acbdada963d]) * (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py > Missing 'hdfs_scheme' variable in params.py file for Apache Knox > > > Key: AMBARI-21765 > URL: https://issues.apache.org/jira/browse/AMBARI-21765 > Project: Ambari > Issue Type: Bug >Reporter: Sandeep More >Assignee: Sandeep More > > Some of the topologies require 'hdfs_scheme' protocol scheme variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136017#comment-16136017 ] Hudson commented on AMBARI-21771: - FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #1824 (See [https://builds.apache.org/job/Ambari-branch-2.5/1824/]) AMBARI-21771 : Maven cleanup of ambari-metrics and ambari-logsearch (avijayan: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=cf5c2f56e21d41f0f71cab5b9c3fb34fd56c5c4c]) * (edit) ambari-logsearch/ambari-logsearch-assembly/pom.xml * (edit) ambari-metrics/ambari-metrics-assembly/pom.xml > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
[
https://issues.apache.org/jira/browse/AMBARI-21767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135991#comment-16135991
]
Mausami Kusum commented on AMBARI-21767:
Yes, My EU hit this towards the end when it does service check. I chose to
ignore and proceed as that seems to be the only option you can do at that stage
and try to fix it later.
> Kerberos cluster - hive service check failed post migration
> ---
>
> Key: AMBARI-21767
> URL: https://issues.apache.org/jira/browse/AMBARI-21767
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Di Li
> Fix For: 2.5.2
>
>
> Post migration hive service check failed with following error:
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> 2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
> 50111 idtest.ambari-qa.1503075103.65.pig
> /etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
> ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
> Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com
> is not allowed to impersonate ambari-qa"}http_code <500>
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> *WORKAROUND:*
> Shailaja suggested to modify all the entries in custom core-site property
> under HDFS service to * and also add hbase. I added hbase key value pair as
> following:
> hadoop.proxyuser.hbase.groups *
> hadoop.proxyuser.hbase.hosts *
> Also modified following properties:
> hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.knox.groups was usersmodified to *
> hadoop.proxyuser.HTTP.groups was usersmodified to *
> hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
> After that hive service check passed. If this work around is not done bigsql
> upgrade will fail during service check. This step is not documented.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-21770:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
> Attachments: AMBARI-21770.patch
>
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-21770:
-
Status: Patch Available (was: Open)
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
> Attachments: AMBARI-21770.patch
>
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135969#comment-16135969
]
Xabriel J Collazo Mojica commented on AMBARI-21730:
---
Got it [~aplusplus]. Thanks for the quick turn around.
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>
> INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
> INFO : Dag submit failed due to There is conflicting local resource
> (guava-14.0.1.jar) between dag local resource and vertex Map 1 local
> resource.
> Resource of dag : resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
> } size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
> Resource of vertex: resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
> }
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is
> not seen anymore.*
> I have reproed this on:
> {{# cat HDP.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.1.0}}
> {{# cat ambari.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.1.0}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xabriel J Collazo Mojica updated AMBARI-21730:
--
Comment: was deleted
(was: [~aplusplus], added the yum repos to the description above.
Let me know if you need further info.)
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>
> INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
> INFO : Dag submit failed due to There is conflicting local resource
> (guava-14.0.1.jar) between dag local resource and vertex Map 1 local
> resource.
> Resource of dag : resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
> } size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
> Resource of vertex: resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
> }
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is
> not seen anymore.*
> I have reproed this on:
> {{# cat HDP.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.1.0}}
> {{# cat ambari.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.1.0}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135965#comment-16135965
]
Xabriel J Collazo Mojica commented on AMBARI-21730:
---
[~aplusplus], added the yum repos to the description above.
Let me know if you need further info.
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>
> INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
> INFO : Dag submit failed due to There is conflicting local resource
> (guava-14.0.1.jar) between dag local resource and vertex Map 1 local
> resource.
> Resource of dag : resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
> } size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
> Resource of vertex: resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
> }
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is
> not seen anymore.*
> I have reproed this on:
> {{# cat HDP.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.1.0}}
> {{# cat ambari.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.1.0}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135941#comment-16135941
]
Hudson commented on AMBARI-21769:
-
SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7916 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/7916/])
AMBARI-21769. zeppelin principal and livy.superusers property do not (rlevas:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=c129baa1f6e4438b7af9e7a01735d8ead05df1be])
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
* (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json
* (edit) ambari-server/src/main/resources/stacks/stack_advisor.py
* (edit) ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json
* (edit)
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/service_advisor.py
* (add)
ambari-server/src/test/python/common-services/SPARK/2.2.0/test_service_advisor.py
* (edit)
ambari-server/src/main/resources/common-services/SPARK/2.2.0/service_advisor.py
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135939#comment-16135939
]
Zhiyuan Yang commented on AMBARI-21730:
---
[~xabriel] Thanks for update! This issue was fixed later than 2.6.1.0.
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>
> INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
> INFO : Dag submit failed due to There is conflicting local resource
> (guava-14.0.1.jar) between dag local resource and vertex Map 1 local
> resource.
> Resource of dag : resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
> } size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
> Resource of vertex: resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
> }
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is
> not seen anymore.*
> I have reproed this on:
> {{# cat HDP.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.1.0}}
> {{# cat ambari.repo | grep baseurl}}
> {{baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.1.0}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21765) Missing 'hdfs_scheme' variable in params.py file for Apache Knox
[ https://issues.apache.org/jira/browse/AMBARI-21765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135943#comment-16135943 ] Hudson commented on AMBARI-21765: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7916 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7916/]) AMBARI-21765. Missing hdfs_scheme variable in params.py file for Apache (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=8e33d8864aa0d1cb5be00241be5900d9fa9310dd]) * (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py * (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/params_linux.py > Missing 'hdfs_scheme' variable in params.py file for Apache Knox > > > Key: AMBARI-21765 > URL: https://issues.apache.org/jira/browse/AMBARI-21765 > Project: Ambari > Issue Type: Bug >Reporter: Sandeep More >Assignee: Sandeep More > > Some of the topologies require 'hdfs_scheme' protocol scheme variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135940#comment-16135940
]
Hudson commented on AMBARI-21173:
-
SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7916 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/7916/])
AMBARI-21173. Not able to start Yarn services after restoring the (vbrodetskyi:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=5895040c8a3374012c11f0e351c8ba631a689ae7])
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135944#comment-16135944 ] Hudson commented on AMBARI-21771: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7916 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7916/]) AMBARI-21771 : Maven cleanup of ambari-metrics and ambari-logsearch (avijayan: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d5904b9a956026b9a0dcb9d5acfd6af22d3dcb67]) * (edit) ambari-metrics/ambari-metrics-assembly/pom.xml * (edit) ambari-logsearch/ambari-logsearch-assembly/pom.xml > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
[
https://issues.apache.org/jira/browse/AMBARI-21767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135925#comment-16135925
]
Di Li commented on AMBARI-21767:
Hello Mausami, [~mmkusum] I didn't hit on an IOP 4.2.0/Kerberos to HDP
2.6.2-193/Ambari 2.5.2-273 migratoin ( local users, not ldap). Also, should
this been failing for you consistently on your cluster, your EU should have
towards the end when all services ran their service checks.
Could you please retest ? and let me know ? Hortonworks have been testing the
migtation extensively with Kerberos, yet no report from their QA regarding this
issue. Right now, I tend to think we should not doc the steps as it's not a
general case in my tests.
> Kerberos cluster - hive service check failed post migration
> ---
>
> Key: AMBARI-21767
> URL: https://issues.apache.org/jira/browse/AMBARI-21767
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Di Li
> Fix For: 2.5.2
>
>
> Post migration hive service check failed with following error:
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> 2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
> 50111 idtest.ambari-qa.1503075103.65.pig
> /etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
> ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
> Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com
> is not allowed to impersonate ambari-qa"}http_code <500>
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> *WORKAROUND:*
> Shailaja suggested to modify all the entries in custom core-site property
> under HDFS service to * and also add hbase. I added hbase key value pair as
> following:
> hadoop.proxyuser.hbase.groups *
> hadoop.proxyuser.hbase.hosts *
> Also modified following properties:
> hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.knox.groups was usersmodified to *
> hadoop.proxyuser.HTTP.groups was usersmodified to *
> hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
> After that hive service check passed. If this work around is not done bigsql
> upgrade will fail during service check. This step is not documented.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xabriel J Collazo Mojica updated AMBARI-21730:
--
Description:
Ambari 2.5+ sets the following Hive Interactive flag by default to:
{{hive.llap.execution.mode=only}}
This mode breaks compatibility with StorageHander based tables. For example,
HBase tables like these fail:
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz",
"hbase.mapred.output.outputtable" = "xyz");
With a stack trace like:
0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
hbase_table_1 SELECT * FROM hbase_table_1;
INFO : Compiling
command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
INFO : Dag submit failed due to There is conflicting local resource
(guava-14.0.1.jar) between dag local resource and vertex Map 1 local resource.
Resource of dag : resource { scheme: "hdfs" host:
"intel68bvt1.fyre.ibm.com" port: 8020 file:
"/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
} size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
Resource of vertex: resource { scheme: "hdfs" host:
"intel68bvt1.fyre.ibm.com" port: 8020 file:
"/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
}
*When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is not
seen anymore.*
I have reproed this on:
{{# cat HDP.repo | grep baseurl}}
{{baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.1.0}}
{{# cat ambari.repo | grep baseurl}}
{{baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.1.0}}
was:
Ambari 2.5+ sets the following Hive Interactive flag by default to:
{{hive.llap.execution.mode=only}}
This mode breaks compatibility with StorageHander based tables. For example,
HBase tables like these fail:
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz",
"hbase.mapred.output.outputtable" = "xyz");
With a stack trace like:
0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
hbase_table_1 SELECT * FROM hbase_table_1;
INFO : Compiling
command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
INFO : Dag submit failed due to There is conflicting local resource
(guava-14.0.1.jar) between dag local resource and vertex Map 1 local resource.
Resource of dag : resource { scheme: "hdfs" host:
"intel68bvt1.fyre.ibm.com" port: 8020 file:
"/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
} size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
Resource of vertex: resource { scheme: "hdfs" host:
"intel68bvt1.fyre.ibm.com" port: 8020 file:
"/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
}
*When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is not
seen anymore.*
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790
[jira] [Created] (AMBARI-21774) NullPointerException if there is no proper HDP/HDF repo
amarnath reddy pappu created AMBARI-21774:
-
Summary: NullPointerException if there is no proper HDP/HDF repo
Key: AMBARI-21774
URL: https://issues.apache.org/jira/browse/AMBARI-21774
Project: Ambari
Issue Type: Bug
Components: ambari-server
Reporter: amarnath reddy pappu
Ambari server fails with "NullPointerException" if cluster is pointing to
specific HDP/HDF/xxx
and if that folder does not exist in resources folders.
Should do NULL check and then print proper meaning full error message so that
customer's can install correct that.
{noformat}
15 Aug 2017 15:54:02,685 INFO [Stack Version Loading Thread]
LatestRepoCallable:80 - Loading latest URL info for stack HDP-2.3 from
http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
15 Aug 2017 15:54:02,691 ERROR [main] AmbariServer:929 - Failed to run the
Ambari Server
org.apache.ambari.server.AmbariException: An error occured during updating
current repository versions with stack repositories.
at
org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:108)
at
org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
at
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:921)
Caused by: java.lang.NullPointerException
at
org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:94)
... 2 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21774) NullPointerException if there is no proper HDP/HDF repo version folder
[
https://issues.apache.org/jira/browse/AMBARI-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amarnath reddy pappu updated AMBARI-21774:
--
Summary: NullPointerException if there is no proper HDP/HDF repo version
folder (was: NullPointerException if there is no proper HDP/HDF repo folder)
> NullPointerException if there is no proper HDP/HDF repo version folder
> --
>
> Key: AMBARI-21774
> URL: https://issues.apache.org/jira/browse/AMBARI-21774
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Reporter: amarnath reddy pappu
>
> Ambari server fails with "NullPointerException" if cluster is pointing to
> specific HDP/HDF/xxx
> and if that folder does not exist in resources folders.
> Should do NULL check and then print proper meaning full error message so that
> customer's can install correct that.
> {noformat}
> 15 Aug 2017 15:54:02,685 INFO [Stack Version Loading Thread]
> LatestRepoCallable:80 - Loading latest URL info for stack HDP-2.3 from
> http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
> 15 Aug 2017 15:54:02,691 ERROR [main] AmbariServer:929 - Failed to run the
> Ambari Server
> org.apache.ambari.server.AmbariException: An error occured during updating
> current repository versions with stack repositories.
> at
> org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:108)
> at
> org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
> at
> org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:921)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:94)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21774) NullPointerException if there is no proper HDP/HDF repo folder
[
https://issues.apache.org/jira/browse/AMBARI-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amarnath reddy pappu updated AMBARI-21774:
--
Summary: NullPointerException if there is no proper HDP/HDF repo folder
(was: NullPointerException if there is no proper HDP/HDF repo)
> NullPointerException if there is no proper HDP/HDF repo folder
> --
>
> Key: AMBARI-21774
> URL: https://issues.apache.org/jira/browse/AMBARI-21774
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Reporter: amarnath reddy pappu
>
> Ambari server fails with "NullPointerException" if cluster is pointing to
> specific HDP/HDF/xxx
> and if that folder does not exist in resources folders.
> Should do NULL check and then print proper meaning full error message so that
> customer's can install correct that.
> {noformat}
> 15 Aug 2017 15:54:02,685 INFO [Stack Version Loading Thread]
> LatestRepoCallable:80 - Loading latest URL info for stack HDP-2.3 from
> http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
> 15 Aug 2017 15:54:02,691 ERROR [main] AmbariServer:929 - Failed to run the
> Ambari Server
> org.apache.ambari.server.AmbariException: An error occured during updating
> current repository versions with stack repositories.
> at
> org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:108)
> at
> org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
> at
> org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:921)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:94)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135888#comment-16135888
]
Hudson commented on AMBARI-21769:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #39 (See
[https://builds.apache.org/job/Ambari-branch-2.6/39/])
AMBARI-21769. zeppelin principal and livy.superusers property do not (rlevas:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=4f27467497d9e109e29424bbff2d1d715a756af8])
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json
* (edit)
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json
* (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
* (edit) ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21675) Add Secure Flag to Cookie / JSESSIONID in Zeppelin
[ https://issues.apache.org/jira/browse/AMBARI-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135887#comment-16135887 ] Hudson commented on AMBARI-21675: - FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #39 (See [https://builds.apache.org/job/Ambari-branch-2.6/39/]) AMBARI-21675.Add Secure Flag to Cookie / JSESSIONID in (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=a143991496f2f5dc64ce35c2e31afa31b9ec404a]) * (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-shiro-ini.xml > Add Secure Flag to Cookie / JSESSIONID in Zeppelin > -- > > Key: AMBARI-21675 > URL: https://issues.apache.org/jira/browse/AMBARI-21675 > Project: Ambari > Issue Type: Bug > Components: ambari-views >Reporter: Prabhjyot Singh >Assignee: Prabhjyot Singh > Fix For: 2.6.0 > > > The secure flag attribute must be set on session cookies containing session > tokens for Zeppelin Application -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21773) Ambari HDP upgrade validation
amarnath reddy pappu created AMBARI-21773: - Summary: Ambari HDP upgrade validation Key: AMBARI-21773 URL: https://issues.apache.org/jira/browse/AMBARI-21773 Project: Ambari Issue Type: Bug Components: ambari-server Reporter: amarnath reddy pappu Priority: Critical Fix For: ambari-server 1. If there is any issues during the upgrade Finalize stage some of the tables left out referring old version of repos Ex: some host_version tables. 2. Some times support/eng is editing/modifying the DB tables to resolve some of the upgrade issues and not doing it correctly. These issues will popup when they attempt to do in the next upgrade or adding a new service. There should be some way to validate all the upgrade related tables to handle these scenarios. May be adding validations during the ambari service start. Options -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21772) zeppelin proxy user settings are not configured in core-site.xml on upgraded cluster from Ambari 2.4.2
[
https://issues.apache.org/jira/browse/AMBARI-21772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-21772:
--
Attachment: AMBARI-21772_branch-2.5_01.patch
AMBARI-21772_branch-2.6_01.patch
> zeppelin proxy user settings are not configured in core-site.xml on upgraded
> cluster from Ambari 2.4.2
> --
>
> Key: AMBARI-21772
> URL: https://issues.apache.org/jira/browse/AMBARI-21772
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21772_branch-2.5_01.patch,
> AMBARI-21772_branch-2.6_01.patch, AMBARI-21772_trunk_01.patch
>
>
> The configurations {{core-site/hadoop.proxyuser.zeppelin.hosts}} and
> {{core-site/hadoop.proxyuser.zeppelin.groups}}, which are required to allow
> zeppelin user to proxy other users in order to support impersonation features
> of few of the zeppelin interpreters, was not added during the upgrade from
> Ambari 2.4.2 to Ambari 2.5.2. Impersonation support for Zeppelin was added
> in AMBARI-20094.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135875#comment-16135875
]
Hudson commented on AMBARI-21768:
-
SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #1823 (See
[https://builds.apache.org/job/Ambari-branch-2.5/1823/])
AMBARI-21768. Spark History Server uses wrong log dir. (Attila Doroszlai
(swagle:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d1df4d9aa658975a82b5cb5e16be893d36268c87])
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
* (edit)
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21675) Add Secure Flag to Cookie / JSESSIONID in Zeppelin
[ https://issues.apache.org/jira/browse/AMBARI-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135874#comment-16135874 ] Hudson commented on AMBARI-21675: - SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #1823 (See [https://builds.apache.org/job/Ambari-branch-2.5/1823/]) Revert "AMBARI-21675.Add Secure Flag to Cookie / JSESSIONID in (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=4124bc30d021277efbd132123a99de9df7707c6e]) * (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-shiro-ini.xml > Add Secure Flag to Cookie / JSESSIONID in Zeppelin > -- > > Key: AMBARI-21675 > URL: https://issues.apache.org/jira/browse/AMBARI-21675 > Project: Ambari > Issue Type: Bug > Components: ambari-views >Reporter: Prabhjyot Singh >Assignee: Prabhjyot Singh > Fix For: 2.6.0 > > > The secure flag attribute must be set on session cookies containing session > tokens for Zeppelin Application -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21772) zeppelin proxy user settings are not configured in core-site.xml on upgraded cluster from Ambari 2.4.2
[
https://issues.apache.org/jira/browse/AMBARI-21772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-21772:
--
Attachment: AMBARI-21772_trunk_01.patch
> zeppelin proxy user settings are not configured in core-site.xml on upgraded
> cluster from Ambari 2.4.2
> --
>
> Key: AMBARI-21772
> URL: https://issues.apache.org/jira/browse/AMBARI-21772
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21772_trunk_01.patch
>
>
> The configurations {{core-site/hadoop.proxyuser.zeppelin.hosts}} and
> {{core-site/hadoop.proxyuser.zeppelin.groups}}, which are required to allow
> zeppelin user to proxy other users in order to support impersonation features
> of few of the zeppelin interpreters, was not added during the upgrade from
> Ambari 2.4.2 to Ambari 2.5.2. Impersonation support for Zeppelin was added
> in AMBARI-20094.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated AMBARI-21771: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2.5, branch-2.6 and trunk. > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-21769:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk
{noformat}
commit c129baa1f6e4438b7af9e7a01735d8ead05df1be
Author: Robert Levas
Date: Mon Aug 21 17:00:52 2017 -0400
{noformat}
Committed to branch-2.6
{noformat}
commit 4f27467497d9e109e29424bbff2d1d715a756af8
Author: Robert Levas
Date: Mon Aug 21 17:02:37 2017 -0400
{noformat}
Committed to branch-2.5
{noformat}
commit 0ae0494dcc264dd2bbb9cb84bad4b13fdfa2a9ef
Author: Robert Levas
Date: Mon Aug 21 17:03:40 2017 -0400
{noformat}
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21076) Move superset as a top-level module in HDP
[
https://issues.apache.org/jira/browse/AMBARI-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135816#comment-16135816
]
Hadoop QA commented on AMBARI-21076:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12882891/AMBARI-21076.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 javac{color}. The patch appears to cause the [build to
fail|https://builds.apache.org/job/Ambari-trunk-test-patch/12037//artifact/patch-work/patchJavacWarnings.txt].
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/12037//console
This message is automatically generated.
> Move superset as a top-level module in HDP
> --
>
> Key: AMBARI-21076
> URL: https://issues.apache.org/jira/browse/AMBARI-21076
> Project: Ambari
> Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
> Attachments: AMBARI-21076.1.patch, AMBARI-21076.2.patch,
> AMBARI-21076.3.patch, AMBARI-21076.patch
>
>
> Superset is a generic UI which can work with multiple data stores e.g HIVE,
> DRUID and any other dataStore that supports SQLALCHEMY dialects.
> Currently superset is installed as a master component under Druid.
> This task is to move superset out of Druid so that it can be installed and
> managed independent of Druid.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135792#comment-16135792 ] Siddharth Wagle commented on AMBARI-21771: -- +1 LGTM > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21718) Unable to remove ZooKeeper on heartbeat lost hosts
[ https://issues.apache.org/jira/browse/AMBARI-21718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135759#comment-16135759 ] Hudson commented on AMBARI-21718: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7915 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7915/]) AMBARI-21718. Unable to remove ZooKeeper on heartbeat lost hosts. (Ishan (jaimin: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d69084461998ad33c233bb33734b7935681df086]) * (edit) ambari-web/test/controllers/main/host/details_test.js * (edit) ambari-web/app/controllers/main/host/details.js * (edit) ambari-web/app/messages.js > Unable to remove ZooKeeper on heartbeat lost hosts > -- > > Key: AMBARI-21718 > URL: https://issues.apache.org/jira/browse/AMBARI-21718 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt > Fix For: trunk, 2.6.0 > > Attachments: AMBARI-21718_branch-2.6.patch, AMBARI-21718.patch > > > When you have a node that is running ZooKeeper Server, and that node dies, > and goes into a heartbeat lost state, it's not possible to remove the > ZooKeeper Server component which makes it impossible to remove that host from > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21772) zeppelin proxy user settings are not configured in core-site.xml on upgraded cluster from Ambari 2.4.2
Robert Levas created AMBARI-21772:
-
Summary: zeppelin proxy user settings are not configured in
core-site.xml on upgraded cluster from Ambari 2.4.2
Key: AMBARI-21772
URL: https://issues.apache.org/jira/browse/AMBARI-21772
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.4.2
Reporter: Robert Levas
Assignee: Robert Levas
Priority: Blocker
Fix For: 2.5.2
The configurations {{core-site/hadoop.proxyuser.zeppelin.hosts}} and
{{core-site/hadoop.proxyuser.zeppelin.groups}}, which are required to allow
zeppelin user to proxy other users in order to support impersonation features
of few of the zeppelin interpreters, was not added during the upgrade from
Ambari 2.4.2 to Ambari 2.5.2. Impersonation support for Zeppelin was added in
AMBARI-20094.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21718) Unable to remove ZooKeeper on heartbeat lost hosts
[ https://issues.apache.org/jira/browse/AMBARI-21718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135754#comment-16135754 ] Hudson commented on AMBARI-21718: - FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #38 (See [https://builds.apache.org/job/Ambari-branch-2.6/38/]) AMBARI-21718. Unable to remove ZooKeeper on heartbeat lost hosts. (Ishan (jaimin: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0d3daa5c6f5353d0e101ba1346464c53812acc5f]) * (edit) ambari-web/app/messages.js * (edit) ambari-web/app/controllers/main/host/details.js * (edit) ambari-web/test/controllers/main/host/details_test.js > Unable to remove ZooKeeper on heartbeat lost hosts > -- > > Key: AMBARI-21718 > URL: https://issues.apache.org/jira/browse/AMBARI-21718 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt > Fix For: trunk, 2.6.0 > > Attachments: AMBARI-21718_branch-2.6.patch, AMBARI-21718.patch > > > When you have a node that is running ZooKeeper Server, and that node dies, > and goes into a heartbeat lost state, it's not possible to remove the > ZooKeeper Server component which makes it impossible to remove that host from > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated AMBARI-21768:
---
Reporter: Yesha Vora (was: Doroszlai, Attila)
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-21770:
-
Attachment: AMBARI-21770.patch
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
> Attachments: AMBARI-21770.patch
>
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Attachment: (was: AMBARI-21173.patch)
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Status: Patch Available (was: Reopened)
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch, AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21675) Add Secure Flag to Cookie / JSESSIONID in Zeppelin
[ https://issues.apache.org/jira/browse/AMBARI-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated AMBARI-21675: --- Fix Version/s: 2.6.0 > Add Secure Flag to Cookie / JSESSIONID in Zeppelin > -- > > Key: AMBARI-21675 > URL: https://issues.apache.org/jira/browse/AMBARI-21675 > Project: Ambari > Issue Type: Bug > Components: ambari-views >Reporter: Prabhjyot Singh >Assignee: Prabhjyot Singh > Fix For: 2.6.0 > > > The secure flag attribute must be set on session cookies containing session > tokens for Zeppelin Application -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Attachment: AMBARI-21173.patch
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch, AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135722#comment-16135722
]
Hadoop QA commented on AMBARI-21768:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12882901/AMBARI-21768.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/12036//console
This message is automatically generated.
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135674#comment-16135674 ] Aravindan Vijayan commented on AMBARI-21771: [~swagle] / [~dsen] / [~oleewere] / [~sumitmohanty] Can you review this simple patch? > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated AMBARI-21771: --- Status: Patch Available (was: Open) > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21770) Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-21770:
-
Summary: Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
(was: spark1 shuffle service not configured properly)
> Spark1 Shuffle Property Is Removed Incorrectly on a Stack Upgrade
> -
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (AMBARI-21770) spark1 shuffle service not configured properly
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley reassigned AMBARI-21770:
Assignee: Jonathan Hurley
> spark1 shuffle service not configured properly
> --
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Assignee: Jonathan Hurley
>Priority: Blocker
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21770) spark1 shuffle service not configured properly
[
https://issues.apache.org/jira/browse/AMBARI-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135663#comment-16135663
]
Jonathan Hurley commented on AMBARI-21770:
--
Here is a summary of the 3 versions of Ambari at play here and what their
installations do:
- HDP 2.5+ with Ambari 2.5.x allows installation of either Spark1 or Spark2.
However, it doesn't matter which one you choose, it sets both {{spark_shuffle}}
and {{spark2_suffle}}. This behavior has been around for a very long time and
we can't change it this late in a release cycle.
{code:title=HDP 2.5 with Ambari 2.5 and Either Spark1 or Spark2}
"yarn.nodemanager.aux-services":
"mapreduce_shuffle,spark_shuffle,spark2_shuffle",
"yarn.nodemanager.aux-services.mapreduce_shuffle.class":
"org.apache.hadoop.mapred.ShuffleHandler",
"yarn.nodemanager.aux-services.spark2_shuffle.class":
"org.apache.spark.network.yarn.YarnShuffleService",
"yarn.nodemanager.aux-services.spark2_shuffle.classpath":
"{{stack_root}}/${hdp.version}/spark2/aux/*",
"yarn.nodemanager.aux-services.spark_shuffle.class":
"org.apache.spark.network.yarn.YarnShuffleService",
"yarn.nodemanager.aux-services.spark_shuffle.classpath":
"{{stack_root}}/${hdp.version}/spark/aux/*",
{code}
- BigInsights 4.2.5 only allows installing Spark2. However it only configures
{{spark_shuffle}} and points it at Spark2:
{code:title=BigInsights 4.2.5 with Spark2}
"yarn.nodemanager.aux-services": "mapreduce_shuffle,spark_shuffle",
"yarn.nodemanager.aux-services.mapreduce_shuffle.class":
"org.apache.hadoop.mapred.ShuffleHandler",
"yarn.nodemanager.aux-services.spark_shuffle.class":
"org.apache.spark.network.yarn.YarnShuffleService",
"yarn.nodemanager.aux-services.spark_shuffle.classpath":
"{{stack_root}}/${iop.version}/spark2/aux/*",
{code}
- BigInsights 4.2.0 only allows installing Spark1. It doesn't even configure
{{spark_shuffle}} by default:
{code:title=BigInsights 4.2.0 with Spark1}
"yarn.nodemanager.aux-services": "mapreduce_shuffle",
"yarn.nodemanager.aux-services.mapreduce_shuffle.class":
"org.apache.hadoop.mapred.ShuffleHandler",
{code}
If these properties are not modified by the administrator, then on upgrade to
HDP, they will be moved to the HDP 2.6 values which include both
{{spark_shuffle}} and {{spark2_shuffle}}.
It seems like the replacement of {{spark_shuffle}} with "" causes problems
post-upgrade when adding spark2 to the cluster after-the-fact.
I think that we need to set these values explicitly on upgrade instead of
having it try to "remove" one.
{code:title=IOP 4.2.0 to HDP 2.6}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
{code}
{code:title=IOP 4.2.5 to HDP 2.6 (no change)}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle,spark2_shuffle
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.nodemanager.aux-services.spark2_shuffle.class =
org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
{code}
> spark1 shuffle service not configured properly
> --
>
> Key: AMBARI-21770
> URL: https://issues.apache.org/jira/browse/AMBARI-21770
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Yesha Vora
>Priority: Blocker
>
> Steps to reproduce:
> * Install BI 4.2.0
> * Upgrade to Ambari-2.5.2.0
> * Upgrade to HDP-2.6
> Here, Spark-shuffle service is not configured for Spark1 component. It only
> configures Spark2-shuffle service
> {code:title=current yarn-site.xml after upgrade}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
> yarn.nodemanager.aux-services.spark2_shuffle.class =
> org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes VALUE>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath VALUE>{code}
> {code:title=expected yarn-site.xml}
> yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
> yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
> yarn.nodemanager.aux-services.spark
[jira] [Commented] (AMBARI-21757) Allow for keytab regeneration to be filtered for hosts
[
https://issues.apache.org/jira/browse/AMBARI-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135646#comment-16135646
]
Hadoop QA commented on AMBARI-21757:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12882910/AMBARI-21757-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
ambari-server.
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/12035//console
This message is automatically generated.
> Allow for keytab regeneration to be filtered for hosts
> --
>
> Key: AMBARI-21757
> URL: https://issues.apache.org/jira/browse/AMBARI-21757
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Eugene Chekanskiy
>Assignee: Eugene Chekanskiy
> Fix For: 3.0.0
>
> Attachments: AMBARI-21757-trunk-2.patch, AMBARI-21757-trunk.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135625#comment-16135625 ] Aravindan Vijayan commented on AMBARI-21771: Manually tested by creating RPM on RHEL6 machine. mvn clean test passed. Verified that removing these artifacts did not affect logfeeder and metrics collector working. > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21718) Unable to remove ZooKeeper on heartbeat lost hosts
[ https://issues.apache.org/jira/browse/AMBARI-21718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaimin Jetly updated AMBARI-21718: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch received +1 on reviewboard. Patch committed to trunk and branch-2.6 > Unable to remove ZooKeeper on heartbeat lost hosts > -- > > Key: AMBARI-21718 > URL: https://issues.apache.org/jira/browse/AMBARI-21718 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Ishan Bhatt >Assignee: Ishan Bhatt > Fix For: trunk, 2.6.0 > > Attachments: AMBARI-21718_branch-2.6.patch, AMBARI-21718.patch > > > When you have a node that is running ZooKeeper Server, and that node dies, > and goes into a heartbeat lost state, it's not possible to remove the > ZooKeeper Server component which makes it impossible to remove that host from > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
[ https://issues.apache.org/jira/browse/AMBARI-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated AMBARI-21771: --- Attachment: AMBARI-21771.patch > Maven cleanup of ambari-metrics and ambari-logsearch modules. > - > > Key: AMBARI-21771 > URL: https://issues.apache.org/jira/browse/AMBARI-21771 > Project: Ambari > Issue Type: Bug > Components: ambari-logsearch, ambari-metrics >Affects Versions: 2.5.2 >Reporter: Aravindan Vijayan >Assignee: Aravindan Vijayan > Fix For: 2.5.2 > > Attachments: AMBARI-21771.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21771) Maven cleanup of ambari-metrics and ambari-logsearch modules.
Aravindan Vijayan created AMBARI-21771: -- Summary: Maven cleanup of ambari-metrics and ambari-logsearch modules. Key: AMBARI-21771 URL: https://issues.apache.org/jira/browse/AMBARI-21771 Project: Ambari Issue Type: Bug Components: ambari-logsearch, ambari-metrics Affects Versions: 2.5.2 Reporter: Aravindan Vijayan Assignee: Aravindan Vijayan Fix For: 2.5.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135585#comment-16135585
]
Hadoop QA commented on AMBARI-21769:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12882906/AMBARI-21769_branch-2.5_01.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/12034//console
This message is automatically generated.
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
[
https://issues.apache.org/jira/browse/AMBARI-21767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135560#comment-16135560
]
Mausami Kusum commented on AMBARI-21767:
I tried the steps as mentioned in the defect by Di:
1. Restore old core-site configurations to the version before your workaround
changes
2. Regenerate keytabs via Ambari web UI
3. hive service check, please update the JIRA with the test results.
After above steps , hive service check started to fail again with the same error
> Kerberos cluster - hive service check failed post migration
> ---
>
> Key: AMBARI-21767
> URL: https://issues.apache.org/jira/browse/AMBARI-21767
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Di Li
> Fix For: 2.5.2
>
>
> Post migration hive service check failed with following error:
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> 2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
> 50111 idtest.ambari-qa.1503075103.65.pig
> /etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
> ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
> Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com
> is not allowed to impersonate ambari-qa"}http_code <500>
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> *WORKAROUND:*
> Shailaja suggested to modify all the entries in custom core-site property
> under HDFS service to * and also add hbase. I added hbase key value pair as
> following:
> hadoop.proxyuser.hbase.groups *
> hadoop.proxyuser.hbase.hosts *
> Also modified following properties:
> hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.knox.groups was usersmodified to *
> hadoop.proxyuser.HTTP.groups was usersmodified to *
> hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
> After that hive service check passed. If this work around is not done bigsql
> upgrade will fail during service check. This step is not documented.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21756) Use latest-vdf for default when version is unspecified
[
https://issues.apache.org/jira/browse/AMBARI-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135549#comment-16135549
]
Hudson commented on AMBARI-21756:
-
SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7914 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/7914/])
AMBARI-21756. Use latest-vdf for default when version is unspecified (ncole:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=15aa0375e8c79bbaab237a28c1f0b78275f551f6])
* (add)
ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.5.xml
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
* (edit) ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/repoinfo.xml
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/repos/repoinfo.xml
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/repos/hdp.json
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java
* (edit) ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
* (add)
ambari-server/src/test/resources/stacks/HDP/2.2.1/services/RANGER/alerts.json
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/metainfo.xml
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
> Use latest-vdf for default when version is unspecified
> --
>
> Key: AMBARI-21756
> URL: https://issues.apache.org/jira/browse/AMBARI-21756
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Reporter: Nate Cole
>Assignee: Nate Cole
>Priority: Critical
> Fix For: 2.6.0
>
> Attachments: AMBARI-21756.patch
>
>
> Use the links in the tag that identify the {{latest-vdf}} that is
> for a version. This is used in blueprint installs when no version is
> specified. This preserves backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21770) spark1 shuffle service not configured properly
Yesha Vora created AMBARI-21770:
---
Summary: spark1 shuffle service not configured properly
Key: AMBARI-21770
URL: https://issues.apache.org/jira/browse/AMBARI-21770
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.2
Reporter: Yesha Vora
Priority: Blocker
Steps to reproduce:
* Install BI 4.2.0
* Upgrade to Ambari-2.5.2.0
* Upgrade to HDP-2.6
Here, Spark-shuffle service is not configured for Spark1 component. It only
configures Spark2-shuffle service
{code:title=current yarn-site.xml after upgrade}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
yarn.nodemanager.aux-services.spark2_shuffle.class =
org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath {code}
{code:title=expected yarn-site.xml}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21756) Use latest-vdf for default when version is unspecified
[
https://issues.apache.org/jira/browse/AMBARI-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nate Cole updated AMBARI-21756:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Use latest-vdf for default when version is unspecified
> --
>
> Key: AMBARI-21756
> URL: https://issues.apache.org/jira/browse/AMBARI-21756
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Reporter: Nate Cole
>Assignee: Nate Cole
>Priority: Critical
> Fix For: 2.6.0
>
> Attachments: AMBARI-21756.patch
>
>
> Use the links in the tag that identify the {{latest-vdf}} that is
> for a version. This is used in blueprint installs when no version is
> specified. This preserves backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
[
https://issues.apache.org/jira/browse/AMBARI-21767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Di Li updated AMBARI-21767:
---
Description:
Post migration hive service check failed with following error:
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
'/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
50111 idtest.ambari-qa.1503075103.65.pig
/etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com is
not allowed to impersonate ambari-qa"}http_code <500>
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
*WORKAROUND:*
Shailaja suggested to modify all the entries in custom core-site property under
HDFS service to * and also add hbase. I added hbase key value pair as
following:
hadoop.proxyuser.hbase.groups *
hadoop.proxyuser.hbase.hosts *
Also modified following properties:
hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.knox.groups was usersmodified to *
hadoop.proxyuser.HTTP.groups was usersmodified to *
hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
After that hive service check passed. If this work around is not done bigsql
upgrade will fail during service check. This step is not documented.
was:
Post migration hive service check failed with following error:
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
'/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
50111 idtest.ambari-qa.1503075103.65.pig
/etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com is
not allowed to impersonate ambari-qa"}http_code <500>
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
Shailaja suggested to modify all the entries in custom core-site property under
HDFS service to * and also add hbase. I added hbase key value pair as
following:
hadoop.proxyuser.hbase.groups *
hadoop.proxyuser.hbase.hosts *
Also modified following properties:
hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.knox.groups was usersmodified to *
hadoop.proxyuser.HTTP.groups was usersmodified to *
hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
After that hive service check passed. If this work around is not done bigsql
upgrade will fail during service check. This step is not documented.
> Kerberos cluster - hive service check failed post migration
> ---
>
> Key: AMBARI-21767
> URL: https://issues.apache.org/jira/browse/AMBARI-21767
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Di Li
> Fix For: 2.5.2
>
>
> Post migration hive service check failed with following error:
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> 2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
> 50111 idtest.ambari-qa.1503075103.65.pig
> /etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
> ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
> Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com
> is not allowed to impersonate ambari-qa"}http_code <500>
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> *WORKAROUND:*
> Shailaja suggested to modify all the entries in custom core-site property
> under HDFS service to * and also add hbase. I added hbase key value pair as
>
[jira] [Commented] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
[
https://issues.apache.org/jira/browse/AMBARI-21767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135434#comment-16135434
]
Siddharth Wagle commented on AMBARI-21767:
--
Hi [~dili], instead of the workaround we should be doing a Regenerate keytabs
right after EU, we would need to hand-edit and fix quite a few config issues
unless we run it. Hence, we recommend Regenerate to fix these issues.
> Kerberos cluster - hive service check failed post migration
> ---
>
> Key: AMBARI-21767
> URL: https://issues.apache.org/jira/browse/AMBARI-21767
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Di Li
> Fix For: 2.5.2
>
>
> Post migration hive service check failed with following error:
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> 2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
> 50111 idtest.ambari-qa.1503075103.65.pig
> /etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
> ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
> Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com
> is not allowed to impersonate ambari-qa"}http_code <500>
> Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
> HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
> ambari-qa"}http_code <500>
> Shailaja suggested to modify all the entries in custom core-site property
> under HDFS service to * and also add hbase. I added hbase key value pair as
> following:
> hadoop.proxyuser.hbase.groups *
> hadoop.proxyuser.hbase.hosts *
> Also modified following properties:
> hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
> hadoop.proxyuser.knox.groups was usersmodified to *
> hadoop.proxyuser.HTTP.groups was usersmodified to *
> hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
> After that hive service check passed. If this work around is not done bigsql
> upgrade will fail during service check. This step is not documented.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21757) Allow for keytab regeneration to be filtered for hosts
[ https://issues.apache.org/jira/browse/AMBARI-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chekanskiy updated AMBARI-21757: --- Attachment: AMBARI-21757-trunk-2.patch > Allow for keytab regeneration to be filtered for hosts > -- > > Key: AMBARI-21757 > URL: https://issues.apache.org/jira/browse/AMBARI-21757 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Eugene Chekanskiy >Assignee: Eugene Chekanskiy > Fix For: 3.0.0 > > Attachments: AMBARI-21757-trunk-2.patch, AMBARI-21757-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21757) Allow for keytab regeneration to be filtered for hosts
[ https://issues.apache.org/jira/browse/AMBARI-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chekanskiy updated AMBARI-21757: --- Attachment: (was: AMBARI-21757-trunk-2.patch) > Allow for keytab regeneration to be filtered for hosts > -- > > Key: AMBARI-21757 > URL: https://issues.apache.org/jira/browse/AMBARI-21757 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Eugene Chekanskiy >Assignee: Eugene Chekanskiy > Fix For: 3.0.0 > > Attachments: AMBARI-21757-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21757) Allow for keytab regeneration to be filtered for hosts
[ https://issues.apache.org/jira/browse/AMBARI-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chekanskiy updated AMBARI-21757: --- Attachment: AMBARI-21757-trunk-2.patch > Allow for keytab regeneration to be filtered for hosts > -- > > Key: AMBARI-21757 > URL: https://issues.apache.org/jira/browse/AMBARI-21757 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Eugene Chekanskiy >Assignee: Eugene Chekanskiy > Fix For: 3.0.0 > > Attachments: AMBARI-21757-trunk-2.patch, AMBARI-21757-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-21769:
--
Status: Patch Available (was: In Progress)
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
[
https://issues.apache.org/jira/browse/AMBARI-21769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-21769:
--
Attachment: AMBARI-21769_branch-2.5_01.patch
AMBARI-21769_branch-2.6_01.patch
AMBARI-21769_trunk_01.patch
> zeppelin principal and livy.superusers property do not match on upgraded
> cluster from Ambari 2.4.2 -and HDP 2.5.5
> -
>
> Key: AMBARI-21769
> URL: https://issues.apache.org/jira/browse/AMBARI-21769
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.4.2
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21769_branch-2.5_01.patch,
> AMBARI-21769_branch-2.6_01.patch, AMBARI-21769_trunk_01.patch
>
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
> installed and Kerberos is enabled, it is expected that that
> {{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
> principal name of the Zeppelin user. However, this value is not always set,
> depending on the order in which the services were installed, when Kerberos
> was enabled, and whether an Ambari or stack upgrade was involved. And if it
> is set, the value may be incorrect since the Kerberos descriptor assumes the
> Zeppelin principal is {{zeppelin-}}
> The solution is to move the logic to set the {{livy-conf/livy.superusers}}
> and {{livy2-conf/livy.superusers}} to the stack advisor to the appropriate
> value can be added as needed. Also while upgrading to Ambari 2.5.2, the
> value(s) should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doroszlai, Attila updated AMBARI-21768:
---
Status: Patch Available (was: In Progress)
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doroszlai, Attila updated AMBARI-21768:
---
Attachment: AMBARI-21768.patch
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
> Attachments: AMBARI-21768.patch
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21768) Spark History Server uses wrong log dir
[
https://issues.apache.org/jira/browse/AMBARI-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doroszlai, Attila updated AMBARI-21768:
---
Description:
Steps to reproduce:
# Install BI 4.2.0
# Upgrade to Ambari 2.5.2
# Upgrade to HDP 2.6
# Run some Spark task (eg. {{spark-shell}})
# Open Spark History Server UI
Result: Spark History Server shows no jobs, because it reads logs from the
wrong directory.
Note: Test with both default and customized {{spark.eventLog.dir}}
configuration.
was:
Steps to reproduce:
# Install BI 4.2.0
# Upgrade to Ambari 2.5.2
# Upgrade to HDP 2.6
Result: Spark History Server shows no jobs, because it reads logs from the
wrong directory.
Note: Test with both default and customized {{spark.eventLog.dir}}
configuration.
> Spark History Server uses wrong log dir
> ---
>
> Key: AMBARI-21768
> URL: https://issues.apache.org/jira/browse/AMBARI-21768
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Blocker
> Fix For: 2.5.2
>
>
> Steps to reproduce:
> # Install BI 4.2.0
> # Upgrade to Ambari 2.5.2
> # Upgrade to HDP 2.6
> # Run some Spark task (eg. {{spark-shell}})
> # Open Spark History Server UI
> Result: Spark History Server shows no jobs, because it reads logs from the
> wrong directory.
> Note: Test with both default and customized {{spark.eventLog.dir}}
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
[
https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135375#comment-16135375
]
Zhiyuan Yang commented on AMBARI-21730:
---
[~xabriel] I wasn't able to reproduce the issue unfortunately. Can you share
more details about your deployment (version and configurations)?
> Default of hive.llap.execution.mode=only on Hive Interactive breaks
> StorageHandler tables.
> --
>
> Key: AMBARI-21730
> URL: https://issues.apache.org/jira/browse/AMBARI-21730
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.5.1
>Reporter: Xabriel J Collazo Mojica
>Assignee: Siddharth Wagle
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example,
> HBase tables like these fail:
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz",
> "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE
> hbase_table_1 SELECT * FROM hbase_table_1;
> INFO : Compiling
> command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>
> INFO : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
> INFO : Dag submit failed due to There is conflicting local resource
> (guava-14.0.1.jar) between dag local resource and vertex Map 1 local
> resource.
> Resource of dag : resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
> } size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
> Resource of vertex: resource { scheme: "hdfs" host:
> "intel68bvt1.fyre.ibm.com" port: 8020 file:
> "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
> }
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is
> not seen anymore.*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21769) zeppelin principal and livy.superusers property do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
Robert Levas created AMBARI-21769:
-
Summary: zeppelin principal and livy.superusers property do not
match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
Key: AMBARI-21769
URL: https://issues.apache.org/jira/browse/AMBARI-21769
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.4.2
Reporter: Robert Levas
Assignee: Robert Levas
Priority: Blocker
Fix For: 2.5.2
In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are
installed and Kerberos is enabled, it is expected that that
{{livy-conf/livy.superusers}} and {{livy2-conf/livy.superusers}} contain the
principal name of the Zeppelin user. However, this value is not always set,
depending on the order in which the services were installed, when Kerberos was
enabled, and whether an Ambari or stack upgrade was involved. And if it is
set, the value may be incorrect since the Kerberos descriptor assumes the
Zeppelin principal is {{zeppelin-}}
The solution is to move the logic to set the {{livy-conf/livy.superusers}} and
{{livy2-conf/livy.superusers}} to the stack advisor to the appropriate value
can be added as needed. Also while upgrading to Ambari 2.5.2, the value(s)
should be fixed if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21768) Spark History Server uses wrong log dir
Doroszlai, Attila created AMBARI-21768:
--
Summary: Spark History Server uses wrong log dir
Key: AMBARI-21768
URL: https://issues.apache.org/jira/browse/AMBARI-21768
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.2
Reporter: Doroszlai, Attila
Assignee: Doroszlai, Attila
Priority: Blocker
Fix For: 2.5.2
Steps to reproduce:
# Install BI 4.2.0
# Upgrade to Ambari 2.5.2
# Upgrade to HDP 2.6
Result: Spark History Server shows no jobs, because it reads logs from the
wrong directory.
Note: Test with both default and customized {{spark.eventLog.dir}}
configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21766) When Matching New VDFs for Parent Repos only Consider STANDARD Types
[
https://issues.apache.org/jira/browse/AMBARI-21766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-21766:
-
Attachment: AMBARI-21766.patch
> When Matching New VDFs for Parent Repos only Consider STANDARD Types
>
>
> Key: AMBARI-21766
> URL: https://issues.apache.org/jira/browse/AMBARI-21766
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.6.0
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.6.0
>
> Attachments: AMBARI-21766.patch
>
>
> When registering a {{MAINT}} VDF where the regex matches both an existing
> {{STANDARD}} and existing {{PATCH}} repository, the new VDF registration will
> fail with:
> {quote}
> {
> "status": 400,
> "message": "Patch 2.5.4.0-121 was found to match more than one repository
> in use: 2.5.0.0-1237, 2.5.4.0-121. Move all services to a common version and
> try again."
> }
> {quote}
> We should only be matching for parents against the {{STANDARD}} repo type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21767) Kerberos cluster - hive service check failed post migration
Di Li created AMBARI-21767:
--
Summary: Kerberos cluster - hive service check failed post
migration
Key: AMBARI-21767
URL: https://issues.apache.org/jira/browse/AMBARI-21767
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.2
Reporter: Di Li
Fix For: 2.5.2
Post migration hive service check failed with following error:
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
2017-08-18 09:51:50,109 - Retrying after 5 seconds. Reason: Execution of
'/var/lib/ambari-agent/tmp/templetonSmoke.sh mmk-m-2.fyre.ibm.com ambari-qa
50111 idtest.ambari-qa.1503075103.65.pig
/etc/security/keytabs/smokeuser.headless.keytab true /usr/bin/kinit
ambari-qa-...@ibm.com /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke
Test (ddl cmd): Failed. : {"error":"User: HTTP/mmk-m-2.fyre.ibm@ibm.com is
not allowed to impersonate ambari-qa"}http_code <500>
Templeton Smoke Test (ddl cmd): Failed. : {"error":"User:
HTTP/mmk-m-2.fyre.ibm@ibm.com is not allowed to impersonate
ambari-qa"}http_code <500>
Shailaja suggested to modify all the entries in custom core-site property under
HDFS service to * and also add hbase. I added hbase key value pair as
following:
hadoop.proxyuser.hbase.groups *
hadoop.proxyuser.hbase.hosts *
Also modified following properties:
hadoop.proxyuser.hcat.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.HTTP.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.yarn.hosts was mmk-m-2.fyre.ibm.com modified to *
hadoop.proxyuser.knox.groups was usersmodified to *
hadoop.proxyuser.HTTP.groups was usersmodified to *
hadoop.proxyuser.root.hosts was mmk-m-1.fyre.ibm.com modified to *
After that hive service check passed. If this work around is not done bigsql
upgrade will fail during service check. This step is not documented.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21076) Move superset as a top-level module in HDP
[ https://issues.apache.org/jira/browse/AMBARI-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated AMBARI-21076: -- Attachment: AMBARI-21076.3.patch > Move superset as a top-level module in HDP > -- > > Key: AMBARI-21076 > URL: https://issues.apache.org/jira/browse/AMBARI-21076 > Project: Ambari > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa > Attachments: AMBARI-21076.1.patch, AMBARI-21076.2.patch, > AMBARI-21076.3.patch, AMBARI-21076.patch > > > Superset is a generic UI which can work with multiple data stores e.g HIVE, > DRUID and any other dataStore that supports SQLALCHEMY dialects. > Currently superset is installed as a master component under Druid. > This task is to move superset out of Druid so that it can be installed and > managed independent of Druid. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21766) When Matching New VDFs for Parent Repos only Consider STANDARD Types
Jonathan Hurley created AMBARI-21766:
Summary: When Matching New VDFs for Parent Repos only Consider
STANDARD Types
Key: AMBARI-21766
URL: https://issues.apache.org/jira/browse/AMBARI-21766
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.6.0
Reporter: Jonathan Hurley
Assignee: Jonathan Hurley
Priority: Critical
Fix For: 2.6.0
When registering a {{MAINT}} VDF where the regex matches both an existing
{{STANDARD}} and existing {{PATCH}} repository, the new VDF registration will
fail with:
{quote}
{
"status": 400,
"message": "Patch 2.5.4.0-121 was found to match more than one repository
in use: 2.5.0.0-1237, 2.5.4.0-121. Move all services to a common version and
try again."
}
{quote}
We should only be matching for parents against the {{STANDARD}} repo type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21765) Missing 'hdfs_scheme' variable in params.py file for Apache Knox
Sandeep More created AMBARI-21765: - Summary: Missing 'hdfs_scheme' variable in params.py file for Apache Knox Key: AMBARI-21765 URL: https://issues.apache.org/jira/browse/AMBARI-21765 Project: Ambari Issue Type: Bug Reporter: Sandeep More Assignee: Sandeep More Some of the topologies require 'hdfs_scheme' protocol scheme variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21756) Use latest-vdf for default when version is unspecified
[
https://issues.apache.org/jira/browse/AMBARI-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135266#comment-16135266
]
Hudson commented on AMBARI-21756:
-
FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #37 (See
[https://builds.apache.org/job/Ambari-branch-2.6/37/])
AMBARI-21756. Use latest-vdf for default when version is unspecified (ncole:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=98be9abf850fd04f1399c4894277b1770a018a5a])
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/metainfo.xml
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/repos/repoinfo.xml
* (add)
ambari-server/src/test/resources/stacks/HDP/2.2.1/services/RANGER/alerts.json
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java
* (add) ambari-server/src/test/resources/stacks/HDP/2.2.1/repos/hdp.json
* (edit) ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/repoinfo.xml
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
* (edit) ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json
* (add)
ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.5.xml
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
> Use latest-vdf for default when version is unspecified
> --
>
> Key: AMBARI-21756
> URL: https://issues.apache.org/jira/browse/AMBARI-21756
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Reporter: Nate Cole
>Assignee: Nate Cole
>Priority: Critical
> Fix For: 2.6.0
>
> Attachments: AMBARI-21756.patch
>
>
> Use the links in the tag that identify the {{latest-vdf}} that is
> for a version. This is used in blueprint installs when no version is
> specified. This preserves backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21758) Prevent New Clusters from Being Provisioned With PATCH/MAINT Repos
[
https://issues.apache.org/jira/browse/AMBARI-21758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135219#comment-16135219
]
Hudson commented on AMBARI-21758:
-
FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7913 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/7913/])
AMBARI-21758 - Prevent New Clusters from Being Provisioned With (jhurley:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=28a42236a1dfc924fed73edfaf4ea6d896f16f02])
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
> Prevent New Clusters from Being Provisioned With PATCH/MAINT Repos
> --
>
> Key: AMBARI-21758
> URL: https://issues.apache.org/jira/browse/AMBARI-21758
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.6.0
>
>
> New clusters should only be able to be provisioned using a {{STANDARD}}
> repository.
> STR:
> - Register a 2.5.0.0-1234 {{STANDARD}} repo
> - Register a 2.5.4.0- {{MAINT} repo
> - Use a blueprint go create a repo for 2.5.4.0-
> The cluster is created and is in a weird state. We should reject these types
> of repos...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Attachment: (was: AMBARI-21173.patch)
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi reopened AMBARI-21173:
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value
[
https://issues.apache.org/jira/browse/AMBARI-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-21173:
---
Attachment: AMBARI-21173.patch
> Not able to start Yarn services after restoring the configs to initial value
>
>
> Key: AMBARI-21173
> URL: https://issues.apache.org/jira/browse/AMBARI-21173
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 3.0.0
>Reporter: Sumana Sathish
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: AMBARI-21173.patch
>
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21763) Cache hashes should not be reset after restart ; some ambari-server restart fixes
[
https://issues.apache.org/jira/browse/AMBARI-21763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135205#comment-16135205
]
Hadoop QA commented on AMBARI-21763:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12882874/AMBARI-21763.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/12033//console
This message is automatically generated.
> Cache hashes should not be reset after restart ; some ambari-server restart
> fixes
> -
>
> Key: AMBARI-21763
> URL: https://issues.apache.org/jira/browse/AMBARI-21763
> Project: Ambari
> Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 3.0.0
>
> Attachments: AMBARI-21763.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21762) Sometimes alerts order is not correctly arranged if ordering by Status
[ https://issues.apache.org/jira/browse/AMBARI-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135203#comment-16135203 ] Hudson commented on AMBARI-21762: - FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #36 (See [https://builds.apache.org/job/Ambari-branch-2.6/36/]) AMBARI-21762 Sometimes alerts order is not correctly arranged if (atkach: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=596b145db79735263fbd88e56f521edfb90fc77b]) * (edit) ambari-web/test/views/common/sort_view_test.js * (edit) ambari-web/app/views/main/alert_definitions_view.js * (edit) ambari-web/app/views/common/sort_view.js > Sometimes alerts order is not correctly arranged if ordering by Status > -- > > Key: AMBARI-21762 > URL: https://issues.apache.org/jira/browse/AMBARI-21762 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Andrii Tkach >Assignee: Andrii Tkach >Priority: Blocker > Fix For: 2.5.2 > > Attachments: AMBARI-21762.patch, AMBARI-21762_trunk.patch, Screen > Shot 2017-08-18 at 11.29.56 AM.png > > > See screenshot. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21762) Sometimes alerts order is not correctly arranged if ordering by Status
[ https://issues.apache.org/jira/browse/AMBARI-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135192#comment-16135192 ] Hudson commented on AMBARI-21762: - SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #1822 (See [https://builds.apache.org/job/Ambari-branch-2.5/1822/]) AMBARI-21762 Sometimes alerts order is not correctly arranged if (atkach: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ba1fa0bfbcfccc1d182ce9f5d5928757c0fc2ea5]) * (edit) ambari-web/app/views/common/sort_view.js * (edit) ambari-web/test/views/common/sort_view_test.js * (edit) ambari-web/app/views/main/alert_definitions_view.js > Sometimes alerts order is not correctly arranged if ordering by Status > -- > > Key: AMBARI-21762 > URL: https://issues.apache.org/jira/browse/AMBARI-21762 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Andrii Tkach >Assignee: Andrii Tkach >Priority: Blocker > Fix For: 2.5.2 > > Attachments: AMBARI-21762.patch, AMBARI-21762_trunk.patch, Screen > Shot 2017-08-18 at 11.29.56 AM.png > > > See screenshot. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21762) Sometimes alerts order is not correctly arranged if ordering by Status
[ https://issues.apache.org/jira/browse/AMBARI-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135188#comment-16135188 ] Hudson commented on AMBARI-21762: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7912 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7912/]) AMBARI-21762 Sometimes alerts order is not correctly arranged if (atkach: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=f901e15bac6728215ab5adea54775565bb8bdd46]) * (edit) ambari-web/test/views/main/alert_definitions_view_test.js * (edit) ambari-web/test/views/common/sort_view_test.js * (edit) ambari-web/app/views/common/sort_view.js * (edit) ambari-web/app/views/main/alert_definitions_view.js > Sometimes alerts order is not correctly arranged if ordering by Status > -- > > Key: AMBARI-21762 > URL: https://issues.apache.org/jira/browse/AMBARI-21762 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.2 >Reporter: Andrii Tkach >Assignee: Andrii Tkach >Priority: Blocker > Fix For: 2.5.2 > > Attachments: AMBARI-21762.patch, AMBARI-21762_trunk.patch, Screen > Shot 2017-08-18 at 11.29.56 AM.png > > > See screenshot. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21764) Patch / Maint UI Assets Need Style Adjustments
[ https://issues.apache.org/jira/browse/AMBARI-21764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonenko Alexander updated AMBARI-21764: - Resolution: Fixed Status: Resolved (was: Patch Available) committed to branch-feature-AMBARI-21450 > Patch / Maint UI Assets Need Style Adjustments > -- > > Key: AMBARI-21764 > URL: https://issues.apache.org/jira/browse/AMBARI-21764 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.3 >Reporter: Antonenko Alexander >Assignee: Antonenko Alexander > Fix For: 2.5.3 > > Attachments: AMBARI-21764.patch > > > The Stacks & Versions UI page needs to have some of its visuals updated to > make the experience of using this page better. See screenshots ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21764) Patch / Maint UI Assets Need Style Adjustments
[ https://issues.apache.org/jira/browse/AMBARI-21764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135182#comment-16135182 ] Antonenko Alexander commented on AMBARI-21764: -- manually tested > Patch / Maint UI Assets Need Style Adjustments > -- > > Key: AMBARI-21764 > URL: https://issues.apache.org/jira/browse/AMBARI-21764 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 2.5.3 >Reporter: Antonenko Alexander >Assignee: Antonenko Alexander > Fix For: 2.5.3 > > Attachments: AMBARI-21764.patch > > > The Stacks & Versions UI page needs to have some of its visuals updated to > make the experience of using this page better. See screenshots ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
