[jira] [Commented] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312641#comment-16312641
 ] 

Hadoop QA commented on AMBARI-22419:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12904731/AMBARI-22419.trunk.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12924//console

This message is automatically generated.

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
> Attachments: AMBARI-22419.branch-2.6.patch, AMBARI-22419.trunk.patch
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22656) Add slot info for Hosts page

2018-01-04 Thread mathildaMa09 (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mathildaMa09 updated AMBARI-22656:
--
Attachment: AMBARI-22656_branch-2.5.patch

> Add slot info for Hosts page
> 
>
> Key: AMBARI-22656
> URL: https://issues.apache.org/jira/browse/AMBARI-22656
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-agent, ambari-server, ambari-web
>Affects Versions: 2.5.2
>Reporter: mathildaMa09
> Fix For: 2.5.2
>
> Attachments: AMBARI-22656.patch, AMBARI-22656_branch-2.5.patch
>
>
> In Hosts Page lists info about hosts like Rack、IP、Cores、DiskUsage .etc, but I 
> need more details about my hosts’ physical position, like slot for instance, 
> to know exactly where my machine locates. So, in hosts page, we can add slot 
> info to describe their location. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Dmytro Grinenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Grinenko updated AMBARI-22419:
-
Attachment: AMBARI-22419.trunk.patch

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
> Attachments: AMBARI-22419.branch-2.6.patch, AMBARI-22419.trunk.patch
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22656) Add slot info for Hosts page

2018-01-04 Thread mathildaMa09 (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mathildaMa09 updated AMBARI-22656:
--
Attachment: AMBARI-22656.patch

> Add slot info for Hosts page
> 
>
> Key: AMBARI-22656
> URL: https://issues.apache.org/jira/browse/AMBARI-22656
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-agent, ambari-server, ambari-web
>Affects Versions: 2.5.2
>Reporter: mathildaMa09
> Fix For: 2.5.2
>
> Attachments: AMBARI-22656.patch
>
>
> In Hosts Page lists info about hosts like Rack、IP、Cores、DiskUsage .etc, but I 
> need more details about my hosts’ physical position, like slot for instance, 
> to know exactly where my machine locates. So, in hosts page, we can add slot 
> info to describe their location. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22656) Add slot info for Hosts page

2018-01-04 Thread mathildaMa09 (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mathildaMa09 updated AMBARI-22656:
--
Attachment: (was: AMBARI-22656.patch)

> Add slot info for Hosts page
> 
>
> Key: AMBARI-22656
> URL: https://issues.apache.org/jira/browse/AMBARI-22656
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-agent, ambari-server, ambari-web
>Affects Versions: 2.5.2
>Reporter: mathildaMa09
> Fix For: 2.5.2
>
> Attachments: AMBARI-22656.patch
>
>
> In Hosts Page lists info about hosts like Rack、IP、Cores、DiskUsage .etc, but I 
> need more details about my hosts’ physical position, like slot for instance, 
> to know exactly where my machine locates. So, in hosts page, we can add slot 
> info to describe their location. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22730) "Enable Launching Docker Containers" should set module.enabled=true

2018-01-04 Thread Yesha Vora (JIRA)
Yesha Vora created AMBARI-22730:
---

 Summary: "Enable Launching Docker Containers" should set 
module.enabled=true
 Key: AMBARI-22730
 URL: https://issues.apache.org/jira/browse/AMBARI-22730
 Project: Ambari
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Yesha Vora


"Enable Launching Docker Containers" is setting module.enabled=True.
It should set module.enabled=true.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22729) "Enable Launching Privileged Containers" should set docker.privileged-containers.enabled to 1

2018-01-04 Thread Yesha Vora (JIRA)
Yesha Vora created AMBARI-22729:
---

 Summary: "Enable Launching Privileged Containers" should set 
docker.privileged-containers.enabled to 1
 Key: AMBARI-22729
 URL: https://issues.apache.org/jira/browse/AMBARI-22729
 Project: Ambari
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Yesha Vora


"Enable Launching Privileged Containers" is setting 
docker.privileged-containers.enabled to True. Instead it should be set to 1.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22728) Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server ports when using HTTP transport.

2018-01-04 Thread Mingjie Tang (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312286#comment-16312286
 ] 

Mingjie Tang commented on AMBARI-22728:
---

The PR is uploaded. 

> Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server 
> ports when using HTTP transport.
> 
>
> Key: AMBARI-22728
> URL: https://issues.apache.org/jira/browse/AMBARI-22728
> Project: Ambari
>  Issue Type: Bug
>Reporter: Mingjie Tang
>Assignee: Mingjie Tang
> Attachments: AMBARI-22728.diff
>
>
> When we are using HTTP transport for the Spark Thrift Server and Spark2 
> Thrift Servers.
> Looking at the 
> /var/lib/ambari-server/resources/common-services/SPARK2/2.0.0/package/scripts/alerts/alert_spark2_thrift_port.py
>  
> and 
> /var/lib/ambari-server/resources/common-services/SPARK/1.2.1/package/scripts/alerts/alert_spark_thrift_port.py
>  
> scripts, they ignore what the cluster administrator has configured 
> hive.server2.thrift.port when http transport is set, and completely ignores 
> what is set in 
> hive.server2.thrift.http.port.
> This causes a false alarm unless you change your http port to what the 
> scripts default to 
> (10002 in Spark2 and 10001 in Spark).
> This is not an ideal solution, especially in cases where the Spark Thrift 
> Server is going to be co-located on the same host as a HiveServer2 because 
> it's likely to result in a port conflict.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22728) Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server ports when using HTTP transport.

2018-01-04 Thread Mingjie Tang (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Tang updated AMBARI-22728:
--
Attachment: AMBARI-22728.diff

> Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server 
> ports when using HTTP transport.
> 
>
> Key: AMBARI-22728
> URL: https://issues.apache.org/jira/browse/AMBARI-22728
> Project: Ambari
>  Issue Type: Bug
>Reporter: Mingjie Tang
>Assignee: Mingjie Tang
> Attachments: AMBARI-22728.diff
>
>
> When we are using HTTP transport for the Spark Thrift Server and Spark2 
> Thrift Servers.
> Looking at the 
> /var/lib/ambari-server/resources/common-services/SPARK2/2.0.0/package/scripts/alerts/alert_spark2_thrift_port.py
>  
> and 
> /var/lib/ambari-server/resources/common-services/SPARK/1.2.1/package/scripts/alerts/alert_spark_thrift_port.py
>  
> scripts, they ignore what the cluster administrator has configured 
> hive.server2.thrift.port when http transport is set, and completely ignores 
> what is set in 
> hive.server2.thrift.http.port.
> This causes a false alarm unless you change your http port to what the 
> scripts default to 
> (10002 in Spark2 and 10001 in Spark).
> This is not an ideal solution, especially in cases where the Spark Thrift 
> Server is going to be co-located on the same host as a HiveServer2 because 
> it's likely to result in a port conflict.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (AMBARI-22728) Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server ports when using HTTP transport.

2018-01-04 Thread Mingjie Tang (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Tang reassigned AMBARI-22728:
-

Assignee: Mingjie Tang

> Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server 
> ports when using HTTP transport.
> 
>
> Key: AMBARI-22728
> URL: https://issues.apache.org/jira/browse/AMBARI-22728
> Project: Ambari
>  Issue Type: Bug
>Reporter: Mingjie Tang
>Assignee: Mingjie Tang
>
> When we are using HTTP transport for the Spark Thrift Server and Spark2 
> Thrift Servers.
> Looking at the 
> /var/lib/ambari-server/resources/common-services/SPARK2/2.0.0/package/scripts/alerts/alert_spark2_thrift_port.py
>  
> and 
> /var/lib/ambari-server/resources/common-services/SPARK/1.2.1/package/scripts/alerts/alert_spark_thrift_port.py
>  
> scripts, they ignore what the cluster administrator has configured 
> hive.server2.thrift.port when http transport is set, and completely ignores 
> what is set in 
> hive.server2.thrift.http.port.
> This causes a false alarm unless you change your http port to what the 
> scripts default to 
> (10002 in Spark2 and 10001 in Spark).
> This is not an ideal solution, especially in cases where the Spark Thrift 
> Server is going to be co-located on the same host as a HiveServer2 because 
> it's likely to result in a port conflict.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22728) Ambari is monitoring the wrong Spark Thrift Server and Spark2 Thrift Server ports when using HTTP transport.

2018-01-04 Thread Mingjie Tang (JIRA)
Mingjie Tang created AMBARI-22728:
-

 Summary: Ambari is monitoring the wrong Spark Thrift Server and 
Spark2 Thrift Server ports when using HTTP transport.
 Key: AMBARI-22728
 URL: https://issues.apache.org/jira/browse/AMBARI-22728
 Project: Ambari
  Issue Type: Bug
Reporter: Mingjie Tang


When we are using HTTP transport for the Spark Thrift Server and Spark2 Thrift 
Servers.
Looking at the 
/var/lib/ambari-server/resources/common-services/SPARK2/2.0.0/package/scripts/alerts/alert_spark2_thrift_port.py
 
and 
/var/lib/ambari-server/resources/common-services/SPARK/1.2.1/package/scripts/alerts/alert_spark_thrift_port.py
 
scripts, they ignore what the cluster administrator has configured 
hive.server2.thrift.port when http transport is set, and completely ignores 
what is set in 
hive.server2.thrift.http.port.
This causes a false alarm unless you change your http port to what the scripts 
default to 
(10002 in Spark2 and 10001 in Spark).
This is not an ideal solution, especially in cases where the Spark Thrift 
Server is going to be co-located on the same host as a HiveServer2 because it's 
likely to result in a port conflict.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22727) "Enable Launching Docker Containers" is not setting module.enabled=True on all nodes

2018-01-04 Thread Yesha Vora (JIRA)
Yesha Vora created AMBARI-22727:
---

 Summary: "Enable Launching Docker Containers" is not setting 
module.enabled=True on all nodes
 Key: AMBARI-22727
 URL: https://issues.apache.org/jira/browse/AMBARI-22727
 Project: Ambari
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Yesha Vora
Priority: Critical


Steps to repro:

* create a cluster with multiple nodes
* select "Enable Launching Docker Containers" from Ambari UI
This action set module.enabled=True only on ambari server host. It does not set 
module.enabled=True on any other node.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22725) Update Hadoop RPC Encryption Properties During Kerberization and Upgrade

2018-01-04 Thread Jonathan Hurley (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hurley updated AMBARI-22725:
-
Attachment: AMBARI-22725.patch

> Update Hadoop RPC Encryption Properties During Kerberization and Upgrade
> 
>
> Key: AMBARI-22725
> URL: https://issues.apache.org/jira/browse/AMBARI-22725
> Project: Ambari
>  Issue Type: Task
>Affects Versions: 2.6.2
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.6.2
>
> Attachments: AMBARI-22725.patch
>
>
> Clients should have the ability to choose encrypted communication over RPC 
> when talking to core hadoop components. Today, the properties that control 
> this are:
> - {{core-site.xml : hadoop.rpc.protection = authentication}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication}}
> The new value of {{privacy}} enables clients to choose an encrypted means of 
> communication. By keeping {{authentication}} first, it will be taken as the 
> default mechanism so that wire encryption is not automatically enabled by 
> accident.
> The following properties should be changed to add {{privacy}}:
> - {{core-site.xml : hadoop.rpc.protection = authentication,privacy}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication,privacy}}
> The following are cases when this needs to be performed:
> - During Kerberization, the above two properties should be automatically 
> reconfigured.
> - During a stack upgrade to any version of HDP 2.6, they should be 
> automatically merged
> Blueprint deployment is not a scenario being covered here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22725) Update Hadoop RPC Encryption Properties During Kerberization and Upgrade

2018-01-04 Thread Jonathan Hurley (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hurley updated AMBARI-22725:
-
Status: Patch Available  (was: Open)

> Update Hadoop RPC Encryption Properties During Kerberization and Upgrade
> 
>
> Key: AMBARI-22725
> URL: https://issues.apache.org/jira/browse/AMBARI-22725
> Project: Ambari
>  Issue Type: Task
>Affects Versions: 2.6.2
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.6.2
>
> Attachments: AMBARI-22725.patch
>
>
> Clients should have the ability to choose encrypted communication over RPC 
> when talking to core hadoop components. Today, the properties that control 
> this are:
> - {{core-site.xml : hadoop.rpc.protection = authentication}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication}}
> The new value of {{privacy}} enables clients to choose an encrypted means of 
> communication. By keeping {{authentication}} first, it will be taken as the 
> default mechanism so that wire encryption is not automatically enabled by 
> accident.
> The following properties should be changed to add {{privacy}}:
> - {{core-site.xml : hadoop.rpc.protection = authentication,privacy}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication,privacy}}
> The following are cases when this needs to be performed:
> - During Kerberization, the above two properties should be automatically 
> reconfigured.
> - During a stack upgrade to any version of HDP 2.6, they should be 
> automatically merged
> Blueprint deployment is not a scenario being covered here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (AMBARI-22725) Update Hadoop RPC Encryption Properties During Kerberization and Upgrade

2018-01-04 Thread Jonathan Hurley (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hurley reassigned AMBARI-22725:


Assignee: Jonathan Hurley

> Update Hadoop RPC Encryption Properties During Kerberization and Upgrade
> 
>
> Key: AMBARI-22725
> URL: https://issues.apache.org/jira/browse/AMBARI-22725
> Project: Ambari
>  Issue Type: Task
>Affects Versions: 2.6.2
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.6.2
>
>
> Clients should have the ability to choose encrypted communication over RPC 
> when talking to core hadoop components. Today, the properties that control 
> this are:
> - {{core-site.xml : hadoop.rpc.protection = authentication}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication}}
> The new value of {{privacy}} enables clients to choose an encrypted means of 
> communication. By keeping {{authentication}} first, it will be taken as the 
> default mechanism so that wire encryption is not automatically enabled by 
> accident.
> The following properties should be changed to add {{privacy}}:
> - {{core-site.xml : hadoop.rpc.protection = authentication,privacy}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication,privacy}}
> The following are cases when this needs to be performed:
> - During Kerberization, the above two properties should be automatically 
> reconfigured.
> - During a stack upgrade to any version of HDP 2.6, they should be 
> automatically merged
> Blueprint deployment is not a scenario being covered here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22726) Test failing with no changes - BlueprintImplTest.testValidateConfigurations__hostGroupForNameNodeHAPositive

2018-01-04 Thread Wing Lam (JIRA)
Wing Lam created AMBARI-22726:
-

 Summary: Test failing with no changes  - 
BlueprintImplTest.testValidateConfigurations__hostGroupForNameNodeHAPositive
 Key: AMBARI-22726
 URL: https://issues.apache.org/jira/browse/AMBARI-22726
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: trunk
 Environment: Ubuntu, 16.04 LTS
Reporter: Wing Lam
Priority: Minor
 Fix For: trunk


When I run just the test 
testValidateConfigurations__hostGroupForNameNodeHAPositive, the test fails.
It appears the problem is that the configuration for the BlueprintValidatorImpl 
is not set up in testValidateConfigurations__hostGroupForNameNodeHAPositive. It 
appears that other tests setup some configuration for 
testValidateConfigurations__hostGroupForNameNodeHAPositive.

Changes that fix this can be seen here:
https://github.com/ReedOei/ambari/commit/09b7dc3f5660325ee0c8010ff4e1d46ca027289c



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22725) Update Hadoop RPC Encryption Properties During Kerberization and Upgrade

2018-01-04 Thread Jonathan Hurley (JIRA)
Jonathan Hurley created AMBARI-22725:


 Summary: Update Hadoop RPC Encryption Properties During 
Kerberization and Upgrade
 Key: AMBARI-22725
 URL: https://issues.apache.org/jira/browse/AMBARI-22725
 Project: Ambari
  Issue Type: Task
Affects Versions: 2.6.2
Reporter: Jonathan Hurley
Priority: Critical
 Fix For: 2.6.2


Clients should have the ability to choose encrypted communication over RPC when 
talking to core hadoop components. Today, the properties that control this are:

- {{core-site.xml : hadoop.rpc.protection = authentication}}
- {{hdfs-site.xml : dfs.data.transfer.protection = authentication}}

The new value of {{privacy}} enables clients to choose an encrypted means of 
communication. By keeping {{authentication}} first, it will be taken as the 
default mechanism so that wire encryption is not automatically enabled by 
accident.

The following properties should be changed to add {{privacy}}:

- {{core-site.xml : hadoop.rpc.protection = authentication,privacy}}
- {{hdfs-site.xml : dfs.data.transfer.protection = authentication,privacy}}

The following are cases when this needs to be performed:
- During Kerberization, the above two properties should be automatically 
reconfigured.
- During a stack upgrade to any version of HDP 2.6, they should be 
automatically merged

Blueprint deployment is not a scenario being covered here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311741#comment-16311741
 ] 

Hadoop QA commented on AMBARI-22699:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12904615/AMBARI-22699.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12923//console

This message is automatically generated.

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311739#comment-16311739
 ] 

Hadoop QA commented on AMBARI-22723:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12904605/AMBARI-22723.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12922//console

This message is automatically generated.

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311727#comment-16311727
 ] 

Hudson commented on AMBARI-22629:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #564 (See 
[https://builds.apache.org/job/Ambari-branch-2.6/564/])
AMBARI-22629 Disabling Kerberos after enabled during Blueprint install 
(echekanskiy: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=a020986728875896df7acd6c2811760ba739d38e])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java


> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error
> -
>
> Key: AMBARI-22629
> URL: https://issues.apache.org/jira/browse/AMBARI-22629
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.0
>Reporter: Robert Levas
>Assignee: Eugene Chekanskiy
>  Labels: blueprints, kerberos
> Fix For: 2.6.2
>
> Attachments: AMBARI-22629.branch-2.6-1.patch, 
> AMBARI-22629.branch-2.6.patch, blueprint_single_node_zk.json, 
> cluster_template_single_node_zk.json, screenshot-error-dialog.png
>
>
> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error:
> {noformat}
> The data directory has not been set.  Generated data can not be stored.
> {noformat}
> !screenshot-error-dialog.png!
> This is caused by an invalid security state set for the installed components 
> since the appropriate state is not set while enabling Kerberos during the 
> installation process:
> {noformat}
> ambari=> select * from hostcomponentstate;
>  id | cluster_id |  component_name  |   version| current_state | host_id 
> | service_name | upgrade_state | security_state
> ++--+--+---+-+--+---+
>   1 |  2 | KERBEROS_CLIENT  | UNKNOWN  | INSTALLED |   1 
> | KERBEROS | NONE  | UNSECURED
>   2 |  2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED |   1 
> | ZOOKEEPER| NONE  | UNSECURED
>   3 |  2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED   |   1 
> | ZOOKEEPER| NONE  | UNSECURED
> {noformat}
> The expected state for each component is {{SECURED}}, not {{UNSECURED}}. 
> Because of this, Ambari _thinks_ there is no work to be done, causing this 
> issue. 
> *Steps to reproduce*:
> # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs 
> are installed on the Ambari server host with the krb5.conf setup properly
> # Install Blueprint - [^blueprint_single_node_zk.json]
> # Create clister - [^cluster_template_single_node_zk.json]
> # When cluster is created, Kerberos should be enabled and all services up
> # Disable Kerberos - error occurs during Unkerberize Cluster task.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311724#comment-16311724
 ] 

Hudson commented on AMBARI-22723:
-

SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8574 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/8574/])
AMBARI-22723. Ambari Upgrade failed in case if database have no (dlysnichenko: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=dfb97855196ff3e40ae4b6742eabe31082560fe6])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java


> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22678) Fix Broken Symlinks on Stack Distribution

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311725#comment-16311725
 ] 

Hudson commented on AMBARI-22678:
-

SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8574 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/8574/])
AMBARI-22678. Fix Broken Symlinks on Stack Distribution (dlysnichenko) 
(dlysnichenko: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=72812bd2bdd496056407d930c4c7e232bd487703])
* (edit) 
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
* (edit) 
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
* (edit) 
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_service.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json


> Fix Broken Symlinks on Stack Distribution
> -
>
> Key: AMBARI-22678
> URL: https://issues.apache.org/jira/browse/AMBARI-22678
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22678.patch
>
>
> There are two scenarios to cover here:
> # Ambari never conf-select'd a component (maybe because of a bug or because 
> the component didn't support it)
> # The conf pointers of a component are broken
> In either event, when distributing a new stack, the code detects this problem 
> (as it would on a first-time install) and tries to fix it:
> {code}
> /etc/component/conf (directory)
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> {code}
> The stack distribution thinks this is a first-time installed and tries to fix 
> the symlinks. We end up with:
> {code}
> /etc/component/conf -> /usr/hdp/current/component
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> /usr/hdp/v2/component -> /etc/component/v2/0
> {code}
> Because we're only conf-selecting v2, v1 never gets corrected since it's 
> already installed. Thus, we have a circular symlink.
> Most likely the proper fix will be:
> - Iterate over the entire known conf-select structure
> - Check to see the state /etc/component/conf - if it's bad, fix it to defaults
> Chances are we can do this directly in 
> {{conf_select.convert_conf_directories_to_symlinks}}:
> {code}
> stack_name = Script.get_stack_name()
> for directory_struct in dirs:
> if not os.path.exists(directory_struct['conf_dir']):
> Logger.info("Skipping the conf-select tool on {0} since {1} does not 
> exist.".format(
> package, directory_struct['conf_dir']))
> return
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error

2018-01-04 Thread Eugene Chekanskiy (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Chekanskiy updated AMBARI-22629:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

b5da60b5ac..a020986728  branch-2.6 -> branch-2.6


> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error
> -
>
> Key: AMBARI-22629
> URL: https://issues.apache.org/jira/browse/AMBARI-22629
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.0
>Reporter: Robert Levas
>Assignee: Eugene Chekanskiy
>  Labels: blueprints, kerberos
> Fix For: 2.6.2
>
> Attachments: AMBARI-22629.branch-2.6-1.patch, 
> AMBARI-22629.branch-2.6.patch, blueprint_single_node_zk.json, 
> cluster_template_single_node_zk.json, screenshot-error-dialog.png
>
>
> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error:
> {noformat}
> The data directory has not been set.  Generated data can not be stored.
> {noformat}
> !screenshot-error-dialog.png!
> This is caused by an invalid security state set for the installed components 
> since the appropriate state is not set while enabling Kerberos during the 
> installation process:
> {noformat}
> ambari=> select * from hostcomponentstate;
>  id | cluster_id |  component_name  |   version| current_state | host_id 
> | service_name | upgrade_state | security_state
> ++--+--+---+-+--+---+
>   1 |  2 | KERBEROS_CLIENT  | UNKNOWN  | INSTALLED |   1 
> | KERBEROS | NONE  | UNSECURED
>   2 |  2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED |   1 
> | ZOOKEEPER| NONE  | UNSECURED
>   3 |  2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED   |   1 
> | ZOOKEEPER| NONE  | UNSECURED
> {noformat}
> The expected state for each component is {{SECURED}}, not {{UNSECURED}}. 
> Because of this, Ambari _thinks_ there is no work to be done, causing this 
> issue. 
> *Steps to reproduce*:
> # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs 
> are installed on the Ambari server host with the krb5.conf setup properly
> # Install Blueprint - [^blueprint_single_node_zk.json]
> # Create clister - [^cluster_template_single_node_zk.json]
> # When cluster is created, Kerberos should be enabled and all services up
> # Disable Kerberos - error occurs during Unkerberize Cluster task.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error

2018-01-04 Thread Eugene Chekanskiy (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Chekanskiy updated AMBARI-22629:
---
Attachment: AMBARI-22629.branch-2.6-1.patch

> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error
> -
>
> Key: AMBARI-22629
> URL: https://issues.apache.org/jira/browse/AMBARI-22629
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.0
>Reporter: Robert Levas
>Assignee: Eugene Chekanskiy
>  Labels: blueprints, kerberos
> Fix For: 2.6.2
>
> Attachments: AMBARI-22629.branch-2.6-1.patch, 
> AMBARI-22629.branch-2.6.patch, blueprint_single_node_zk.json, 
> cluster_template_single_node_zk.json, screenshot-error-dialog.png
>
>
> Disabling Kerberos after enabled during Blueprint install fails with missing 
> data directory error:
> {noformat}
> The data directory has not been set.  Generated data can not be stored.
> {noformat}
> !screenshot-error-dialog.png!
> This is caused by an invalid security state set for the installed components 
> since the appropriate state is not set while enabling Kerberos during the 
> installation process:
> {noformat}
> ambari=> select * from hostcomponentstate;
>  id | cluster_id |  component_name  |   version| current_state | host_id 
> | service_name | upgrade_state | security_state
> ++--+--+---+-+--+---+
>   1 |  2 | KERBEROS_CLIENT  | UNKNOWN  | INSTALLED |   1 
> | KERBEROS | NONE  | UNSECURED
>   2 |  2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED |   1 
> | ZOOKEEPER| NONE  | UNSECURED
>   3 |  2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED   |   1 
> | ZOOKEEPER| NONE  | UNSECURED
> {noformat}
> The expected state for each component is {{SECURED}}, not {{UNSECURED}}. 
> Because of this, Ambari _thinks_ there is no work to be done, causing this 
> issue. 
> *Steps to reproduce*:
> # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs 
> are installed on the Ambari server host with the krb5.conf setup properly
> # Install Blueprint - [^blueprint_single_node_zk.json]
> # Create clister - [^cluster_template_single_node_zk.json]
> # When cluster is created, Kerberos should be enabled and all services up
> # Disable Kerberos - error occurs during Unkerberize Cluster task.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22724) Idempotent issue on Ambari Upgrade, renameServiceDeletedColumn failed with column already exists exception

2018-01-04 Thread Vitaly Brodetskyi (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-22724:
---
Attachment: AMBARI-22724.patch

> Idempotent issue on Ambari Upgrade, renameServiceDeletedColumn failed with 
> column already exists exception
> --
>
> Key: AMBARI-22724
> URL: https://issues.apache.org/jira/browse/AMBARI-22724
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22724.patch
>
>
> Issue discovered alongside with another bugs which leads upgrade to fail in 
> the middle with partially applying changes to DB. It leads to exception in 
> code: 
> {code}
>   private void renameServiceDeletedColumn() throws AmbariException, 
> SQLException {
> if (dbAccessor.tableHasColumn(CLUSTER_CONFIG_TABLE, 
> SERVICE_DELETED_COLUMN)) {
>   dbAccessor.renameColumn(CLUSTER_CONFIG_TABLE, SERVICE_DELETED_COLUMN, 
> new DBAccessor.DBColumnInfo(UNMAPPED_COLUMN, Short.class, null, 0, false));
> }
>   }
> {code}
> Exception:  ERROR: column "unmapped" of relation "clusterconfig" already 
> exists   (err.png)
> Table generated DDL from current state: 
> {code}
> -- auto-generated definition
> CREATE TABLE clusterconfig
> (
>   config_id  INT8(19)   NOT NULL
> CONSTRAINT pk_clusterconfig
> PRIMARY KEY,
>   version_tagVARCHAR(255)   NOT NULL,
>   versionINT8(19)   NOT NULL,
>   type_name  VARCHAR(255)   NOT NULL,
>   cluster_id INT8(19)   NOT NULL
> CONSTRAINT fk_clusterconfig_cluster_id
> REFERENCES clusters,
>   stack_id   INT8(19)   NOT NULL
> CONSTRAINT fk_clusterconfig_stack_id
> REFERENCES stack,
>   config_dataTEXT(max)  NOT NULL,
>   config_attributes  TEXT(max),
>   create_timestamp   INT8(19)   NOT NULL,
>   unmapped   INT2(5) DEFAULT 0  NOT NULL,
>   selected   INT2(5) DEFAULT 0  NOT NULL,
>   selected_timestamp INT8(19) DEFAULT 0 NOT NULL,
>   service_deletedINT2(5) DEFAULT 0  NOT NULL
> );
> CREATE UNIQUE INDEX uq_config_type_tag
>   ON clusterconfig (cluster_id, type_name, version_tag);
> CREATE UNIQUE INDEX uq_config_type_version
>   ON clusterconfig (cluster_id, type_name, version);
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22724) Idempotent issue on Ambari Upgrade, renameServiceDeletedColumn failed with column already exists exception

2018-01-04 Thread Vitaly Brodetskyi (JIRA)
Vitaly Brodetskyi created AMBARI-22724:
--

 Summary: Idempotent issue on Ambari Upgrade, 
renameServiceDeletedColumn failed with column already exists exception
 Key: AMBARI-22724
 URL: https://issues.apache.org/jira/browse/AMBARI-22724
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.6.2
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Blocker
 Fix For: 2.6.2


Issue discovered alongside with another bugs which leads upgrade to fail in the 
middle with partially applying changes to DB. It leads to exception in code: 

{code}
  private void renameServiceDeletedColumn() throws AmbariException, 
SQLException {
if (dbAccessor.tableHasColumn(CLUSTER_CONFIG_TABLE, 
SERVICE_DELETED_COLUMN)) {
  dbAccessor.renameColumn(CLUSTER_CONFIG_TABLE, SERVICE_DELETED_COLUMN, new 
DBAccessor.DBColumnInfo(UNMAPPED_COLUMN, Short.class, null, 0, false));
}
  }
{code}


Exception:  ERROR: column "unmapped" of relation "clusterconfig" already exists 
  (err.png)

Table generated DDL from current state: 
{code}
-- auto-generated definition
CREATE TABLE clusterconfig
(
  config_id  INT8(19)   NOT NULL
CONSTRAINT pk_clusterconfig
PRIMARY KEY,
  version_tagVARCHAR(255)   NOT NULL,
  versionINT8(19)   NOT NULL,
  type_name  VARCHAR(255)   NOT NULL,
  cluster_id INT8(19)   NOT NULL
CONSTRAINT fk_clusterconfig_cluster_id
REFERENCES clusters,
  stack_id   INT8(19)   NOT NULL
CONSTRAINT fk_clusterconfig_stack_id
REFERENCES stack,
  config_dataTEXT(max)  NOT NULL,
  config_attributes  TEXT(max),
  create_timestamp   INT8(19)   NOT NULL,
  unmapped   INT2(5) DEFAULT 0  NOT NULL,
  selected   INT2(5) DEFAULT 0  NOT NULL,
  selected_timestamp INT8(19) DEFAULT 0 NOT NULL,
  service_deletedINT2(5) DEFAULT 0  NOT NULL
);

CREATE UNIQUE INDEX uq_config_type_tag
  ON clusterconfig (cluster_id, type_name, version_tag);

CREATE UNIQUE INDEX uq_config_type_version
  ON clusterconfig (cluster_id, type_name, version);


{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311630#comment-16311630
 ] 

Hudson commented on AMBARI-22699:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8573 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/8573/])
AMBARI-22699. Update FE to initiate regenerate keytab file operations 
(aantonenko: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=c088e59ebcf099b01603d883dce7700d44cabc68])
* (edit) ambari-web/app/messages.js
* (edit) ambari-web/test/utils/date/timezone_test.js
* (edit) ambari-web/app/models/host_component.js
* (edit) ambari-web/app/utils/ajax/ajax.js
* (edit) ambari-web/app/views/main/host/details.js
* (edit) ambari-web/app/views/main/service/item.js
* (edit) ambari-web/app/controllers/main/host/details.js
* (edit) ambari-web/app/controllers/main/service/item.js


> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (AMBARI-22678) Fix Broken Symlinks on Stack Distribution

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311511#comment-16311511
 ] 

Dmitry Lysnichenko edited comment on AMBARI-22678 at 1/4/18 4:43 PM:
-

Committed
   6e58eccdfc..41c362eb97  branch-2.6 -> branch-2.6
   dfb9785519..72812bd2bd  trunk -> trunk


was (Author: dmitriusan):
Committed
   6e58eccdfc..41c362eb97  branch-2.6 -> branch-2.6


> Fix Broken Symlinks on Stack Distribution
> -
>
> Key: AMBARI-22678
> URL: https://issues.apache.org/jira/browse/AMBARI-22678
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22678.patch
>
>
> There are two scenarios to cover here:
> # Ambari never conf-select'd a component (maybe because of a bug or because 
> the component didn't support it)
> # The conf pointers of a component are broken
> In either event, when distributing a new stack, the code detects this problem 
> (as it would on a first-time install) and tries to fix it:
> {code}
> /etc/component/conf (directory)
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> {code}
> The stack distribution thinks this is a first-time installed and tries to fix 
> the symlinks. We end up with:
> {code}
> /etc/component/conf -> /usr/hdp/current/component
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> /usr/hdp/v2/component -> /etc/component/v2/0
> {code}
> Because we're only conf-selecting v2, v1 never gets corrected since it's 
> already installed. Thus, we have a circular symlink.
> Most likely the proper fix will be:
> - Iterate over the entire known conf-select structure
> - Check to see the state /etc/component/conf - if it's bad, fix it to defaults
> Chances are we can do this directly in 
> {{conf_select.convert_conf_directories_to_symlinks}}:
> {code}
> stack_name = Script.get_stack_name()
> for directory_struct in dirs:
> if not os.path.exists(directory_struct['conf_dir']):
> Logger.info("Skipping the conf-select tool on {0} since {1} does not 
> exist.".format(
> package, directory_struct['conf_dir']))
> return
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311611#comment-16311611
 ] 

Hudson commented on AMBARI-22723:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #563 (See 
[https://builds.apache.org/job/Ambari-branch-2.6/563/])
AMBARI-22723. Ambari Upgrade failed in case if database have no (dlysnichenko: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=b5da60b5ac51637e0448c3df0662a58f477cefbf])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java


> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311569#comment-16311569
 ] 

Dmitry Lysnichenko edited comment on AMBARI-22723 at 1/4/18 4:37 PM:
-

Committed 
To https://git-wip-us.apache.org/repos/asf/ambari.git
   41c362eb97..b5da60b5ac  branch-2.6 -> branch-2.6
   c088e59ebc..dfb9785519  trunk -> trunk




was (Author: dmitriusan):
Committed 
To https://git-wip-us.apache.org/repos/asf/ambari.git
   41c362eb97..b5da60b5ac  branch-2.6 -> branch-2.6


> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22403) Read the JAVA_HOME depending on the OS family during Service install (action & command) and upgrade

2018-01-04 Thread Yussuf Shaikh (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yussuf Shaikh updated AMBARI-22403:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

changes has been merged into branch-feature-AMBARI-21674

> Read the JAVA_HOME depending on the OS family during Service install (action 
> & command) and upgrade
> ---
>
> Key: AMBARI-22403
> URL: https://issues.apache.org/jira/browse/AMBARI-22403
> Project: Ambari
>  Issue Type: Sub-task
>  Components: ambari-server
>Reporter: Yussuf Shaikh
>Assignee: Yussuf Shaikh
> Attachments: AMBARI-22403.001.patch
>
>
> Use of proper java_home value during service install and configuration base 
> on host OS type.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311569#comment-16311569
 ] 

Dmitry Lysnichenko commented on AMBARI-22723:
-

Committed 
To https://git-wip-us.apache.org/repos/asf/ambari.git
   41c362eb97..b5da60b5ac  branch-2.6 -> branch-2.6


> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Aleksandr Kovalenko (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311559#comment-16311559
 ] 

Aleksandr Kovalenko commented on AMBARI-22699:
--

+1 for the patch

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22678) Fix Broken Symlinks on Stack Distribution

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311556#comment-16311556
 ] 

Hudson commented on AMBARI-22678:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #562 (See 
[https://builds.apache.org/job/Ambari-branch-2.6/562/])
AMBARI-22678. Fix Broken Symlinks on Stack Distribution (dlysnichenko) 
(dlysnichenko: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=41c362eb97b2ed396b3f413581b9c9344f78d649])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
* (edit) 
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_service.py
* (edit) 
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
* (edit) 
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py


> Fix Broken Symlinks on Stack Distribution
> -
>
> Key: AMBARI-22678
> URL: https://issues.apache.org/jira/browse/AMBARI-22678
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22678.patch
>
>
> There are two scenarios to cover here:
> # Ambari never conf-select'd a component (maybe because of a bug or because 
> the component didn't support it)
> # The conf pointers of a component are broken
> In either event, when distributing a new stack, the code detects this problem 
> (as it would on a first-time install) and tries to fix it:
> {code}
> /etc/component/conf (directory)
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> {code}
> The stack distribution thinks this is a first-time installed and tries to fix 
> the symlinks. We end up with:
> {code}
> /etc/component/conf -> /usr/hdp/current/component
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> /usr/hdp/v2/component -> /etc/component/v2/0
> {code}
> Because we're only conf-selecting v2, v1 never gets corrected since it's 
> already installed. Thus, we have a circular symlink.
> Most likely the proper fix will be:
> - Iterate over the entire known conf-select structure
> - Check to see the state /etc/component/conf - if it's bad, fix it to defaults
> Chances are we can do this directly in 
> {{conf_select.convert_conf_directories_to_symlinks}}:
> {code}
> stack_name = Script.get_stack_name()
> for directory_struct in dirs:
> if not os.path.exists(directory_struct['conf_dir']):
> Logger.info("Skipping the conf-select tool on {0} since {1} does not 
> exist.".format(
> package, directory_struct['conf_dir']))
> return
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311537#comment-16311537
 ] 

Hadoop QA commented on AMBARI-22699:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12904615/AMBARI-22699.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-web.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12921//console

This message is automatically generated.

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22678) Fix Broken Symlinks on Stack Distribution

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311511#comment-16311511
 ] 

Dmitry Lysnichenko commented on AMBARI-22678:
-

Committed
   6e58eccdfc..41c362eb97  branch-2.6 -> branch-2.6


> Fix Broken Symlinks on Stack Distribution
> -
>
> Key: AMBARI-22678
> URL: https://issues.apache.org/jira/browse/AMBARI-22678
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22678.patch
>
>
> There are two scenarios to cover here:
> # Ambari never conf-select'd a component (maybe because of a bug or because 
> the component didn't support it)
> # The conf pointers of a component are broken
> In either event, when distributing a new stack, the code detects this problem 
> (as it would on a first-time install) and tries to fix it:
> {code}
> /etc/component/conf (directory)
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> {code}
> The stack distribution thinks this is a first-time installed and tries to fix 
> the symlinks. We end up with:
> {code}
> /etc/component/conf -> /usr/hdp/current/component
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> /usr/hdp/v2/component -> /etc/component/v2/0
> {code}
> Because we're only conf-selecting v2, v1 never gets corrected since it's 
> already installed. Thus, we have a circular symlink.
> Most likely the proper fix will be:
> - Iterate over the entire known conf-select structure
> - Check to see the state /etc/component/conf - if it's bad, fix it to defaults
> Chances are we can do this directly in 
> {{conf_select.convert_conf_directories_to_symlinks}}:
> {code}
> stack_name = Script.get_stack_name()
> for directory_struct in dirs:
> if not os.path.exists(directory_struct['conf_dir']):
> Logger.info("Skipping the conf-select tool on {0} since {1} does not 
> exist.".format(
> package, directory_struct['conf_dir']))
> return
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-21559) Alert definition paths should be updated on upgrade

2018-01-04 Thread Doroszlai, Attila (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-21559:
---
Fix Version/s: 3.0.0

> Alert definition paths should be updated on upgrade
> ---
>
> Key: AMBARI-21559
> URL: https://issues.apache.org/jira/browse/AMBARI-21559
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Critical
> Fix For: 3.0.0, 2.5.2
>
> Attachments: AMBARI-21559.patch
>
>
> Upon upgrading Ambari, alert scripts may be relocated in the new version.  
> Alert definitions in the database should be updated to the new paths to avoid 
> errors like:
> {noformat}
> Unable to execute alert. Unable to find 
> 'AMBARI_METRICS/0.1.0.4.1/package/alerts/alert_ambari_metrics_monitor.py' as 
> an absolute path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'FLUME/1.6.0.4.2/package/alerts/alert_flume_agent_status.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_checkpoint_time.py' as an absolute path 
> or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_datanode_unmounted_data_dir.py' as an 
> absolute path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_ha_namenode_health.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_upgrade_finalized.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'YARN/2.7.2.4.2/package/alerts/alert_nodemanager_health.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'YARN/2.7.2.4.2/package/alerts/alert_nodemanagers_summary.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-21559) Alert definition paths should be updated on upgrade

2018-01-04 Thread Doroszlai, Attila (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311505#comment-16311505
 ] 

Doroszlai, Attila commented on AMBARI-21559:


Committed to 
[trunk|http://git-wip-us.apache.org/repos/asf/ambari/commit/217620b0a2].

> Alert definition paths should be updated on upgrade
> ---
>
> Key: AMBARI-21559
> URL: https://issues.apache.org/jira/browse/AMBARI-21559
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Critical
> Fix For: 3.0.0, 2.5.2
>
> Attachments: AMBARI-21559.patch
>
>
> Upon upgrading Ambari, alert scripts may be relocated in the new version.  
> Alert definitions in the database should be updated to the new paths to avoid 
> errors like:
> {noformat}
> Unable to execute alert. Unable to find 
> 'AMBARI_METRICS/0.1.0.4.1/package/alerts/alert_ambari_metrics_monitor.py' as 
> an absolute path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'FLUME/1.6.0.4.2/package/alerts/alert_flume_agent_status.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_checkpoint_time.py' as an absolute path 
> or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_datanode_unmounted_data_dir.py' as an 
> absolute path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_ha_namenode_health.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'HDFS/2.7.2.4.2/package/alerts/alert_upgrade_finalized.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'YARN/2.7.2.4.2/package/alerts/alert_nodemanager_health.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> Unable to execute alert. Unable to find 
> 'YARN/2.7.2.4.2/package/alerts/alert_nodemanagers_summary.py' as an absolute 
> path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Attachment: (was: AMBARI-22699.patch)

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Attachment: AMBARI-22699.patch

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Status: Open  (was: Patch Available)

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Status: Patch Available  (was: Open)

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311472#comment-16311472
 ] 

Hadoop QA commented on AMBARI-22699:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12904610/AMBARI-22699.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 core tests{color}.  The test build failed in 
[ambari-web|https://builds.apache.org/job/Ambari-trunk-test-patch/12920//artifact/patch-work/testrun_ambari-web.txt]
 

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12920//console

This message is automatically generated.

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Status: Patch Available  (was: Open)

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22699) Update FE to initiate regenerate keytab file operations for a service and a host

2018-01-04 Thread Antonenko Alexander (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antonenko Alexander updated AMBARI-22699:
-
Attachment: AMBARI-22699.patch

> Update FE to initiate regenerate keytab file operations for a service and a 
> host
> 
>
> Key: AMBARI-22699
> URL: https://issues.apache.org/jira/browse/AMBARI-22699
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 3.0.0
>Reporter: Antonenko Alexander
>Assignee: Antonenko Alexander
> Fix For: 3.0.0
>
> Attachments: AMBARI-22699.patch
>
>
> Update FE to initiate regenerate keytab file operations for a service and a 
> host.
> An option needs to be added to the Actions menu of a service to regenerate 
> all keytab file for that service. The Ambari REST API call to initiate this 
> is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_components=SERVICENAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}
> NOTE:  CLUSTERNAME and SERVICENAME need to be replaced with the appropriate 
> values.
> An option needs to be added to the Host Actions menu of a host to regenerate 
> all keytab file for that host. The Ambari REST API call to initiate this is:
> {code}
> PUT 
> /api/v1/clusters/CLUSTERNAME?regenerate_keytabs=all_hosts=HOSTNAME:*
> {
>   "Clusters": {
> "security_type" : "KERBEROS"
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-22723:

Affects Version/s: 2.6.1

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-22723:

Status: Patch Available  (was: Open)

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-22723:

Fix Version/s: 2.6.2

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-22723:

Attachment: AMBARI-22723.patch

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.6.1
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
> Fix For: 2.6.2
>
> Attachments: AMBARI-22723.patch
>
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)
Dmitry Lysnichenko created AMBARI-22723:
---

 Summary: Ambari Upgrade failed in case if database have no 
cluster_version table already. The issue with idempotence
 Key: AMBARI-22723
 URL: https://issues.apache.org/jira/browse/AMBARI-22723
 Project: Ambari
  Issue Type: Bug
Reporter: Dmitry Lysnichenko
Assignee: Dmitry Lysnichenko
Priority: Blocker



AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
upgrade tries, failing to follow the Idempotent upgrade paradigm.

Issue is with:
{code}
protected void executeDDLUpdates() throws AmbariException, SQLException {
Integer currentVersionID = getCurrentVersionID();<--
dropBrokenFK();
updateServiceComponentDesiredStateTable(currentVersionID); 
<--
updateServiceDesiredStateTable(currentVersionID); <--
addSelectedCollumsToClusterconfigTable();
updateHostComponentDesiredStateTable();
updateHostComponentStateTable();
dropStaleTables();  <--
updateUpgradeTable();
createUpgradeHistoryTable();
updateRepositoryVersionTable();
renameServiceDeletedColumn();
addLegacyColumn();
expandUpgradeItemItemTextColumn();
addViewUrlPKConstraint();
removeStaleConstraints();
}
{code}

{{getCurrentVersionID()}} trying to query non-existing table and failing whole 
upgrade





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22723) Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence

2018-01-04 Thread Dmitry Lysnichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-22723:

Component/s: ambari-server

> Ambari Upgrade failed in case if database have no cluster_version table 
> already. The issue with idempotence
> ---
>
> Key: AMBARI-22723
> URL: https://issues.apache.org/jira/browse/AMBARI-22723
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
>Priority: Blocker
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several 
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID); 
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables();  <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
> {{getCurrentVersionID()}} trying to query non-existing table and failing 
> whole upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22656) Add slot info for Hosts page

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311384#comment-16311384
 ] 

Hadoop QA commented on AMBARI-22656:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12904510/AMBARI-22656.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12919//console

This message is automatically generated.

> Add slot info for Hosts page
> 
>
> Key: AMBARI-22656
> URL: https://issues.apache.org/jira/browse/AMBARI-22656
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-agent, ambari-server, ambari-web
>Affects Versions: 2.5.2
>Reporter: mathildaMa09
> Fix For: 2.5.2
>
> Attachments: AMBARI-22656.patch
>
>
> In Hosts Page lists info about hosts like Rack、IP、Cores、DiskUsage .etc, but I 
> need more details about my hosts’ physical position, like slot for instance, 
> to know exactly where my machine locates. So, in hosts page, we can add slot 
> info to describe their location. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311380#comment-16311380
 ] 

Hadoop QA commented on AMBARI-22419:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12904555/AMBARI-22419.branch-2.6.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 javac{color}.  The patch appears to cause the [build to 
fail|https://builds.apache.org/job/Ambari-trunk-test-patch/12918//artifact/patch-work/patchJavacWarnings.txt].

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/12918//console

This message is automatically generated.

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
> Attachments: AMBARI-22419.branch-2.6.patch
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22718) Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with Kerberos

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311290#comment-16311290
 ] 

Hudson commented on AMBARI-22718:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8571 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/8571/])
AMBARI-22718. Cannot set security.inter.broker.protocol: SASL_SSL via 
(adoroszlai: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=13056cf2b0bf4ff20192860e2bfee556c1c57b4a])
* (add) 
ambari-server/src/test/python/stacks/2.6/configs/secure_kafka_sasl_ssl.json
* (edit) 
ambari-server/src/test/python/stacks/2.6/KAFKA/test_kafka_broker_other_sasl.py
* (edit) 
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py


> Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with 
> Kerberos
> ---
>
> Key: AMBARI-22718
> URL: https://issues.apache.org/jira/browse/AMBARI-22718
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22718.branch-2.6.patch, AMBARI-22718.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Tweak {{stack_features.json}} to lower {{min_version}} for 
> {{kafka_extended_sasl_support}}
> # Create secure cluster via blueprint (ZooKeeper + Kafka) with 
> {{security.inter.broker.protocol: SASL_SSL}}
> Result: Kafka Brokers stopped due to missing {{kafka_jaas.conf}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22719) Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311291#comment-16311291
 ] 

Hudson commented on AMBARI-22719:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8571 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/8571/])
AMBARI-22719. Regenerate Keytabs resets security.inter.broker.protocol 
(adoroszlai: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=e2be62d598054c822ecc0c7039ed8a433e1212d9])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py
* (edit) ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py


> Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL
> -
>
> Key: AMBARI-22719
> URL: https://issues.apache.org/jira/browse/AMBARI-22719
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22719.branch-2.6.patch, AMBARI-22719.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Deploy secure cluster (ZooKeeper + Kafka)
> # Set {{security.inter.broker.protocol}} to {{SASL_PLAINTEXT}} (or 
> {{SASL_SSL}})
> # Remove {{security.inter.broker.protocol}} from cluster Kerberos descriptor 
> or update it to {{SASL_PLAINTEXT}} (or {{SASL_SSL}})
>  
> ([steps|https://community.hortonworks.com/articles/89713/updating-the-user-sepecified-kerberos-descriptor.html])
> # Regenerate Keytabs
> Result: {{security.inter.broker.protocol}} is reset to {{PLAINTEXTSASL}}
> This is caused by stack advisor forcing {{PLAINTEXTSASL}} for secure cluster, 
> although {{SASL_PLAINTEXT}} (or {{SASL_SSL}}) is also valid.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22719) Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL

2018-01-04 Thread Doroszlai, Attila (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-22719:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 
[trunk|https://git1-us-west.apache.org/repos/asf?p=ambari.git;a=commit;h=e2be62d598054c822ecc0c7039ed8a433e1212d9]
 and 
[branch-2.6|https://git1-us-west.apache.org/repos/asf?p=ambari.git;a=commit;h=6e58eccdfc71f81c3d60ee66e305626589277e69].

> Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL
> -
>
> Key: AMBARI-22719
> URL: https://issues.apache.org/jira/browse/AMBARI-22719
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22719.branch-2.6.patch, AMBARI-22719.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Deploy secure cluster (ZooKeeper + Kafka)
> # Set {{security.inter.broker.protocol}} to {{SASL_PLAINTEXT}} (or 
> {{SASL_SSL}})
> # Remove {{security.inter.broker.protocol}} from cluster Kerberos descriptor 
> or update it to {{SASL_PLAINTEXT}} (or {{SASL_SSL}})
>  
> ([steps|https://community.hortonworks.com/articles/89713/updating-the-user-sepecified-kerberos-descriptor.html])
> # Regenerate Keytabs
> Result: {{security.inter.broker.protocol}} is reset to {{PLAINTEXTSASL}}
> This is caused by stack advisor forcing {{PLAINTEXTSASL}} for secure cluster, 
> although {{SASL_PLAINTEXT}} (or {{SASL_SSL}}) is also valid.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22718) Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with Kerberos

2018-01-04 Thread Doroszlai, Attila (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-22718:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 
[trunk|https://git1-us-west.apache.org/repos/asf?p=ambari.git;a=commit;h=13056cf2b0bf4ff20192860e2bfee556c1c57b4a]
 and 
[branch-2.6|https://git1-us-west.apache.org/repos/asf?p=ambari.git=commit=8fce2a17417acd92c4f5acdc5b9a90ce37cb2a5d]

> Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with 
> Kerberos
> ---
>
> Key: AMBARI-22718
> URL: https://issues.apache.org/jira/browse/AMBARI-22718
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22718.branch-2.6.patch, AMBARI-22718.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Tweak {{stack_features.json}} to lower {{min_version}} for 
> {{kafka_extended_sasl_support}}
> # Create secure cluster via blueprint (ZooKeeper + Kafka) with 
> {{security.inter.broker.protocol: SASL_SSL}}
> Result: Kafka Brokers stopped due to missing {{kafka_jaas.conf}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22722) Enabling CPU Scheduling in YARN does not show restart option

2018-01-04 Thread Prabhu Joseph (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated AMBARI-22722:
---
Description: 
When enabling CPU Scheduling under YARN -> Config, it prompts only for YARN 
Refresh Queue but not the restart of RM. CPU Scheduling needs RM Restart.



  was:
When enabling CPU Scheduling under YARN -> Config, it prompts for only YARN 
Refresh Queue but not the restart of RM. CPU Scheduling needs RM Restart.




> Enabling CPU Scheduling in YARN does not show restart option
> 
>
> Key: AMBARI-22722
> URL: https://issues.apache.org/jira/browse/AMBARI-22722
> Project: Ambari
>  Issue Type: Bug
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Prabhu Joseph
>Priority: Critical
>
> When enabling CPU Scheduling under YARN -> Config, it prompts only for YARN 
> Refresh Queue but not the restart of RM. CPU Scheduling needs RM Restart.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22719) Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311219#comment-16311219
 ] 

Hudson commented on AMBARI-22719:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #561 (See 
[https://builds.apache.org/job/Ambari-branch-2.6/561/])
AMBARI-22719. Regenerate Keytabs resets security.inter.broker.protocol 
(adoroszlai: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=6e58eccdfc71f81c3d60ee66e305626589277e69])
* (edit) ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py


> Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL
> -
>
> Key: AMBARI-22719
> URL: https://issues.apache.org/jira/browse/AMBARI-22719
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22719.branch-2.6.patch, AMBARI-22719.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Deploy secure cluster (ZooKeeper + Kafka)
> # Set {{security.inter.broker.protocol}} to {{SASL_PLAINTEXT}} (or 
> {{SASL_SSL}})
> # Remove {{security.inter.broker.protocol}} from cluster Kerberos descriptor 
> or update it to {{SASL_PLAINTEXT}} (or {{SASL_SSL}})
>  
> ([steps|https://community.hortonworks.com/articles/89713/updating-the-user-sepecified-kerberos-descriptor.html])
> # Regenerate Keytabs
> Result: {{security.inter.broker.protocol}} is reset to {{PLAINTEXTSASL}}
> This is caused by stack advisor forcing {{PLAINTEXTSASL}} for secure cluster, 
> although {{SASL_PLAINTEXT}} (or {{SASL_SSL}}) is also valid.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (AMBARI-22718) Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with Kerberos

2018-01-04 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-22718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311218#comment-16311218
 ] 

Hudson commented on AMBARI-22718:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #561 (See 
[https://builds.apache.org/job/Ambari-branch-2.6/561/])
AMBARI-22718. Cannot set security.inter.broker.protocol: SASL_SSL via 
(adoroszlai: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=8fce2a17417acd92c4f5acdc5b9a90ce37cb2a5d])
* (add) 
ambari-server/src/test/python/stacks/2.6/configs/secure_kafka_sasl_ssl.json
* (edit) 
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
* (edit) 
ambari-server/src/test/python/stacks/2.6/KAFKA/test_kafka_broker_other_sasl.py


> Cannot set security.inter.broker.protocol: SASL_SSL via Blueprint with 
> Kerberos
> ---
>
> Key: AMBARI-22718
> URL: https://issues.apache.org/jira/browse/AMBARI-22718
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.6.1
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 3.0.0, 2.6.2
>
> Attachments: AMBARI-22718.branch-2.6.patch, AMBARI-22718.trunk.patch
>
>
> STR:
> # Install Ambari 2.6.1
> # Tweak {{stack_features.json}} to lower {{min_version}} for 
> {{kafka_extended_sasl_support}}
> # Create secure cluster via blueprint (ZooKeeper + Kafka) with 
> {{security.inter.broker.protocol: SASL_SSL}}
> Result: Kafka Brokers stopped due to missing {{kafka_jaas.conf}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (AMBARI-22722) Enabling CPU Scheduling in YARN does not show restart option

2018-01-04 Thread Prabhu Joseph (JIRA)
Prabhu Joseph created AMBARI-22722:
--

 Summary: Enabling CPU Scheduling in YARN does not show restart 
option
 Key: AMBARI-22722
 URL: https://issues.apache.org/jira/browse/AMBARI-22722
 Project: Ambari
  Issue Type: Bug
Affects Versions: 2.5.0, 2.6.0
Reporter: Prabhu Joseph
Priority: Critical


When enabling CPU Scheduling under YARN -> Config, it prompts for only YARN 
Refresh Queue but not the restart of RM. CPU Scheduling needs RM Restart.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Dmytro Grinenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Grinenko updated AMBARI-22419:
-
Status: Patch Available  (was: In Progress)

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
> Attachments: AMBARI-22419.branch-2.6.patch
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Dmytro Grinenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Grinenko updated AMBARI-22419:
-
Attachment: AMBARI-22419.branch-2.6.patch

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
> Attachments: AMBARI-22419.branch-2.6.patch
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Dmytro Grinenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Grinenko updated AMBARI-22419:
-
Fix Version/s: 2.6.2

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
> Fix For: 2.6.2
>
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (AMBARI-22419) Ambari upgrade failed

2018-01-04 Thread Dmytro Grinenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-22419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Grinenko reassigned AMBARI-22419:


Assignee: Dmytro Grinenko

> Ambari upgrade failed
> -
>
> Key: AMBARI-22419
> URL: https://issues.apache.org/jira/browse/AMBARI-22419
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-sever
>Affects Versions: 2.6.0
>Reporter: amarnath reddy pappu
>Assignee: Dmytro Grinenko
>
> Ambari upgrade would fail during the DMLStatements execution if there is no 
> service definition in the stacks.
> For example: Customer database has entry for GANGLIA in alert_definition 
> table but there is no stack definition - hence its getting failed with below 
> exception.
> {noformat}
> org.apache.ambari.server.AmbariException: Service not found, 
> clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:234)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:424)
>  
> Caused by: org.apache.ambari.server.ServiceNotFoundException: Service not 
> found, clusterName=comcastdrcluster, serviceName=GANGLIA 
> at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
>  
> at 
> org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
>  
> at 
> org.apache.ambari.server.upgrade.UpdateAlertScriptPaths.executeDMLUpdates(UpdateAlertScriptPaths.java:46)
>  
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:938)
>  
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:231)
>  
> ... 1 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)