subject:"Review Request 51713\: After upgrading cluster from HDP\-2.4.x to HDP\-2.5.x and added atlas service \- missing kafka security properties"

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 473
> > 
> >
> > If you use KERBEROS and KERBEROS_SERVER as service/component then this 
> > group should get skipped.
> 
> Robert Levas wrote:
> Kerberos can be enabled and not require the KERBEROS services and the 
> KERBEROS_CLIENT to be installed.  In the manual case, they are not since we 
> do not need to distribute keytab files.  Is there a way to condtionally 
> execute an action based on a config property (`cluster-env/security_enabled`) 
> or the cluster's security state.  This would only be to prevent the task from 
> showing up in the list of tasks shown during the upgrade process.  
> Internally, if there is no user-defined Kerberos descriptor, nothing happens 
> - but I can fix the log messages to clarifiy this.
> 
> Nate Cole wrote:
> Hmm, I see what you're saying.  Can you file a future jira to add a way 
> to compute whether an execute-stage should be included, other than the simple 
> service/component check?

Created [AMBARI-18352](https://issues.apache.org/jira/browse/AMBARI-18352) to 
address this


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 9, 2016, 2:29 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 2:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 2:29 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:20:19.796s
[INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
[INFO] Final Memory: 62M/1882M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148340
---


Ship it!




Ship It!

- Jonathan Hurley


On Sept. 9, 2016, 9:49 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:49 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:20:19.796s
> [INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
> [INFO] Final Memory: 62M/1882M
> [INFO] 
> 
> 
> #

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148339
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 (line 134)


Just add a bit more information to some of these messages, if possible:

"The Kerberos descriptor for stack %s was not found."

etc.  These show as stdout/stderr for the task, so as much detail that we 
can get there, the easier it is to pinpoint what happened.


- Nate Cole


On Sept. 9, 2016, 9:49 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:49 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas



> On Sept. 8, 2016, 9:07 p.m., Sumit Mohanty wrote:
> > Impressive set of changes. One question:
> > Some of the config changes for secure cluster are fed from the stack 
> > advisor. With this capability, new artifacts in kerberos.json file will be 
> > added during HDP upgrade. I assume there is no stack advisor call to go 
> > with it. Will that be an issue?

There are no stack advisor updates needed related to this. Any updates to the 
stack advisor related to HDP 2.5 would have been added in a different patch. 
This issue is only related to the user-specified Kerberos descritor in how it 
becomes out of sync with the stack Kerberos descriptor once the stack version 
changes.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148295
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 9:49 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing (updated)
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:20:19.796s
[INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
[INFO] Final Memory: 62M/1882M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas



> On Sept. 8, 2016, 12:06 p.m., Nate Cole wrote:
> > What will happen if, say, doing an Upgrade after this code has run, the 
> > user decides to Downgrade?  Will the old data be intact or does it get 
> > overwritten?
> 
> Robert Levas wrote:
> Looking at the *-upgrade-2.5.xml files, will 
> `UpgradeUserKerberosDescriptor` execute for downgrades? I need to test that. 
> However, theorectially, the Kerberos descriptor will be updated to work with 
> the target stack.  So it should be fine.

I tested this scneario with changes in rev3.  It works as expected in both 
directions now.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148199
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas



> On Sept. 8, 2016, 1:07 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java,
> >  lines 555-558
> > 
> >
> > I'm worried about this logic on downgrade. When upgrading from 2.3 to 
> > 2.5, the previous value is from 2.3 and the new value is from 2.5.
> > 
> > However a downgrade also has the same values (from=2.3, to=2.5) but the 
> > direction changes to DOWNGRADE.
> > 
> > Doesn't that mean that this needs to check direction as well?
> 
> Robert Levas wrote:
> On a downgrade (kicked off during the middle of the upgrade process), the 
> previous and new versions appear to both the original version.  Is there a 
> way to know the versions explicity rather than asking the cluster object?
> 
> Jonathan Hurley wrote:
> I don't think so - you'd need to pass in information from the upgrade 
> context into the Kerberos helper and query that object for the stacks and 
> direction.

Thanks to Nate (@ncole), I found the context properties that explicitly 
indicates target and source versions and the upgrade direction.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148193
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 472
> > 
> >
> > Can remove the "(if needed)" in text.

done.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-09 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 9:42 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

changes made due to reviewer comments

- fixed logger's associated class name in `UpgradeUserKerberosDescriptor`
- fixed messages logged in `UpgradeUserKerberosDescriptor`
- added ability to properly handle downgrades


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:22:13.782s
[INFO] Finished at: Wed Sep 07 22:51:30 EDT 2016
[INFO] Final Memory: 60M/1835M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148295
---



Impressive set of changes. One question:
Some of the config changes for secure cluster are fed from the stack advisor. 
With this capability, new artifacts in kerberos.json file will be added during 
HDP upgrade. I assume there is no stack advisor call to go with it. Will that 
be an issue?

- Sumit Mohanty


On Sept. 8, 2016, 3:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 1:07 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java,
> >  lines 555-558
> > 
> >
> > I'm worried about this logic on downgrade. When upgrading from 2.3 to 
> > 2.5, the previous value is from 2.3 and the new value is from 2.5.
> > 
> > However a downgrade also has the same values (from=2.3, to=2.5) but the 
> > direction changes to DOWNGRADE.
> > 
> > Doesn't that mean that this needs to check direction as well?

On a downgrade (kicked off during the middle of the upgrade process), the 
previous and new versions appear to both the original version.  Is there a way 
to know the versions explicity rather than asking the cluster object?


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148193
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 1:07 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java,
> >  line 101
> > 
> >
> > Can this possibly be null somehow can cause a NPE later on?

If this was an issue... we would have NPEs thrown a lot. I dont think it is 
expected that stack name or version would be null at this point.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148193
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 1:07 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json,
> >  lines 107-108
> > 
> >
> > Does this change require work in the next upgrade catalog?

This is already in the upgrade catalog.  Just reconcilling here.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148193
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Nate Cole



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 473
> > 
> >
> > If you use KERBEROS and KERBEROS_SERVER as service/component then this 
> > group should get skipped.
> 
> Robert Levas wrote:
> Kerberos can be enabled and not require the KERBEROS services and the 
> KERBEROS_CLIENT to be installed.  In the manual case, they are not since we 
> do not need to distribute keytab files.  Is there a way to condtionally 
> execute an action based on a config property (`cluster-env/security_enabled`) 
> or the cluster's security state.  This would only be to prevent the task from 
> showing up in the list of tasks shown during the upgrade process.  
> Internally, if there is no user-defined Kerberos descriptor, nothing happens 
> - but I can fix the log messages to clarifiy this.

Hmm, I see what you're saying.  Can you file a future jira to add a way to 
compute whether an execute-stage should be included, other than the simple 
service/component check?


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java,
> >  line 99
> > 
> >
> > log statement should use previous version oldVersion.toString()

This is already fixed in rev 2.


> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 473
> > 
> >
> > If you use KERBEROS and KERBEROS_SERVER as service/component then this 
> > group should get skipped.

Kerberos can be enabled and not require the KERBEROS services and the 
KERBEROS_CLIENT to be installed.  In the manual case, they are not since we do 
not need to distribute keytab files.  Is there a way to condtionally execute an 
action based on a config property (`cluster-env/security_enabled`) or the 
cluster's security state.  This would only be to prevent the task from showing 
up in the list of tasks shown during the upgrade process.  Internally, if there 
is no user-defined Kerberos descriptor, nothing happens - but I can fix the log 
messages to clarifiy this.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java,
> >  lines 85-89
> > 
> >
> > Suggest making KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster) 
> > public and use an injected helper instance.

I can call 
`org.apache.ambari.server.controller.KerberosHelper#getKerberosDescriptor(org.apache.ambari.server.controller.KerberosHelper.KerberosDescriptorType,
 org.apache.ambari.server.state.Cluster, boolean, 
java.util.Collection)` to do that job; however I need to get 
the ArtifactEntity to update it, so the current implementation saves an 
injection, a method call, and multiple JPA calls to retrieve the same artfact 
record. The only real duplicated code is 

```
  KerberosDescriptor userDescriptor = (entity == null) ? null : 
kerberosDescriptorFactory.createInstance(entity.getArtifactData());
```


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 12:06 p.m., Nate Cole wrote:
> > What will happen if, say, doing an Upgrade after this code has run, the 
> > user decides to Downgrade?  Will the old data be intact or does it get 
> > overwritten?

Looking at the *-upgrade-2.5.xml files, will `UpgradeUserKerberosDescriptor` 
execute for downgrades? I need to test that. However, theorectially, the 
Kerberos descriptor will be updated to work with the target stack.  So it 
should be fine.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148199
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148199
---



What will happen if, say, doing an Upgrade after this code has run, the user 
decides to Downgrade?  Will the old data be intact or does it get overwritten?

- Nate Cole


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas



> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java,
> >  line 50
> > 
> >
> > Does not match class name, is that on purpose?

Yikes... nice catch.  This is what you get when you rush to get a patch out. :(


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 8, 2016, 11:09 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 8, 2016, 11:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  d472b79 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 8, 2016, 11:09 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

Fixed log message showing the wrong version data.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 d472b79 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:22:13.782s
[INFO] Finished at: Wed Sep 07 22:51:30 EDT 2016
[INFO] Final Memory: 60M/1835M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-08 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 8, 2016, 10:23 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 d472b79 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing (updated)
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:22:13.782s
[INFO] Finished at: Wed Sep 07 22:51:30 EDT 2016
[INFO] Final Memory: 60M/1835M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

2016-09-07 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 d472b79 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

23 matches

Site Navigation

Mail list logo

Footer information