Re: Review Request 59701: AMBARI-21154 : Add JAAS config properties for Atlas Hive hook in HiveCli to use kerberos ticket-cache

2017-06-23 Thread Vishal Suvagia via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59701/
---

(Updated June 23, 2017, 12:38 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, 
Mugdha Varadkar, Nixon Rodrigues, Robert Levas, and Sumit Mohanty.


Changes
---

Updated patch to change approach by removing kerberos.json and adding required 
parameters in stack-advisor logic.


Bugs: AMBARI-21154
https://issues.apache.org/jira/browse/AMBARI-21154


Repository: ambari


Description
---

In a kerberized environment, Atlas hook uses JAAS configuration section named 
"KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
this configuration section is set to use the keytab and principal of 
HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
with Kafka if the user can't read the configured keytab.

Given that HiveCLI users would have performed kinit, the hook in HiveCLI should 
use the ticket-cache generated by kinit. When ticket cache is not available 
(for example in HiveServer2), the hook should use the configuration provided in 
KafkaClient JAAS section

As a solution need to add below in hive atlas-application.properties by default 
if atlas-hive hook is enabled in secure mode

atlas.jaas.ticketBased-KafkaClient.loginModuleControlFlag=required
atlas.jaas.ticketBased-KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.ticketBased-KafkaClient.option.useTicketCache=true


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/service_advisor.py
 6d3e13d 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
a29f74b 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
 8c659ee 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
3054ca3 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
f8bbca5 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
1cbd78b 
  
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
 ede267a 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
b70943b 
  ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py d4d28c9 


Diff: https://reviews.apache.org/r/59701/diff/2/

Changes: https://reviews.apache.org/r/59701/diff/1-2/


Testing
---

Verified fresh install and upgrade on Cent-OS-6.


Thanks,

Vishal Suvagia



Re: Review Request 59701: AMBARI-21154 : Add JAAS config properties for Atlas Hive hook in HiveCli to use kerberos ticket-cache

2017-06-23 Thread Vishal Suvagia via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59701/
---

(Updated June 23, 2017, 12:44 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, 
Mugdha Varadkar, Nixon Rodrigues, Robert Levas, and Sumit Mohanty.


Changes
---

Patch for branch-2.5 is attached to Apache JIRA


Bugs: AMBARI-21154
https://issues.apache.org/jira/browse/AMBARI-21154


Repository: ambari


Description (updated)
---

In a kerberized environment, Atlas hook uses JAAS configuration section named 
"KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
this configuration section is set to use the keytab and principal of 
HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
with Kafka if the user can't read the configured keytab.

Given that HiveCLI users would have performed kinit, the hook in HiveCLI should 
use the ticket-cache generated by kinit. When ticket cache is not available 
(for example in HiveServer2), the hook should use the configuration provided in 
KafkaClient JAAS section

As a solution need to add below in hive atlas-application.properties by default 
if atlas-hive hook is enabled in secure mode

atlas.jaas.ticketBased-KafkaClient.loginModuleControlFlag=required
atlas.jaas.ticketBased-KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.ticketBased-KafkaClient.option.useTicketCache=true

The attached patch is for trunk branch, patch for branch-2.5 is attached to 
Apache Jira


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/service_advisor.py
 6d3e13d 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
a29f74b 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
 8c659ee 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
3054ca3 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
f8bbca5 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
1cbd78b 
  
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
 ede267a 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
b70943b 
  ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py d4d28c9 


Diff: https://reviews.apache.org/r/59701/diff/2/


Testing
---

Verified fresh install and upgrade on Cent-OS-6.


Thanks,

Vishal Suvagia



Re: Review Request 59701: AMBARI-21154 : Add JAAS config properties for Atlas Hive hook in HiveCli to use kerberos ticket-cache

2017-06-23 Thread Vishal Suvagia via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59701/
---

(Updated June 23, 2017, 1:04 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, 
Mugdha Varadkar, Nixon Rodrigues, Robert Levas, and Sumit Mohanty.


Bugs: AMBARI-21154
https://issues.apache.org/jira/browse/AMBARI-21154


Repository: ambari


Description
---

In a kerberized environment, Atlas hook uses JAAS configuration section named 
"KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
this configuration section is set to use the keytab and principal of 
HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
with Kafka if the user can't read the configured keytab.

Given that HiveCLI users would have performed kinit, the hook in HiveCLI should 
use the ticket-cache generated by kinit. When ticket cache is not available 
(for example in HiveServer2), the hook should use the configuration provided in 
KafkaClient JAAS section

As a solution need to add below in hive atlas-application.properties by default 
if atlas-hive hook is enabled in secure mode

atlas.jaas.ticketBased-KafkaClient.loginModuleControlFlag=required
atlas.jaas.ticketBased-KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.ticketBased-KafkaClient.option.useTicketCache=true

The attached patch is for trunk branch, patch for branch-2.5 is attached to 
Apache Jira


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/service_advisor.py
 6d3e13d 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
a29f74b 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
 8c659ee 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
3054ca3 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
f8bbca5 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
1cbd78b 
  
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
 ede267a 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
b70943b 
  ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py d4d28c9 


Diff: https://reviews.apache.org/r/59701/diff/3/

Changes: https://reviews.apache.org/r/59701/diff/2-3/


Testing
---

Verified fresh install and upgrade on Cent-OS-6.


Thanks,

Vishal Suvagia



Review Request 60388: AMBARI-21256 : As part of START_ALL of services Ranger kms starts after hbase and hive causing operation failure

2017-06-23 Thread Vishal Suvagia via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60388/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
Mugdha Varadkar, Sumit Mohanty, and Velmurugan Periasamy.


Bugs: AMBARI-21256
https://issues.apache.org/jira/browse/AMBARI-21256


Repository: ambari


Description
---

As part of START_ALL of services Ranger-KMS starts after Hbase and Hive causing 
their start failure.
In an encrypted cluster where TDE is on, Hbase service seems to be dependent on 
the Ranger-KMS service, but for a start-all services action Hbase service 
starts before Ranger-KMS.
So Hbase master goes down while connecting to KMS, since at the time of Hbase 
start KMS is down , connection to KMS fails, and Hbase-Master fails to become 
active.
This in-turn causes Atlas service start failure which actually depends on HBase 
and start_all flow fails for HBASE and HIVE.

Attached patch is for branch-2.5, addendum trunk patch is attached to Apache 
JIRA.


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/role_command_order.json
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60388/diff/1/


Testing
---

Verified with installation on Cent-OS 6.


Thanks,

Vishal Suvagia



Re: Review Request 59701: AMBARI-21154 : Add JAAS config properties for Atlas Hive hook in HiveCli to use kerberos ticket-cache

2017-06-23 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59701/#review178795
---


Ship it!




Ship It!

- Robert Levas


On June 23, 2017, 9:04 a.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59701/
> ---
> 
> (Updated June 23, 2017, 9:04 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan 
> Neethiraj, Mugdha Varadkar, Nixon Rodrigues, Robert Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21154
> https://issues.apache.org/jira/browse/AMBARI-21154
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In a kerberized environment, Atlas hook uses JAAS configuration section named 
> "KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
> this configuration section is set to use the keytab and principal of 
> HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
> with Kafka if the user can't read the configured keytab.
> 
> Given that HiveCLI users would have performed kinit, the hook in HiveCLI 
> should use the ticket-cache generated by kinit. When ticket cache is not 
> available (for example in HiveServer2), the hook should use the configuration 
> provided in KafkaClient JAAS section
> 
> As a solution need to add below in hive atlas-application.properties by 
> default if atlas-hive hook is enabled in secure mode
> 
> atlas.jaas.ticketBased-KafkaClient.loginModuleControlFlag=required
> atlas.jaas.ticketBased-KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.ticketBased-KafkaClient.option.useTicketCache=true
> 
> The attached patch is for trunk branch, patch for branch-2.5 is attached to 
> Apache Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/service_advisor.py
>  6d3e13d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> a29f74b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
>  8c659ee 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
> 3054ca3 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
> f8bbca5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 1cbd78b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
>  ede267a 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
> b70943b 
>   ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 
> d4d28c9 
> 
> 
> Diff: https://reviews.apache.org/r/59701/diff/3/
> 
> 
> Testing
> ---
> 
> Verified fresh install and upgrade on Cent-OS-6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>



Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor

2017-06-23 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60389/
---

Review request for Ambari, Oliver Szabo and Robert Nettleton.


Bugs: AMBARI-21333
https://issues.apache.org/jira/browse/AMBARI-21333


Repository: ambari


Description
---

Extract Log Search service advisor from the stack advisor.
Also created an empty Ambari Infra service advisor.


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
726514b 


Diff: https://reviews.apache.org/r/60389/diff/1/


Testing
---

Tested locally.


Thanks,

Miklos Gergely



Re: Review Request 60388: AMBARI-21256 : As part of START_ALL of services Ranger kms starts after hbase and hive causing operation failure

2017-06-23 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60388/#review178797
---


Ship it!




Ship It!

- Sumit Mohanty


On June 23, 2017, 1:13 p.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60388/
> ---
> 
> (Updated June 23, 2017, 1:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> Mugdha Varadkar, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-21256
> https://issues.apache.org/jira/browse/AMBARI-21256
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of START_ALL of services Ranger-KMS starts after Hbase and Hive 
> causing their start failure.
> In an encrypted cluster where TDE is on, Hbase service seems to be dependent 
> on the Ranger-KMS service, but for a start-all services action Hbase service 
> starts before Ranger-KMS.
> So Hbase master goes down while connecting to KMS, since at the time of Hbase 
> start KMS is down , connection to KMS fails, and Hbase-Master fails to become 
> active.
> This in-turn causes Atlas service start failure which actually depends on 
> HBase and start_all flow fails for HBASE and HIVE.
> 
> Attached patch is for branch-2.5, addendum trunk patch is attached to Apache 
> JIRA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/role_command_order.json
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60388/diff/1/
> 
> 
> Testing
> ---
> 
> Verified with installation on Cent-OS 6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>



Re: Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor

2017-06-23 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60389/#review178798
---


Ship it!




The patch looks fine to me.

Can you also please add Alejandro to the reviewer list? 

Thanks.

- Robert Nettleton


On June 23, 2017, 1:54 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60389/
> ---
> 
> (Updated June 23, 2017, 1:54 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21333
> https://issues.apache.org/jira/browse/AMBARI-21333
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Extract Log Search service advisor from the stack advisor.
> Also created an empty Ambari Infra service advisor.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 726514b 
> 
> 
> Diff: https://reviews.apache.org/r/60389/diff/1/
> 
> 
> Testing
> ---
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Review Request 60390: AMBARI-21327 Ambari server to print error messages if NN HA namenode services properties use diff FQDN (dual network cards) than FQDN in the HostComponentState table

2017-06-23 Thread Di Li

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60390/
---

Review request for Ambari and Tim Thorpe.


Bugs: AMBARI-21327
https://issues.apache.org/jira/browse/AMBARI-21327


Repository: ambari


Description
---

For hosts with dual network cards and configured with two FQDN, it's possible 
to manually config NN HA with alternative FQDN than the ones recorded in the 
Ambari database. This caused an issue for EU/RU when upon kicking off the 
upgrade via UI, the UI immediately posts the following error:
"An internal system exception occurred: Request specifies host X but it is 
not a valid host based on the target service=null and component=null"


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
 427a5f5 


Diff: https://reviews.apache.org/r/60390/diff/1/


Testing
---

existing unit tests.


Thanks,

Di Li



Review Request 60399: AMBARI-21329 : No data on templated Grafana dashboards on HDF cluster.

2017-06-23 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60399/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21329
https://issues.apache.org/jira/browse/AMBARI-21329


Repository: ambari


Description
---

On a cluster with HDF 3.0 and Ambari 2.5.1.0, no data is seen in Grafana 
dashboards with Hosts dropdown selection.


Diffs
-

  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 b8cba25 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 32fe32e 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 18a940d 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 beb0904 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
 767398b 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DatabaseMetricsSource.java
 fa04f74 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
 dbb51cf 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
 d7d905c 


Diff: https://reviews.apache.org/r/60399/diff/1/


Testing
---

Manually tested.
mvn clean test passed.


Thanks,

Aravindan Vijayan



Re: Review Request 60399: AMBARI-21329 : No data on templated Grafana dashboards on HDF cluster.

2017-06-23 Thread Sid Wagle

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60399/#review178808
---


Ship it!




Ship It!

- Sid Wagle


On June 23, 2017, 7:42 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60399/
> ---
> 
> (Updated June 23, 2017, 7:42 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21329
> https://issues.apache.org/jira/browse/AMBARI-21329
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On a cluster with HDF 3.0 and Ambari 2.5.1.0, no data is seen in Grafana 
> dashboards with Hosts dropdown selection.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
>  b8cba25 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
>  32fe32e 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  18a940d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  beb0904 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
>  767398b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DatabaseMetricsSource.java
>  fa04f74 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
>  dbb51cf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
>  d7d905c 
> 
> 
> Diff: https://reviews.apache.org/r/60399/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> mvn clean test passed.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>



Review Request 60403: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

2017-06-23 Thread Qin Liu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/
---

Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, 
Sid Wagle, and Tim Thorpe.


Bugs: AMBARI-17898
https://issues.apache.org/jira/browse/AMBARI-17898


Repository: ambari


Description
---

This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
authenticate".
In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But 
Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does 
not support Kerberos HTTP SPNEGO authentication.
/var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
http://metrics-collector:6188/ws/v1/timeline/metrics
2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
server. HTTP Error 401: Authentication required
2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...


Diffs
-

  ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
7fe7397 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 d1429ed 
  ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
77b8c23 
  ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py 
PRE-CREATION 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
 4056ae3 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
 51f541f 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml
 9031b46 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
 f49d47d 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
 b8c14f4 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
 b7dee50 


Diff: https://reviews.apache.org/r/60403/diff/1/


Testing
---

1. Unit tests passed
2. manually tested with HDP/trunk


Thanks,

Qin Liu



Review Request 60408: Cannot start ambari server

2017-06-23 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60408/
---

Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21335
https://issues.apache.org/jira/browse/AMBARI-21335


Repository: ambari


Description
---

ambari-server start fails with following error
{Code}
ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.Unable to determine server PID. Retrying...
..Unable to determine server PID. Retrying...
..Unable to determine server PID. Retrying...
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process died with exitcode 1. Check 
/var/log/ambari-server/ambari-server.out for more information.
{Code}

logs in /var/log/ambari-server/ambari-server.out  shows
{code}
Jun 22, 2017 7:29:55 PM org.apache.ambari.server.configuration.Configuration 

INFO: Reading password from existing file
Jun 22, 2017 7:29:55 PM org.apache.ambari.server.configuration.Configuration 
getHostsMapFile
INFO: Hosts Mapping File null
Jun 22, 2017 7:29:55 PM org.apache.ambari.server.controller.HostsMap setupMap
INFO: Using hostsmap file null
Exception in thread "main" java.lang.NoSuchMethodError: 
org.slf4j.helpers.MessageFormatter.format(Ljava/lang/String;Ljava/lang/Object;)Ljava/lang/String;
at org.slf4j.impl.JDK14LoggerAdapter.info(JDK14LoggerAdapter.java:303)
at 
org.apache.ambari.server.controller.ControllerModule.getPersistenceProperties(ControllerModule.java:221)
at 
org.apache.ambari.server.controller.ControllerModule.buildJpaPersistModule(ControllerModule.java:416)
at 
org.apache.ambari.server.controller.ControllerModule.configure(ControllerModule.java:343)
at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
at 
com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
at com.google.inject.spi.Elements.getElements(Elements.java:101)
at 
com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133)
at 
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
at com.google.inject.Guice.createInjector(Guice.java:95)
at com.google.inject.Guice.createInjector(Guice.java:72)
at com.google.inject.Guice.createInjector(Guice.java:62)
at 
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1060)
~
{code}

Ambari build used - 3.0.0.0-871


Diffs
-

  ambari-server/src/main/assemblies/server.xml 53e65fd 


Diff: https://reviews.apache.org/r/60408/diff/1/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi



Re: Review Request 60408: Cannot start ambari server

2017-06-23 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60408/#review178832
---


Ship it!




Ship It!

- Sumit Mohanty


On June 24, 2017, 12:42 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60408/
> ---
> 
> (Updated June 24, 2017, 12:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21335
> https://issues.apache.org/jira/browse/AMBARI-21335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server start fails with following error
> {Code}
> ambari-server start
> Using python  /usr/bin/python
> Starting ambari-server
> Ambari Server running with administrator privileges.
> Organizing resource files at /var/lib/ambari-server/resources...
> Ambari database consistency check started...
> Server PID at: /var/run/ambari-server/ambari-server.pid
> Server out at: /var/log/ambari-server/ambari-server.out
> Server log at: /var/log/ambari-server/ambari-server.log
> Waiting for server start.Unable to determine server PID. Retrying...
> ..Unable to determine server PID. Retrying...
> ..Unable to determine server PID. Retrying...
> ERROR: Exiting with exit code -1.
> REASON: Ambari Server java process died with exitcode 1. Check 
> /var/log/ambari-server/ambari-server.out for more information.
> {Code}
> 
> logs in /var/log/ambari-server/ambari-server.out  shows
> {code}
> Jun 22, 2017 7:29:55 PM org.apache.ambari.server.configuration.Configuration 
> 
> INFO: Reading password from existing file
> Jun 22, 2017 7:29:55 PM org.apache.ambari.server.configuration.Configuration 
> getHostsMapFile
> INFO: Hosts Mapping File null
> Jun 22, 2017 7:29:55 PM org.apache.ambari.server.controller.HostsMap setupMap
> INFO: Using hostsmap file null
> Exception in thread "main" java.lang.NoSuchMethodError: 
> org.slf4j.helpers.MessageFormatter.format(Ljava/lang/String;Ljava/lang/Object;)Ljava/lang/String;
> at org.slf4j.impl.JDK14LoggerAdapter.info(JDK14LoggerAdapter.java:303)
> at 
> org.apache.ambari.server.controller.ControllerModule.getPersistenceProperties(ControllerModule.java:221)
> at 
> org.apache.ambari.server.controller.ControllerModule.buildJpaPersistModule(ControllerModule.java:416)
> at 
> org.apache.ambari.server.controller.ControllerModule.configure(ControllerModule.java:343)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
> at 
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
> at com.google.inject.spi.Elements.getElements(Elements.java:101)
> at 
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133)
> at 
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
> at com.google.inject.Guice.createInjector(Guice.java:95)
> at com.google.inject.Guice.createInjector(Guice.java:72)
> at com.google.inject.Guice.createInjector(Guice.java:62)
> at 
> org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1060)
> ~
> {code}
> 
> Ambari build used - 3.0.0.0-871
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/assemblies/server.xml 53e65fd 
> 
> 
> Diff: https://reviews.apache.org/r/60408/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>