Re: Review Request 48634: File browser : File preview show first character only when ssl is enabled.

2016-06-13 Thread Nitiraj Rathore

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48634/#review137452
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 13, 2016, 1:18 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48634/
> ---
> 
> (Updated June 13, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17194
> https://issues.apache.org/jira/browse/AMBARI-17194
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Using IOUtils.read to read from file.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java
>  3585516 
> 
> Diff: https://reviews.apache.org/r/48634/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48308: [AMBARI-17078] Make Spark2-ThriftServer and Livy Server as optional by default

2016-06-13 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48308/#review137449
---


Ship it!




Ship It!

- Sumit Mohanty


On June 14, 2016, 4:10 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48308/
> ---
> 
> (Updated June 14, 2016, 4:10 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17078
> https://issues.apache.org/jira/browse/AMBARI-17078
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Spark2-ThriftServer and Livy Server are not being selected by default, but 
> should be unselected by default.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> db2986b 
> 
> Diff: https://reviews.apache.org/r/48308/diff/
> 
> 
> Testing
> ---
> 
> Manullay verified
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Re: Review Request 48308: [AMBARI-17078] Make Spark2-ThriftServer and Livy Server as optional by default

2016-06-13 Thread Jeff Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48308/
---

(Updated June 14, 2016, 4:10 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

Move the change to HDP 2.5


Bugs: AMBARI-17078
https://issues.apache.org/jira/browse/AMBARI-17078


Repository: ambari


Description
---

Spark2-ThriftServer and Livy Server are not being selected by default, but 
should be unselected by default.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
db2986b 

Diff: https://reviews.apache.org/r/48308/diff/


Testing
---

Manullay verified


Thanks,

Jeff Zhang



Re: Review Request 48308: [AMBARI-17078] Make Spark2-ThriftServer and Livy Server as optional by default

2016-06-13 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48308/#review137445
---




ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
910)


The new components should be in HDP/2.5 stack advisor as they are added in 
that version.


- Sumit Mohanty


On June 7, 2016, 2:59 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48308/
> ---
> 
> (Updated June 7, 2016, 2:59 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17078
> https://issues.apache.org/jira/browse/AMBARI-17078
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Spark2-ThriftServer and Livy Server are not being selected by default, but 
> should be unselected by default.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 36fe066b65b12326a128fdb5e8caf9ad56a8211c 
> 
> Diff: https://reviews.apache.org/r/48308/diff/
> 
> 
> Testing
> ---
> 
> Manullay verified
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Review Request 48670: Return well formatted error response while deleting host with clients installed.

2016-06-13 Thread Ajit Kumar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48670/
---

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Bugs: AMBARI-17210
https://issues.apache.org/jira/browse/AMBARI-17210


Repository: ambari


Description
---

Return well formatted error response while deleting host with clients installed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 46fc65295f72bae39c534cd9684f2d76d1e30224 

Diff: https://reviews.apache.org/r/48670/diff/


Testing
---

Manual testing

delete 
http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/hosts/c6403.ambari.apache.org
{
  "status" : 500,
  "message" : "org.apache.ambari.server.controller.spi.SystemException: An 
internal system exception occurred: Cannot remove host c6403.ambari.apache.org 
from c1.  The following roles exist, and these components must be stopped if 
running, and then deleted: HDFS_CLIENT, MAPREDUCE2_CLIENT, YARN_CLIENT, 
ZOOKEEPER_CLIENT"
}


Thanks,

Ajit Kumar



Re: Review Request 48664: Add logging for the command executed during PXF service check

2016-06-13 Thread Matt

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48664/#review137415
---


Ship it!




Ship It!

- Matt


On June 13, 2016, 3:30 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48664/
> ---
> 
> (Updated June 13, 2016, 3:30 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov and Matt.
> 
> 
> Bugs: AMBARI-17208
> https://issues.apache.org/jira/browse/AMBARI-17208
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add logging for the command executed during PXF service check
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
>  c39a85e 
> 
> Diff: https://reviews.apache.org/r/48664/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>



Re: Review Request 48663: Export PGHOST before any HAWQ Master or Standby custom command is executed

2016-06-13 Thread bhuvnesh chaudhary

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48663/#review137412
---


Ship it!




Ship It!

- bhuvnesh chaudhary


On June 13, 2016, 10:25 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48663/
> ---
> 
> (Updated June 13, 2016, 10:25 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-17207
> https://issues.apache.org/jira/browse/AMBARI-17207
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Export PGHOST before any HAWQ Master or Standby custom command is executed
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
>  1804f11 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
>  dc9bb12 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py
>  119f2c7 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
>  12d3511 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py aa63004 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 185bde8 
> 
> Diff: https://reviews.apache.org/r/48663/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> Updated unit tests:
> ```
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove 

Review Request 48664: Add logging for the command executed during PXF service check

2016-06-13 Thread bhuvnesh chaudhary

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48664/
---

Review request for Ambari, Alexander Denissov and Matt.


Bugs: AMBARI-17208
https://issues.apache.org/jira/browse/AMBARI-17208


Repository: ambari


Description
---

Add logging for the command executed during PXF service check


Diffs
-

  
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
 c39a85e 

Diff: https://reviews.apache.org/r/48664/diff/


Testing
---

yes


Thanks,

bhuvnesh chaudhary



Review Request 48663: Export PGHOST before any HAWQ Master or Standby custom command is executed

2016-06-13 Thread Matt

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48663/
---

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav Jain.


Bugs: AMBARI-17207
https://issues.apache.org/jira/browse/AMBARI-17207


Repository: ambari


Description
---

Export PGHOST before any HAWQ Master or Standby custom command is executed


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
 1804f11 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
 dc9bb12 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py
 119f2c7 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
 12d3511 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py aa63004 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 185bde8 

Diff: https://reviews.apache.org/r/48663/diff/


Testing
---

Manually tested.

Updated unit tests:
```
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-06-13 15:24:28,647 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-06-13 15:24:28,652 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-06-13 15:24:28,656 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource 

Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/#review137388
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 8:57 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48654/
> ---
> 
> (Updated June 13, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17202
> https://issues.apache.org/jira/browse/AMBARI-17202
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When upgrading from earlier versions of Ambari, the alert definitions do not 
> include some of the new parameters / connection timeout values which were 
> added to the alert definitions. This is normally OK since the alert framework 
> has sensible defaults of the values are not specified.
> However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
> properly render/save the values.
> STR:
> Install Ambari 2.2.0
> Upgrade to Ambari 2.2.2
> You should now have alert definitions, such as DataNode Web UI, which do not 
> have a connection_timeout value.
> Navigate to the DataNode Web UI alert definition page and notice that you are 
> not able to update and save the "Connection Timeout" property correctly.
> Attachments
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java
>  c269927 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48654/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-13 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review137384
---



Does this handle UI based cluster deployments?
Probably there is no good way for UI based deplyments to know if a specific 
request is a cluster create request or not.

What happens after cluster creation is over - does the code continue to handle 
Heartbeat processing completed event?
Would it have been possible to create an event for Request-Completed and 
process that rather than Heartbeat processing completed?

- Sumit Mohanty


On June 11, 2016, 6:43 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 11, 2016, 6:43 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/HeartbeatProcessingFinishedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  77419d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  324a397 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Succeeded locally
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/#review137379
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java 
(line 75)


This is probably for the best as the Jackson and Gson parsers used for the 
REST API and agents keep turning this into 0.0 ... Of course, if we used 
objects and not wonk Map ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48654/
> ---
> 
> (Updated June 13, 2016, 4:57 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17202
> https://issues.apache.org/jira/browse/AMBARI-17202
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When upgrading from earlier versions of Ambari, the alert definitions do not 
> include some of the new parameters / connection timeout values which were 
> added to the alert definitions. This is normally OK since the alert framework 
> has sensible defaults of the values are not specified.
> However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
> properly render/save the values.
> STR:
> Install Ambari 2.2.0
> Upgrade to Ambari 2.2.2
> You should now have alert definitions, such as DataNode Web UI, which do not 
> have a connection_timeout value.
> Navigate to the DataNode Web UI alert definition page and notice that you are 
> not able to update and save the "Connection Timeout" property correctly.
> Attachments
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java
>  c269927 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48654/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Re: Review Request 48549: AMBARI-17165 Handle Java patches execution during Ranger upgrade

2016-06-13 Thread Alejandro Fernandez


> On June 10, 2016, 5:40 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml,
> >  line 889
> > 
> >
> > hosts="all" is the default, you can remove it.
> > sequential="true" isn't needed since it is only useful when two 
> > consecutive tasks need to run one after the other even though they are on 
> > different hosts.
> > 
> > Why is there a stop and start in post-upgrade?
> > What are you trying to achieve here?
> > 
> > Pre-upgrade is any steps to execute before restarting the component, 
> > Post-upgrade is after restarting. So if there are any other stop-start 
> > steps, it means the component is being restarted twice!
> 
> Nate Cole wrote:
> I agree with Alejandro here, it looks like you're stopping all admins, 
> java-patching only one, then starting them all.  They should all need 
> patching, no?  And if that's the case that can all be handled on the python 
> side with nothing but restart for the component in the UP.
> 
> Jonathan Hurley wrote:
> I think that stopping all and running the patch on 1 is correct. The 
> patch executes `db_setup.py -javapatch`, so I'd expect that since it's done 
> on the database that you'd only want it done once. But you'd also want all of 
> them down.
> 
> With that said, I still don't understand why this is being done 
> post-upgrade.
> 
> Mugdha Varadkar wrote:
> Yes db_setup.py -javapatch needs to run only at one host.
> 
> The reason behind this addition is java patches failures: when we try to 
> run Java patches with Ranger services started. That's why I have stopped 
> service (from all hosts, in case of HA) before starting to execute Java 
> patches. 
> 
> It needs to be a part of post upgrade, because it needs to be applied 
> only on one host also it needs to have config properties of new version.

Please setup a quick 10 min discussion with Jonathan or myself to go over this.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48549/#review137042
---


On June 10, 2016, 12:47 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48549/
> ---
> 
> (Updated June 10, 2016, 12:47 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17165
> https://issues.apache.org/jira/browse/AMBARI-17165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Revisiting execution of java patches during upgrade for Ranger Service
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
>  4fd5801 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
>  272a3cc 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> b0cff68 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> 0b72254 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
>  111b432 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
>  9365646 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  f40f760 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 712241b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 4187d64 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> ea5ff5a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
>  e83b54b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  7fb03dc 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> 4065e87 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 7f988e3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  460e6b3 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 6ce4c81 
> 
> Diff: https://reviews.apache.org/r/48549/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger upgrade from stack 2.4 to 2.5
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>



Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/
---

(Updated Червень 13, 2016, 8:57 після полудня)


Review request for Ambari, Dmytro Sen and Jonathan Hurley.


Bugs: AMBARI-17202
https://issues.apache.org/jira/browse/AMBARI-17202


Repository: ambari


Description
---

When upgrading from earlier versions of Ambari, the alert definitions do not 
include some of the new parameters / connection timeout values which were added 
to the alert definitions. This is normally OK since the alert framework has 
sensible defaults of the values are not specified.
However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
properly render/save the values.
STR:
Install Ambari 2.2.0
Upgrade to Ambari 2.2.2
You should now have alert definitions, such as DataNode Web UI, which do not 
have a connection_timeout value.
Navigate to the DataNode Web UI alert definition page and notice that you are 
not able to update and save the "Connection Timeout" property correctly.
Attachments


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java 
c269927 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 3ee8bba 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 7451bbe 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 c221138 

Diff: https://reviews.apache.org/r/48654/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi



Re: Review Request 48609: AMBARI-17183: client.properties for Falcon should be configurable via Ambari

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48609/#review137377
---



Has this been tested on an actual stack? These configs are being added to HDP 
2.3 or higher.
For an existing cluster with HDP 2.3 or higher, I believe ambari upgrade will 
add this as a new config type.


ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-client.properties.xml
 (line 28)


Based on the latest code, we no longer need 

I believe you want,
 and take any other defaults.


- Alejandro Fernandez


On June 13, 2016, 4:02 a.m., Venkat Ranganathan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48609/
> ---
> 
> (Updated June 13, 2016, 4:02 a.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-17183
> https://issues.apache.org/jira/browse/AMBARI-17183
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently falcon-client.properties is generated without functionality for 
> user changes.   We should fix it for users of mirroring etc
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/FALCON/package/scripts/falcon.py
>  9a72af1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-client.properties.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48609/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>



Re: Review Request 48532: AMBARI-17157 Storm 1.0 log4j config update

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48532/#review137375
---



Pushed to trunk, commit bb87fe175aedb7c8eb6400a15b230abfae55da9d
branch-2.4, commit 3535972f1b98f0898b808ee6a0efd5c97f8f8964

- Alejandro Fernandez


On June 10, 2016, 2:45 a.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48532/
> ---
> 
> (Updated June 10, 2016, 2:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17157
> https://issues.apache.org/jira/browse/AMBARI-17157
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-17157  Storm 1.0 log4j config update
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-cluster-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-worker-log4j.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48532/diff/
> 
> 
> Testing
> ---
> 
> Installed storm 1.0 using Ambari. Tested if the logs are being written in new 
> format.
> 
> 
> Thanks,
> 
> Sriharsha Chintalapani
> 
>



Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review137374
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Re: Review Request 48655: ATLAS conf dir needs to be present in all ATLAS hook deployed hosts

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48655/#review137373
---




ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
 (line 238)


Does this need to change for existing clusters?
What about the different stacks?
I don't believe ATLAS is even available on HDP 2.2


- Alejandro Fernandez


On June 13, 2016, 5:51 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48655/
> ---
> 
> (Updated June 13, 2016, 5:51 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Nate Cole.
> 
> 
> Bugs: AMBARI-17203
> https://issues.apache.org/jira/browse/AMBARI-17203
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Atlas atlas-application.properties configuration is required by the Atlas 
> hooks (Falcon, Storm, Sqoop and Hive).
> Currently the atlas configuration is not being pushed to all the hook hosts 
> which could lead to issues if any config is changed. The Atlas configuration 
> changes needs to be pushed to all dependent hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  441f0da 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
>  67077c4 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  c20523d 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  fea0635 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
>  e78190f 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  c154a16 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
>  d18d820 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  eadbd4a 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
>  6c3e91f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/metainfo.xml 
> 1998131 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> ac1f9e7 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/metainfo.xml 
> eb67d63 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/metainfo.xml 
> 3faadc0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 1bcc09e 
> 
> Diff: https://reviews.apache.org/r/48655/diff/
> 
> 
> Testing
> ---
> 
> Manual test deploy Atlas with all hook integration services on multi-host 
> cluster.  Verify Atlas configuration locations.  Update Atlas configuration.  
> Verify dependant service restart suggestion from Ambari.
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-13 Thread Robert Nettleton


> On June 13, 2016, 4:51 p.m., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java,
> >  line 191
> > 
> >
> > I'm a little concerned that this approach could have performance issues 
> > down the road.
> > 
> > I'm wondering if it might be worth considering launching a separate 
> > task thread when the cluster is created by the TopologyManager, that can 
> > monitor the state of the LogicalRequest as the deployment progresses.
> > 
> > I would recommend asking Sumit to review this as well, since it's not 
> > clear to me that this is the approach that should be taken.
> 
> Sebastian Toader wrote:
> This loop simply iterates through the received reports extracts cluster 
> id and publishes an event for further processing. The event is consumed on a 
> separate thread 'ambari-event-bus' which is set up in 
> ```AmbariEventPublisher```. Do you think that the extraction of cluster name 
> and cluster id may slow the the heartbeat handler?

Hi Sebastian, Thanks for this information.

I guess I'd be concerned about any extraneous additions to the 
HeartBeatHandler, especially given that it seems like there would be other 
alternatives.

Was the possibility of launching a thread in the TopologyManager to monitor a 
cluster deployment's status considered?  Since the TopologyManager already has 
access to the LogicalRequest and the underlying physical requests as well, it 
seems like the most natural fit for monitoring status, and providing a way to 
log this information.

My overall concern is that the heartbeat handlers are being modified just to 
support logging a message when the cluster has completed.  As I mentioned 
above, it seems like it would be simpler and more straightforward to resolve 
this with a thread that monitors the progress, using the resource APIs that 
already exist in Ambari.  

If the approach I've just detailed was considered, can you post why it was not 
chosen?  Are there some technical concerns that make using a separate thread 
not feasible? 

Thanks.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review137326
---


On June 11, 2016, 6:43 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 11, 2016, 6:43 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/HeartbeatProcessingFinishedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  77419d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  324a397 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Succeeded locally
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Dmytro Grinenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/#review137371
---


Ship it!




Ship It!

- Dmytro Grinenko


On June 13, 2016, 8:15 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48654/
> ---
> 
> (Updated June 13, 2016, 8:15 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17202
> https://issues.apache.org/jira/browse/AMBARI-17202
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When upgrading from earlier versions of Ambari, the alert definitions do not 
> include some of the new parameters / connection timeout values which were 
> added to the alert definitions. This is normally OK since the alert framework 
> has sensible defaults of the values are not specified.
> However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
> properly render/save the values.
> STR:
> Install Ambari 2.2.0
> Upgrade to Ambari 2.2.2
> You should now have alert definitions, such as DataNode Web UI, which do not 
> have a connection_timeout value.
> Navigate to the DataNode Web UI alert definition page and notice that you are 
> not able to update and save the "Connection Timeout" property correctly.
> Attachments
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  5016325 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48654/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/
---

(Updated Червень 13, 2016, 8:15 після полудня)


Review request for Ambari, Dmytro Sen and Jonathan Hurley.


Bugs: AMBARI-17202
https://issues.apache.org/jira/browse/AMBARI-17202


Repository: ambari


Description
---

When upgrading from earlier versions of Ambari, the alert definitions do not 
include some of the new parameters / connection timeout values which were added 
to the alert definitions. This is normally OK since the alert framework has 
sensible defaults of the values are not specified.
However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
properly render/save the values.
STR:
Install Ambari 2.2.0
Upgrade to Ambari 2.2.2
You should now have alert definitions, such as DataNode Web UI, which do not 
have a connection_timeout value.
Navigate to the DataNode Web UI alert definition page and notice that you are 
not able to update and save the "Connection Timeout" property correctly.
Attachments


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 3ee8bba 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 5016325 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 c221138 

Diff: https://reviews.apache.org/r/48654/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi



Re: Review Request 48655: ATLAS conf dir needs to be present in all ATLAS hook deployed hosts

2016-06-13 Thread John Speidel

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48655/#review137363
---


Ship it!




Ship It!

- John Speidel


On June 13, 2016, 5:51 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48655/
> ---
> 
> (Updated June 13, 2016, 5:51 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Nate Cole.
> 
> 
> Bugs: AMBARI-17203
> https://issues.apache.org/jira/browse/AMBARI-17203
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Atlas atlas-application.properties configuration is required by the Atlas 
> hooks (Falcon, Storm, Sqoop and Hive).
> Currently the atlas configuration is not being pushed to all the hook hosts 
> which could lead to issues if any config is changed. The Atlas configuration 
> changes needs to be pushed to all dependent hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  441f0da 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
>  67077c4 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  c20523d 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  fea0635 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
>  e78190f 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  c154a16 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
>  d18d820 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  eadbd4a 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
>  6c3e91f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/metainfo.xml 
> 1998131 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> ac1f9e7 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/metainfo.xml 
> eb67d63 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/metainfo.xml 
> 3faadc0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 1bcc09e 
> 
> Diff: https://reviews.apache.org/r/48655/diff/
> 
> 
> Testing
> ---
> 
> Manual test deploy Atlas with all hook integration services on multi-host 
> cluster.  Verify Atlas configuration locations.  Update Atlas configuration.  
> Verify dependant service restart suggestion from Ambari.
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>



Re: Review Request 48640: Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to SERVICE.ADMINISTRATOR role and above

2016-06-13 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48640/#review137361
---


Ship it!




Ship It!

- Jonathan Hurley


On June 13, 2016, 2:50 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48640/
> ---
> 
> (Updated June 13, 2016, 2:50 p.m.)
> 
> 
> Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Nate Cole, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17177
> https://issues.apache.org/jira/browse/AMBARI-17177
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to the following roles:
> 
> * AMBARI.ADMINISTRATOR 
> * CLUSTER.ADMINISTRATOR 
> * CLUSTER.OPERATOR 
> * SERVICE.ADMINISTRATOR
> 
> This is a DB change adding an authorization record to the `roleauthorization` 
> table and relevant records for the different roles to the 
> `permission_roleauthorization` table. 
> 
> The description/name of the `SERVICE.VIEW_OPERATIONAL_LOGS` authorization 
> should be
> ```
> View service operational logs
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
> 33ed7ed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RoleAuthorizationDAO.java
>  aa74224 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java
>  ee948fe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java
>  6b038f4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  5016325 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fff1716 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql e7eff93 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b90c7da 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql b9181d2 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> c840b9c 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 742eaa3 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 87ef7fb 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog230Test.java
>  d7e13ea 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48640/diff/
> 
> 
> Testing
> ---
> 
> Manually tested upgrade and new cluster creation
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:16:17.022s
> [INFO] Finished at: Mon Jun 13 09:41:19 EDT 2016
> [INFO] Final Memory: 59M/1881M
> [INFO] 
> 
> 
> # Jenkins test results: 
> 
> Tests run: 4478, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 01:38 h
> [INFO] Finished at: 2016-06-13T17:03:59+00:00
> [INFO] Final Memory: 152M/527M
> [INFO] 
> 
> 
> 
> {color:green}+1 overall{color}.  Here are the results of testing the latest 
> attachment 
>   
> http://issues.apache.org/jira/secure/attachment/12809853/AMBARI-17177_trunk_01.patch
>   against trunk revision .
> 
> {color:green}+1 @author{color}.  The patch does not contain any @author 
> tags.
> 
> {color:green}+1 tests included{color}.  The patch appears to include 2 
> new or modified test files.
> 
> {color:green}+1 javac{color}.  The applied patch does not increase the 
> total number of javac compiler warnings.
> 
> {color:green}+1 release audit{color}.  The applied patch does not 
> increase the total number of release audit warnings.
> 
> {color:green}+1 core tests{color}.  The patch passed unit tests in 
> ambari-admin ambari-server.
> 
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//console
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 48557: Fixed implementation of on-ambari-upgrade support. Patch 2: add logic for ambari-upgrade

2016-06-13 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48557/#review137360
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 (line 36)


Can we initialie these in their declaration since they are optional?


- Jonathan Hurley


On June 13, 2016, 9:22 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48557/
> ---
> 
> (Updated June 13, 2016, 9:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17112
> https://issues.apache.org/jira/browse/AMBARI-17112
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch implements logic for on-ambari-upgrade attributes: add, update, delete
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  de5147d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> c570ab3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  f6791ee 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-site.xml
>  6793e4e 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  fd68ae0 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-startup.properties.xml
>  aacf10a 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml
>  4e56084 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/core-site.xml
>  7f67b8a 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml
>  274163e 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  b0f36e7 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
>  444b8b4 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml
>  cfdd989 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  7f474f6 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml
>  f3a6bcf 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml
>  47c900e 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml
>  a309fa4 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-site.xml
>  2d0047c 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/ranger-env.xml
>  5082277 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  121a797 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  ae56f8b 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  4317cfa 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  7eb78e5 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  a747dde 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  35910ee 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6eb312f 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml
>  9870b8b 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  633cbda 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  daacdee 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  21cd096 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  a8f1584 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
>  d7c87e9 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  7aa18cc 
>   
> 

Re: Review Request 48655: ATLAS conf dir needs to be present in all ATLAS hook deployed hosts

2016-06-13 Thread Nate Cole

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48655/#review137359
---


Ship it!




Ship It!

- Nate Cole


On June 13, 2016, 1:51 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48655/
> ---
> 
> (Updated June 13, 2016, 1:51 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Nate Cole.
> 
> 
> Bugs: AMBARI-17203
> https://issues.apache.org/jira/browse/AMBARI-17203
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Atlas atlas-application.properties configuration is required by the Atlas 
> hooks (Falcon, Storm, Sqoop and Hive).
> Currently the atlas configuration is not being pushed to all the hook hosts 
> which could lead to issues if any config is changed. The Atlas configuration 
> changes needs to be pushed to all dependent hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  441f0da 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
>  67077c4 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  c20523d 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  fea0635 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
>  e78190f 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  c154a16 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
>  d18d820 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  eadbd4a 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
>  6c3e91f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/metainfo.xml 
> 1998131 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> ac1f9e7 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/metainfo.xml 
> eb67d63 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/metainfo.xml 
> 3faadc0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 1bcc09e 
> 
> Diff: https://reviews.apache.org/r/48655/diff/
> 
> 
> Testing
> ---
> 
> Manual test deploy Atlas with all hook integration services on multi-host 
> cluster.  Verify Atlas configuration locations.  Update Atlas configuration.  
> Verify dependant service restart suggestion from Ambari.
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>



Re: Review Request 48640: Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to SERVICE.ADMINISTRATOR role and above

2016-06-13 Thread Nate Cole

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48640/#review137358
---


Ship it!




Ship It!

- Nate Cole


On June 13, 2016, 2:50 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48640/
> ---
> 
> (Updated June 13, 2016, 2:50 p.m.)
> 
> 
> Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Nate Cole, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17177
> https://issues.apache.org/jira/browse/AMBARI-17177
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to the following roles:
> 
> * AMBARI.ADMINISTRATOR 
> * CLUSTER.ADMINISTRATOR 
> * CLUSTER.OPERATOR 
> * SERVICE.ADMINISTRATOR
> 
> This is a DB change adding an authorization record to the `roleauthorization` 
> table and relevant records for the different roles to the 
> `permission_roleauthorization` table. 
> 
> The description/name of the `SERVICE.VIEW_OPERATIONAL_LOGS` authorization 
> should be
> ```
> View service operational logs
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
> 33ed7ed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RoleAuthorizationDAO.java
>  aa74224 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java
>  ee948fe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java
>  6b038f4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  5016325 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fff1716 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql e7eff93 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b90c7da 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql b9181d2 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> c840b9c 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 742eaa3 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 87ef7fb 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog230Test.java
>  d7e13ea 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48640/diff/
> 
> 
> Testing
> ---
> 
> Manually tested upgrade and new cluster creation
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:16:17.022s
> [INFO] Finished at: Mon Jun 13 09:41:19 EDT 2016
> [INFO] Final Memory: 59M/1881M
> [INFO] 
> 
> 
> # Jenkins test results: 
> 
> Tests run: 4478, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 01:38 h
> [INFO] Finished at: 2016-06-13T17:03:59+00:00
> [INFO] Final Memory: 152M/527M
> [INFO] 
> 
> 
> 
> {color:green}+1 overall{color}.  Here are the results of testing the latest 
> attachment 
>   
> http://issues.apache.org/jira/secure/attachment/12809853/AMBARI-17177_trunk_01.patch
>   against trunk revision .
> 
> {color:green}+1 @author{color}.  The patch does not contain any @author 
> tags.
> 
> {color:green}+1 tests included{color}.  The patch appears to include 2 
> new or modified test files.
> 
> {color:green}+1 javac{color}.  The applied patch does not increase the 
> total number of javac compiler warnings.
> 
> {color:green}+1 release audit{color}.  The applied patch does not 
> increase the total number of release audit warnings.
> 
> {color:green}+1 core tests{color}.  The patch passed unit tests in 
> ambari-admin ambari-server.
> 
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//console
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 48557: Fixed implementation of on-ambari-upgrade support. Patch 2: add logic for ambari-upgrade

2016-06-13 Thread Nate Cole

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48557/#review137357
---


Ship it!




Ship It!

- Nate Cole


On June 13, 2016, 9:22 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48557/
> ---
> 
> (Updated June 13, 2016, 9:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17112
> https://issues.apache.org/jira/browse/AMBARI-17112
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch implements logic for on-ambari-upgrade attributes: add, update, delete
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  de5147d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> c570ab3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  f6791ee 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-site.xml
>  6793e4e 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  fd68ae0 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-startup.properties.xml
>  aacf10a 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml
>  4e56084 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/core-site.xml
>  7f67b8a 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml
>  274163e 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  b0f36e7 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
>  444b8b4 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml
>  cfdd989 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  7f474f6 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml
>  f3a6bcf 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml
>  47c900e 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml
>  a309fa4 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-site.xml
>  2d0047c 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/ranger-env.xml
>  5082277 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  121a797 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  ae56f8b 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  4317cfa 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  7eb78e5 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  a747dde 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  35910ee 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6eb312f 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml
>  9870b8b 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  633cbda 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  daacdee 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  21cd096 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  a8f1584 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
>  d7c87e9 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  7aa18cc 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  fcaecaa 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
>  85a1eba 
>   
> 

Re: Review Request 48568: AMBARI-17176 VDF: include default version definition in list, even if Internet Access is available

2016-06-13 Thread Jaimin Jetly

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48568/#review137356
---


Ship it!




Ship It!

- Jaimin Jetly


On June 11, 2016, 12:01 a.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48568/
> ---
> 
> (Updated June 11, 2016, 12:01 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Xi Wang, and Yusaku 
> Sako.
> 
> 
> Bugs: AMBARI-17176
> https://issues.apache.org/jira/browse/AMBARI-17176
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need to include the Default Version Definition in the list of dropdown 
> Versions along with the discovered versions.
> Current experience: we only include Default if we believe we do not have 
> internet access
> New experience: include Default in the list regardless of internet or not.
> This is for backwards compatibility, and in the case where the user "just 
> does not know" their version (and therefore, needs Ambari to just figure it 
> out best it can).
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsCreateCtrl.js
>  5433bd6 
>   ambari-web/app/controllers/wizard/step1_controller.js 3af28de 
>   ambari-web/app/templates/wizard/step1.hbs 59ff21b 
> 
> Diff: https://reviews.apache.org/r/48568/diff/
> 
> 
> Testing
> ---
> 
> ambari-web:
> 28670 tests complete (24 seconds)
> 154 tests pending
> ambari-admin:
> Executed 71 of 71 SUCCESS (0.142 secs / 0.356 secs)
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>



Re: Review Request 48624: Ambari upgrade from 2.2.2 to 2.4.0 fails if cluster name was changed

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48624/#review137354
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 9:41 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48624/
> ---
> 
> (Updated June 13, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17188
> https://issues.apache.org/jira/browse/AMBARI-17188
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrade cluster_handle column to have current cluster_id if it is not null.
> Check null or empty configuration properties for custom capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
> 
> Diff: https://reviews.apache.org/r/48624/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48624: Ambari upgrade from 2.2.2 to 2.4.0 fails if cluster name was changed

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48624/#review137353
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 (line 2439)


Sorry, I read it too quickly, I thought these were Ambari usernames and 
passwords.


- Alejandro Fernandez


On June 13, 2016, 9:41 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48624/
> ---
> 
> (Updated June 13, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17188
> https://issues.apache.org/jira/browse/AMBARI-17188
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrade cluster_handle column to have current cluster_id if it is not null.
> Check null or empty configuration properties for custom capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
> 
> Diff: https://reviews.apache.org/r/48624/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48624: Ambari upgrade from 2.2.2 to 2.4.0 fails if cluster name was changed

2016-06-13 Thread Gaurav Nagar


> On June 13, 2016, 6:37 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 2440
> > 
> >
> > Move these properties out of the for-loop.
> > If they are empty, should be be an error instead of a "continue"

These are the instance properties, as we are iterating over instances these 
cannot be move out of for-loop. 
Ideally this should never happen. It can happen only when custom configured 
instance is in bad state. In this case, the view instance is not working before 
upgrade and we are skipping the upgrade for it instead of aborting upgrade.


- Gaurav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48624/#review137351
---


On June 13, 2016, 9:41 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48624/
> ---
> 
> (Updated June 13, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17188
> https://issues.apache.org/jira/browse/AMBARI-17188
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrade cluster_handle column to have current cluster_id if it is not null.
> Check null or empty configuration properties for custom capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
> 
> Diff: https://reviews.apache.org/r/48624/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48640: Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to SERVICE.ADMINISTRATOR role and above

2016-06-13 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48640/
---

(Updated June 13, 2016, 2:50 p.m.)


Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Nate Cole, and 
Vitalyi Brodetskyi.


Changes
---

updated test results


Bugs: AMBARI-17177
https://issues.apache.org/jira/browse/AMBARI-17177


Repository: ambari


Description
---

Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to the following roles:

* AMBARI.ADMINISTRATOR 
* CLUSTER.ADMINISTRATOR 
* CLUSTER.OPERATOR 
* SERVICE.ADMINISTRATOR

This is a DB change adding an authorization record to the `roleauthorization` 
table and relevant records for the different roles to the 
`permission_roleauthorization` table. 

The description/name of the `SERVICE.VIEW_OPERATIONAL_LOGS` authorization 
should be
```
View service operational logs
```


Diffs
-

  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
33ed7ed 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RoleAuthorizationDAO.java
 aa74224 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java
 ee948fe 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 3ee8bba 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java
 6b038f4 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 5016325 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fff1716 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql e7eff93 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b90c7da 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql b9181d2 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
c840b9c 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 742eaa3 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 87ef7fb 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog230Test.java
 d7e13ea 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 c221138 

Diff: https://reviews.apache.org/r/48640/diff/


Testing (updated)
---

Manually tested upgrade and new cluster creation

# Local test results:
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:16:17.022s
[INFO] Finished at: Mon Jun 13 09:41:19 EDT 2016
[INFO] Final Memory: 59M/1881M
[INFO] 

# Jenkins test results: 

Tests run: 4478, Failures: 0, Errors: 0, Skipped: 34

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 01:38 h
[INFO] Finished at: 2016-06-13T17:03:59+00:00
[INFO] Final Memory: 152M/527M
[INFO] 


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12809853/AMBARI-17177_trunk_01.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-admin ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7322//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7322//console


Thanks,

Robert Levas



Re: Review Request 48624: Ambari upgrade from 2.2.2 to 2.4.0 fails if cluster name was changed

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48624/#review137351
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 (line 2439)


Move these properties out of the for-loop.
If they are empty, should be be an error instead of a "continue"


- Alejandro Fernandez


On June 13, 2016, 9:41 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48624/
> ---
> 
> (Updated June 13, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17188
> https://issues.apache.org/jira/browse/AMBARI-17188
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrade cluster_handle column to have current cluster_id if it is not null.
> Check null or empty configuration properties for custom capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
> 
> Diff: https://reviews.apache.org/r/48624/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

2016-06-13 Thread Alejandro Fernandez


> On June 7, 2016, 6:49 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py,
> >  line 130
> > 
> >
> > Please consult with Jayush Luniya.
> > The stack already has a config for the stack root, so hdp_select (or 
> > distro select as it should be called) shouldn't be hardcoding paths.
> 
> Laszlo Puskas wrote:
> This patch is for 2.2-next branch that doesn't have those improvements. 
> (Tried to do this following one of your previous notes - seems that those 
> weren't backported to this branch)

Does this patch also need to go into trunk?


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136517
---


On June 13, 2016, 9:22 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 13, 2016, 9:22 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jayush Luniya, Sandor Magyari, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installation related information.
> 
> The patch only applies for 2.2-next versions.
> Based on the git logs the affected codebase has been refactored as part of 
> AMBARI-15329.
> (This addition should be ported to the newer versions isf applicable)
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>



Re: Review Request 48657: Allow option to skip duplicate URL checking when creating VDF (part 2)

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48657/#review137344
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 5:57 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48657/
> ---
> 
> (Updated June 13, 2016, 5:57 p.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17173
> https://issues.apache.org/jira/browse/AMBARI-17173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> My original patch had the for loop inside the check block, preventing an 
> addition to the critical Set.  Sigh.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  62568cf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
>  3bc4aec 
> 
> Diff: https://reviews.apache.org/r/48657/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>



Re: Review Request 48659: Fix Spark2 thriftserver Ambari definition bug

2016-06-13 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48659/#review137343
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 6:04 p.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48659/
> ---
> 
> (Updated June 13, 2016, 6:04 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17204
> https://issues.apache.org/jira/browse/AMBARI-17204
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix Spark2 thriftserver Ambari definition bug
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
> 
> Diff: https://reviews.apache.org/r/48659/diff/
> 
> 
> Testing
> ---
> 
> Unit test is done.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>



Review Request 48659: Fix Spark2 thriftserver Ambari definition bug

2016-06-13 Thread Saisai Shao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48659/
---

Review request for Ambari and Jayush Luniya.


Bugs: AMBARI-17204
https://issues.apache.org/jira/browse/AMBARI-17204


Repository: ambari


Description
---

Fix Spark2 thriftserver Ambari definition bug


Diffs
-

  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 

Diff: https://reviews.apache.org/r/48659/diff/


Testing
---

Unit test is done.


Thanks,

Saisai Shao



Re: Review Request 48657: Allow option to skip duplicate URL checking when creating VDF (part 2)

2016-06-13 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48657/#review137342
---


Ship it!




Ship It!

- Jonathan Hurley


On June 13, 2016, 1:57 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48657/
> ---
> 
> (Updated June 13, 2016, 1:57 p.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17173
> https://issues.apache.org/jira/browse/AMBARI-17173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> My original patch had the for loop inside the check block, preventing an 
> addition to the critical Set.  Sigh.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  62568cf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
>  3bc4aec 
> 
> Diff: https://reviews.apache.org/r/48657/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>



Re: Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/#review137339
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 (line 280)


Probably not a big deal since this is for upgrades, but you could just 
instantiate a single Json parser once and use it for all alerts. Might cut down 
on upgrade time if there are a lot of alerts to go through and change.



ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 (line 283)


Maybe make `connection_timeout` static (or at least declared once and 
re-used within the method?


- Jonathan Hurley


On June 13, 2016, 1:34 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48654/
> ---
> 
> (Updated June 13, 2016, 1:34 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17202
> https://issues.apache.org/jira/browse/AMBARI-17202
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When upgrading from earlier versions of Ambari, the alert definitions do not 
> include some of the new parameters / connection timeout values which were 
> added to the alert definitions. This is normally OK since the alert framework 
> has sensible defaults of the values are not specified.
> However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
> properly render/save the values.
> STR:
> Install Ambari 2.2.0
> Upgrade to Ambari 2.2.2
> You should now have alert definitions, such as DataNode Web UI, which do not 
> have a connection_timeout value.
> Navigate to the DataNode Web UI alert definition page and notice that you are 
> not able to update and save the "Connection Timeout" property correctly.
> Attachments
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  3ee8bba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  5016325 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c221138 
> 
> Diff: https://reviews.apache.org/r/48654/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Review Request 48657: Allow option to skip duplicate URL checking when creating VDF (part 2)

2016-06-13 Thread Nate Cole

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48657/
---

Review request for Ambari and Jonathan Hurley.


Bugs: AMBARI-17173
https://issues.apache.org/jira/browse/AMBARI-17173


Repository: ambari


Description
---

My original patch had the for loop inside the check block, preventing an 
addition to the critical Set.  Sigh.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 62568cf 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
 3bc4aec 

Diff: https://reviews.apache.org/r/48657/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole



Review Request 48655: ATLAS conf dir needs to be present in all ATLAS hook deployed hosts

2016-06-13 Thread Tom Beerbower

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48655/
---

Review request for Ambari, John Speidel and Nate Cole.


Bugs: AMBARI-17203
https://issues.apache.org/jira/browse/AMBARI-17203


Repository: ambari


Description
---

Atlas atlas-application.properties configuration is required by the Atlas hooks 
(Falcon, Storm, Sqoop and Hive).
Currently the atlas configuration is not being pushed to all the hook hosts 
which could lead to issues if any config is changed. The Atlas configuration 
changes needs to be pushed to all dependent hosts.


Diffs
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 441f0da 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
 67077c4 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
 c20523d 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 fea0635 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
 e78190f 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
 c154a16 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
 d18d820 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 eadbd4a 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
 6c3e91f 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/metainfo.xml 
1998131 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
ac1f9e7 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/metainfo.xml 
eb67d63 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/metainfo.xml 
3faadc0 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
1bcc09e 

Diff: https://reviews.apache.org/r/48655/diff/


Testing
---

Manual test deploy Atlas with all hook integration services on multi-host 
cluster.  Verify Atlas configuration locations.  Update Atlas configuration.  
Verify dependant service restart suggestion from Ambari.

mvn clean test


Thanks,

Tom Beerbower



Review Request 48654: Upgrading Ambari Causes Alert Fields Not To Save Thresholds

2016-06-13 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48654/
---

Review request for Ambari, Dmytro Sen and Jonathan Hurley.


Bugs: AMBARI-17202
https://issues.apache.org/jira/browse/AMBARI-17202


Repository: ambari


Description
---

When upgrading from earlier versions of Ambari, the alert definitions do not 
include some of the new parameters / connection timeout values which were added 
to the alert definitions. This is normally OK since the alert framework has 
sensible defaults of the values are not specified.
However, it was observed on Ambari 2.2.2 that the web client wasn't able to 
properly render/save the values.
STR:
Install Ambari 2.2.0
Upgrade to Ambari 2.2.2
You should now have alert definitions, such as DataNode Web UI, which do not 
have a connection_timeout value.
Navigate to the DataNode Web UI alert definition page and notice that you are 
not able to update and save the "Connection Timeout" property correctly.
Attachments


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 3ee8bba 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 5016325 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 c221138 

Diff: https://reviews.apache.org/r/48654/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi



Re: Review Request 48650: Add recently added logs to HadoopServiceConfig.json too

2016-06-13 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48650/#review137329
---


Ship it!




Ship It!

- Robert Nettleton


On June 13, 2016, 4:28 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48650/
> ---
> 
> (Updated June 13, 2016, 4:28 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17201
> https://issues.apache.org/jira/browse/AMBARI-17201
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Also fixed indentations.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/HadoopServiceConfig.json
>  829839f 
> 
> Diff: https://reviews.apache.org/r/48650/diff/
> 
> 
> Testing
> ---
> 
> Tested locally, now these logs are also visible on the Troubleshooting screen.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-13 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review137326
---



Thanks for providing this patch.

Most of the changes seem straightforward to me, but I'm concerned about 
attaching this check to the Ambari Agent HeartBeat handler.  My concern is that 
this could have performance implications later on.

I'd request that perhaps an alternate approach be considered, since there might 
be ways to implement this as a separate task thread in the TopologyManager, 
that could then add the required logging once the cluster deployment completes.

I think it makes sense to ask Sumit Mohanty to review this as well.  

Thanks.


ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
 (line 191)


I'm a little concerned that this approach could have performance issues 
down the road.

I'm wondering if it might be worth considering launching a separate task 
thread when the cluster is created by the TopologyManager, that can monitor the 
state of the LogicalRequest as the deployment progresses.

I would recommend asking Sumit to review this as well, since it's not clear 
to me that this is the approach that should be taken.


- Robert Nettleton


On June 11, 2016, 6:43 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 11, 2016, 6:43 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/HeartbeatProcessingFinishedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  77419d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  324a397 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Succeeded locally
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48650: Add recently added logs to HadoopServiceConfig.json too

2016-06-13 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48650/#review137321
---


Ship it!




Ship It!

- Oliver Szabo


On June 13, 2016, 4:28 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48650/
> ---
> 
> (Updated June 13, 2016, 4:28 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17201
> https://issues.apache.org/jira/browse/AMBARI-17201
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Also fixed indentations.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/HadoopServiceConfig.json
>  829839f 
> 
> Diff: https://reviews.apache.org/r/48650/diff/
> 
> 
> Testing
> ---
> 
> Tested locally, now these logs are also visible on the Troubleshooting screen.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Review Request 48650: Add recently added logs to HadoopServiceConfig.json too

2016-06-13 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48650/
---

Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Bugs: AMBARI-17201
https://issues.apache.org/jira/browse/AMBARI-17201


Repository: ambari


Description
---

Also fixed indentations.


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/HadoopServiceConfig.json
 829839f 

Diff: https://reviews.apache.org/r/48650/diff/


Testing
---

Tested locally, now these logs are also visible on the Troubleshooting screen.


Thanks,

Miklos Gergely



Review Request 48651: Add unit tests for Spark2 service definition

2016-06-13 Thread Saisai Shao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48651/
---

Review request for Ambari and Jayush Luniya.


Bugs: AMBARI-16864
https://issues.apache.org/jira/browse/AMBARI-16864


Repository: ambari


Description
---

Add unit tests for Spark2 service definition


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 7e85115 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
 2c19b88 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 8ad53da 
  ambari-server/src/test/python/stacks/2.5/SPARK2/test_job_history_server.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_client.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_service_check.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_thrift_server.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/configs/secured.json PRE-CREATION 
  
ambari-server/src/test/python/stacks/2.5/configs/spark2-job-history-server.json 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/configs/spark2_default.json 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.5/configs/spark2_thriftserver.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/48651/diff/


Testing
---

Local unit test is done.


Thanks,

Saisai Shao



Re: Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-13 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/#review137312
---


Ship it!




Ship It!

- Robert Levas


On June 13, 2016, 11:04 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48642/
> ---
> 
> (Updated June 13, 2016, 11:04 a.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17196
> https://issues.apache.org/jira/browse/AMBARI-17196
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When creating a cluster with kerberos, ambari-server.out contains 
> NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
> belong to logical tasks.
> NPE source is a Long->long conversion when logical task id is null when 
> AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.
> 
> The fix is a NPE check that skips the registerPhysicalTaskId method. I added 
> it also to the StartHostsTask class.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  00ecb98 
> 
> Diff: https://reviews.apache.org/r/48642/diff/
> 
> 
> Testing
> ---
> 
> Changes are done in private classes, so they cannot be accessed from unit 
> tests.
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-13 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/#review137308
---


Ship it!




Ship It!

- Oliver Szabo


On June 13, 2016, 3:04 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48642/
> ---
> 
> (Updated June 13, 2016, 3:04 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17196
> https://issues.apache.org/jira/browse/AMBARI-17196
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When creating a cluster with kerberos, ambari-server.out contains 
> NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
> belong to logical tasks.
> NPE source is a Long->long conversion when logical task id is null when 
> AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.
> 
> The fix is a NPE check that skips the registerPhysicalTaskId method. I added 
> it also to the StartHostsTask class.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  00ecb98 
> 
> Diff: https://reviews.apache.org/r/48642/diff/
> 
> 
> Testing
> ---
> 
> Changes are done in private classes, so they cannot be accessed from unit 
> tests.
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48643: Ambari uses too small a window for region server shutdown

2016-06-13 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48643/#review137306
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 13, 2016, 6:21 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48643/
> ---
> 
> (Updated June 13, 2016, 6:21 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-16914
> https://issues.apache.org/jira/browse/AMBARI-16914
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari seems to issue a formal shutdown to a Region server but quickly (30
> seconds) follows it up with SIGKILL. On a full loaded HBase system with about
> 200 regions per region server and active transaction flow, there is no way a
> RS can stop in 30 seconds. This has caused many issues in production including
> a memstore corruption. Why not use the shutdown script that comes with HBase?
> 
> 
> 
> 2016-05-24 15:36:19,191 - 
> Execute['/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config 
> /usr/hdp/current/hbase-regionserver/conf stop regionserver']
> {'only_if': 'ambari-sudo.sh -H -E test -f 
> /var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E 
> cat /var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1', 
> 'on_timeout': '! ( ambari-sudo.sh -H -E test -f 
> /var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E 
> cat /var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1 ) || 
> ambari-sudo.sh -H -E kill -9 `ambari-sudo.sh -H -E cat 
> /var/run/hbase/hbase-hbase-regionserver.pid`', 'timeout': 30, 'user': 'hbase'}
> 
> 2016-05-24 15:36:50,982 - Executing '! ( ambari-sudo.sh -H -E test -f 
> /var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E 
> cat /var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1 ) || 
> ambari-sudo.sh -H -E kill -9 `ambari-sudo.sh -H -E cat 
> /var/run/hbase/hbase-hbase-regionserver.pid`'. Reason: Execution of 
> 'ambari-sudo.sh su hbase -l -s /bin/bash -c 'export 
> PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/sbin:/usr/sbin:/bin:/usr/bin:/var/lib/ambari-agent'"'"'
>  ; /usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config 
> /usr/hdp/current/hbase-regionserver/conf stop regionserver'' was killed due 
> timeout after 30 seconds
> 2016-05-24 15:36:51,053 - 
> File['/var/run/hbase/hbase-hbase-regionserver.pid']
> {'action': ['delete']}
> 
> 2016-05-24 15:36:51,054 - Deleting 
> File['/var/run/hbase/hbase-hbase-regionserver.pid'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b40923a 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase_service.py
>  4d0d7f3 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py
>  21b491d 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
>  eaee3cf 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py
>  a6904f6 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  05bad1c 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_service.py
>  0a42cda 
>   
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 
> ac8b153 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 04aa828 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 02f982e 
> 
> Diff: https://reviews.apache.org/r/48643/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>



Re: Review Request 48641: yarncli throws log4j error "FileNotFoundException : /grid/0/log/yarn/hrt_qa/rm-audit.log"

2016-06-13 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48641/#review137305
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 13, 2016, 2:31 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48641/
> ---
> 
> (Updated Июнь 13, 2016, 2:31 п.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17148
> https://issues.apache.org/jira/browse/AMBARI-17148
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> yarn cli are throwing "FileNotFoundException : /grid/0/log/yarn/hrt_qa/rm-
> audit.log" log4j errors.  
> Find yarn configuration at  yarn-ha/conf/hadoop/> and service logs at
> 
> 
> 
> 
> 
> 2016-05-05 
> 12:15:15,412|beaver.machine|INFO|32706|140642029037312|MainThread|RUNNING: 
> /usr/hdp/current/hadoop-yarn-client/bin/yarn application -list -appStates 
> RUNNING
> 2016-05-05 
> 12:15:17,162|beaver.machine|INFO|32706|140642029037312|MainThread|log4j: 
> ERROR setFile(null,true) call failed .
> 2016-05-05 
> 12:15:17,166|beaver.machine|INFO|32706|140642029037312|MainThread|java.io. 
> FileNotFoundException : /grid/0/log/yarn/hrt_qa/nm-audit.log (No such file or 
> directory)
> 2016-05-05 
> 12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> java.io.FileOutputStream.open0(Native Method)
> 2016-05-05 
> 12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> java.io.FileOutputStream.open(FileOutputStream.java:270)
> 2016-05-05 
> 12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> java.io.FileOutputStream.(FileOutputStream.java:213)
> 2016-05-05 
> 12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> java.io.FileOutputStream.(FileOutputStream.java:133)
> 2016-05-05 
> 12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
> 2016-05-05 
> 12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
> 2016-05-05 
> 12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
> 2016-05-05 
> 12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
> 2016-05-05 
> 12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
> 2016-05-05 
> 12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
> 2016-05-05 
> 12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
> 2016-05-05 
> 12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> 2016-05-05 
> 12:15:17,174|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> 2016-05-05 
> 12:15:17,174|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> 2016-05-05 
> 12:15:17,178|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> 2016-05-05 
> 12:15:17,179|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> 2016-05-05 
> 12:15:17,179|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.LogManager.(LogManager.java:127)
> 2016-05-05 
> 12:15:17,181|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.log4j.Logger.getLogger(Logger.java:104)
> 2016-05-05 
> 12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262)
> 2016-05-05 
> 12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:108)
> 2016-05-05 
> 12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
> 

Review Request 48643: Ambari uses too small a window for region server shutdown

2016-06-13 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48643/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-16914
https://issues.apache.org/jira/browse/AMBARI-16914


Repository: ambari


Description
---

Ambari seems to issue a formal shutdown to a Region server but quickly (30
seconds) follows it up with SIGKILL. On a full loaded HBase system with about
200 regions per region server and active transaction flow, there is no way a
RS can stop in 30 seconds. This has caused many issues in production including
a memstore corruption. Why not use the shutdown script that comes with HBase?



2016-05-24 15:36:19,191 - 
Execute['/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config 
/usr/hdp/current/hbase-regionserver/conf stop regionserver']
{'only_if': 'ambari-sudo.sh -H -E test -f 
/var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E cat 
/var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1', 'on_timeout': '! 
( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-regionserver.pid && 
ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-regionserver.pid` 
>/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `ambari-sudo.sh -H -E cat 
/var/run/hbase/hbase-hbase-regionserver.pid`', 'timeout': 30, 'user': 'hbase'}

2016-05-24 15:36:50,982 - Executing '! ( ambari-sudo.sh -H -E test -f 
/var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E cat 
/var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1 ) || 
ambari-sudo.sh -H -E kill -9 `ambari-sudo.sh -H -E cat 
/var/run/hbase/hbase-hbase-regionserver.pid`'. Reason: Execution of 
'ambari-sudo.sh su hbase -l -s /bin/bash -c 'export 
PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/sbin:/usr/sbin:/bin:/usr/bin:/var/lib/ambari-agent'"'"'
 ; /usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config 
/usr/hdp/current/hbase-regionserver/conf stop regionserver'' was killed due 
timeout after 30 seconds
2016-05-24 15:36:51,053 - 
File['/var/run/hbase/hbase-hbase-regionserver.pid']
{'action': ['delete']}

2016-05-24 15:36:51,054 - Deleting 
File['/var/run/hbase/hbase-hbase-regionserver.pid'


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
 b40923a 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase_service.py
 4d0d7f3 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py
 21b491d 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
 eaee3cf 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py
 a6904f6 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 05bad1c 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_service.py
 0a42cda 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 
ac8b153 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 04aa828 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 02f982e 

Diff: https://reviews.apache.org/r/48643/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk



Re: Review Request 48636: Zeppelin Views are not working with Custom and Remote cluster view configuration

2016-06-13 Thread Rohit Choudhary

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48636/#review137303
---


Ship it!




Ship It!

- Rohit Choudhary


On June 13, 2016, 1:22 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48636/
> ---
> 
> (Updated June 13, 2016, 1:22 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17191
> https://issues.apache.org/jira/browse/AMBARI-17191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Two patches attached in the jira issue
> AMBARI-17191_branch-2.4_v1.patch
> AMBARI-17191_trunk_v1.patch
> 
> 
> - fix zeppelin view
> 
> 
> Diffs
> -
> 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServiceCheck.java
>  e80f884 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServlet.java
>  f497599 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/index.jsp 493473e 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/web.xml 2ca5664 
>   contrib/views/zeppelin/src/main/resources/view.xml 3c5c5cf 
> 
> Diff: https://reviews.apache.org/r/48636/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-13 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/
---

Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
Sebastian Toader.


Bugs: AMBARI-17196
https://issues.apache.org/jira/browse/AMBARI-17196


Repository: ambari


Description
---

When creating a cluster with kerberos, ambari-server.out contains 
NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
belong to logical tasks.
NPE source is a Long->long conversion when logical task id is null when 
AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.

The fix is a NPE check that skips the registerPhysicalTaskId method. I added it 
also to the StartHostsTask class.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 
00ecb98 

Diff: https://reviews.apache.org/r/48642/diff/


Testing
---

Changes are done in private classes, so they cannot be accessed from unit tests.


Thanks,

Daniel Gergely



Re: Review Request 48636: Zeppelin Views are not working with Custom and Remote cluster view configuration

2016-06-13 Thread Gaurav Nagar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48636/#review137301
---


Ship it!




Ship It!

- Gaurav Nagar


On June 13, 2016, 1:22 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48636/
> ---
> 
> (Updated June 13, 2016, 1:22 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17191
> https://issues.apache.org/jira/browse/AMBARI-17191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Two patches attached in the jira issue
> AMBARI-17191_branch-2.4_v1.patch
> AMBARI-17191_trunk_v1.patch
> 
> 
> - fix zeppelin view
> 
> 
> Diffs
> -
> 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServiceCheck.java
>  e80f884 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServlet.java
>  f497599 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/index.jsp 493473e 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/web.xml 2ca5664 
>   contrib/views/zeppelin/src/main/resources/view.xml 3c5c5cf 
> 
> Diff: https://reviews.apache.org/r/48636/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Review Request 48641: yarncli throws log4j error "FileNotFoundException : /grid/0/log/yarn/hrt_qa/rm-audit.log"

2016-06-13 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48641/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17148
https://issues.apache.org/jira/browse/AMBARI-17148


Repository: ambari


Description
---

yarn cli are throwing "FileNotFoundException : /grid/0/log/yarn/hrt_qa/rm-
audit.log" log4j errors.  
Find yarn configuration at  and service logs at





2016-05-05 
12:15:15,412|beaver.machine|INFO|32706|140642029037312|MainThread|RUNNING: 
/usr/hdp/current/hadoop-yarn-client/bin/yarn application -list -appStates 
RUNNING
2016-05-05 
12:15:17,162|beaver.machine|INFO|32706|140642029037312|MainThread|log4j: ERROR 
setFile(null,true) call failed .
2016-05-05 
12:15:17,166|beaver.machine|INFO|32706|140642029037312|MainThread|java.io. 
FileNotFoundException : /grid/0/log/yarn/hrt_qa/nm-audit.log (No such file or 
directory)
2016-05-05 
12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
java.io.FileOutputStream.open0(Native Method)
2016-05-05 
12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
java.io.FileOutputStream.open(FileOutputStream.java:270)
2016-05-05 
12:15:17,167|beaver.machine|INFO|32706|140642029037312|MainThread|at 
java.io.FileOutputStream.(FileOutputStream.java:213)
2016-05-05 
12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
java.io.FileOutputStream.(FileOutputStream.java:133)
2016-05-05 
12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
2016-05-05 
12:15:17,168|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
2016-05-05 
12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
2016-05-05 
12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
2016-05-05 
12:15:17,169|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
2016-05-05 
12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
2016-05-05 
12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
2016-05-05 
12:15:17,170|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
2016-05-05 
12:15:17,174|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
2016-05-05 
12:15:17,174|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
2016-05-05 
12:15:17,178|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
2016-05-05 
12:15:17,179|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
2016-05-05 
12:15:17,179|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.LogManager.(LogManager.java:127)
2016-05-05 
12:15:17,181|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.log4j.Logger.getLogger(Logger.java:104)
2016-05-05 
12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262)
2016-05-05 
12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:108)
2016-05-05 
12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2016-05-05 
12:15:17,182|beaver.machine|INFO|32706|140642029037312|MainThread|at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2016-05-05 
12:15:17,183|beaver.machine|INFO|32706|140642029037312|MainThread|at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2016-05-05 
12:15:17,183|beaver.machine|INFO|32706|140642029037312|MainThread|at 
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2016-05-05 

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-13 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/
---

(Updated June 13, 2016, 2:21 p.m.)


Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.


Bugs: AMBARI-17180
https://issues.apache.org/jira/browse/AMBARI-17180


Repository: ambari


Description
---

base_alert.py puts a warning into the log if there are properties referenced in 
the HA nameservice or the alias which are not present in the configuration. The 
absence of these properties is an indicator that the HA is not enabled, it is 
not a cause for warning.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
  ambari-agent/src/test/python/ambari_agent/TestAlerts.py e114daa 

Diff: https://reviews.apache.org/r/48589/diff/


Testing (updated)
---

Tested on local cluster works fine. Unit tests are running fine too.


Thanks,

Miklos Gergely



Re: Review Request 48414: AMBARI-17118 Incorrect formated external url in ranger configuration - causes Namenode startup failure

2016-06-13 Thread Mugdha Varadkar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48414/
---

(Updated June 13, 2016, 2:15 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Sumit Mohanty, 
Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17118
https://issues.apache.org/jira/browse/AMBARI-17118


Repository: ambari


Description
---

Ranger Admin url when configured with trailing slash causes Namenode startup 
failure


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions.py
 48ae225 
  
ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions_v2.py
 cfdd6f7 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
 260f018 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
 e5faf4b 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 05bad1c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 9af87d4 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 fea0635 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 09878ba 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 4d30f55 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 29ac561 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
 2ba3794 
  ambari-server/src/main/resources/common-services/RANGER/0.6.0/metainfo.xml 
6f1460a 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
 cbe2a31 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 17f71fb 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 978ad92 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 29fb3c1 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
36fe066 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
53cdcfe 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py c077a9f 

Diff: https://reviews.apache.org/r/48414/diff/


Testing
---

Tested Ranger Installation on centos6


Thanks,

Mugdha Varadkar



Re: Review Request 48485: AMBARI-17044 Optimize LogSearch Solr for cache setting and default values

2016-06-13 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48485/#review137298
---


Ship it!




Ship It!

- Miklos Gergely


On June 13, 2016, 10:24 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48485/
> ---
> 
> (Updated June 13, 2016, 10:24 a.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17044
> https://issues.apache.org/jira/browse/AMBARI-17044
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed default settings to not to wait for cache to be fully build before 
> addressing request
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2
>  c0d8b2f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  c870edc 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  a06a900 
> 
> Diff: https://reviews.apache.org/r/48485/diff/
> 
> 
> Testing
> ---
> 
> Create 3 node Hadoop cluster and tested the change
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Re: Review Request 48485: AMBARI-17044 Optimize LogSearch Solr for cache setting and default values

2016-06-13 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48485/#review137297
---


Ship it!




Ship It!

- Oliver Szabo


On June 13, 2016, 10:24 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48485/
> ---
> 
> (Updated June 13, 2016, 10:24 a.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17044
> https://issues.apache.org/jira/browse/AMBARI-17044
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed default settings to not to wait for cache to be fully build before 
> addressing request
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2
>  c0d8b2f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  c870edc 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  a06a900 
> 
> Diff: https://reviews.apache.org/r/48485/diff/
> 
> 
> Testing
> ---
> 
> Create 3 node Hadoop cluster and tested the change
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Re: Review Request 48523: AMBARI-17145 Unformatted configs remain in zeppelin-env.sh

2016-06-13 Thread Masahiro Tanaka


> On June 10, 2016, 9:35 p.m., Jayush Luniya wrote:
> > Ship It!

Thank you! Could you commit this ?


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48523/#review137073
---


On June 9, 2016, 11:49 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48523/
> ---
> 
> (Updated June 9, 2016, 11:49 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jayush Luniya, Pallav 
> Kulshreshtha, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17145
> https://issues.apache.org/jira/browse/AMBARI-17145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we just installed Zeppelin Notebook, /etc/zeppelin/conf/zeppelin-env.sh 
> is like this
> 
> ```
> # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to 
> use local mode
> export MASTER=yarn-client
> export SPARK_YARN_JAR={spark_jar_dir}/zeppelin-spark-0.5.5-SNAPSHOT.jar
> 
> (snip)
> ```
> 
> It maybe 
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  doesn't import from resource_management.libraries.functions.format import 
> format
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  a4efd72 
> 
> Diff: https://reviews.apache.org/r/48523/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?
> 
> Masahiro Tanaka wrote:
> Accoding to the `man systemctl`:
> ```
> is-active PATTERN...
> Check whether any of the specified units are active (i.e. running). 
> Returns an exit
> code 0 if at least one is active, or non-zero otherwise. Unless 
> --quiet is specified,
> this will also print the current unit state to standard output.
> ```
> 
> We can expect that if at least one unit is active, then return code (exit 
> code) equals 0.
> 
> BUT there is a bug in *systemd*. See [this PR on 
> GitHub](https://github.com/systemd/systemd/pull/2430)
> 
> So the actual behavior on CentOS 7.2 is below:
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> # systemctl is-active iptables firewalld
> inactive
> active
> # echo $?
> 3
> # systemctl is-active firewalld iptables
> active
> inactive
> # echo $?
> 3
> ```
> 
> That's why I checked if stdout includes the string `active` in my code.
> 
> Andrew Onischuk wrote:
> What will happen is firewalld or iptables is absent as a services, could 
> that be the case for centos7?
> 
> Andrew Onischuk wrote:
> Basically try doing something like this:
> 
> # systemctl is-active firewalld somethingunknown
> # echo $?
> 
> Masahiro Tanaka wrote:
> Here
> ```
> # systemctl is-active firewalld somethingunknown
> active
> unknown
> # echo $?
> 3
> # systemctl is-active somethingunknown firewalld
> unknown
> active
> # echo $?
> 3
> ```

Thanks!


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137293
---


Ship it!




Ship It!

- Andrew Onischuk


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Masahiro Tanaka


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?
> 
> Masahiro Tanaka wrote:
> Accoding to the `man systemctl`:
> ```
> is-active PATTERN...
> Check whether any of the specified units are active (i.e. running). 
> Returns an exit
> code 0 if at least one is active, or non-zero otherwise. Unless 
> --quiet is specified,
> this will also print the current unit state to standard output.
> ```
> 
> We can expect that if at least one unit is active, then return code (exit 
> code) equals 0.
> 
> BUT there is a bug in *systemd*. See [this PR on 
> GitHub](https://github.com/systemd/systemd/pull/2430)
> 
> So the actual behavior on CentOS 7.2 is below:
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> # systemctl is-active iptables firewalld
> inactive
> active
> # echo $?
> 3
> # systemctl is-active firewalld iptables
> active
> inactive
> # echo $?
> 3
> ```
> 
> That's why I checked if stdout includes the string `active` in my code.
> 
> Andrew Onischuk wrote:
> What will happen is firewalld or iptables is absent as a services, could 
> that be the case for centos7?
> 
> Andrew Onischuk wrote:
> Basically try doing something like this:
> 
> # systemctl is-active firewalld somethingunknown
> # echo $?
> 
> Masahiro Tanaka wrote:
> Here
> ```
> # systemctl is-active firewalld somethingunknown
> active
> unknown
> # echo $?
> 3
> # systemctl is-active somethingunknown firewalld
> unknown
> active
> # echo $?
> 3
> ```
> 
> Andrew Onischuk wrote:
> Thanks!

And the exact environment is this
```
# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
# systemctl --version
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP 
+GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
```


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Masahiro Tanaka


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?
> 
> Masahiro Tanaka wrote:
> Accoding to the `man systemctl`:
> ```
> is-active PATTERN...
> Check whether any of the specified units are active (i.e. running). 
> Returns an exit
> code 0 if at least one is active, or non-zero otherwise. Unless 
> --quiet is specified,
> this will also print the current unit state to standard output.
> ```
> 
> We can expect that if at least one unit is active, then return code (exit 
> code) equals 0.
> 
> BUT there is a bug in *systemd*. See [this PR on 
> GitHub](https://github.com/systemd/systemd/pull/2430)
> 
> So the actual behavior on CentOS 7.2 is below:
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> # systemctl is-active iptables firewalld
> inactive
> active
> # echo $?
> 3
> # systemctl is-active firewalld iptables
> active
> inactive
> # echo $?
> 3
> ```
> 
> That's why I checked if stdout includes the string `active` in my code.
> 
> Andrew Onischuk wrote:
> What will happen is firewalld or iptables is absent as a services, could 
> that be the case for centos7?
> 
> Andrew Onischuk wrote:
> Basically try doing something like this:
> 
> # systemctl is-active firewalld somethingunknown
> # echo $?

Here
```
# systemctl is-active firewalld somethingunknown
active
unknown
# echo $?
3
# systemctl is-active somethingunknown firewalld
unknown
active
# echo $?
3
```


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?
> 
> Masahiro Tanaka wrote:
> Accoding to the `man systemctl`:
> ```
> is-active PATTERN...
> Check whether any of the specified units are active (i.e. running). 
> Returns an exit
> code 0 if at least one is active, or non-zero otherwise. Unless 
> --quiet is specified,
> this will also print the current unit state to standard output.
> ```
> 
> We can expect that if at least one unit is active, then return code (exit 
> code) equals 0.
> 
> BUT there is a bug in *systemd*. See [this PR on 
> GitHub](https://github.com/systemd/systemd/pull/2430)
> 
> So the actual behavior on CentOS 7.2 is below:
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> # systemctl is-active iptables firewalld
> inactive
> active
> # echo $?
> 3
> # systemctl is-active firewalld iptables
> active
> inactive
> # echo $?
> 3
> ```
> 
> That's why I checked if stdout includes the string `active` in my code.
> 
> Andrew Onischuk wrote:
> What will happen is firewalld or iptables is absent as a services, could 
> that be the case for centos7?

Basically try doing something like this:

# systemctl is-active firewalld somethingunknown
# echo $?


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?
> 
> Masahiro Tanaka wrote:
> Accoding to the `man systemctl`:
> ```
> is-active PATTERN...
> Check whether any of the specified units are active (i.e. running). 
> Returns an exit
> code 0 if at least one is active, or non-zero otherwise. Unless 
> --quiet is specified,
> this will also print the current unit state to standard output.
> ```
> 
> We can expect that if at least one unit is active, then return code (exit 
> code) equals 0.
> 
> BUT there is a bug in *systemd*. See [this PR on 
> GitHub](https://github.com/systemd/systemd/pull/2430)
> 
> So the actual behavior on CentOS 7.2 is below:
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> # systemctl is-active iptables firewalld
> inactive
> active
> # echo $?
> 3
> # systemctl is-active firewalld iptables
> active
> inactive
> # echo $?
> 3
> ```
> 
> That's why I checked if stdout includes the string `active` in my code.

What will happen is firewalld or iptables is absent as a services, could that 
be the case for centos7?


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Masahiro Tanaka


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```
> 
> Andrew Onischuk wrote:
> Oh, I see.
> 
> So:
> if first  is active and second is inactive what is the return code of 
> command?
> if second is active and first is  inactive what is the return code of 
> command?

Accoding to the `man systemctl`:
```
is-active PATTERN...
Check whether any of the specified units are active (i.e. running). Returns 
an exit
code 0 if at least one is active, or non-zero otherwise. Unless --quiet is 
specified,
this will also print the current unit state to standard output.
```

We can expect that if at least one unit is active, then return code (exit code) 
equals 0.

BUT there is a bug in *systemd*. See [this PR on 
GitHub](https://github.com/systemd/systemd/pull/2430)

So the actual behavior on CentOS 7.2 is below:
```
# systemctl is-active iptables firewalld
inactive
active
# systemctl is-active iptables
inactive
# systemctl is-active firewalld
active
# systemctl is-active iptables firewalld
inactive
active
# echo $?
3
# systemctl is-active firewalld iptables
active
inactive
# echo $?
3
```

That's why I checked if stdout includes the string `active` in my code.


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing.
> Acutually I didn't remove the check for firewalld. `systemctl is-active` 
> can take multiple arguments.
> This is a sample output on CentOS7.2
> 
> ```
> # systemctl is-active iptables firewalld
> inactive
> active
> # systemctl is-active iptables
> inactive
> # systemctl is-active firewalld
> active
> ```

Oh, I see.

So:
if first  is active and second is inactive what is the return code of command?
if second is active and first is  inactive what is the return code of command?


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Review Request 48636: Zeppelin Views are not working with Custom and Remote cluster view configuration

2016-06-13 Thread Renjith Kamath

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48636/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Jayush Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17191
https://issues.apache.org/jira/browse/AMBARI-17191


Repository: ambari


Description
---

Two patches attached in the jira issue
AMBARI-17191_branch-2.4_v1.patch
AMBARI-17191_trunk_v1.patch


- fix zeppelin view


Diffs
-

  
contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServiceCheck.java
 e80f884 
  
contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServlet.java
 f497599 
  contrib/views/zeppelin/src/main/resources/WEB-INF/index.jsp 493473e 
  contrib/views/zeppelin/src/main/resources/WEB-INF/web.xml 2ca5664 
  contrib/views/zeppelin/src/main/resources/view.xml 3c5c5cf 

Diff: https://reviews.apache.org/r/48636/diff/


Testing
---

Manually tested on CentOS


Thanks,

Renjith Kamath



Re: Review Request 48557: Fixed implementation of on-ambari-upgrade support. Patch 2: add logic for ambari-upgrade

2016-06-13 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48557/
---

(Updated June 13, 2016, 4:22 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

Added unit tests, javadoc, some other fixes.


Bugs: AMBARI-17112
https://issues.apache.org/jira/browse/AMBARI-17112


Repository: ambari


Description
---

Patch implements logic for on-ambari-upgrade attributes: add, update, delete


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
 de5147d 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
c570ab3 
  
ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 f6791ee 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 3ee8bba 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-site.xml
 6793e4e 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 fd68ae0 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-startup.properties.xml
 aacf10a 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml
 4e56084 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/core-site.xml
 7f67b8a 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml
 274163e 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 b0f36e7 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
 444b8b4 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml
 cfdd989 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
 7f474f6 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml
 f3a6bcf 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml
 47c900e 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml
 a309fa4 
  
ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-site.xml
 2d0047c 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/ranger-env.xml
 5082277 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
 121a797 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
 ae56f8b 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
 4317cfa 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
 7eb78e5 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
 a747dde 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
 35910ee 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
 6eb312f 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml
 9870b8b 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
 633cbda 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
 daacdee 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
 21cd096 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
 a8f1584 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
 d7c87e9 
  
ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
 7aa18cc 
  
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
 fcaecaa 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
 85a1eba 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/capacity-scheduler.xml
 b4fedd8 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-site.xml
 b1b30aa 
  ambari-server/src/main/resources/configuration-schema.xsd 6716dd5 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/configuration/hive-site.xml
 8dc02c5 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/configuration/oozie-site.xml
 7e746c0 
  

Re: Review Request 48634: File browser : File preview show first character only when ssl is enabled.

2016-06-13 Thread Gaurav Nagar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48634/
---

(Updated June 13, 2016, 1:18 p.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17194
https://issues.apache.org/jira/browse/AMBARI-17194


Repository: ambari


Description
---

Using IOUtils.read to read from file.


Diffs (updated)
-

  
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java
 3585516 

Diff: https://reviews.apache.org/r/48634/diff/


Testing
---

Manually Tested


Thanks,

Gaurav Nagar



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Masahiro Tanaka


> On June 13, 2016, 1:03 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/firewall.py, line 123
> > 
> >
> > Can you please explain why did we remove the check for firewalld?

Thank you for reviewing.
Acutually I didn't remove the check for firewalld. `systemctl is-active` can 
take multiple arguments.
This is a sample output on CentOS7.2

```
# systemctl is-active iptables firewalld
inactive
active
# systemctl is-active iptables
inactive
# systemctl is-active firewalld
active
```


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Review Request 48634: File browser : File preview show first character only when ssl is enabled.

2016-06-13 Thread Gaurav Nagar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48634/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17194
https://issues.apache.org/jira/browse/AMBARI-17194


Repository: ambari


Description
---

Using IOUtils.read to read from file.


Diffs
-

  
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java
 3585516 

Diff: https://reviews.apache.org/r/48634/diff/


Testing
---

Manually Tested


Thanks,

Gaurav Nagar



Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-06-13 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review137283
---




ambari-common/src/main/python/ambari_commons/firewall.py (line 123)


Can you please explain why did we remove the check for firewalld?


- Andrew Onischuk


On June 7, 2016, 11:11 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated June 7, 2016, 11:11 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Florian Barca, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48549: AMBARI-17165 Handle Java patches execution during Ranger upgrade

2016-06-13 Thread Mugdha Varadkar


> On June 10, 2016, 5:40 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml,
> >  line 889
> > 
> >
> > hosts="all" is the default, you can remove it.
> > sequential="true" isn't needed since it is only useful when two 
> > consecutive tasks need to run one after the other even though they are on 
> > different hosts.
> > 
> > Why is there a stop and start in post-upgrade?
> > What are you trying to achieve here?
> > 
> > Pre-upgrade is any steps to execute before restarting the component, 
> > Post-upgrade is after restarting. So if there are any other stop-start 
> > steps, it means the component is being restarted twice!
> 
> Nate Cole wrote:
> I agree with Alejandro here, it looks like you're stopping all admins, 
> java-patching only one, then starting them all.  They should all need 
> patching, no?  And if that's the case that can all be handled on the python 
> side with nothing but restart for the component in the UP.
> 
> Jonathan Hurley wrote:
> I think that stopping all and running the patch on 1 is correct. The 
> patch executes `db_setup.py -javapatch`, so I'd expect that since it's done 
> on the database that you'd only want it done once. But you'd also want all of 
> them down.
> 
> With that said, I still don't understand why this is being done 
> post-upgrade.

Yes db_setup.py -javapatch needs to run only at one host.

The reason behind this addition is java patches failures: when we try to run 
Java patches with Ranger services started. That's why I have stopped service 
(from all hosts, in case of HA) before starting to execute Java patches. 

It needs to be a part of post upgrade, because it needs to be applied only on 
one host also it needs to have config properties of new version.


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48549/#review137042
---


On June 10, 2016, 12:47 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48549/
> ---
> 
> (Updated June 10, 2016, 12:47 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17165
> https://issues.apache.org/jira/browse/AMBARI-17165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Revisiting execution of java patches during upgrade for Ranger Service
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
>  4fd5801 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
>  272a3cc 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> b0cff68 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> 0b72254 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
>  111b432 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
>  9365646 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  f40f760 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 712241b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 4187d64 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> ea5ff5a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
>  e83b54b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  7fb03dc 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> 4065e87 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 7f988e3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  460e6b3 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 6ce4c81 
> 
> Diff: https://reviews.apache.org/r/48549/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger upgrade from stack 2.4 to 2.5
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>



Re: Review Request 48540: Ubuntu 16, Hive Metastore Start failed

2016-06-13 Thread Myroslav Papirkovskyy

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48540/#review137277
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Червень 10, 2016, 1:23 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48540/
> ---
> 
> (Updated Червень 10, 2016, 1:23 після полудня)
> 
> 
> Review request for Ambari and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-17161
> https://issues.apache.org/jira/browse/AMBARI-17161
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> /var/lib/ambari-agent/tmp/addMysqlUser.sh didn't add mysql user.
> 
> 
> 
> + echo 'Adding user hive@% and removing users with empty name'
> Adding user hive@% and removing users with empty name
> + /var/lib/ambari-agent/ambari-sudo.sh su mysql -s /bin/bash - -c 'mysql 
> -u root -e "CREATE USER '\''hive'\''@'\''%'\'' IDENTIFIED BY '\''h'\'';"'
> ERROR 1698 (28000): Access denied for user 'root'@'localhost'
> + /var/lib/ambari-agent/ambari-sudo.sh su mysql -s /bin/bash - -c 'mysql 
> -u root -e "GRANT ALL PRIVILEGES ON *.* TO '\''hive'\''@'\''%'\'';"'
> ERROR 1698 (28000): Access denied for user 'root'@'localhost'
> + /var/lib/ambari-agent/ambari-sudo.sh su mysql -s /bin/bash - -c 'mysql 
> -u root -e "DELETE FROM mysql.user WHERE user='\'''\'';"'
> ERROR 1698 (28000): Access denied for user 'root'@'localhost'
> + /var/lib/ambari-agent/ambari-sudo.sh su mysql -s /bin/bash - -c 'mysql 
> -u root -e "flush privileges;"'
> ERROR 1698 (28000): Access denied for user 'root'@'localhost'
> + /var/lib/ambari-agent/ambari-sudo.sh service mysql stop
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/addMysqlUser.sh
>  36ed58f 
> 
> Diff: https://reviews.apache.org/r/48540/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>



Re: Review Request 48490: Last button in the log search pagination panel does not take the user to the last page and few more fixes

2016-06-13 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48490/#review137275
---


Ship it!




Ship It!

- Oliver Szabo


On June 13, 2016, 6:36 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48490/
> ---
> 
> (Updated June 13, 2016, 6:36 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17140
> https://issues.apache.org/jira/browse/AMBARI-17140
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains
> 1) Last button in the log search pagination panel does not take the user to 
> the last page.
> 2) Solr warming feature added.
> 3) Suffix in columns mapping issue fixed.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
>  917956f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
>  53e386e 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
>  ca63671 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/MgrBase.java
>  1d069d3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigMgr.java
>  3b23f2a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
>  4647c7b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
>  92bfb01 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
>  1e107ed 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/ConfigUtil.java
>  44d184d 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/view/VUserConfig.java
>  55ec1c0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties
>  25b4f1a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/META-INF/applicationContext.xml
>  6a73d60 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collections/BaseCollection.js
>  24b6974 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Intro.js
>  869a1d4 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js
>  5f93c5a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dialog/ApplySearchFilterView.js
>  4babebc 
> 
> Diff: https://reviews.apache.org/r/48490/diff/
> 
> 
> Testing
> ---
> 
> Setup Logsearch on 2 node cluster and tested the above feature.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>



Re: Review Request 48622: AMBARI-17136: If Solr is down or not ready, then LogFeeder to should retry

2016-06-13 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48622/#review137274
---


Ship it!




Ship It!

- Oliver Szabo


On June 13, 2016, 9:28 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48622/
> ---
> 
> (Updated June 13, 2016, 9:28 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17136
> https://issues.apache.org/jira/browse/AMBARI-17136
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Uploaded new patch based on feedback from previous review board requests
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
> 
> Diff: https://reviews.apache.org/r/48622/diff/
> 
> 
> Testing
> ---
> 
> Tested on 3 node hadoop cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Re: Review Request 48622: AMBARI-17136: If Solr is down or not ready, then LogFeeder to should retry

2016-06-13 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48622/#review137273
---


Ship it!




Ship It!

- Miklos Gergely


On June 13, 2016, 9:28 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48622/
> ---
> 
> (Updated June 13, 2016, 9:28 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17136
> https://issues.apache.org/jira/browse/AMBARI-17136
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Uploaded new patch based on feedback from previous review board requests
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
> 
> Diff: https://reviews.apache.org/r/48622/diff/
> 
> 
> Testing
> ---
> 
> Tested on 3 node hadoop cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Review Request 48629: AMBARI-17189 : Change in Atlas authorization from class based to value based

2016-06-13 Thread Gautam Borad

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48629/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Madhan 
Neethiraj, Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17189
https://issues.apache.org/jira/browse/AMBARI-17189


Repository: ambari


Description
---

Need to update values of atlas.authorizer.impl from ::
  atlas.authorizer.impl = org.apache.atlas.authorize.SimpleAtlasAuthorizer 
(Simple Authorization)
  atlas.authorizer.impl = 
org.apache.ranger.authorization.atlas.authorizer.RangerAtlasAuthorizer (Ranger 
Authorization)
  
To 
  
  atlas.authorizer.impl = simple (Simple Authorization)
  atlas.authorizer.impl = ranger (Ranger Authorization)


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
53cdcfe 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 4e6dcda 

Diff: https://reviews.apache.org/r/48629/diff/


Testing
---

Verified enable / disable of Ranger Atlas Plugin.


Thanks,

Gautam Borad



Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Jeff Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 13, 2016, 10:45 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description (updated)
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang



Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Jeff Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 13, 2016, 10:28 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

address comments and rebase patch


Summary (updated)
-

[AMBARI-16920] Follow up issue for Spark2 stack definition


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang



Review Request 48628: AMBARI-17184: HBase doesn't start because of lacking of variable

2016-06-13 Thread Masahiro Tanaka

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48628/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sid Wagle.


Bugs: AMBARI-17184
https://issues.apache.org/jira/browse/AMBARI-17184


Repository: ambari


Description
---

When we wanted to restart HBase, we got an error
```
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
 line 159, in 
HbaseMaster().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 686, in restart
self.start(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
 line 86, in start
self.configure(env) # for security
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
 line 41, in configure
hbase(name='master')
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py",
 line 166, in hbase
tag = 'GANGLIA-MASTER' if name == 'master' else 'GANGLIA-RS'
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py",
 line 233, in hbase_TemplateConfig
template_tag = tag
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/template_config.py",
 line 46, in action_create
content = Template(template_name, extra_imports=self.resource.extra_imports)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 123, in action_create
content = self._get_content()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 160, in _get_content
return content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", 
line 51, in __call__
return self.get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", 
line 142, in get_content
rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 
891, in render
return self.environment.handle_exception(exc_info, True)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2",
 line 76, in top-level template code
hbase.sink.timeline.truststore.password = {{metric_truststore_password}}
ambari_jinja2.exceptions.UndefinedError: 'metric_collector_hosts' is undefined
```


We should define metric_collector_hosts in 
ambari/ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 05bad1c 

Diff: https://reviews.apache.org/r/48628/diff/


Testing
---

mvn clean test && manual test


Thanks,

Masahiro Tanaka



Re: Review Request 48485: AMBARI-17044 Optimize LogSearch Solr for cache setting and default values

2016-06-13 Thread Don Bosco Durai

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48485/
---

(Updated June 13, 2016, 10:24 a.m.)


Review request for Ambari, Oliver Szabo and Sumit Mohanty.


Changes
---

Updated solrconfig.xml.j2 in the Ambari stack location


Bugs: AMBARI-17044
https://issues.apache.org/jira/browse/AMBARI-17044


Repository: ambari


Description
---

Changed default settings to not to wait for cache to be fully build before 
addressing request


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2
 c0d8b2f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
 c870edc 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
 a06a900 

Diff: https://reviews.apache.org/r/48485/diff/


Testing
---

Create 3 node Hadoop cluster and tested the change


Thanks,

Don Bosco Durai



Re: Review Request 48624: Ambari upgrade from 2.2.2 to 2.4.0 fails if cluster name was changed

2016-06-13 Thread Nitiraj Rathore

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48624/#review137264
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 13, 2016, 9:41 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48624/
> ---
> 
> (Updated June 13, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17188
> https://issues.apache.org/jira/browse/AMBARI-17188
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrade cluster_handle column to have current cluster_id if it is not null.
> Check null or empty configuration properties for custom capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7451bbe 
> 
> Diff: https://reviews.apache.org/r/48624/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48545: View config- Allows remote cluster creation using a user having no permissions on the cluster

2016-06-13 Thread Nitiraj Rathore

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48545/#review137262
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 10, 2016, 12:15 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48545/
> ---
> 
> (Updated June 10, 2016, 12:15 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17162
> https://issues.apache.org/jira/browse/AMBARI-17162
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added permission check for user (ambari/cluster admin) before save/update of 
> remote cluster.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java
>  1135424 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariClusterRegistry.java
>  c0c887f 
>   ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
> 90144e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/view/RemoteAmbariClusterTest.java
>  2b6e014 
> 
> Diff: https://reviews.apache.org/r/48545/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>



Re: Review Request 48622: AMBARI-17136: If Solr is down or not ready, then LogFeeder to should retry

2016-06-13 Thread Don Bosco Durai


> On June 13, 2016, 9:56 a.m., Miklos Gergely wrote:
> > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java,
> >  line 354
> > 
> >
> > addRouterField should be called outside of the loop. The purpose of 
> > this function is to distribute the uploaded documents evenly among the 
> > shards, so it is not important to ensure that the ROUTER_FIELD is 
> > calculated based on the actual sending time, there is no need to re-set it 
> > in every iteration. The addRouterField function may not throw an exception 
> > so it is not a problem if it is not in any try-catch block.
> > 
> > Using setField instead of addField is a good decision, it describes 
> > better what the program's intention is.

In a multi-node SolrCloud cluster, each shard might be in different nodes. I 
have seen in some cases, if particular shard has any long running issue, then 
it is better to move on to next shard so that we don't block forever.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48622/#review137260
---


On June 13, 2016, 9:28 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48622/
> ---
> 
> (Updated June 13, 2016, 9:28 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17136
> https://issues.apache.org/jira/browse/AMBARI-17136
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Uploaded new patch based on feedback from previous review board requests
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
> 
> Diff: https://reviews.apache.org/r/48622/diff/
> 
> 
> Testing
> ---
> 
> Tested on 3 node hadoop cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Re: Review Request 48622: AMBARI-17136: If Solr is down or not ready, then LogFeeder to should retry

2016-06-13 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48622/#review137260
---




ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
 (line 341)


addRouterField should be called outside of the loop. The purpose of this 
function is to distribute the uploaded documents evenly among the shards, so it 
is not important to ensure that the ROUTER_FIELD is calculated based on the 
actual sending time, there is no need to re-set it in every iteration. The 
addRouterField function may not throw an exception so it is not a problem if it 
is not in any try-catch block.

Using setField instead of addField is a good decision, it describes better 
what the program's intention is.


- Miklos Gergely


On June 13, 2016, 9:28 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48622/
> ---
> 
> (Updated June 13, 2016, 9:28 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17136
> https://issues.apache.org/jira/browse/AMBARI-17136
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Uploaded new patch based on feedback from previous review board requests
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
> 
> Diff: https://reviews.apache.org/r/48622/diff/
> 
> 
> Testing
> ---
> 
> Tested on 3 node hadoop cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>



Re: Review Request 48623: Fix configuration xml files that don't pass validation

2016-06-13 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48623/#review137253
---


Ship it!




Ship It!

- Andrew Onischuk


On June 13, 2016, 9:24 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48623/
> ---
> 
> (Updated June 13, 2016, 9:24 a.m.)
> 
> 
> Review request for Ambari and Andrew Onischuk.
> 
> 
> Bugs: AMBARI-17187
> https://issues.apache.org/jira/browse/AMBARI-17187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> ServicePropertiesTest.validatePropertySchemaOfServiceXMLs
> src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml 
> didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
> content of element 'property' is not complete. One of '{filename, deleted, 
> final, on-ambari-upgrade, property-type, depends-on, property_depended_by}' 
> is expected.
> at 
> org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:48)
> --- Stdout: ---
> 2016-06-12 18:50:23,380 ERROR [main] stack.StackManager 
> (StackManager.java:validateAllPropertyXmlsInFolderRecursively(318)) - File 
> /opt/teamcity-agent/work/660cf0c01ac51af8/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
> content of element 'property' is not complete. One of '{filename, deleted, 
> final, on-ambari-upgrade, property-type, depends-on, property_depended_by}' 
> is expected.
> 
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  960c575 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  20f3173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
>  9c4ad88 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
>  2fa9448 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
>  41c8e6a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
>  f520455 
> 
> Diff: https://reviews.apache.org/r/48623/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>



Review Request 48623: Fix configuration xml files that don't pass validation

2016-06-13 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48623/
---

Review request for Ambari and Andrew Onischuk.


Bugs: AMBARI-17187
https://issues.apache.org/jira/browse/AMBARI-17187


Repository: ambari


Description
---

{code}
ServicePropertiesTest.validatePropertySchemaOfServiceXMLs
src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml 
didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
content of element 'property' is not complete. One of '{filename, deleted, 
final, on-ambari-upgrade, property-type, depends-on, property_depended_by}' is 
expected.
at 
org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:48)
--- Stdout: ---
2016-06-12 18:50:23,380 ERROR [main] stack.StackManager 
(StackManager.java:validateAllPropertyXmlsInFolderRecursively(318)) - File 
/opt/teamcity-agent/work/660cf0c01ac51af8/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
 didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
content of element 'property' is not complete. One of '{filename, deleted, 
final, on-ambari-upgrade, property-type, depends-on, property_depended_by}' is 
expected.

{code}


Diffs
-

  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
 960c575 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 20f3173 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
 9c4ad88 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
 2fa9448 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
 41c8e6a 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
 f520455 

Diff: https://reviews.apache.org/r/48623/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko



Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

2016-06-13 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 13, 2016, 9:22 a.m.)


Review request for Ambari, Andrew Onischuk, Jayush Luniya, Sandor Magyari, 
Sumit Mohanty, and Sebastian Toader.


Changes
---

Added Jayush.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description (updated)
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installation related information.

The patch only applies for 2.2-next versions.
Based on the git logs the affected codebase has been refactored as part of 
AMBARI-15329.
(This addition should be ported to the newer versions isf applicable)


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Unit tests passed.


Thanks,

Laszlo Puskas



Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

2016-06-13 Thread Laszlo Puskas


> On June 7, 2016, 6:49 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py,
> >  line 130
> > 
> >
> > Please consult with Jayush Luniya.
> > The stack already has a config for the stack root, so hdp_select (or 
> > distro select as it should be called) shouldn't be hardcoding paths.

This patch is for 2.2-next branch that doesn't have those improvements. (Tried 
to do this following one of your previous notes - seems that those weren't 
backported to this branch)


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136517
---


On June 13, 2016, 8:28 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 13, 2016, 8:28 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Daniel Gergely, Sandor Magyari, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installation related information.
> 
> The patch only applies for 2.2-next versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>



Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

2016-06-13 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 13, 2016, 8:28 a.m.)


Review request for Ambari, Andrew Onischuk, Daniel Gergely, Oliver Szabo, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Changes
---

Applied review notes.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installation related information.

The patch only applies for 2.2-next versions.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Unit tests passed.


Thanks,

Laszlo Puskas



Re: Review Request 48490: Last button in the log search pagination panel does not take the user to the last page and few more fixes

2016-06-13 Thread Dharmesh Makwana

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48490/
---

(Updated June 13, 2016, 6:36 a.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
Mohanty, and Sebastian Toader.


Changes
---

Fixed review comments.


Bugs: AMBARI-17140
https://issues.apache.org/jira/browse/AMBARI-17140


Repository: ambari


Description
---

Patch contains
1) Last button in the log search pagination panel does not take the user to the 
last page.
2) Solr warming feature added.
3) Suffix in columns mapping issue fixed.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
 917956f 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 147e148 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
 53e386e 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
 ca63671 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/MgrBase.java
 1d069d3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigMgr.java
 3b23f2a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
 4647c7b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
 92bfb01 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
 1e107ed 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/ConfigUtil.java
 44d184d 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/view/VUserConfig.java
 55ec1c0 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
25b4f1a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/META-INF/applicationContext.xml
 6a73d60 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collections/BaseCollection.js
 24b6974 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Intro.js 
869a1d4 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js
 5f93c5a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dialog/ApplySearchFilterView.js
 4babebc 

Diff: https://reviews.apache.org/r/48490/diff/


Testing
---

Setup Logsearch on 2 node cluster and tested the above feature.


Thanks,

Dharmesh Makwana