Review Request 52979: NPE when a non-existent host is provided as part of the host filter

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52979/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18623
https://issues.apache.org/jira/browse/AMBARI-18623


Repository: ambari


Description
---

When a non-existent host is provided as part of the host filter then server
throws an NPE.

API call




curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
'{"RequestInfo":{"context":"YARN Service 
Check","command":"YARN_SERVICE_CHECK"},"Requests/resource_filters":[{"service_name":"YARN","hosts":"hn0-d7d2bf.hdinsight.net"}]}'
 http://localhost:8080/api/v1/clusters/jeezraspark2zeppelin18/requests
{
  "status": 500,
  "message": "Server Error"
}




15 Oct 2016 01:10:28,655  INFO [ambari-client-thread-1378] 
AmbariManagementControllerImpl:3762 - Received action execution request, 
clusterName=jeezraspark2zeppelin18, request=isCommand :true, action :null, 
command :YARN_SERVICE_CHECK, inputs :{}, resourceFilters: 
[RequestResourceFilter{serviceName='YARN', componentName='null', 
hostNames=[hn0-d7d2bf.hdinsight.net]}], exclusive: false, clusterName 
:jeezraspark2zeppelin18
15 Oct 2016 01:10:28,658 ERROR [ambari-client-thread-1378] 
BaseManagementHandler:71 - Caught a runtime exception while attempting to 
create a resource: null
java.lang.NullPointerException
at 
org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.findHostAndAddServiceCheckAction(AmbariCustomCommandExecutionHelper.java:557)
at 
org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.addExecutionCommandsToStage(AmbariCustomCommandExecutionHelper.java:997)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:3818)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 

Diff: https://reviews.apache.org/r/52979/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 52932: Ambari MIB Should Be Included with Installation Bits

[Dmytro Sen]

> On Окт. 17, 2016, 9:10 п.п., Jonathan Hurley wrote:
> > ambari-server/pom.xml, line 44
> > 
> >
> > I think that perhaps we should now move this out of contrib and into 
> > resources directly since it's a first-class part of the product.

Do you mean moving whole alert-snmp-mib directory to 
/var/lib/ambari-server/resources ? Or only APACHE-AMBARI-MIB.txt ?


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52932/#review152957
---


On Окт. 17, 2016, 4:05 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52932/
> ---
> 
> (Updated Окт. 17, 2016, 4:05 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18611
> https://issues.apache.org/jira/browse/AMBARI-18611
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The Ambari MIB located at 
> https://github.com/apache/ambari/blob/trunk/contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt
>  should be included on the file system when Ambari is installed.
> It's location should be alongside other resources at:
> /var/lib/ambari-server/resources
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml e02b7a5 
>   ambari-server/src/main/assemblies/server.xml 9252c56 
> 
> Diff: https://reviews.apache.org/r/52932/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 52979: NPE when a non-existent host is provided as part of the host filter

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52979/#review153064
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 18, 2016, 10:37 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52979/
> ---
> 
> (Updated Oct. 18, 2016, 10:37 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18623
> https://issues.apache.org/jira/browse/AMBARI-18623
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a non-existent host is provided as part of the host filter then server
> throws an NPE.
> 
> API call
> 
> 
> 
> 
> curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
> '{"RequestInfo":{"context":"YARN Service 
> Check","command":"YARN_SERVICE_CHECK"},"Requests/resource_filters":[{"service_name":"YARN","hosts":"hn0-d7d2bf.hdinsight.net"}]}'
>  http://localhost:8080/api/v1/clusters/jeezraspark2zeppelin18/requests
> {
>   "status": 500,
>   "message": "Server Error"
> }
> 
> 
> 
> 
> 15 Oct 2016 01:10:28,655  INFO [ambari-client-thread-1378] 
> AmbariManagementControllerImpl:3762 - Received action execution request, 
> clusterName=jeezraspark2zeppelin18, request=isCommand :true, action :null, 
> command :YARN_SERVICE_CHECK, inputs :{}, resourceFilters: 
> [RequestResourceFilter{serviceName='YARN', componentName='null', 
> hostNames=[hn0-d7d2bf.hdinsight.net]}], exclusive: false, clusterName 
> :jeezraspark2zeppelin18
> 15 Oct 2016 01:10:28,658 ERROR [ambari-client-thread-1378] 
> BaseManagementHandler:71 - Caught a runtime exception while attempting to 
> create a resource: null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.findHostAndAddServiceCheckAction(AmbariCustomCommandExecutionHelper.java:557)
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.addExecutionCommandsToStage(AmbariCustomCommandExecutionHelper.java:997)
> at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:3818)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
> 
> Diff: https://reviews.apache.org/r/52979/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153069
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 18, 2016, 3:36 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 3:36 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 52983: Can't open Hive view in Internet explorer 11 in HDP 2.5

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52983/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj Rathore, and 
Rohit Choudhary.


Bugs: AMBARI-18625
https://issues.apache.org/jira/browse/AMBARI-18625


Repository: ambari


Description
---

Replaced ES6 code with ES5 code for making it work on IE 11.


Diffs
-

  
contrib/views/hive-next/src/main/resources/ui/hive-web/app/components/date-range-widget.js
 a9315ba 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/components/date-range-widget.js
 a9315ba 

Diff: https://reviews.apache.org/r/52983/diff/


Testing
---

Manually tested.


Thanks,

Pallav Kulshreshtha

Review Request 52987: Remove Unnecessary Locks Inside Of SCH Business Object Implementations

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52987/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
Sid Wagle.


Bugs: AMBARI-18614
https://issues.apache.org/jira/browse/AMBARI-18614


Repository: ambari


Description
---

A continuation of AMBARI-18456...

The {{ServiceComponentHost}} business object should have the following work 
performed:
- Remove locking around areas where its no longer required
- Replace collections with thread-safe concurrent versions
- Remove some reliance on state-full business objects (caches)


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/state/HostConfig.java 
fc22ba5 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
 586134c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 5566343 
  
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
 a6f5040 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
 dcffece 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
 cd41929 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
 c4d735d 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
 b082145 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 7b3f671 
  ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 
43de27f 
  ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
fffda6c 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java
 6d19113 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java
 60027c2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java
 59bb393 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java
 fa5491e 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
 ca12826 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 64d8184 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java
 3e526d9 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java
 9e285c6 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java
 fbed6e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java
 bb55597 
  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 48279af 

Diff: https://reviews.apache.org/r/52987/diff/


Testing
---

Tests run: 4688, Failures: 0, Errors: 0, Skipped: 42

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 22:28 min
[INFO] Finished at: 2016-10-17T18:20:53-04:00
[INFO] Final Memory: 53M/475M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 52958: Fix Log Search User Config bug

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52958/
---

(Updated Oct. 18, 2016, 12:48 p.m.)


Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Changes
---

Some more refactorings for a cleaner code


Bugs: AMBARI-18616
https://issues.apache.org/jira/browse/AMBARI-18616


Repository: ambari


Description
---

Log Search User Config UI screen does not offer the chance to modify the filter 
settings for those logs which don't have any log entries loaded so far. For 
these logs the initial settings will be in effect until the first modification 
of the filters when all of these settings will be erased, and thus every log 
will be loaded for them without filtering.

The User Config UI should always show all the logs, first those which already 
have some log entries in the solr, then the rest.

Also some minor refactoring for cleaner code.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/HadoopServiceConfigHelper.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 58337f7 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java
 45d66cf 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
 d276769 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/common/LogFeederDataMap.java
 b09610c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/filter/CreateLogfeederFilterView.js
 9bdf0fa 

Diff: https://reviews.apache.org/r/52958/diff/


Testing
---

Done


Thanks,

Miklos Gergely

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Dmytro Grinenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153079
---


Ship it!




Ship It!

- Dmytro Grinenko


On Oct. 18, 2016, 12:36 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 12:36 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52983: Can't open Hive view in Internet explorer 11 in HDP 2.5

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52983/#review153082
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On Oct. 18, 2016, 11:07 a.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52983/
> ---
> 
> (Updated Oct. 18, 2016, 11:07 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj Rathore, 
> and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-18625
> https://issues.apache.org/jira/browse/AMBARI-18625
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Replaced ES6 code with ES5 code for making it work on IE 11.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive-next/src/main/resources/ui/hive-web/app/components/date-range-widget.js
>  a9315ba 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/components/date-range-widget.js
>  a9315ba 
> 
> Diff: https://reviews.apache.org/r/52983/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153083
---


Ship it!




Ship It!

- Di Li


On Oct. 18, 2016, 12:36 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 12:36 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153086
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 (line 616)


Why is this a Map ... Seems kind of odd that we track the # of 
commands running as a long.


- Jonathan Hurley


On Oct. 17, 2016, 8:36 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 17, 2016, 8:36 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52932: Ambari MIB Should Be Included with Installation Bits

[Jonathan Hurley]

> On Oct. 17, 2016, 5:10 p.m., Jonathan Hurley wrote:
> > ambari-server/pom.xml, line 44
> > 
> >
> > I think that perhaps we should now move this out of contrib and into 
> > resources directly since it's a first-class part of the product.
> 
> Dmytro Sen wrote:
> Do you mean moving whole alert-snmp-mib directory to 
> /var/lib/ambari-server/resources ? Or only APACHE-AMBARI-MIB.txt ?

Only the MIB - The idea here is that we are shipping a first-class MIB with 
known OIDs which can be imported into any organization's existing SNMP 
structure. I don't think we need the rest of alert-snmp-mib since it contains 
examples for setting up a quick ad-hoc SNMP infra.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52932/#review152957
---


On Oct. 17, 2016, 12:05 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52932/
> ---
> 
> (Updated Oct. 17, 2016, 12:05 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18611
> https://issues.apache.org/jira/browse/AMBARI-18611
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The Ambari MIB located at 
> https://github.com/apache/ambari/blob/trunk/contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt
>  should be included on the file system when Ambari is installed.
> It's location should be alongside other resources at:
> /var/lib/ambari-server/resources
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml e02b7a5 
>   ambari-server/src/main/assemblies/server.xml 9252c56 
> 
> Diff: https://reviews.apache.org/r/52932/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 52932: Ambari MIB Should Be Included with Installation Bits

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52932/
---

(Updated Окт. 18, 2016, 2:27 п.п.)


Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle.


Bugs: AMBARI-18611
https://issues.apache.org/jira/browse/AMBARI-18611


Repository: ambari


Description
---

The Ambari MIB located at 
https://github.com/apache/ambari/blob/trunk/contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt
 should be included on the file system when Ambari is installed.
It's location should be alongside other resources at:
/var/lib/ambari-server/resources


Diffs (updated)
-

  ambari-server/src/main/assemblies/server.xml 9252c56 
  ambari-server/src/main/resources/APACHE-AMBARI-MIB.txt PRE-CREATION 
  contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt 85ec3cd 
  contrib/alert-snmp-mib/README.md 37fbc0a 

Diff: https://reviews.apache.org/r/52932/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 52958: Fix Log Search User Config bug

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52958/#review153091
---


Ship it!




Ship It!

- Robert Nettleton


On Oct. 18, 2016, 12:48 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52958/
> ---
> 
> (Updated Oct. 18, 2016, 12:48 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18616
> https://issues.apache.org/jira/browse/AMBARI-18616
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Log Search User Config UI screen does not offer the chance to modify the 
> filter settings for those logs which don't have any log entries loaded so 
> far. For these logs the initial settings will be in effect until the first 
> modification of the filters when all of these settings will be erased, and 
> thus every log will be loaded for them without filtering.
> 
> The User Config UI should always show all the logs, first those which already 
> have some log entries in the solr, then the rest.
> 
> Also some minor refactoring for cleaner code.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/HadoopServiceConfigHelper.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  58337f7 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java
>  45d66cf 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
>  d276769 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/common/LogFeederDataMap.java
>  b09610c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/filter/CreateLogfeederFilterView.js
>  9bdf0fa 
> 
> Diff: https://reviews.apache.org/r/52958/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Review Request 52989: HDFS goes down after installing cluster

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18629
https://issues.apache.org/jira/browse/AMBARI-18629


Repository: ambari


Description
---

After cluster install, HDFS goes down after it is started.  
Here is a repro cluster : 172.27.35.0 (with https)

Also, this cluster runs on all the configs mentioned in the Environment
description.  
No particular exceptions are found in HDFS/ambari server logs


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
  ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 

Diff: https://reviews.apache.org/r/52989/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 52989: HDFS goes down after installing cluster

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153095
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Жов. 18, 2016, 3:09 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Жов. 18, 2016, 3:09 після полудня)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52989: HDFS goes down after installing cluster

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153097
---




ambari-agent/src/main/python/ambari_agent/ActionQueue.py (line 242)


Should the default value be 2 seconds - I was looking at runs yesterday and 
NN status check was taking more than 2 seconds and I assume it will translate 
to reporting the status as INSTALLED.

Wonder if 5 seconds is a better default? We should still log if command 
takes more than 2 seconds.


- Sumit Mohanty


On Oct. 18, 2016, 3:09 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Oct. 18, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52932: Ambari MIB Should Be Included with Installation Bits

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52932/#review153102
---


Ship it!




Ship It!

- Jonathan Hurley


On Oct. 18, 2016, 10:27 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52932/
> ---
> 
> (Updated Oct. 18, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18611
> https://issues.apache.org/jira/browse/AMBARI-18611
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The Ambari MIB located at 
> https://github.com/apache/ambari/blob/trunk/contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt
>  should be included on the file system when Ambari is installed.
> It's location should be alongside other resources at:
> /var/lib/ambari-server/resources
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/assemblies/server.xml 9252c56 
>   ambari-server/src/main/resources/APACHE-AMBARI-MIB.txt PRE-CREATION 
>   contrib/alert-snmp-mib/APACHE-AMBARI-MIB.txt 85ec3cd 
>   contrib/alert-snmp-mib/README.md 37fbc0a 
> 
> Diff: https://reviews.apache.org/r/52932/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49484/#review153107
---


Ship it!




Ship It!

- Jayush Luniya


On Oct. 17, 2016, 6:40 a.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49484/
> ---
> 
> (Updated Oct. 17, 2016, 6:40 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17489
> https://issues.apache.org/jira/browse/AMBARI-17489
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove spark.yarn.max.executor.failures configuration in Spark Ambari 
> definition
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  6465b72 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> e8a2e2f 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> e127dff 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  0a14581 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> fa1a555 
> 
> Diff: https://reviews.apache.org/r/49484/diff/
> 
> 
> Testing
> ---
> 
> Local verification.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>

Re: Review Request 52860: Rename service to "Microsoft R Server" and component to "Microsoft R Node Client"

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52860/#review153109
---


Ship it!




Ship It!

- Jayush Luniya


On Oct. 14, 2016, 10:07 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52860/
> ---
> 
> (Updated Oct. 14, 2016, 10:07 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jayush Luniya, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18597
> https://issues.apache.org/jira/browse/AMBARI-18597
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Rename service to "Microsoft R Server" and component to "Microsoft R Node 
> Client" in the Microsoft R management pack.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  397c22d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/kerberos.json
>  86fab2d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/metainfo.xml
>  6c5f062 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/files/microsoft_r_serviceCheck.r
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/microsoft_r.py
>  f6eb33e 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_linux.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_windows.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/service_check.py
>  5368e94 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/role_command_order.json
>  6edbc2c 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/service_advisor.py
>  d48afec 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R_SERVER/8.0.5/role_command_order.json
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/metainfo.xml
>  9470d22 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/repos/repoinfo.xml
>   
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> eb7076c 
> 
> Diff: https://reviews.apache.org/r/52860/diff/
> 
> 
> Testing
> ---
> 
> Manually tested installation and kerberization.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review153108
---


Ship it!




Ship It!

- Aravindan Vijayan


On Sept. 22, 2016, 12:44 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 22, 2016, 12:44 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector (in AMS distributed mode) web-console will be Kerberos HTTP 
> SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics 
> Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> 
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml d73a0af 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> e98c65c 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  3ca3a31 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
> 050af16 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/faked_kerberos.py
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
>  51f541f 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  6934924 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  3823912 
> 
> Diff: https://reviews.apache.org/r/51724/diff/
> 
> 
> Testing
> ---
> 
> 1. passed existing unittest cases.
> 2. manually tested with HDP-2.5.0.0/trunk 
>1) Non http spnego enabled clusters
>2) http spnego enabled cluster - tested the trunk's monitor code against a 
> http spnego enabled cluster 
>   http spnego enabled cluster is currently broken in trunk: the 
> distributed metrics collector's http server currently will not started with 
> http spnego enabled. so I had to manually test the trunk's monitor code with 
> http spnego enabled collector on a branch2.4 cluster. Need to mention that 
> the testing should be valid because the code change area is monitor code and 
> the trunk and branch2.4's collectors are same.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 52860: Rename service to "Microsoft R Server" and component to "Microsoft R Node Client"

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52860/#review153111
---




contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/metainfo.xml
 (line 35)


Rename to microsoft_r_node_client.py?


- Jayush Luniya


On Oct. 14, 2016, 10:07 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52860/
> ---
> 
> (Updated Oct. 14, 2016, 10:07 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jayush Luniya, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18597
> https://issues.apache.org/jira/browse/AMBARI-18597
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Rename service to "Microsoft R Server" and component to "Microsoft R Node 
> Client" in the Microsoft R management pack.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  397c22d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/kerberos.json
>  86fab2d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/metainfo.xml
>  6c5f062 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/files/microsoft_r_serviceCheck.r
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/microsoft_r.py
>  f6eb33e 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_linux.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_windows.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/service_check.py
>  5368e94 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/role_command_order.json
>  6edbc2c 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/service_advisor.py
>  d48afec 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R_SERVER/8.0.5/role_command_order.json
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/metainfo.xml
>  9470d22 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/repos/repoinfo.xml
>   
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> eb7076c 
> 
> Diff: https://reviews.apache.org/r/52860/diff/
> 
> 
> Testing
> ---
> 
> Manually tested installation and kerberization.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 52989: HDFS goes down after installing cluster

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153113
---



1. Agree with SUmit on increasing timeout to 5 sicne we are killing the thread.
2. Instead of cryptic way of kill is there a graceful way to stop using 
separate Stoppable Thread extension which can be called from the parent ?

- Sid Wagle


On Oct. 18, 2016, 3:09 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Oct. 18, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52860: Rename service to "Microsoft R Server" and component to "Microsoft R Node Client"

[Balázs Bence Sári]

> On Oct. 18, 2016, 4:41 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/metainfo.xml,
> >  line 35
> > 
> >
> > Rename to microsoft_r_node_client.py?

I didn't change the convention (the script was named after the service). Do you 
want this to be changed in a new patch?


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52860/#review153111
---


On Oct. 14, 2016, 10:07 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52860/
> ---
> 
> (Updated Oct. 14, 2016, 10:07 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jayush Luniya, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18597
> https://issues.apache.org/jira/browse/AMBARI-18597
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Rename service to "Microsoft R Server" and component to "Microsoft R Node 
> Client" in the Microsoft R management pack.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  397c22d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/kerberos.json
>  86fab2d 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/metainfo.xml
>  6c5f062 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/files/microsoft_r_serviceCheck.r
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/microsoft_r.py
>  f6eb33e 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_linux.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/params_windows.py
>   
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/package/scripts/service_check.py
>  5368e94 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/role_command_order.json
>  6edbc2c 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.5/service_advisor.py
>  d48afec 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R_SERVER/8.0.5/role_command_order.json
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/metainfo.xml
>  9470d22 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.5/repos/repoinfo.xml
>   
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> eb7076c 
> 
> Diff: https://reviews.apache.org/r/52860/diff/
> 
> 
> Testing
> ---
> 
> Manually tested installation and kerberization.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 52987: Remove Unnecessary Locks Inside Of SCH Business Object Implementations

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52987/#review153116
---


Ship it!





ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java
 


Was this meant to create h3?


- Alejandro Fernandez


On Oct. 18, 2016, 12:23 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52987/
> ---
> 
> (Updated Oct. 18, 2016, 12:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-18614
> https://issues.apache.org/jira/browse/AMBARI-18614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A continuation of AMBARI-18456...
> 
> The {{ServiceComponentHost}} business object should have the following work 
> performed:
> - Remove locking around areas where its no longer required
> - Replace collections with thread-safe concurrent versions
> - Remove some reliance on state-full business objects (caches)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/java/org/apache/ambari/server/state/HostConfig.java 
> fc22ba5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  586134c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  5566343 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  a6f5040 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  dcffece 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  cd41929 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  c4d735d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  b082145 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  7b3f671 
>   ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 
> 43de27f 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
> fffda6c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java
>  6d19113 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java
>  60027c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java
>  59bb393 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java
>  fa5491e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  ca12826 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  64d8184 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java
>  3e526d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java
>  9e285c6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java
>  fbed6e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java
>  bb55597 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
>  48279af 
> 
> Diff: https://reviews.apache.org/r/52987/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4688, Failures: 0, Errors: 0, Skipped: 42
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 22:28 min
> [INFO] Finished at: 2016-10-17T18:20:53-04:00
> [INFO] Final Memory: 53M/475M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52987: Remove Unnecessary Locks Inside Of SCH Business Object Implementations

[Jonathan Hurley]

> On Oct. 18, 2016, 1:39 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java,
> >  line 224
> > 
> >
> > Was this meant to create h3?

No, it was purposfully trying to create another h2, causing a failure. However, 
since objects from the factory are created immediately, going through the 
ServiceComponent object to add hosts takes care of this. See this new code 
below it ...

```
try {
  sc.addServiceComponentHost("h2");
  fail("Expected error for dups");
} catch (Exception e) {
  // Expected
}
```


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52987/#review153116
---


On Oct. 18, 2016, 8:23 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52987/
> ---
> 
> (Updated Oct. 18, 2016, 8:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-18614
> https://issues.apache.org/jira/browse/AMBARI-18614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A continuation of AMBARI-18456...
> 
> The {{ServiceComponentHost}} business object should have the following work 
> performed:
> - Remove locking around areas where its no longer required
> - Replace collections with thread-safe concurrent versions
> - Remove some reliance on state-full business objects (caches)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/java/org/apache/ambari/server/state/HostConfig.java 
> fc22ba5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  586134c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  5566343 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  a6f5040 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  dcffece 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  cd41929 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  c4d735d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  b082145 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  7b3f671 
>   ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 
> 43de27f 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
> fffda6c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java
>  6d19113 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java
>  60027c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java
>  59bb393 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java
>  fa5491e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  ca12826 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  64d8184 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java
>  3e526d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java
>  9e285c6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java
>  fbed6e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java
>  bb55597 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
>  48279af 
> 
> Diff: https://reviews.apache.org/r/52987/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4688, Failures: 0, Errors: 0, Skipped: 42
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 22:28 min
> [INFO] Finished at: 2016-10-17T18:20:53-04:00
> [INFO] Final Memory: 53M/475M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153117
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 (line 389)


The log statements will already contain the timestamp, no need for new 
Date().



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 (line 55)


May want to remove any leading/trailing whitespace after splitting.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 (line 103)


Can "topn" be a constant, plus the other strings used in these files? Case 
insensitive equals?



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
 (line 112)


May still be good to  put " ASC" in the else clause to make it clear.


- Alejandro Fernandez


On Oct. 13, 2016, 8:40 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 13, 2016, 8:40 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  c2aea8c 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/
---

(Updated Oct. 18, 2016, 6:17 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo Puskas, 
Nate Cole, Sumit Mohanty, and Sebastian Toader.


Changes
---

Add comments, minor fixes.


Bugs: AMBARI-18553
https://issues.apache.org/jira/browse/AMBARI-18553


Repository: ambari


Description
---

Today, the START commands stored in role_command_orders table are bound to 
multiple stages created based on dependencies between components - a component 
in the second stage can only begin its START after the entire first stage is 
done as opposed to just its dependencies in the first stage. This eventually 
increases the overall blueprint deployment time.
The goal is to be able to configure a direct dependency based execution model 
of commands, for now only for Blueprint based deployment commands.

Implementation:
---
When creating stages we set the commandExecutionType to RoleGraph. In case 
commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
created. commandExecutionType is persisted into Stage object / entity as well, 
so ActionScheduler can decide based on commandExecutionType how to execute the 
stage. In case commandExecutionType is set to DEPENDENCY_ORDERED it will filter 
out commands having dependencies on other commands IN_PROGRESS. By default 
commandExecutionType is STAGE_BASED which works as before, creating one or more 
stages dependening on dependecies.
DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
initiated by Blueprint deployment and if Ambari property 
server.stage.command.execution_type = DEPENDENCY_ORDERED.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 8cbfb1e 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
f03d8ea 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 5d8f279 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 5afaba8 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 bbdb808 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 eaea913 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c9ab6f9 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
 625b168 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
 5ca4d88 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 34331ee 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 bd23e00 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 a49fc09 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewURLResourceProviderTest.java
 e223d0f 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java
 a03f05b 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleGraphTest.java
 8bead43 
  
ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java
 d26816c 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java
 2aaa2cf 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  lines 874-875
> > 
> >
> > Why check both here? Isn't it enough to simply check the stage's 
> > configure role ordering? 
> > 
> > Also - you can just return false immediately if it's not 
> > DEPENDENCY_ORDERED so you don't have to if-block out the entire logic of 
> > the method.

Normally would be enough to check only on stage, but I want to be able I can 
enable/disable this feature from ambari.properties.


> On Oct. 17, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java,
> >  line 69
> > 
> >
> > This feels very wrong to hard-code into something like a scheduler. It 
> > should be declarative somehow. Same for the other components below...

Earlier this was hard coded in RoleCommandOrder, I've take out from there. 
Ideally this would come from outside of Ambari, but may be we could added 
methods like isNamenodeHA etc to the Cluster object. May be an ideal sultion 
could be created in a separate patch, as I don't want to make big modifications 
on normal STAGE flow.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152962
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplan

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 2:19 p.m., Robert Levas wrote:
> > Has this been tested using both models while enabling Kerberos?

I've tested BP deployment of a secure cluster and Enable/Disable Kerberos, also 
Upgrade. Since this DEPENDENCY_ORDER execution applies only for requests marked 
with property INITIAL_START - theese are START requests created via Blueprints 
- everything elso should execute in same manner as before.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152841
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 2:36 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java,
> >  lines 3074-3076
> > 
> >
> > Need a default-if-not-null here

COMMAND_EXECUTION_TYPE is ConfigurationProperty and has a default value set to 
CommandExecutionType.STAGE.


> On Oct. 17, 2016, 2:36 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java,
> >  lines 43-46
> > 
> >
> > Use Objects.equals() for simplicity

In case of using Objects.equals you still had to cast and compare two fields 
separately. There's also EqualsBuilder but given that there are only two fields 
overall wouldn't be simplier.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152873
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgre

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/
---

(Updated Oct. 18, 2016, 6:25 p.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.


Changes
---

Addressed Jonathan's concern.


Bugs: AMBARI-18619
https://issues.apache.org/jira/browse/AMBARI-18619


Repository: ambari


Description
---

STR:

* Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
host.
* Run multiple service checks in parallel, but notice that they typically run 
on the same 1 or 2 hosts.

Currently, Ambari relies on getting the list of candidate hosts from the DB and 
excludes all hosts that are in maintenance mode. From that list, it picks the 
first host that is healthy (i.e., heartbeating). This means that the logic does 
not pick a random host.

Instead, Ambari should always pick a random host and prefer to schedule on 
hosts that have 0 in-progress commands.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 746bca4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 5d8f279 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 17760c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 e82900d 

Diff: https://reviews.apache.org/r/52964/diff/


Testing
---

Waiting for unit test results.


Thanks,

Alejandro Fernandez

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Alejandro Fernandez]

> On Oct. 18, 2016, 1:58 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java,
> >  line 623
> > 
> >
> > Why is this a Map ... Seems kind of odd that we track the # 
> > of commands running as a long.

Changed it to Map


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153086
---


On Oct. 18, 2016, 6:25 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

> On Oct. 14, 2016, 8:59 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java,
> >  line 2430
> > 
> >
> > How about using `TimeUnit.MINUTES.toMillis(5)` instead of `30L`?
> 
> Amarnath reddy pappu wrote:
> not using this - because will loose option of setting in milliseconds in 
> ambari.properties
> 
> Attila Doroszlai wrote:
> Using `TimeUnit.MINUTES.toMillis(5)` for the default value will not 
> affect the ability to get the setting from `ambari.properties`.  It's just a 
> more readable form of the same time length in milliseconds.

implemented


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review152643
---


On Oct. 17, 2016, 11:07 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 17, 2016, 11:07 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

> On Oct. 18, 2016, 5:37 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  lines 950-951
> > 
> >
> > Whitespace: missing between `){`, and extra blank line at start of 
> > method.

don't see any space between ){ - however please check latest diff.
taken out blank line at start


> On Oct. 18, 2016, 5:37 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  lines 953-954
> > 
> >
> > Variable names `monitor_delay` and `monitor_dealy_long` should be 
> > camelCase, ie. `monitorDelay`.
> > 
> > Also, `monitor_dealy_long` has typo, should be `monitorDelayLong`.

implemented


> On Oct. 18, 2016, 5:37 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  line 958
> > 
> >
> > Whitespace

don't see any whitespace , however please check latest diff


> On Oct. 18, 2016, 5:37 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  line 962
> > 
> >
> > Whitespace

don't see any whitespace , however please check latest diff


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review153034
---


On Oct. 17, 2016, 11:07 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 17, 2016, 11:07 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/
---

(Updated Oct. 18, 2016, 6:29 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sid Wagle.


Changes
---

implemented minor comments and attached diff


Bugs: AMBARI-18592
https://issues.apache.org/jira/browse/AMBARI-18592


Repository: ambari


Description
---

Ability to enable the DEBUG logs in Ambari-server without server restart

1. PropertyConfigurator would check log4j.properties update every 5mins. if 
there is any change in the file/timestamp it will reload the log4j 
configurations.
2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
3. Have modified only for ambari-server
4.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 c05840e 

Diff: https://reviews.apache.org/r/52839/diff/


Testing
---

1. Change the log4j.properties from INFO to DEBUG
[Expected Result] server should start logging the DEBUG logs with in 5mins of 
time.
[Resutl] PASS

2. Change the log4j.properties from DEBUG to INFO
[Expected Result] server should reduce the logging to INFO mode with in 5mins 
of time.
[Resutl] PASS


Thanks,

Amarnath reddy pappu

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/
---

(Updated Oct. 18, 2016, 6:34 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18593
https://issues.apache.org/jira/browse/AMBARI-18593


Repository: ambari


Description
---

HDFS exposes top user activity broken down by operations in jmx (nntop).
These metrics should be captured in AMS and exposed in Grafana's HDFS 
dashboards.

Downsampling should likely be a function like MIN, MAX, AVG, SUM of underlying 
timeseries specified from the client.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 ba7807b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
 c056d79 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
 118c695 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
 177e444 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
 f7060e0 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 b6e5da9 

Diff: https://reviews.apache.org/r/52842/diff/


Testing
---

Manually tested.
Unit tests added.
mvn clean test on ambari-metrics pending.


Thanks,

Aravindan Vijayan

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

[Aravindan Vijayan]

> On Oct. 18, 2016, 6:01 p.m., Alejandro Fernandez wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java,
> >  line 55
> > 
> >
> > May want to remove any leading/trailing whitespace after splitting.

Done.


> On Oct. 18, 2016, 6:01 p.m., Alejandro Fernandez wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java,
> >  line 103
> > 
> >
> > Can "topn" be a constant, plus the other strings used in these files? 
> > Case insensitive equals?

Done. Will move to an enum if more downsampling types come up in the future.


> On Oct. 18, 2016, 6:01 p.m., Alejandro Fernandez wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java,
> >  line 112
> > 
> >
> > May still be good to  put " ASC" in the else clause to make it clear.

Done.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153117
---


On Oct. 18, 2016, 6:34 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:34 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153135
---




ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 (line 605)


This can be really simplified with 1 Query rerun first host with nothing 
running on it that matches cretieria of component, randomness does not dd 
anything to the equation of runnign service check on a candidate host.

No point maintaining local list of what hosts have running commands and 
throwing them away.


- Sid Wagle


On Oct. 18, 2016, 6:25 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Jonathan Hurley]

> On Oct. 18, 2016, 2:36 p.m., Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java,
> >  line 612
> > 
> >
> > This can be really simplified with 1 Query rerun first host with 
> > nothing running on it that matches cretieria of component, randomness does 
> > not dd anything to the equation of runnign service check on a candidate 
> > host.
> > 
> > No point maintaining local list of what hosts have running commands and 
> > throwing them away.

I thought about this as well, but:

- Didn't want the choice being made in the DB
- What happens if there are tasks in progress on all hosts?


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153135
---


On Oct. 18, 2016, 2:25 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 2:25 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52987: Remove Unnecessary Locks Inside Of SCH Business Object Implementations

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52987/#review153146
---


Ship it!




Ship It!

- Nate Cole


On Oct. 18, 2016, 8:23 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52987/
> ---
> 
> (Updated Oct. 18, 2016, 8:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-18614
> https://issues.apache.org/jira/browse/AMBARI-18614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A continuation of AMBARI-18456...
> 
> The {{ServiceComponentHost}} business object should have the following work 
> performed:
> - Remove locking around areas where its no longer required
> - Replace collections with thread-safe concurrent versions
> - Remove some reliance on state-full business objects (caches)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/java/org/apache/ambari/server/state/HostConfig.java 
> fc22ba5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  586134c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  5566343 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  a6f5040 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  dcffece 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  cd41929 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  c4d735d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  b082145 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  7b3f671 
>   ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 
> 43de27f 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
> fffda6c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java
>  6d19113 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java
>  60027c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java
>  59bb393 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java
>  fa5491e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  ca12826 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  64d8184 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java
>  3e526d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java
>  9e285c6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java
>  fbed6e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java
>  bb55597 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
>  48279af 
> 
> Diff: https://reviews.apache.org/r/52987/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4688, Failures: 0, Errors: 0, Skipped: 42
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 22:28 min
> [INFO] Finished at: 2016-10-17T18:20:53-04:00
> [INFO] Final Memory: 53M/475M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Sid Wagle]

> On Oct. 18, 2016, 6:36 p.m., Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java,
> >  line 612
> > 
> >
> > This can be really simplified with 1 Query rerun first host with 
> > nothing running on it that matches cretieria of component, randomness does 
> > not dd anything to the equation of runnign service check on a candidate 
> > host.
> > 
> > No point maintaining local list of what hosts have running commands and 
> > throwing them away.
> 
> Jonathan Hurley wrote:
> I thought about this as well, but:
> 
> - Didn't want the choice being made in the DB
> - What happens if there are tasks in progress on all hosts?

Agree with that point. Also I talked to Alejandro about value of randomizing 
and to some extent I buy the point of not always picking same host, thereby 
droping the issue.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153135
---


On Oct. 18, 2016, 6:25 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153148
---


Ship it!




Ship It!

- Sid Wagle


On Oct. 18, 2016, 6:25 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 18, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153154
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review153176
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 (line 2428)


nit:  "Indicates the delay, in milliseconds, for the log4j monitor to check 
for changes"

Something like that.  We need to be very detailed so users/admins can 
understand exactly what a config knob is for.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 (lines 963 - 964)


These can be collapsed:
LOG.error("Exception in setting log4j monitor delay of {} for {}", 
monitorDelay, log4jpath, e);


- Nate Cole


On Oct. 18, 2016, 2:29 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 18, 2016, 2:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Review Request 52996: For rolling upgrade of Kafka 0.10.0.1, set configs for backward compatibility

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52996/
---

Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-18630
https://issues.apache.org/jira/browse/AMBARI-18630


Repository: ambari


Description
---

In the 2.5 upgrade pack, the following properties need to be set
{code}
inter.broker.protocol.version=0.9.0.0
log.message.format.version=0.9.0.0
{code}

after upgrade is done we should delete the inter.broker.protocol.version.
Users should remove log.message.format.version once they update their clients.

(Still waiting on final confirmation for behavior from Kafka team)


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
e8a2e2f 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
e89d62b 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
e127dff 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
fa1a555 

Diff: https://reviews.apache.org/r/52996/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Re: Review Request 52989: HDFS goes down after installing cluster

[Andrew Onischuk]

> On Oct. 18, 2016, 4:50 p.m., Sid Wagle wrote:
> > 1. Agree with SUmit on increasing timeout to 5 sicne we are killing the 
> > thread.
> > 2. Instead of cryptic way of kill is there a graceful way to stop using 
> > separate Stoppable Thread extension which can be called from the parent ?

1. Sure this is what will go as the second patch (already talked with Sumit)
2. We cannot do that. Since we need to interrupt external commands like 
subprocess etc. also this won't interrupt loops, sleep commands, etc. unless we 
add check for that everywhere. In every function which has that. And even with 
that subprocess/sleep is still something which woudln't interrupt.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153113
---


On Oct. 18, 2016, 3:09 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Oct. 18, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 52999: Incorporate database consistency check into main Ambari process

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52999/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18631
https://issues.apache.org/jira/browse/AMBARI-18631


Repository: ambari


Description
---

Currently the database consistency check runs as a separate process prior 
Ambari process. The database consistency checker load various modules needed 
for performing the validations. (e.g. load stack definitions to be able to 
compare service configs from stack with configs from db).
Once database consistency checker completed Ambari server is started. Ambari 
server beside others loads the same modules as database consistency checker.
This double initialisation adds time to the ambari server startup time which 
could be reduced if the database consistency check is moved into the ambari 
server process.
Moreover the database consistency check may check at the beginning if the 
database is empty and perform the checks only if there is data in the database.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 2d91eca 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
 535d74f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 eee7fee 
  ambari-server/src/main/python/ambari_server_main.py 57ec58d 

Diff: https://reviews.apache.org/r/52999/diff/


Testing
---

UT will be added and checked after patch draft review


Thanks,

Vitalyi Brodetskyi

Re: Review Request 52989: HDFS goes down after installing cluster

[Sid Wagle]

> On Oct. 18, 2016, 4:50 p.m., Sid Wagle wrote:
> > 1. Agree with SUmit on increasing timeout to 5 sicne we are killing the 
> > thread.
> > 2. Instead of cryptic way of kill is there a graceful way to stop using 
> > separate Stoppable Thread extension which can be called from the parent ?
> 
> Andrew Onischuk wrote:
> 1. Sure this is what will go as the second patch (already talked with 
> Sumit)
> 2. We cannot do that. Since we need to interrupt external commands like 
> subprocess etc. also this won't interrupt loops, sleep commands, etc. unless 
> we add check for that everywhere. In every function which has that. And even 
> with that subprocess/sleep is still something which woudln't interrupt.

How about spawning a subprocess that is long running until we hit a bad command 
and have to kill and respawn to execute non-blacklisted commands? This might 
need to have a separate queue for blacklisted commands that are tried for 
whitllisting withought affecting main thread.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153113
---


On Oct. 18, 2016, 3:09 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Oct. 18, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52999: Incorporate database consistency check into main Ambari process

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52999/#review153199
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 (line 304)


Can we move this to a seaparate method instead of adding all code to run()?



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 (lines 304 - 325)


Minor nit: Error available sounds like is stderr available for writing, 
ifErrorsFound sounds more nuanced.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 (line 312)


formatting



ambari-server/src/main/python/ambari_server_main.py (line 105)


This should be configurable if it isn't.


- Sid Wagle


On Oct. 18, 2016, 9:50 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52999/
> ---
> 
> (Updated Oct. 18, 2016, 9:50 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18631
> https://issues.apache.org/jira/browse/AMBARI-18631
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the database consistency check runs as a separate process prior 
> Ambari process. The database consistency checker load various modules needed 
> for performing the validations. (e.g. load stack definitions to be able to 
> compare service configs from stack with configs from db).
> Once database consistency checker completed Ambari server is started. Ambari 
> server beside others loads the same modules as database consistency checker.
> This double initialisation adds time to the ambari server startup time which 
> could be reduced if the database consistency check is moved into the ambari 
> server process.
> Moreover the database consistency check may check at the beginning if the 
> database is empty and perform the checks only if there is data in the 
> database.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  2d91eca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
>  535d74f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  eee7fee 
>   ambari-server/src/main/python/ambari_server_main.py 57ec58d 
> 
> Diff: https://reviews.apache.org/r/52999/diff/
> 
> 
> Testing
> ---
> 
> UT will be added and checked after patch draft review
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 52989: HDFS goes down after installing cluster

[Andrew Onischuk]

> On Oct. 18, 2016, 4:50 p.m., Sid Wagle wrote:
> > 1. Agree with SUmit on increasing timeout to 5 sicne we are killing the 
> > thread.
> > 2. Instead of cryptic way of kill is there a graceful way to stop using 
> > separate Stoppable Thread extension which can be called from the parent ?
> 
> Andrew Onischuk wrote:
> 1. Sure this is what will go as the second patch (already talked with 
> Sumit)
> 2. We cannot do that. Since we need to interrupt external commands like 
> subprocess etc. also this won't interrupt loops, sleep commands, etc. unless 
> we add check for that everywhere. In every function which has that. And even 
> with that subprocess/sleep is still something which woudln't interrupt.
> 
> Sid Wagle wrote:
> How about spawning a subprocess that is long running until we hit a bad 
> command and have to kill and respawn to execute non-blacklisted commands? 
> This might need to have a separate queue for blacklisted commands that are 
> tried for whitllisting withought affecting main thread.

good point. Will do.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52989/#review153113
---


On Oct. 18, 2016, 3:09 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52989/
> ---
> 
> (Updated Oct. 18, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18629
> https://issues.apache.org/jira/browse/AMBARI-18629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After cluster install, HDFS goes down after it is started.  
> Here is a repro cluster : 172.27.35.0 (with https)
> 
> Also, this cluster runs on all the configs mentioned in the Environment
> description.  
> No particular exceptions are found in HDFS/ambari server logs
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-common/src/main/python/ambari_commons/thread_utils.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52989/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52495: AMBARI-18245 Upgrade node to version 4.x

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52495/
---

(Updated Oct. 18, 2016, 11:37 p.m.)


Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
Subramanian, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-18245
https://issues.apache.org/jira/browse/AMBARI-18245


Repository: ambari


Description
---

We are currently using 0.10, which is very old and going EOL 2016-10-01: 
https://github.com/nodejs/LTS
We should upgrade to Node 4.x:
Upgrade Node on Ambari Web
Upgrade Node on Ambari Admin
Upgrade Node on contrib/views/* modules


Diffs (updated)
-

  .gitignore d9f23de 
  ambari-admin/pom.xml e057178 
  ambari-admin/src/main/resources/ui/admin-web/package.json 42f584e 
  ambari-web/app/styles/common.less a053d28 
  ambari-web/karma.conf.js 7442b95 
  ambari-web/package.json 94ded60 
  ambari-web/pom.xml d3de055 
  contrib/views/capacity-scheduler/pom.xml cd97dcf 
  contrib/views/capacity-scheduler/src/main/resources/ui/package.json 2cdc672 
  contrib/views/files/pom.xml 383d90d 
  contrib/views/hawq/pom.xml 9a563f6 
  contrib/views/hive-next/pom.xml 921990a 
  contrib/views/hive-next/src/main/resources/ui/hive-web/package.json 595b1f2 
  contrib/views/hive/pom.xml 20b23f6 
  contrib/views/hive/src/main/resources/ui/hive-web/package.json 595b1f2 
  contrib/views/hueambarimigration/pom.xml 3b28276 
  contrib/views/jobs/pom.xml bd01753 
  contrib/views/pig/pom.xml 595537a 
  contrib/views/pig/src/main/resources/ui/pig-web/package.json 3221483 
  contrib/views/wfmanager/pom.xml 389dc6c 

Diff: https://reviews.apache.org/r/52495/diff/


Testing
---

Ran "mvn clean package", Ambari Server failed, which is irralevant to this 
patch.
Then run "mvn clean install" individually:

Ambari Web:
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 3:08.144s
[INFO] Finished at: Mon Oct 03 17:13:25 PDT 2016
[INFO] Final Memory: 16M/407M
[INFO] 

30366 tests complete (27 seconds)
151 tests pending

Ambari Admin:
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 14.924s
[INFO] Finished at: Mon Oct 03 17:47:01 PDT 2016
[INFO] Final Memory: 20M/284M
[INFO] 

PhantomJS 1.9.7 (Mac OS X): Executed 86 of 86 SUCCESS (0.267 secs / 0.475 secs)

contrib/views/ modules:
[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Contrib Views .. SUCCESS [1:00.758s]
[INFO] Ambari View Utils . SUCCESS [4.714s]
[INFO] Ambari View Commons ... SUCCESS [0.286s]
[INFO] Files . SUCCESS [52.631s]
[INFO] Jobs .. SUCCESS [6.712s]
[INFO] Pig ... SUCCESS [1:07.799s]
[INFO] Slider  SUCCESS [36.043s]
[INFO] Capacity Scheduler  SUCCESS [53.446s]
[INFO] Tez ... SUCCESS [3.659s]
[INFO] HAWQ .. SUCCESS [2:24.060s]
[INFO] Storm_Monitoring .. SUCCESS [0.432s]
[INFO] Zeppelin .. SUCCESS [2.080s]
[INFO] hueambarimigration  SUCCESS [20.801s]
[INFO] Hive Jdbc . SUCCESS [1:00.664s]
[INFO] WF Manager View ... SUCCESS [32.911s]
[INFO] Hive .. SUCCESS [51.542s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 9:58.964s
[INFO] Finished at: Mon Oct 03 17:22:59 PDT 2016
[INFO] Final Memory: 90M/404M
[INFO] 


Thanks,

Zhe (Joe) Wang

Re: Review Request 52495: AMBARI-18245 Upgrade node to version 4.x

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52495/#review153220
---


Ship it!




Ship It!

- Richard Zang


On Oct. 18, 2016, 11:37 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52495/
> ---
> 
> (Updated Oct. 18, 2016, 11:37 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18245
> https://issues.apache.org/jira/browse/AMBARI-18245
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We are currently using 0.10, which is very old and going EOL 2016-10-01: 
> https://github.com/nodejs/LTS
> We should upgrade to Node 4.x:
> Upgrade Node on Ambari Web
> Upgrade Node on Ambari Admin
> Upgrade Node on contrib/views/* modules
> 
> 
> Diffs
> -
> 
>   .gitignore d9f23de 
>   ambari-admin/pom.xml e057178 
>   ambari-admin/src/main/resources/ui/admin-web/package.json 42f584e 
>   ambari-web/app/styles/common.less a053d28 
>   ambari-web/karma.conf.js 7442b95 
>   ambari-web/package.json 94ded60 
>   ambari-web/pom.xml d3de055 
>   contrib/views/capacity-scheduler/pom.xml cd97dcf 
>   contrib/views/capacity-scheduler/src/main/resources/ui/package.json 2cdc672 
>   contrib/views/files/pom.xml 383d90d 
>   contrib/views/hawq/pom.xml 9a563f6 
>   contrib/views/hive-next/pom.xml 921990a 
>   contrib/views/hive-next/src/main/resources/ui/hive-web/package.json 595b1f2 
>   contrib/views/hive/pom.xml 20b23f6 
>   contrib/views/hive/src/main/resources/ui/hive-web/package.json 595b1f2 
>   contrib/views/hueambarimigration/pom.xml 3b28276 
>   contrib/views/jobs/pom.xml bd01753 
>   contrib/views/pig/pom.xml 595537a 
>   contrib/views/pig/src/main/resources/ui/pig-web/package.json 3221483 
>   contrib/views/wfmanager/pom.xml 389dc6c 
> 
> Diff: https://reviews.apache.org/r/52495/diff/
> 
> 
> Testing
> ---
> 
> Ran "mvn clean package", Ambari Server failed, which is irralevant to this 
> patch.
> Then run "mvn clean install" individually:
> 
> Ambari Web:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 3:08.144s
> [INFO] Finished at: Mon Oct 03 17:13:25 PDT 2016
> [INFO] Final Memory: 16M/407M
> [INFO] 
> 
> 
> 30366 tests complete (27 seconds)
> 151 tests pending
> 
> Ambari Admin:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 14.924s
> [INFO] Finished at: Mon Oct 03 17:47:01 PDT 2016
> [INFO] Final Memory: 20M/284M
> [INFO] 
> 
> 
> PhantomJS 1.9.7 (Mac OS X): Executed 86 of 86 SUCCESS (0.267 secs / 0.475 
> secs)
> 
> contrib/views/ modules:
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Contrib Views .. SUCCESS [1:00.758s]
> [INFO] Ambari View Utils . SUCCESS [4.714s]
> [INFO] Ambari View Commons ... SUCCESS [0.286s]
> [INFO] Files . SUCCESS [52.631s]
> [INFO] Jobs .. SUCCESS [6.712s]
> [INFO] Pig ... SUCCESS [1:07.799s]
> [INFO] Slider  SUCCESS [36.043s]
> [INFO] Capacity Scheduler  SUCCESS [53.446s]
> [INFO] Tez ... SUCCESS [3.659s]
> [INFO] HAWQ .. SUCCESS [2:24.060s]
> [INFO] Storm_Monitoring .. SUCCESS [0.432s]
> [INFO] Zeppelin .. SUCCESS [2.080s]
> [INFO] hueambarimigration  SUCCESS [20.801s]
> [INFO] Hive Jdbc . SUCCESS [1:00.664s]
> [INFO] WF Manager View ... SUCCESS [32.911s]
> [INFO] Hive .. SUCCESS [51.542s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 9:58.964s
> [INFO] Finished at: Mon Oct 03 17:22:59 PDT 2016
> [INFO] Final

Re: Review Request 52996: For rolling upgrade of Kafka 0.10.0.1, set configs for backward compatibility

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52996/
---

(Updated Oct. 18, 2016, 8:57 p.m.)


Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Changes
---

Update for direction from Kafka team.


Bugs: AMBARI-18630
https://issues.apache.org/jira/browse/AMBARI-18630


Repository: ambari


Description
---

In the 2.5 upgrade pack, the following properties need to be set
{code}
inter.broker.protocol.version=0.9.0.0
log.message.format.version=0.9.0.0
{code}

after upgrade is done we should delete the inter.broker.protocol.version.
Users should remove log.message.format.version once they update their clients.

(Still waiting on final confirmation for behavior from Kafka team)


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
e8a2e2f 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
e89d62b 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
e127dff 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
fa1a555 

Diff: https://reviews.apache.org/r/52996/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Re: Review Request 52996: For rolling upgrade of Kafka 0.10.0.1, set configs for backward compatibility

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52996/#review153232
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 19, 2016, 12:57 a.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52996/
> ---
> 
> (Updated Oct. 19, 2016, 12:57 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18630
> https://issues.apache.org/jira/browse/AMBARI-18630
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the 2.5 upgrade pack, the following properties need to be set
> {code}
> inter.broker.protocol.version=0.9.0.0
> log.message.format.version=0.9.0.0
> {code}
> 
> after upgrade is done we should delete the inter.broker.protocol.version.
> Users should remove log.message.format.version once they update their clients.
> 
> (Still waiting on final confirmation for behavior from Kafka team)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> e8a2e2f 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> e89d62b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> e127dff 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> fa1a555 
> 
> Diff: https://reviews.apache.org/r/52996/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/
---

(Updated Oct. 19, 2016, 1:34 a.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.


Changes
---

Fixed unit tests.


Bugs: AMBARI-18619
https://issues.apache.org/jira/browse/AMBARI-18619


Repository: ambari


Description
---

STR:

* Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
host.
* Run multiple service checks in parallel, but notice that they typically run 
on the same 1 or 2 hosts.

Currently, Ambari relies on getting the list of candidate hosts from the DB and 
excludes all hosts that are in maintenance mode. From that list, it picks the 
first host that is healthy (i.e., heartbeating). This means that the logic does 
not pick a random host.

Instead, Ambari should always pick a random host and prefer to schedule on 
hosts that have 0 in-progress commands.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 746bca4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 5d8f279 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 17760c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 e82900d 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 197b925 

Diff: https://reviews.apache.org/r/52964/diff/


Testing
---

Waiting for unit test results.


Thanks,

Alejandro Fernandez

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/
---

(Updated Oct. 19, 2016, 1:35 a.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18619
https://issues.apache.org/jira/browse/AMBARI-18619


Repository: ambari


Description
---

STR:

* Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
host.
* Run multiple service checks in parallel, but notice that they typically run 
on the same 1 or 2 hosts.

Currently, Ambari relies on getting the list of candidate hosts from the DB and 
excludes all hosts that are in maintenance mode. From that list, it picks the 
first host that is healthy (i.e., heartbeating). This means that the logic does 
not pick a random host.

Instead, Ambari should always pick a random host and prefer to schedule on 
hosts that have 0 in-progress commands.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 746bca4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 5d8f279 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 17760c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 e82900d 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 197b925 

Diff: https://reviews.apache.org/r/52964/diff/


Testing (updated)
---

mvn clean test


Thanks,

Alejandro Fernandez

Re: Review Request 50803: [Ambari-19781] Integrate Druid With Ambari

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50803/#review152194
---


Ship it!




Ship It!

- Swapan Shridhar


On Oct. 14, 2016, 11:15 a.m., Nishant Bangarwa wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50803/
> ---
> 
> (Updated Oct. 14, 2016, 11:15 a.m.)
> 
> 
> Review request for Ambari, Renjith Kamath and Swapan Shridhar.
> 
> 
> Bugs: Ambari-19781
> https://issues.apache.org/jira/browse/Ambari-19781
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Defines Druid as a new service in HDP 2.6
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-broker.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-coordinator.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-historical.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-logrotate.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-middlemanager.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-overlord.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-router.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/common-services/DRUID/0.9.2/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/broker.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/coordinator.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/historical.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/middlemanager.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/overlord.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/router.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/quicklinks/quicklinks.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/themes/theme.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/role_command_order.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/metainfo.xml 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
> 1f722dc 
>   ambari-server/src/test/python/stacks/2.6/DRUID/test_druid_historical.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.6/configs/default.json PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50803/diff/
> 
> 
> Testing
> ---
> 
> Tested it locally by installing ambari and adding the newly added resources. 
> Was able to install druid and start all newly added services.
> 
> 
> Thanks,
> 
> Nishant Bangarwa
> 
>

Re: Review Request 52996: For rolling upgrade of Kafka 0.10.0.1, set configs for backward compatibility

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52996/#review153239
---


Ship it!




Out of curiousity, instead of providing a message about removing the two 
properties, could we just have a new stage right before finalization that does 
it automatically? I know it won't restart anything, but at least the configs 
will be correct.

- Jonathan Hurley


On Oct. 18, 2016, 8:57 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52996/
> ---
> 
> (Updated Oct. 18, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18630
> https://issues.apache.org/jira/browse/AMBARI-18630
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the 2.5 upgrade pack, the following properties need to be set
> {code}
> inter.broker.protocol.version=0.9.0.0
> log.message.format.version=0.9.0.0
> {code}
> 
> after upgrade is done we should delete the inter.broker.protocol.version.
> Users should remove log.message.format.version once they update their clients.
> 
> (Still waiting on final confirmation for behavior from Kafka team)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> e8a2e2f 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> e89d62b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> e127dff 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> fa1a555 
> 
> Diff: https://reviews.apache.org/r/52996/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>