Re: Review Request 54334: Ambari Contrib View Has Inconsistent Version in pom.xml

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54334/
---

(Updated Dec. 2, 2016, 10:35 p.m.)


Review request for Ambari.


Bugs: AMBARI-19073
https://issues.apache.org/jira/browse/AMBARI-19073


Repository: ambari


Description
---

{{contrib/views/ambari-views-package/pom.xml}} has the wrong version defined 
which can cause build failures under certain conditions. The parent POM from 
{{contrib/views/pom.xml}} defines 2.0.0.0.0 while 
{{contrib/views/ambari-views-package/pom.xml}} defines 2.5.0.0.0.

{code}
git diff pom.xml
diff --git a/contrib/views/ambari-views-package/pom.xml 
b/contrib/views/ambari-views-package/pom.xml
index cb070e6..16c9985 100644
--- a/contrib/views/ambari-views-package/pom.xml
+++ b/contrib/views/ambari-views-package/pom.xml
@@ -26,7 +26,7 @@
   
 org.apache.ambari.contrib.views
 ambari-contrib-views
-2.5.0.0.0
+2.0.0.0-SNAPSHOT
   
{code}

This now matches {{contrib/views/pom.xml}}:}
{code}
http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
  
org.apache.ambari
ambari-project
2.0.0.0-SNAPSHOT
../../ambari-project
  
{code}


Diffs
-

  contrib/views/ambari-views-package/pom.xml cb070e6 

Diff: https://reviews.apache.org/r/54334/diff/


Testing (updated)
---

[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main  SUCCESS [  2.889 s]
[INFO] Apache Ambari Project POM .. SUCCESS [  0.005 s]
[INFO] Ambari Web . SUCCESS [ 54.477 s]
[INFO] Ambari Views ... SUCCESS [  1.011 s]
[INFO] Ambari Admin View .. SUCCESS [ 49.458 s]
[INFO] utility  SUCCESS [  0.260 s]
[INFO] ambari-metrics . SUCCESS [  0.097 s]
[INFO] Ambari Metrics Common .. SUCCESS [  0.671 s]
[INFO] Ambari Metrics Hadoop Sink . SUCCESS [  0.393 s]
[INFO] Ambari Metrics Flume Sink .. SUCCESS [  0.187 s]
[INFO] Ambari Metrics Kafka Sink .. SUCCESS [  0.183 s]
[INFO] Ambari Metrics Storm Sink .. SUCCESS [  4.063 s]
[INFO] Ambari Metrics Storm Sink (Legacy) . SUCCESS [  0.372 s]
[INFO] Ambari Metrics Collector ... SUCCESS [ 44.707 s]
[INFO] Ambari Metrics Monitor . SUCCESS [  0.047 s]
[INFO] Ambari Metrics Grafana . SUCCESS [  3.368 s]
[INFO] Ambari Metrics Assembly  SUCCESS [ 23.896 s]
[INFO] Ambari Server .. SUCCESS [  9.294 s]
[INFO] Ambari Functional Tests  SUCCESS [  0.191 s]
[INFO] Ambari Agent ... SUCCESS [  0.300 s]
[INFO] Ambari Client .. SUCCESS [  0.008 s]
[INFO] Ambari Python Client ... SUCCESS [  0.004 s]
[INFO] Ambari Groovy Client ... SUCCESS [  1.912 s]
[INFO] Ambari Shell ... SUCCESS [  0.008 s]
[INFO] Ambari Python Shell  SUCCESS [  0.003 s]
[INFO] Ambari Groovy Shell  SUCCESS [  0.340 s]
[INFO] ambari-logsearch ... SUCCESS [  0.034 s]
[INFO] Ambari Logsearch Appender .. SUCCESS [  0.111 s]
[INFO] Ambari Logsearch Solr Client ... SUCCESS [  0.166 s]
[INFO] Ambari Logsearch Portal  SUCCESS [ 35.417 s]
[INFO] Ambari Logsearch Log Feeder  SUCCESS [  0.520 s]
[INFO] Ambari Logsearch Assembly .. SUCCESS [  0.051 s]
[INFO] Ambari Logsearch Integration Test .. SUCCESS [ 16.689 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 54334: Ambari Contrib View Has Inconsistent Version in pom.xml

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54334/#review157884
---


Ship it!




Ship It!

- Sid Wagle


On Dec. 3, 2016, 2:24 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54334/
> ---
> 
> (Updated Dec. 3, 2016, 2:24 a.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-19073
> https://issues.apache.org/jira/browse/AMBARI-19073
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {{contrib/views/ambari-views-package/pom.xml}} has the wrong version defined 
> which can cause build failures under certain conditions. The parent POM from 
> {{contrib/views/pom.xml}} defines 2.0.0.0.0 while 
> {{contrib/views/ambari-views-package/pom.xml}} defines 2.5.0.0.0.
> 
> {code}
> git diff pom.xml
> diff --git a/contrib/views/ambari-views-package/pom.xml 
> b/contrib/views/ambari-views-package/pom.xml
> index cb070e6..16c9985 100644
> --- a/contrib/views/ambari-views-package/pom.xml
> +++ b/contrib/views/ambari-views-package/pom.xml
> @@ -26,7 +26,7 @@
>
>  org.apache.ambari.contrib.views
>  ambari-contrib-views
> -2.5.0.0.0
> +2.0.0.0-SNAPSHOT
>
> {code}
> 
> This now matches {{contrib/views/pom.xml}}:}
> {code}
> http://maven.apache.org/POM/4.0.0"; 
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
> http://maven.apache.org/maven-v4_0_0.xsd";>
>   
> org.apache.ambari
> ambari-project
> 2.0.0.0-SNAPSHOT
> ../../ambari-project
>   
> {code}
> 
> 
> Diffs
> -
> 
>   contrib/views/ambari-views-package/pom.xml cb070e6 
> 
> Diff: https://reviews.apache.org/r/54334/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 54334: Ambari Contrib View Has Inconsistent Version in pom.xml

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54334/
---

Review request for Ambari.


Bugs: AMBARI-19073
https://issues.apache.org/jira/browse/AMBARI-19073


Repository: ambari


Description
---

{{contrib/views/ambari-views-package/pom.xml}} has the wrong version defined 
which can cause build failures under certain conditions. The parent POM from 
{{contrib/views/pom.xml}} defines 2.0.0.0.0 while 
{{contrib/views/ambari-views-package/pom.xml}} defines 2.5.0.0.0.

{code}
git diff pom.xml
diff --git a/contrib/views/ambari-views-package/pom.xml 
b/contrib/views/ambari-views-package/pom.xml
index cb070e6..16c9985 100644
--- a/contrib/views/ambari-views-package/pom.xml
+++ b/contrib/views/ambari-views-package/pom.xml
@@ -26,7 +26,7 @@
   
 org.apache.ambari.contrib.views
 ambari-contrib-views
-2.5.0.0.0
+2.0.0.0-SNAPSHOT
   
{code}

This now matches {{contrib/views/pom.xml}}:}
{code}
http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
  
org.apache.ambari
ambari-project
2.0.0.0-SNAPSHOT
../../ambari-project
  
{code}


Diffs
-

  contrib/views/ambari-views-package/pom.xml cb070e6 

Diff: https://reviews.apache.org/r/54334/diff/


Testing
---


Thanks,

Jonathan Hurley

Re: Review Request 54270: Perf: Deploy 3000 Agent cluster and find perf bugs

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54270/#review157873
---


Ship it!




Ship It!

- Alejandro Fernandez


On Dec. 3, 2016, 12:32 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54270/
> ---
> 
> (Updated Dec. 3, 2016, 12:32 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19058
> https://issues.apache.org/jira/browse/AMBARI-19058
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Use Ambari 2.5 branch instead of trunk to deploy 2000-3000 agents on GCE and 
> start finding perf bugs in the following areas.
> 
> 
> Diffs
> -
> 
>   contrib/utils/perf/deploy-gce-perf-cluster.py 4737c6f 
> 
> Diff: https://reviews.apache.org/r/54270/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 54270: Perf: Deploy 3000 Agent cluster and find perf bugs

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54270/
---

(Updated Гру. 3, 2016, 12:32 до полудня)


Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.


Bugs: AMBARI-19058
https://issues.apache.org/jira/browse/AMBARI-19058


Repository: ambari


Description
---

Use Ambari 2.5 branch instead of trunk to deploy 2000-3000 agents on GCE and 
start finding perf bugs in the following areas.


Diffs (updated)
-

  contrib/utils/perf/deploy-gce-perf-cluster.py 4737c6f 

Diff: https://reviews.apache.org/r/54270/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 54329: Ambari Server Unit Test failure on branch-2.5 and trunk: org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54329/#review157859
---


Ship it!




Ship It!

- Jonathan Hurley


On Dec. 2, 2016, 6:43 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54329/
> ---
> 
> (Updated Dec. 2, 2016, 6:43 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19071
> https://issues.apache.org/jira/browse/AMBARI-19071
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This error is happening on both trunk and branch-2.5.
> 
> {code}
> Running org.apache.ambari.server.state.ServiceTest
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.352 sec <<< 
> FAILURE! - in org.apache.ambari.server.state.ServicePropertiesTest
> validatePropertySchemaOfServiceXMLs(org.apache.ambari.server.state.ServicePropertiesTest)
>   Time elapsed: 7.281 sec  <<< ERROR!
> org.apache.ambari.server.AmbariException: File 
> /home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
>  didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
> content of element 'property' is not complete. One of '{display-name, 
> filename, deleted, final, on-ambari-upgrade, on-stack-upgrade, property-type, 
> value-attributes, depends-on, property_depended_by}' is expected.
>   at 
> org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:50)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml
>  760e706 
> 
> Diff: https://reviews.apache.org/r/54329/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 54329: Ambari Server Unit Test failure on branch-2.5 and trunk: org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs

[Mahadev Konar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54329/#review157858
---


Ship it!




Ship It!

- Mahadev Konar


On Dec. 2, 2016, 11:43 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54329/
> ---
> 
> (Updated Dec. 2, 2016, 11:43 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19071
> https://issues.apache.org/jira/browse/AMBARI-19071
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This error is happening on both trunk and branch-2.5.
> 
> {code}
> Running org.apache.ambari.server.state.ServiceTest
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.352 sec <<< 
> FAILURE! - in org.apache.ambari.server.state.ServicePropertiesTest
> validatePropertySchemaOfServiceXMLs(org.apache.ambari.server.state.ServicePropertiesTest)
>   Time elapsed: 7.281 sec  <<< ERROR!
> org.apache.ambari.server.AmbariException: File 
> /home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
>  didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
> content of element 'property' is not complete. One of '{display-name, 
> filename, deleted, final, on-ambari-upgrade, on-stack-upgrade, property-type, 
> value-attributes, depends-on, property_depended_by}' is expected.
>   at 
> org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:50)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml
>  760e706 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml
>  760e706 
> 
> Diff: https://reviews.apache.org/r/54329/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 54329: Ambari Server Unit Test failure on branch-2.5 and trunk: org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54329/
---

Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Sumit Mohanty, 
and Sid Wagle.


Bugs: AMBARI-19071
https://issues.apache.org/jira/browse/AMBARI-19071


Repository: ambari


Description
---

This error is happening on both trunk and branch-2.5.

{code}
Running org.apache.ambari.server.state.ServiceTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.352 sec <<< 
FAILURE! - in org.apache.ambari.server.state.ServicePropertiesTest
validatePropertySchemaOfServiceXMLs(org.apache.ambari.server.state.ServicePropertiesTest)
  Time elapsed: 7.281 sec  <<< ERROR!
org.apache.ambari.server.AmbariException: File 
/home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
 didn't pass the validation. Error message is : cvc-complex-type.2.4.b: The 
content of element 'property' is not complete. One of '{display-name, filename, 
deleted, final, on-ambari-upgrade, on-stack-upgrade, property-type, 
value-attributes, depends-on, property_depended_by}' is expected.
at 
org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:50)
{code}


Diffs
-

  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml
 760e706 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml
 760e706 

Diff: https://reviews.apache.org/r/54329/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 54326: Remove 'llap_queue_capacity' references from UI code.

[Yusaku Sako]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54326/#review157846
---


Ship it!




Ship It!

- Yusaku Sako


On Dec. 2, 2016, 10:33 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54326/
> ---
> 
> (Updated Dec. 2, 2016, 10:33 p.m.)
> 
> 
> Review request for Ambari, Xi Wang and Yusaku Sako.
> 
> 
> Bugs: AMBARI-19069
> https://issues.apache.org/jira/browse/AMBARI-19069
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Config 'llap_queue_capacity' exists in hive-interactive-env. But with 
> AMBARI-18901 work, this config will be removed.
> ambari-web should also remove any special handling that it does with 
> 'llap_queue_capacity' as of now
> 
> 
> Diffs
> -
> 
>   ambari-web/app/messages.js 22720a9 
>   ambari-web/app/models/configs/objects/service_config_property.js 5d85ae0 
>   ambari-web/app/views/common/configs/widgets/slider_config_widget_view.js 
> 71156f4 
> 
> Diff: https://reviews.apache.org/r/54326/diff/
> 
> 
> Testing
> ---
> 
> Verified the patch on the cluster.
> Verified that the patch does not affect ambari-web unit test run:
> ambari-web (branch-2.5) unit test result with the patch:
> 25637 tests complete (23 seconds)
> 57 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Review Request 54326: Remove 'llap_queue_capacity' references from UI code.

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54326/
---

Review request for Ambari, Xi Wang and Yusaku Sako.


Bugs: AMBARI-19069
https://issues.apache.org/jira/browse/AMBARI-19069


Repository: ambari


Description
---

Config 'llap_queue_capacity' exists in hive-interactive-env. But with 
AMBARI-18901 work, this config will be removed.
ambari-web should also remove any special handling that it does with 
'llap_queue_capacity' as of now


Diffs
-

  ambari-web/app/messages.js 22720a9 
  ambari-web/app/models/configs/objects/service_config_property.js 5d85ae0 
  ambari-web/app/views/common/configs/widgets/slider_config_widget_view.js 
71156f4 

Diff: https://reviews.apache.org/r/54326/diff/


Testing
---

Verified the patch on the cluster.
Verified that the patch does not affect ambari-web unit test run:
ambari-web (branch-2.5) unit test result with the patch:
25637 tests complete (23 seconds)
57 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157825
---


Ship it!




Ship It!

- Alejandro Fernandez


On Dec. 2, 2016, 2:43 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  59dd9d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157822
---


Ship it!




Ship It!

- Jonathan Hurley


On Dec. 2, 2016, 3:45 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 3:45 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  0448b9ff6397e391ddc778f0f3f76bc37190299d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent .. SUCCESS [25.738s]
> [INFO] Ambari Client . SUCCESS [0.050s]
> [INFO] Ambari Python Client .. SUCCESS [0.945s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.984s]
> [INFO] Ambari Shell .. SUCCESS [0.035s]
> [INFO] Ambari Python Shell ... SUCCESS [0.653s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
> [INFO] ambari-logsearch .. SUCCESS [0.299s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.200s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.170s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.326s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.665s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.091s]
> [INFO] Ambari Logsearch Integration Test . SUCCESS [0.372s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:23.315s
> [INFO] Finished at: Mon Nov 14 13:44:45 PST 2016
> [INFO] Final Memory: 327M/1187M
> [INFO] 
> 

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Jonathan Hurley]

> On Dec. 2, 2016, 2:41 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java,
> >  line 335
> > 
> >
> > What if the cluster name is provided, but doesn't exist (due to a 
> > rename) - now we'd lose heartbeats then too, right?
> 
> Nahappan Somasundaram wrote:
> What if the cluster name is provided, but doesn't exist (due to a rename) 
>  
> >> Isn't that a bug by itself?
> 
> Moreover, ClustersImpl.getCluster() and ClusterImpl.getService() throw an 
> exception if the retrieved cluster or service object is null, which is a bad 
> design. Getters/setters should not throw exceptions, just return the object 
> (null or not). Because of this, there is no way of verifying if the cluster 
> or service objects are null, other than to handle the exception and ignoring 
> it (which is bad). 
> 
> Stage.java::addGenericExecutionCommand() could be a better place to put 
> this logic.
> 
> Nahappan Somasundaram wrote:
> AmbariManagementControllerImpl.java::createHostAction() works better for 
> execution commands targeted on a host. The cluster and service exist at this 
> point.

Commands come back scoped by cluster name - so a rename with running ops or 
status commands could cause a loss of heartbeat if the heartbeats can't be 
delivered. It's a yucky problem.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157803
---


On Dec. 2, 2016, 3:45 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 3:45 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  0448b9ff6397e391ddc778f0f3f76bc37190299d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent ..

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/
---

(Updated Dec. 2, 2016, 12:45 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit 
Mohanty.


Changes
---

Set the value of credential store enabled on the execution command in 
AmbariManagementController::createHostAction() where the cluster and service 
are available for the host.
Please ignore revision 7.


Bugs: AMBARI-1
https://issues.apache.org/jira/browse/AMBARI-1


Repository: ambari


Description
---

AMBARI-1: Ambari-agent: Create configuration files with JCEKS information


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini 
61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
11c8cbedbf4e3c9d3325090421dc2011a8e56668 
  ambari-agent/src/packages/tarball/all.xml 
c4812087976f9ec38c20d92c20747eb8988d8290 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 75bef3027bf00117c1d423c01af936cf8f45ed1c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 0448b9ff6397e391ddc778f0f3f76bc37190299d 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
 a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 

Diff: https://reviews.apache.org/r/53747/diff/


Testing
---

** 1. mvn clean install -DskipTests **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [7.433s]
[INFO] Apache Ambari Project POM . SUCCESS [0.040s]
[INFO] Ambari Web  SUCCESS [1:11.223s]
[INFO] Ambari Views .. SUCCESS [1.128s]
[INFO] Ambari Admin View . SUCCESS [6.366s]
[INFO] utility ... SUCCESS [0.359s]
[INFO] ambari-metrics  SUCCESS [0.721s]
[INFO] Ambari Metrics Common . SUCCESS [6.490s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
[INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
[INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
[INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
[INFO] Ambari Server . SUCCESS [2:28.842s]
[INFO] Ambari Functional Tests ... SUCCESS [1.071s]
[INFO] Ambari Agent .. SUCCESS [25.738s]
[INFO] Ambari Client . SUCCESS [0.050s]
[INFO] Ambari Python Client .. SUCCESS [0.945s]
[INFO] Ambari Groovy Client .. SUCCESS [1.984s]
[INFO] Ambari Shell .. SUCCESS [0.035s]
[INFO] Ambari Python Shell ... SUCCESS [0.653s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
[INFO] ambari-logsearch .. SUCCESS [0.299s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.200s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.170s]
[INFO] Ambari Logsearch Portal ... SUCCESS [7.326s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.665s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.091s]
[INFO] Ambari Logsearch Integration Test . SUCCESS [0.372s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:23.315s
[INFO] Finished at: Mon Nov 14 13:44:45 PST 2016
[INFO] Final Memory: 327M/1187M
[INFO] 

** 2. mvn test -DskipPythonTests -Dtest=*ExecutionCommand*,*HeartBeatHandler* **

---
 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=tr

Re: Review Request 54276: AMBARI-19038: Support migration of LDAP users & groups to PAM

[Vishal Ghugare]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54276/
---

(Updated Dec. 2, 2016, 12:19 p.m.)


Review request for Ambari, Alejandro Fernandez and Di Li.


Bugs: AMBARI-19038
https://issues.apache.org/jira/browse/AMBARI-19038


Repository: ambari


Description
---

Story to address migration of LDAP users & groups to PAM.
Note: LDAP usesids that collide with existing PAM userids in Ambari metastore 
will not be migrated.


Diffs
-

  ambari-server/sbin/ambari-server 8afabb1 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/LdapToPamMigrationHelper.java
 PRE-CREATION 
  ambari-server/src/main/python/ambari-server.py 21bd0bb 
  ambari-server/src/main/python/ambari_server/setupActions.py 7ea0752 
  ambari-server/src/main/python/ambari_server/setupSecurity.py 1508d27 

Diff: https://reviews.apache.org/r/54276/diff/


Testing (updated)
---

1.  Install Cluster and Configure for LDAP
2.  Perform LDAP sync -  you can verify by logging with LDAP users
3.  Setup the switch to PAM (ambari-server setup-pam)
4.  Perform Migrate from LDAP to PAM (ambari-server migrate-ldap-pam)
5.  Login with PAM users


Thanks,

Vishal Ghugare

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Nahappan Somasundaram]

> On Dec. 2, 2016, 11:41 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java,
> >  line 335
> > 
> >
> > What if the cluster name is provided, but doesn't exist (due to a 
> > rename) - now we'd lose heartbeats then too, right?
> 
> Nahappan Somasundaram wrote:
> What if the cluster name is provided, but doesn't exist (due to a rename) 
>  
> >> Isn't that a bug by itself?
> 
> Moreover, ClustersImpl.getCluster() and ClusterImpl.getService() throw an 
> exception if the retrieved cluster or service object is null, which is a bad 
> design. Getters/setters should not throw exceptions, just return the object 
> (null or not). Because of this, there is no way of verifying if the cluster 
> or service objects are null, other than to handle the exception and ignoring 
> it (which is bad). 
> 
> Stage.java::addGenericExecutionCommand() could be a better place to put 
> this logic.

AmbariManagementControllerImpl.java::createHostAction() works better for 
execution commands targeted on a host. The cluster and service exist at this 
point.


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157803
---


On Dec. 2, 2016, 10:18 a.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 10:18 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent .. SUCCESS [25.738s]
> [INFO] Ambari Client . SUCCESS [0.050s]
> [INFO] Ambari Python Client .. SUCCESS [0.945s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.984s]
> [INFO] Ambari Shell .. SUCCESS [0.035s]
> [INFO] Ambari Python Shell ...

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Nahappan Somasundaram]

> On Dec. 2, 2016, 11:41 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java,
> >  line 335
> > 
> >
> > What if the cluster name is provided, but doesn't exist (due to a 
> > rename) - now we'd lose heartbeats then too, right?

What if the cluster name is provided, but doesn't exist (due to a rename)  
>> Isn't that a bug by itself?

Moreover, ClustersImpl.getCluster() and ClusterImpl.getService() throw an 
exception if the retrieved cluster or service object is null, which is a bad 
design. Getters/setters should not throw exceptions, just return the object 
(null or not). Because of this, there is no way of verifying if the cluster or 
service objects are null, other than to handle the exception and ignoring it 
(which is bad). 

Stage.java::addGenericExecutionCommand() could be a better place to put this 
logic.


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157803
---


On Dec. 2, 2016, 10:18 a.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 10:18 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent .. SUCCESS [25.738s]
> [INFO] Ambari Client . SUCCESS [0.050s]
> [INFO] Ambari Python Client .. SUCCESS [0.945s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.984s]
> [INFO] Ambari Shell .. SUCCESS [0.035s]
> [INFO] Ambari Python Shell ... SUCCESS [0.653s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
> [INFO] ambari-logsearch .. SUCCESS [0.299s]
> [INFO] Ambari Logsearch Appender ...

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Jonathan Hurley]

> On Dec. 2, 2016, 1:26 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java,
> >  line 57
> > 
> >
> > I think this will work most of the time. However, if users are deleting 
> > x records out of the host_role_command table, then there's still a chance 
> > that an RU/EU will still have tasks that aren't deleted, and hence the 
> > upgrade will still show up as PENDING.
> > 
> > In my opinion, we need a better way of deleting old records. What do 
> > you think?
> > 
> > It would be ideal if the user can be prompted to delete data older than 
> > x days. If an EU/RU falls in that category, then we find the requestIDs and 
> > then delete all tasks that belong to those requestIDs. This way, there is 
> > no "partial deletion" of data, which can be problematic.
> 
> Nate Cole wrote:
> Seems to me that people shouldn't be removing records out of the DB that 
> can orphan requests like this.  We have a way to clean records, it just 
> doesn't look like it's implemented for R/S/T (only for Alerts for some 
> reason).
> 
> We can even make it such that only COMPLETED requests are fully purged.  
> Who cares about the history that all DN were successfully restarted three 
> months ago?
> 
> Alejandro Fernandez wrote:
> I believe we have a PreCheck that finds the last Upgrade and makes sure 
> that it completed (not sure if it checks the request status or the last 3 
> tasks during Finalize).
> Either way, I don't think users should leave orphan records. Our story 
> for cleaning up history shouldn't be through manually deleting records. I 
> would prefer if this could be done through an API and eventually UI.

The problem is that we're facing this issue now - with databases that have been 
manually purged. The commands being run are done so to include all tasks within 
a given time period as long as they don't spill over - they run queries before 
to figure out if their query will cut requests in half or if they'll clean them 
out cleanly.

That's a different Jira - this was to address the fact that a 0-task request is 
COMPLETED, not PENDING (unless its logical)


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review15
---


On Dec. 2, 2016, 9:43 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 9:43 a.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
>  

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157803
---




ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 (line 335)


What if the cluster name is provided, but doesn't exist (due to a rename) - 
now we'd lose heartbeats then too, right?


- Jonathan Hurley


On Dec. 2, 2016, 1:18 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent .. SUCCESS [25.738s]
> [INFO] Ambari Client . SUCCESS [0.050s]
> [INFO] Ambari Python Client .. SUCCESS [0.945s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.984s]
> [INFO] Ambari Shell .. SUCCESS [0.035s]
> [INFO] Ambari Python Shell ... SUCCESS [0.653s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
> [INFO] ambari-logsearch .. SUCCESS [0.299s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.200s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.170s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.326s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.665s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.091s]
> [INFO] Ambari Logsearch Integration Test . SUCCESS [0.372s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:23.315s
> [INFO] Finished at: Mon Nov 14 13:44:45 PST 2

Re: Review Request 54314: AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA

[Sangeeta Ravindran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54314/#review157801
---


Ship it!




Ship It!

- Sangeeta Ravindran


On Dec. 2, 2016, 6:18 p.m., Jesus Alvarez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54314/
> ---
> 
> (Updated Dec. 2, 2016, 6:18 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran.
> 
> 
> Bugs: AMBARI-18926
> https://issues.apache.org/jira/browse/AMBARI-18926
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/admin/kerberos/step1_controller.js 2feeb7f 
>   ambari-web/test/controllers/main/admin/kerberos/step1_controller_test.js 
> 9280367f 
> 
> Diff: https://reviews.apache.org/r/54314/diff/
> 
> 
> Testing
> ---
> 
> Tested enabling kerberos using an MIT KDC, and enabling kerberos with freeipa.
> Tested navigating back from steps 2,3,4,5 to step 1 and closing wizard all 
> worked ok.
> 
> 
> Thanks,
> 
> Jesus Alvarez
> 
>

Re: Review Request 54259: In stack version, build number should not be mandatory

[Amruta Borkar]

> On Dec. 2, 2016, 7:16 p.m., Di Li wrote:
> > Ship It!

Thanks Di, 
Could you please help me push this to trunk?


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54259/#review157793
---


On Dec. 2, 2016, 5:44 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54259/
> ---
> 
> (Updated Dec. 2, 2016, 5:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Di Li.
> 
> 
> Bugs: AMBARI-19039
> https://issues.apache.org/jira/browse/AMBARI-19039
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In stack version, build number should not be mandatory. mapreduce.tar.gz file 
> does not get copied during blueprint deployment when stack version does not 
> have build version.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  519c88b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  ca8fe19 
> 
> Diff: https://reviews.apache.org/r/54259/diff/
> 
> 
> Testing
> ---
> 
> Test manually with stack version with build number and without build number.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Review Request 54318: Escaped usernames passed to the post-user creation script

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54318/
---

Review request for Ambari, Attila Doroszlai, Sandor Magyari, and Sebastian 
Toader.


Bugs: AMBARI-19067
https://issues.apache.org/jira/browse/AMBARI-19067


Repository: ambari


Description
---

The initial implementation of the post user creation hook generates a json file 
that is passed to the fast hdfs tool to perform user home directory creation in 
bulk. If usernames that come from the UI or from LDAPsync contain special 
caracters the generated JSON may be corrupted.

The post user creation hook is amended by this fix to escape usernames in the 
generated json.


Diffs
-

  ambari-server/src/main/resources/scripts/post-user-creation-hook.sh 34169c1 

Diff: https://reviews.apache.org/r/54318/diff/


Testing
---

Manually in progress.


Thanks,

Laszlo Puskas

Re: Review Request 54259: In stack version, build number should not be mandatory

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54259/#review157793
---


Ship it!




Ship It!

- Di Li


On Dec. 2, 2016, 5:44 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54259/
> ---
> 
> (Updated Dec. 2, 2016, 5:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Di Li.
> 
> 
> Bugs: AMBARI-19039
> https://issues.apache.org/jira/browse/AMBARI-19039
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In stack version, build number should not be mandatory. mapreduce.tar.gz file 
> does not get copied during blueprint deployment when stack version does not 
> have build version.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  519c88b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  ca8fe19 
> 
> Diff: https://reviews.apache.org/r/54259/diff/
> 
> 
> Testing
> ---
> 
> Test manually with stack version with build number and without build number.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 54314: AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54314/#review157794
---


Ship it!




Ship It!

- Di Li


On Dec. 2, 2016, 6:18 p.m., Jesus Alvarez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54314/
> ---
> 
> (Updated Dec. 2, 2016, 6:18 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran.
> 
> 
> Bugs: AMBARI-18926
> https://issues.apache.org/jira/browse/AMBARI-18926
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/admin/kerberos/step1_controller.js 2feeb7f 
>   ambari-web/test/controllers/main/admin/kerberos/step1_controller_test.js 
> 9280367f 
> 
> Diff: https://reviews.apache.org/r/54314/diff/
> 
> 
> Testing
> ---
> 
> Tested enabling kerberos using an MIT KDC, and enabling kerberos with freeipa.
> Tested navigating back from steps 2,3,4,5 to step 1 and closing wizard all 
> worked ok.
> 
> 
> Thanks,
> 
> Jesus Alvarez
> 
>

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Alejandro Fernandez]

> On Dec. 2, 2016, 6:26 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java,
> >  line 57
> > 
> >
> > I think this will work most of the time. However, if users are deleting 
> > x records out of the host_role_command table, then there's still a chance 
> > that an RU/EU will still have tasks that aren't deleted, and hence the 
> > upgrade will still show up as PENDING.
> > 
> > In my opinion, we need a better way of deleting old records. What do 
> > you think?
> > 
> > It would be ideal if the user can be prompted to delete data older than 
> > x days. If an EU/RU falls in that category, then we find the requestIDs and 
> > then delete all tasks that belong to those requestIDs. This way, there is 
> > no "partial deletion" of data, which can be problematic.
> 
> Nate Cole wrote:
> Seems to me that people shouldn't be removing records out of the DB that 
> can orphan requests like this.  We have a way to clean records, it just 
> doesn't look like it's implemented for R/S/T (only for Alerts for some 
> reason).
> 
> We can even make it such that only COMPLETED requests are fully purged.  
> Who cares about the history that all DN were successfully restarted three 
> months ago?

I believe we have a PreCheck that finds the last Upgrade and makes sure that it 
completed (not sure if it checks the request status or the last 3 tasks during 
Finalize).
Either way, I don't think users should leave orphan records. Our story for 
cleaning up history shouldn't be through manually deleting records. I would 
prefer if this could be done through an API and eventually UI.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review15
---


On Dec. 2, 2016, 2:43 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_ro

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157786
---


Ship it!




Ship It!

- Nate Cole


On Dec. 2, 2016, 9:43 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 9:43 a.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  59dd9d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Nate Cole]

> On Dec. 2, 2016, 1:26 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java,
> >  line 57
> > 
> >
> > I think this will work most of the time. However, if users are deleting 
> > x records out of the host_role_command table, then there's still a chance 
> > that an RU/EU will still have tasks that aren't deleted, and hence the 
> > upgrade will still show up as PENDING.
> > 
> > In my opinion, we need a better way of deleting old records. What do 
> > you think?
> > 
> > It would be ideal if the user can be prompted to delete data older than 
> > x days. If an EU/RU falls in that category, then we find the requestIDs and 
> > then delete all tasks that belong to those requestIDs. This way, there is 
> > no "partial deletion" of data, which can be problematic.

Seems to me that people shouldn't be removing records out of the DB that can 
orphan requests like this.  We have a way to clean records, it just doesn't 
look like it's implemented for R/S/T (only for Alerts for some reason).

We can even make it such that only COMPLETED requests are fully purged.  Who 
cares about the history that all DN were successfully restarted three months 
ago?


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review15
---


On Dec. 2, 2016, 9:43 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 9:43 a.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  59dd9d9 
>   
> ambari-server/sr

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/#review157783
---


Ship it!




Ship It!

- Nate Cole


On Dec. 2, 2016, 1:18 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53747/
> ---
> 
> (Updated Dec. 2, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-1
> https://issues.apache.org/jira/browse/AMBARI-1
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-1: Ambari-agent: Create configuration files with JCEKS information
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 
> 61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 11c8cbedbf4e3c9d3325090421dc2011a8e56668 
>   ambari-agent/src/packages/tarball/all.xml 
> c4812087976f9ec38c20d92c20747eb8988d8290 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  75bef3027bf00117c1d423c01af936cf8f45ed1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 
> 
> Diff: https://reviews.apache.org/r/53747/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.433s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.040s]
> [INFO] Ambari Web  SUCCESS [1:11.223s]
> [INFO] Ambari Views .. SUCCESS [1.128s]
> [INFO] Ambari Admin View . SUCCESS [6.366s]
> [INFO] utility ... SUCCESS [0.359s]
> [INFO] ambari-metrics  SUCCESS [0.721s]
> [INFO] Ambari Metrics Common . SUCCESS [6.490s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
> [INFO] Ambari Server . SUCCESS [2:28.842s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.071s]
> [INFO] Ambari Agent .. SUCCESS [25.738s]
> [INFO] Ambari Client . SUCCESS [0.050s]
> [INFO] Ambari Python Client .. SUCCESS [0.945s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.984s]
> [INFO] Ambari Shell .. SUCCESS [0.035s]
> [INFO] Ambari Python Shell ... SUCCESS [0.653s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
> [INFO] ambari-logsearch .. SUCCESS [0.299s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.200s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.170s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.326s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.665s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.091s]
> [INFO] Ambari Logsearch Integration Test . SUCCESS [0.372s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:23.315s
> [INFO] Finished at: Mon Nov 14 13:44:45 PST 2016
> [INFO] Final Memory: 327M/1187M
> [INFO] 
> 
> 
> ** 2. mvn test -DskipPythonTests -Dtest=*ExecutionCommand*,*HeartBeatHandler* 
> **
> 
> ---

Re: Review Request 54259: In stack version, build number should not be mandatory

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54259/#review157779
---


Ship it!




Ship It!

- Alejandro Fernandez


On Dec. 2, 2016, 5:44 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54259/
> ---
> 
> (Updated Dec. 2, 2016, 5:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Di Li.
> 
> 
> Bugs: AMBARI-19039
> https://issues.apache.org/jira/browse/AMBARI-19039
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In stack version, build number should not be mandatory. mapreduce.tar.gz file 
> does not get copied during blueprint deployment when stack version does not 
> have build version.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  519c88b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  ca8fe19 
> 
> Diff: https://reviews.apache.org/r/54259/diff/
> 
> 
> Testing
> ---
> 
> Test manually with stack version with build number and without build number.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 54175: AMBARI-19020 Ubuntu14/16 Add Support for Zookeeper on HDP 2.5

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54175/#review157778
---



This has been pushed to trunk and branch-2.5.  Please close this review and 
associated JIRA.

- Nate Cole


On Nov. 30, 2016, 3:08 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54175/
> ---
> 
> (Updated Nov. 30, 2016, 3:08 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-19020
> https://issues.apache.org/jira/browse/AMBARI-19020
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-19020 Ubuntu14/16 Add Support for Zookeeper on HDP 2.5
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.6/metainfo.xml 
> 525078e 
> 
> Diff: https://reviews.apache.org/r/54175/diff/
> 
> 
> Testing
> ---
> 
> Manifest change only. Unit test not affected. Tested E2E.
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review15
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
 (line 57)


I think this will work most of the time. However, if users are deleting x 
records out of the host_role_command table, then there's still a chance that an 
RU/EU will still have tasks that aren't deleted, and hence the upgrade will 
still show up as PENDING.

In my opinion, we need a better way of deleting old records. What do you 
think?

It would be ideal if the user can be prompted to delete data older than x 
days. If an EU/RU falls in that category, then we find the requestIDs and then 
delete all tasks that belong to those requestIDs. This way, there is no 
"partial deletion" of data, which can be problematic.


- Alejandro Fernandez


On Dec. 2, 2016, 2:43 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  59dd9d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> --

Re: Review Request 54169: AMBARI-19007 Atlas to support configuration of hooks from separate cluster

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157776
---


Ship it!




Ship It!

- Alejandro Fernandez


On Dec. 2, 2016, 1:01 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> ---
> 
> (Updated Dec. 2, 2016, 1:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan 
> Neethiraj, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) Introduced new flag for all the hook suported 
> components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set 
> the hook flag. This flag value is used to recommend the expected 
> configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add 
> common-atlas application properties under each hook config-type eg: 
> hive-atlas-application.properties config-type which are require to 
> communicate with external ATLAS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a1d2f95 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  ec846f8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  52de784 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
>  09cced6 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  4429253 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  0fb21d0 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  150f629 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  5e2c709 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  bcc598a 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  01e5f00 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  fe3f34a 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
>  f682e97 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  f7823d2 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  283f54d 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  68f06db 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
>  9547335 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  aca0681 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  bda4fe2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  ab350dc 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  b71f4a9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 775dbab 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  f2dd099 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  ce0b387 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 39cbbce 
> 
> Diff: https://reviews.apache.org/r/54169/diff/
> 
> 
> Testing
> ---
> 
> Tested Atlas installation via UI and blue-print. With blueprint used 
> config_recommendation_strategy as NEVER_APPLY and 
> ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
> 
> 
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - 
> in 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> 
> Results :
> 
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> [INFO] 
> 

Re: Review Request 53747: AMBARI-18888: Ambari-agent: Create configuration files with JCEKS information

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53747/
---

(Updated Dec. 2, 2016, 10:18 a.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit 
Mohanty.


Changes
---

HeartBeatHandler.java::sendCommands(): Verify that cluster name and service 
name are not null before accessing the cluster and service objects.


Bugs: AMBARI-1
https://issues.apache.org/jira/browse/AMBARI-1


Repository: ambari


Description
---

AMBARI-1: Ambari-agent: Create configuration files with JCEKS information


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini 
61948d4b1f5db37f0c9800a01187ede2cb8f92f2 
  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
11c8cbedbf4e3c9d3325090421dc2011a8e56668 
  ambari-agent/src/packages/tarball/all.xml 
c4812087976f9ec38c20d92c20747eb8988d8290 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 ef1ee4fb4f7480821c78f2149f438ce3fcd249de 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 75bef3027bf00117c1d423c01af936cf8f45ed1c 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
 a50a1162a4b1ee6604e641a1a21ecf3f15fe96fa 

Diff: https://reviews.apache.org/r/53747/diff/


Testing
---

** 1. mvn clean install -DskipTests **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [7.433s]
[INFO] Apache Ambari Project POM . SUCCESS [0.040s]
[INFO] Ambari Web  SUCCESS [1:11.223s]
[INFO] Ambari Views .. SUCCESS [1.128s]
[INFO] Ambari Admin View . SUCCESS [6.366s]
[INFO] utility ... SUCCESS [0.359s]
[INFO] ambari-metrics  SUCCESS [0.721s]
[INFO] Ambari Metrics Common . SUCCESS [6.490s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [3.222s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [1.221s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [1.192s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.991s]
[INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.501s]
[INFO] Ambari Metrics Collector .. SUCCESS [9.142s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.627s]
[INFO] Ambari Metrics Grafana  SUCCESS [0.828s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:13.892s]
[INFO] Ambari Server . SUCCESS [2:28.842s]
[INFO] Ambari Functional Tests ... SUCCESS [1.071s]
[INFO] Ambari Agent .. SUCCESS [25.738s]
[INFO] Ambari Client . SUCCESS [0.050s]
[INFO] Ambari Python Client .. SUCCESS [0.945s]
[INFO] Ambari Groovy Client .. SUCCESS [1.984s]
[INFO] Ambari Shell .. SUCCESS [0.035s]
[INFO] Ambari Python Shell ... SUCCESS [0.653s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.782s]
[INFO] ambari-logsearch .. SUCCESS [0.299s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.200s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.170s]
[INFO] Ambari Logsearch Portal ... SUCCESS [7.326s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.665s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.091s]
[INFO] Ambari Logsearch Integration Test . SUCCESS [0.372s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:23.315s
[INFO] Finished at: Mon Nov 14 13:44:45 PST 2016
[INFO] Final Memory: 327M/1187M
[INFO] 

** 2. mvn test -DskipPythonTests -Dtest=*ExecutionCommand*,*HeartBeatHandler* **

---
 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.actionmanager.ExecutionCommandWrapperTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.124 sec - in 
org.apache.ambari.server.actionmanager.Execut

Re: Review Request 54314: AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA

[Jesus Alvarez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54314/
---

(Updated Dec. 2, 2016, 6:18 p.m.)


Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran.


Bugs: AMBARI-18926
https://issues.apache.org/jira/browse/AMBARI-18926


Repository: ambari


Description
---

AMBARI-18926: Kerberos Wizard UI creates duplicate radio buttons for FreeIPA


Diffs
-

  ambari-web/app/controllers/main/admin/kerberos/step1_controller.js 2feeb7f 
  ambari-web/test/controllers/main/admin/kerberos/step1_controller_test.js 
9280367f 

Diff: https://reviews.apache.org/r/54314/diff/


Testing
---

Tested enabling kerberos using an MIT KDC, and enabling kerberos with freeipa.
Tested navigating back from steps 2,3,4,5 to step 1 and closing wizard all 
worked ok.


Thanks,

Jesus Alvarez

Re: Review Request 54175: AMBARI-19020 Ubuntu14/16 Add Support for Zookeeper on HDP 2.5

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54175/#review157773
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 30, 2016, 3:08 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54175/
> ---
> 
> (Updated Nov. 30, 2016, 3:08 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-19020
> https://issues.apache.org/jira/browse/AMBARI-19020
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-19020 Ubuntu14/16 Add Support for Zookeeper on HDP 2.5
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.6/metainfo.xml 
> 525078e 
> 
> Diff: https://reviews.apache.org/r/54175/diff/
> 
> 
> Testing
> ---
> 
> Manifest change only. Unit test not affected. Tested E2E.
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 54259: In stack version, build number should not be mandatory

[Amruta Borkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54259/
---

(Updated Dec. 2, 2016, 5:44 p.m.)


Review request for Ambari, Alejandro Fernandez and Di Li.


Bugs: AMBARI-19039
https://issues.apache.org/jira/browse/AMBARI-19039


Repository: ambari


Description
---

In stack version, build number should not be mandatory. mapreduce.tar.gz file 
does not get copied during blueprint deployment when stack version does not 
have build version.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 519c88b 
  
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
 ca8fe19 

Diff: https://reviews.apache.org/r/54259/diff/


Testing
---

Test manually with stack version with build number and without build number.


Thanks,

Amruta Borkar

Re: Review Request 54259: In stack version, build number should not be mandatory

[Amruta Borkar]

> On Dec. 1, 2016, 7:07 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py,
> >  line 191
> > 
> >
> > If there's a dash, then we should have at least one number after it.
> > 
> > ([\d.]+(-\d+)?)
> > 
> > If you do this, then need to change it to
> > "if matches and len(matches) == 2:"

Hello Alejandro,
Thanks for pointing that out. I modified the regex accordingly, also 
len(matches)=2 was causing due to grouping. So I modified it so that it does 
not return sub groups from the string and we get only one string with complete 
version number. So We don't have to modify the IF condition.


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54259/#review157628
---


On Dec. 1, 2016, 6:13 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54259/
> ---
> 
> (Updated Dec. 1, 2016, 6:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Di Li.
> 
> 
> Bugs: AMBARI-19039
> https://issues.apache.org/jira/browse/AMBARI-19039
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In stack version, build number should not be mandatory. mapreduce.tar.gz file 
> does not get copied during blueprint deployment when stack version does not 
> have build version.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  519c88b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  ca8fe19 
> 
> Diff: https://reviews.apache.org/r/54259/diff/
> 
> 
> Testing
> ---
> 
> Test manually with stack version with build number and without build number.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 54276: AMBARI-19038: Support migration of LDAP users & groups to PAM

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54276/#review157761
---



hi Vishal, 

Could you please described how you have tested it?

- Di Li


On Dec. 2, 2016, 12:29 a.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54276/
> ---
> 
> (Updated Dec. 2, 2016, 12:29 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Di Li.
> 
> 
> Bugs: AMBARI-19038
> https://issues.apache.org/jira/browse/AMBARI-19038
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Story to address migration of LDAP users & groups to PAM.
> Note: LDAP usesids that collide with existing PAM userids in Ambari metastore 
> will not be migrated.
> 
> 
> Diffs
> -
> 
>   ambari-server/sbin/ambari-server 8afabb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/LdapToPamMigrationHelper.java
>  PRE-CREATION 
>   ambari-server/src/main/python/ambari-server.py 21bd0bb 
>   ambari-server/src/main/python/ambari_server/setupActions.py 7ea0752 
>   ambari-server/src/main/python/ambari_server/setupSecurity.py 1508d27 
> 
> Diff: https://reviews.apache.org/r/54276/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vishal Ghugare
> 
>

Re: Review Request 54267: RU: wrong version exposed when Downgrade is going

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54267/
---

(Updated Dec. 2, 2016, 11:18 a.m.)


Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-19053
https://issues.apache.org/jira/browse/AMBARI-19053


Repository: ambari


Description
---

We are incorrectly using the current cluster_version as the from_version on the 
entity.  For a downgrade, the from_ and to_ versions were the same.  Also added 
downgrade_allowed=false when downgrading, as that is more logically correct.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 9034989 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 14e3d08 

Diff: https://reviews.apache.org/r/54267/diff/


Testing (updated)
---

Manual.  Automated:

Tests run: 4791, Failures: 0, Errors: 0, Skipped: 37

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 34:31.727s
[INFO] Finished at: Fri Dec 02 10:56:52 EST 2016
[INFO] Final Memory: 37M/677M
[INFO] 


Thanks,

Nate Cole

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157760
---


Ship it!




Ship It!

- Robert Levas


On Dec. 2, 2016, 9:43 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 2, 2016, 9:43 a.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  59dd9d9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53943: Improve remoteIp in audit log

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53943/#review157758
---



Pushed to trunk and branch-2.5 - please close this review.

- Jonathan Hurley


On Nov. 29, 2016, 7:16 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53943/
> ---
> 
> (Updated Nov. 29, 2016, 7:16 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Oliver 
> Szabo, Robert Levas, Robert Nettleton, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18944
> https://issues.apache.org/jira/browse/AMBARI-18944
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Mutil-agent were configured in some production environments, RemoteIp 
> therefore will obtain mutilple IP addresses, like 
> (RemoteIp(172.16.0.66,172.16.193.56),)which causes errors in log output.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/RequestUtils.java 
> 0ac782f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/utils/RequestUtilsTest.java
>  595127e 
> 
> Diff: https://reviews.apache.org/r/53943/diff/
> 
> 
> Testing
> ---
> 
> RequestUtilsTest All Tests Passsed ,Process finished with exit code 0
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 52155: ambari server upgrade ambari to 2.1.1 duplicate key error

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52155/#review157756
---



This has been pushed to trunk and branch-2.5 - please close the review.

- Jonathan Hurley


On Nov. 21, 2016, 2:52 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52155/
> ---
> 
> (Updated Nov. 21, 2016, 2:52 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, Robert Nettleton, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18441
> https://issues.apache.org/jira/browse/AMBARI-18441
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> My first upgrade test failed due to my environment issue. Since in real 
> production environment this failure would  occur again.
> 
> ambari upgrade to 2.1.1 and later version, if the first time failed , excute 
> ambari-server upgrade again will error duplicate key value violates unique 
> constraint "pk_hostcomponentstate".
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog211.java
>  db13612 
> 
> Diff: https://reviews.apache.org/r/52155/diff/
> 
> 
> Testing
> ---
> 
> the unit test is UpgradeCatalog211Test.java: Process finished with exit code 
> 0.
> done !
> 
> 
> File Attachments
> 
> 
> AMBARI-18441-02.patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/11/16/4c6f3d15-96cc-4c22-babe-b45b103a8c30__AMBARI-18441-02.patch
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 54308: Refactor widgets on Dashboard page

[Aleksandr Kovalenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54308/#review157753
---


Ship it!




Ship It!

- Aleksandr Kovalenko


On Дек. 2, 2016, 2:47 п.п., Andrii Tkach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54308/
> ---
> 
> (Updated Дек. 2, 2016, 2:47 п.п.)
> 
> 
> Review request for Ambari and Aleksandr Kovalenko.
> 
> 
> Bugs: AMBARI-19064
> https://issues.apache.org/jira/browse/AMBARI-19064
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Refactor widgets on Dashboard page
> 
> 
> Diffs
> -
> 
>   ambari-web/app/mixins/main/dashboard/widgets/editable.js f330ec3 
>   ambari-web/app/templates/main/dashboard/edit_widget_popup.hbs 3a36ed6 
>   
> ambari-web/app/templates/main/dashboard/edit_widget_popup_single_threshold.hbs
>  d80e601 
>   ambari-web/app/templates/main/dashboard/plus_button_filter.hbs df7714c 
>   ambari-web/app/templates/main/dashboard/widgets.hbs a778033 
>   ambari-web/app/templates/main/dashboard/widgets/pie_chart.hbs fdeafd1 
>   ambari-web/app/views/common/not-scrollable-textarea.js 1898714 
>   ambari-web/app/views/main/dashboard/widget.js cc5be28 
>   ambari-web/app/views/main/dashboard/widgets.js 999fc08 
>   ambari-web/app/views/main/dashboard/widgets/datanode_live.js 184b046 
>   ambari-web/app/views/main/dashboard/widgets/flume_agent_live.js e8169c9 
>   ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js 04e7d66 
>   ambari-web/app/views/main/dashboard/widgets/hbase_average_load.js 48c4de0 
>   ambari-web/app/views/main/dashboard/widgets/hbase_links.js 4f2053a 
>   ambari-web/app/views/main/dashboard/widgets/hbase_master_heap.js 63b1cc2 
>   ambari-web/app/views/main/dashboard/widgets/hbase_master_uptime.js 0801ac2 
>   ambari-web/app/views/main/dashboard/widgets/hbase_regions_in_transition.js 
> 1f225c6 
>   ambari-web/app/views/main/dashboard/widgets/hdfs_capacity.js e64534d 
>   ambari-web/app/views/main/dashboard/widgets/hdfs_links.js cfe5eb1 
>   ambari-web/app/views/main/dashboard/widgets/metrics_cpu.js 72ef1c2 
>   ambari-web/app/views/main/dashboard/widgets/metrics_load.js c357934 
>   ambari-web/app/views/main/dashboard/widgets/metrics_memory.js 659d647 
>   ambari-web/app/views/main/dashboard/widgets/metrics_network.js 40fdb0b 
>   ambari-web/app/views/main/dashboard/widgets/namenode_cpu.js 161b5c0 
>   ambari-web/app/views/main/dashboard/widgets/namenode_heap.js 731143c 
>   ambari-web/app/views/main/dashboard/widgets/namenode_rpc.js 58b86d9 
>   ambari-web/app/views/main/dashboard/widgets/namenode_uptime.js e55f871 
>   ambari-web/app/views/main/dashboard/widgets/node_managers_live.js d5bf50b 
>   ambari-web/app/views/main/dashboard/widgets/pie_chart_widget.js 8f2dc90 
>   ambari-web/app/views/main/dashboard/widgets/pxf_live.js 74b2096 
>   ambari-web/app/views/main/dashboard/widgets/resource_manager_heap.js 
> 9509053 
>   ambari-web/app/views/main/dashboard/widgets/resource_manager_uptime.js 
> 6d87741 
>   ambari-web/app/views/main/dashboard/widgets/supervisor_live.js b8e36ca 
>   ambari-web/app/views/main/dashboard/widgets/text_widget.js 388ea52 
>   ambari-web/app/views/main/dashboard/widgets/text_widget_single_threshold.js 
> 69c5821 
>   ambari-web/app/views/main/dashboard/widgets/uptime_text_widget.js 792aa70 
>   ambari-web/app/views/main/dashboard/widgets/yarn_links.js d7b6488 
>   ambari-web/app/views/main/dashboard/widgets/yarn_memory.js b4dc9b7 
>   ambari-web/test/views/main/dashboard/widget_test.js 0544ff6 
>   ambari-web/test/views/main/dashboard/widgets/hbase_average_load_test.js 
> e3a08c9 
>   
> ambari-web/test/views/main/dashboard/widgets/hbase_regions_in_transition_test.js
>  8ad1531 
>   ambari-web/test/views/main/dashboard/widgets/namenode_rpc_test.js 474d099 
>   
> ambari-web/test/views/main/dashboard/widgets/text_widget_single_threshold_test.js
>  e564922 
>   ambari-web/test/views/main/dashboard/widgets/text_widget_test.js 730e209 
>   ambari-web/test/views/main/dashboard/widgets/uptime_text_widget_test.js 
> fbe2694 
>   ambari-web/test/views/main/dashboard/widgets_test.js 0781c79 
> 
> Diff: https://reviews.apache.org/r/54308/diff/
> 
> 
> Testing
> ---
> 
> 19919 tests complete (29 seconds)
>   155 tests pending
> 
> 
> Thanks,
> 
> Andrii Tkach
> 
>

Review Request 54308: Refactor widgets on Dashboard page

[Andrii Tkach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54308/
---

Review request for Ambari and Aleksandr Kovalenko.


Bugs: AMBARI-19064
https://issues.apache.org/jira/browse/AMBARI-19064


Repository: ambari


Description
---

Refactor widgets on Dashboard page


Diffs
-

  ambari-web/app/mixins/main/dashboard/widgets/editable.js f330ec3 
  ambari-web/app/templates/main/dashboard/edit_widget_popup.hbs 3a36ed6 
  
ambari-web/app/templates/main/dashboard/edit_widget_popup_single_threshold.hbs 
d80e601 
  ambari-web/app/templates/main/dashboard/plus_button_filter.hbs df7714c 
  ambari-web/app/templates/main/dashboard/widgets.hbs a778033 
  ambari-web/app/templates/main/dashboard/widgets/pie_chart.hbs fdeafd1 
  ambari-web/app/views/common/not-scrollable-textarea.js 1898714 
  ambari-web/app/views/main/dashboard/widget.js cc5be28 
  ambari-web/app/views/main/dashboard/widgets.js 999fc08 
  ambari-web/app/views/main/dashboard/widgets/datanode_live.js 184b046 
  ambari-web/app/views/main/dashboard/widgets/flume_agent_live.js e8169c9 
  ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js 04e7d66 
  ambari-web/app/views/main/dashboard/widgets/hbase_average_load.js 48c4de0 
  ambari-web/app/views/main/dashboard/widgets/hbase_links.js 4f2053a 
  ambari-web/app/views/main/dashboard/widgets/hbase_master_heap.js 63b1cc2 
  ambari-web/app/views/main/dashboard/widgets/hbase_master_uptime.js 0801ac2 
  ambari-web/app/views/main/dashboard/widgets/hbase_regions_in_transition.js 
1f225c6 
  ambari-web/app/views/main/dashboard/widgets/hdfs_capacity.js e64534d 
  ambari-web/app/views/main/dashboard/widgets/hdfs_links.js cfe5eb1 
  ambari-web/app/views/main/dashboard/widgets/metrics_cpu.js 72ef1c2 
  ambari-web/app/views/main/dashboard/widgets/metrics_load.js c357934 
  ambari-web/app/views/main/dashboard/widgets/metrics_memory.js 659d647 
  ambari-web/app/views/main/dashboard/widgets/metrics_network.js 40fdb0b 
  ambari-web/app/views/main/dashboard/widgets/namenode_cpu.js 161b5c0 
  ambari-web/app/views/main/dashboard/widgets/namenode_heap.js 731143c 
  ambari-web/app/views/main/dashboard/widgets/namenode_rpc.js 58b86d9 
  ambari-web/app/views/main/dashboard/widgets/namenode_uptime.js e55f871 
  ambari-web/app/views/main/dashboard/widgets/node_managers_live.js d5bf50b 
  ambari-web/app/views/main/dashboard/widgets/pie_chart_widget.js 8f2dc90 
  ambari-web/app/views/main/dashboard/widgets/pxf_live.js 74b2096 
  ambari-web/app/views/main/dashboard/widgets/resource_manager_heap.js 9509053 
  ambari-web/app/views/main/dashboard/widgets/resource_manager_uptime.js 
6d87741 
  ambari-web/app/views/main/dashboard/widgets/supervisor_live.js b8e36ca 
  ambari-web/app/views/main/dashboard/widgets/text_widget.js 388ea52 
  ambari-web/app/views/main/dashboard/widgets/text_widget_single_threshold.js 
69c5821 
  ambari-web/app/views/main/dashboard/widgets/uptime_text_widget.js 792aa70 
  ambari-web/app/views/main/dashboard/widgets/yarn_links.js d7b6488 
  ambari-web/app/views/main/dashboard/widgets/yarn_memory.js b4dc9b7 
  ambari-web/test/views/main/dashboard/widget_test.js 0544ff6 
  ambari-web/test/views/main/dashboard/widgets/hbase_average_load_test.js 
e3a08c9 
  
ambari-web/test/views/main/dashboard/widgets/hbase_regions_in_transition_test.js
 8ad1531 
  ambari-web/test/views/main/dashboard/widgets/namenode_rpc_test.js 474d099 
  
ambari-web/test/views/main/dashboard/widgets/text_widget_single_threshold_test.js
 e564922 
  ambari-web/test/views/main/dashboard/widgets/text_widget_test.js 730e209 
  ambari-web/test/views/main/dashboard/widgets/uptime_text_widget_test.js 
fbe2694 
  ambari-web/test/views/main/dashboard/widgets_test.js 0781c79 

Diff: https://reviews.apache.org/r/54308/diff/


Testing
---

19919 tests complete (29 seconds)
  155 tests pending


Thanks,

Andrii Tkach

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Jonathan Hurley]

> On Dec. 2, 2016, 7:42 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java,
> >  line 93
> > 
> >
> > Shouldn't this return a previously instantiated constant (`private 
> > static final ...)`?

Yes, yes it should ... I'll change this one and the other one.


> On Dec. 2, 2016, 7:42 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java,
> >  line 753
> > 
> >
> > Wouldn't it be _cheaper_ to put this in an `else` block rather and 
> > perform the calculation only to overwrite the result if it is a logical 
> > topolpgy request?

Yep, I'll make that change.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157733
---


On Dec. 1, 2016, 5:45 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 1, 2016, 5:45 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38

Re: Review Request 54252: Data model and Json parser for quick link profiles.

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54252/#review157751
---


Ship it!




Ship It!

- Attila Doroszlai


On Dec. 2, 2016, 3:10 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54252/
> ---
> 
> (Updated Dec. 2, 2016, 3:10 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Attila Magyar, Jayush Luniya, 
> Laszlo Puskas, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18978
> https://issues.apache.org/jira/browse/AMBARI-18978
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Data model and JSON parser.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/AcceptAllFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Component.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Filter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/LinkNameFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/PropertyFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfile.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParser.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Service.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java
>  f44f741 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParserTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/example_quicklinks_profile.json 
> PRE-CREATION 
>   ambari-server/src/test/resources/inconsistent_quicklinks_profile.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54252/diff/
> 
> 
> Testing
> ---
> 
> - Wrote new unit tests
> - Run the tests for Ambari-server. Failures are irrelevant.
> 
> Failed tests:
>   UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1118
>   Unexpected method call 
> AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for interface 
> org.apache.ambari.server.state.Cluster, EasyMock for interface 
> org.apache.ambari.server.state.Service):
> AmbariManagementController.getClusters(): expected: at least 0, actual: 1
> AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for 
> interface org.apache.ambari.server.state.Cluster, EasyMock for interface 
> org.apache.ambari.server.state.Service): expected: 1, actual: 0
>   DataStoreImplTest.testFind:526 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testRemove:475 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testStore_update:357 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testStore_update_longStringValue:426
>   Expectation failure on verify:
> DynamicEntity.set("DS_id", 99): expected: 1, actual: 0
> Tests in error:
>   KerberosCheckerTest.testCheckFailed »  Unexpected exception, 
> expected   KerberosCheckerTest.testCheckPassed:62 » ClassCast class 
> sun.security.provider...
> 
> Tests run: 4785, Failures: 5, Errors: 4, Skipped: 37
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/
---

(Updated Dec. 2, 2016, 9:43 a.m.)


Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
Sebastian Toader.


Changes
---

Updated with reviewer's comments.


Bugs: AMBARI-19055
https://issues.apache.org/jira/browse/AMBARI-19055


Repository: ambari


Description
---

It may be necessary to remove entries from the {{host_role_command}} table if 
the size of the table has grown excessively large in order to reduce the query 
times for "IN_PROGRESS" requests.

However, if you remove all tasks for an upgrade, but you leave the stages, 
request, and upgrade items, the upgrade will appear as though it is now 
{{PENDING}}.

{noformat:title=Before Removal}
{
  "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
  "Upgrade": {
...
"request_status": "COMPLETED",
"skip_failures": false,
"skip_service_check_failures": false,
"start_time": 1480517560950,
"suspended": false,
"to_version": "2.5.2.0-67",
"type": "INTERNAL_REQUEST",
"upgrade_type": "NON_ROLLING"
  },
  "upgrade_groups": [
  ...
{noformat}

{noformat:title=After Removal}
{
  "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
  "Upgrade": {
...
"request_status": "PENDING",
"skip_failures": false,
"skip_service_check_failures": false,
"start_time": 1480517560950,
"suspended": false,
"to_version": "2.5.2.0-67",
"type": "INTERNAL_REQUEST",
"upgrade_type": "NON_ROLLING"
  },
  "upgrade_groups": [
  ...
{noformat}

The actual request is showing as {{COMPLETED}} correctly:
{noformat}
{
  "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
  "Requests": {
"aborted_task_count": 0,
"cluster_name": "c1",
"completed_task_count": 0,
"create_time": 1480517560897,
"end_time": 1480517643350,
"exclusive": false,
"failed_task_count": 0,
"id": 12,
"inputs": null,
"operation_level": null,
"progress_percent": 100,
"queued_task_count": 0,
"request_context": "Upgrading to 2.5.2.0-67",
"request_schedule": null,
"request_status": "COMPLETED",
"resource_filters": [],
"start_time": 1480517560950,
"task_count": 0,
"timed_out_task_count": 0,
"type": "INTERNAL_REQUEST"
  },
  "tasks": [],
  ...
{noformat}

STR:
- Perform a simple upgrade of a ZK-only cluster
- Find out the ID of the upgrade request (say its 12)
- Remove tasks
{code}
DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
host_role_command WHERE request_id = 12)

DELETE FROM host_role_command WHERE request_id = 12
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
 3a86aef 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
 8c1bc57 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
 59dd9d9 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
 6f592cd 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
 5dfc74d 

Diff: https://reviews.apache.org/r/54264/diff/


Testing
---

Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 26:38 min
[INFO] Finished at: 2016-12-01T16:51:11-05:00
[INFO] Final Memory: 57M/704M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157749
---


Ship it!




Ship It!

- Robert Nettleton


On Dec. 1, 2016, 10:45 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 1, 2016, 10:45 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 54249: Remove unnecessary Log Feeder Date Mapper tests

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54249/
---

(Updated Dec. 2, 2016, 2:16 p.m.)


Review request for Ambari, Oliver Szabo, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

remove branch-2.4 as the test wasn't present there


Bugs: AMBARI-19045
https://issues.apache.org/jira/browse/AMBARI-19045


Repository: ambari


Description
---

Some unit tests for Date Mapper in the Log Feeder 
(testMapperDate_patternWithoutYear_previousYearLog, 
testMapperDate_patternWithoutYear_currentYearLog) make no sense, but fail in 
December. Remove these tests.
Options


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperDateTest.java
 667c9ff 

Diff: https://reviews.apache.org/r/54249/diff/


Testing
---

-


Thanks,

Miklos Gergely

Re: Review Request 54252: Data model and Json parser for quick link profiles.

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54252/
---

(Updated Dec. 2, 2016, 2:10 p.m.)


Review request for Ambari, Attila Doroszlai, Attila Magyar, Jayush Luniya, 
Laszlo Puskas, Sumit Mohanty, and Sebastian Toader.


Changes
---

Fixed syntax error.


Bugs: AMBARI-18978
https://issues.apache.org/jira/browse/AMBARI-18978


Repository: ambari


Description
---

Data model and JSON parser.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/AcceptAllFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Component.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Filter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/LinkNameFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/PropertyFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfile.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParser.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Service.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java
 f44f741 
  
ambari-server/src/test/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParserTest.java
 PRE-CREATION 
  ambari-server/src/test/resources/example_quicklinks_profile.json PRE-CREATION 
  ambari-server/src/test/resources/inconsistent_quicklinks_profile.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/54252/diff/


Testing
---

- Wrote new unit tests
- Run the tests for Ambari-server. Failures are irrelevant.

Failed tests:
  UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1118
  Unexpected method call 
AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for interface 
org.apache.ambari.server.state.Cluster, EasyMock for interface 
org.apache.ambari.server.state.Service):
AmbariManagementController.getClusters(): expected: at least 0, actual: 1
AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for 
interface org.apache.ambari.server.state.Cluster, EasyMock for interface 
org.apache.ambari.server.state.Service): expected: 1, actual: 0
  DataStoreImplTest.testFind:526 expected: but was:
  DataStoreImplTest.testRemove:475 expected: but was:
  DataStoreImplTest.testStore_update:357 expected: but was:
  DataStoreImplTest.testStore_update_longStringValue:426
  Expectation failure on verify:
DynamicEntity.set("DS_id", 99): expected: 1, actual: 0
Tests in error:
  KerberosCheckerTest.testCheckFailed »  Unexpected exception, 
expected

Re: Review Request 54252: Data model and Json parser for quick link profiles.

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54252/
---

(Updated Dec. 2, 2016, 2:06 p.m.)


Review request for Ambari, Attila Doroszlai, Attila Magyar, Jayush Luniya, 
Laszlo Puskas, Sumit Mohanty, and Sebastian Toader.


Changes
---

Fixed review comments, removed unneeded import


Bugs: AMBARI-18978
https://issues.apache.org/jira/browse/AMBARI-18978


Repository: ambari


Description
---

Data model and JSON parser.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/AcceptAllFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Component.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Filter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/LinkNameFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/PropertyFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfile.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParser.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Service.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java
 f44f741 
  
ambari-server/src/test/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParserTest.java
 PRE-CREATION 
  ambari-server/src/test/resources/example_quicklinks_profile.json PRE-CREATION 
  ambari-server/src/test/resources/inconsistent_quicklinks_profile.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/54252/diff/


Testing
---

- Wrote new unit tests
- Run the tests for Ambari-server. Failures are irrelevant.

Failed tests:
  UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1118
  Unexpected method call 
AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for interface 
org.apache.ambari.server.state.Cluster, EasyMock for interface 
org.apache.ambari.server.state.Service):
AmbariManagementController.getClusters(): expected: at least 0, actual: 1
AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for 
interface org.apache.ambari.server.state.Cluster, EasyMock for interface 
org.apache.ambari.server.state.Service): expected: 1, actual: 0
  DataStoreImplTest.testFind:526 expected: but was:
  DataStoreImplTest.testRemove:475 expected: but was:
  DataStoreImplTest.testStore_update:357 expected: but was:
  DataStoreImplTest.testStore_update_longStringValue:426
  Expectation failure on verify:
DynamicEntity.set("DS_id", 99): expected: 1, actual: 0
Tests in error:
  KerberosCheckerTest.testCheckFailed »  Unexpected exception, 
expected

Re: Review Request 54252: Data model and Json parser for quick link profiles.

[Balázs Bence Sári]

> On Dec. 1, 2016, 9:15 p.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParser.java,
> >  line 81
> > 
> >
> > `throws IOEXception` does not match `@throws JsonParseException`

IOException is declared by superclass signatures and can be thrown by method 
calls within the method


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54252/#review157638
---


On Dec. 1, 2016, 2:25 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54252/
> ---
> 
> (Updated Dec. 1, 2016, 2:25 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Attila Magyar, Jayush Luniya, 
> Laszlo Puskas, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18978
> https://issues.apache.org/jira/browse/AMBARI-18978
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Data model and JSON parser.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/AcceptAllFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Component.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Filter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/LinkNameFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/PropertyFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfile.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParser.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinksprofile/Service.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java
>  f44f741 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/quicklinksprofile/QuickLinksProfileParserTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/example_quicklinks_profile.json 
> PRE-CREATION 
>   ambari-server/src/test/resources/inconsistent_quicklinks_profile.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54252/diff/
> 
> 
> Testing
> ---
> 
> - Wrote new unit tests
> - Run the tests for Ambari-server. Failures are irrelevant.
> 
> Failed tests:
>   UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1118
>   Unexpected method call 
> AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for interface 
> org.apache.ambari.server.state.Cluster, EasyMock for interface 
> org.apache.ambari.server.state.Service):
> AmbariManagementController.getClusters(): expected: at least 0, actual: 1
> AmbariManagementController.initializeWidgetsAndLayouts(EasyMock for 
> interface org.apache.ambari.server.state.Cluster, EasyMock for interface 
> org.apache.ambari.server.state.Service): expected: 1, actual: 0
>   DataStoreImplTest.testFind:526 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testRemove:475 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testStore_update:357 expected: DS_DataStoreImplTest$TestEntity_1> but was: DS_DataStoreImplTest$TestSubEntity_1>
>   DataStoreImplTest.testStore_update_longStringValue:426
>   Expectation failure on verify:
> DynamicEntity.set("DS_id", 99): expected: 1, actual: 0
> Tests in error:
>   KerberosCheckerTest.testCheckFailed »  Unexpected exception, 
> expected   KerberosCheckerTest.testCheckPassed:62 » ClassCast class 
> sun.security.provider...
> 
> Tests run: 4785, Failures: 5, Errors: 4, Skipped: 37
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 54306: Do not change timeline.metrics.service.webapp.address in move wizard

[Andriy Babiichuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54306/#review157738
---


Ship it!




Ship It!

- Andriy Babiichuk


On Гру. 2, 2016, 1:53 після полудня, Aleksandr Kovalenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54306/
> ---
> 
> (Updated Гру. 2, 2016, 1:53 після полудня)
> 
> 
> Review request for Ambari and Andriy Babiichuk.
> 
> 
> Bugs: AMBARI-19063
> https://issues.apache.org/jira/browse/AMBARI-19063
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Do not change timeline.metrics.service.webapp.address when moving Metrics 
> Collector via Move Wizard.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/service/reassign/step4_controller.js 
> 26ac68c 
>   ambari-web/test/controllers/main/service/reassign/step4_controller_test.js 
> 5e7b0e3 
> 
> Diff: https://reviews.apache.org/r/54306/diff/
> 
> 
> Testing
> ---
> 
> 19966 tests complete (35 seconds)
>   155 tests pending
> 
> 
> Thanks,
> 
> Aleksandr Kovalenko
> 
>

Review Request 54306: Do not change timeline.metrics.service.webapp.address in move wizard

[Aleksandr Kovalenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54306/
---

Review request for Ambari and Andriy Babiichuk.


Bugs: AMBARI-19063
https://issues.apache.org/jira/browse/AMBARI-19063


Repository: ambari


Description
---

Do not change timeline.metrics.service.webapp.address when moving Metrics 
Collector via Move Wizard.


Diffs
-

  ambari-web/app/controllers/main/service/reassign/step4_controller.js 26ac68c 
  ambari-web/test/controllers/main/service/reassign/step4_controller_test.js 
5e7b0e3 

Diff: https://reviews.apache.org/r/54306/diff/


Testing
---

19966 tests complete (35 seconds)
  155 tests pending


Thanks,

Aleksandr Kovalenko

Re: Review Request 54169: AMBARI-19007 Atlas to support configuration of hooks from separate cluster

[Mugdha Varadkar]

> On Dec. 1, 2016, 5:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java,
> >  line 329
> > 
> >
> > Sorry to nitpick here, these 4 functions are nearly identical. Can you 
> > parametrize the service and config type?

Updated in latest patch


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157617
---


On Dec. 2, 2016, 1:01 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> ---
> 
> (Updated Dec. 2, 2016, 1:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan 
> Neethiraj, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) Introduced new flag for all the hook suported 
> components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set 
> the hook flag. This flag value is used to recommend the expected 
> configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add 
> common-atlas application properties under each hook config-type eg: 
> hive-atlas-application.properties config-type which are require to 
> communicate with external ATLAS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a1d2f95 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  ec846f8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  52de784 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
>  09cced6 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  4429253 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  0fb21d0 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  150f629 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  5e2c709 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  bcc598a 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  01e5f00 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  fe3f34a 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
>  f682e97 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  f7823d2 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  283f54d 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  68f06db 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
>  9547335 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  aca0681 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  bda4fe2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  ab350dc 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  b71f4a9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 775dbab 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  f2dd099 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  ce0b387 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 39cbbce 
> 
> Diff: https://reviews.apache.org/r/54169/diff/
> 
> 
> Testing
> ---
> 
> Tested Atlas installation via UI and blue-print. With blueprint used 
> config_recommendation_strategy as NEVER_APPLY and 
> ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
> 
> 
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true

Re: Review Request 54169: AMBARI-19007 Atlas to support configuration of hooks from separate cluster

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/
---

(Updated Dec. 2, 2016, 1:01 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, 
and Sumit Mohanty.


Bugs: AMBARI-19007
https://issues.apache.org/jira/browse/AMBARI-19007


Repository: ambari


Description
---

1) Introduced new flag for all the hook suported 
components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
2) If ATLAS service is present/selected for install, stack-advisor will set the 
hook flag. This flag value is used to recommend the expected configuration 
required for hook to work.
3) For Blue-print based installation user need to set hook flag and add 
common-atlas application properties under each hook config-type eg: 
hive-atlas-application.properties config-type which are require to communicate 
with external ATLAS


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 a1d2f95 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 ec846f8 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 52de784 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
 09cced6 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 4429253 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 0fb21d0 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
 150f629 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 5e2c709 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 bcc598a 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 01e5f00 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
 fe3f34a 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
 f682e97 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
 f7823d2 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
 283f54d 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
 68f06db 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
 9547335 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 aca0681 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 bda4fe2 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
 ab350dc 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
 b71f4a9 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
775dbab 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 f2dd099 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 ce0b387 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce 

Diff: https://reviews.apache.org/r/54169/diff/


Testing
---

Tested Atlas installation via UI and blue-print. With blueprint used 
config_recommendation_strategy as NEVER_APPLY and 
ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES


mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test

---
 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - 
in 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest

Results :

Tests run: 167, Failures: 0, Errors: 0, Skipped: 0


[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 24.484s
[INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
[INFO] Final Memory: 62M/797M
[INFO] 


Thanks,

Mugdha Varadkar

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157733
---


Fix it, then Ship it!




Ship It!


ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
 (line 93)


Shouldn't this return a previously instantiated constant (`private static 
final ...)`?



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
 (line 751)


Wouldn't it be _cheaper_ to put this in an `else` block rather and perform 
the calculation only to overwrite the result if it is a logical topolpgy 
request?


- Robert Levas


On Dec. 1, 2016, 5:45 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 1, 2016, 5:45 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 54264: Removing Tasks From host_role_command Causes Upgrades To Show As PENDING

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54264/#review157734
---


Ship it!




Ship It!

- Sebastian Toader


On Dec. 1, 2016, 11:45 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54264/
> ---
> 
> (Updated Dec. 1, 2016, 11:45 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-19055
> https://issues.apache.org/jira/browse/AMBARI-19055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It may be necessary to remove entries from the {{host_role_command}} table if 
> the size of the table has grown excessively large in order to reduce the 
> query times for "IN_PROGRESS" requests.
> 
> However, if you remove all tasks for an upgrade, but you leave the stages, 
> request, and upgrade items, the upgrade will appear as though it is now 
> {{PENDING}}.
> 
> {noformat:title=Before Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "COMPLETED",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> {noformat:title=After Removal}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/upgrades/12";,
>   "Upgrade": {
> ...
> "request_status": "PENDING",
> "skip_failures": false,
> "skip_service_check_failures": false,
> "start_time": 1480517560950,
> "suspended": false,
> "to_version": "2.5.2.0-67",
> "type": "INTERNAL_REQUEST",
> "upgrade_type": "NON_ROLLING"
>   },
>   "upgrade_groups": [
>   ...
> {noformat}
> 
> The actual request is showing as {{COMPLETED}} correctly:
> {noformat}
> {
>   "href": "http://localhost:8080/api/v1/clusters/c1/requests/12";,
>   "Requests": {
> "aborted_task_count": 0,
> "cluster_name": "c1",
> "completed_task_count": 0,
> "create_time": 1480517560897,
> "end_time": 1480517643350,
> "exclusive": false,
> "failed_task_count": 0,
> "id": 12,
> "inputs": null,
> "operation_level": null,
> "progress_percent": 100,
> "queued_task_count": 0,
> "request_context": "Upgrading to 2.5.2.0-67",
> "request_schedule": null,
> "request_status": "COMPLETED",
> "resource_filters": [],
> "start_time": 1480517560950,
> "task_count": 0,
> "timed_out_task_count": 0,
> "type": "INTERNAL_REQUEST"
>   },
>   "tasks": [],
>   ...
> {noformat}
> 
> STR:
> - Perform a simple upgrade of a ZK-only cluster
> - Find out the ID of the upgrade request (say its 12)
> - Remove tasks
> {code}
> DELETE FROM execution_command WHERE task_id IN (SELECT task_id FROM 
> host_role_command WHERE request_id = 12)
> 
> DELETE FROM host_role_command WHERE request_id = 12
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3a86aef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java
>  8c1bc57 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java
>  6f592cd 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  5dfc74d 
> 
> Diff: https://reviews.apache.org/r/54264/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4785, Failures: 0, Errors: 0, Skipped: 37
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 26:38 min
> [INFO] Finished at: 2016-12-01T16:51:11-05:00
> [INFO] Final Memory: 57M/704M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 54305: Hive view history tab does not list queries, if there are large number of queries

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54305/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, 
Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-19062
https://issues.apache.org/jira/browse/AMBARI-19062


Repository: ambari


Description
---

Earlier : ATS calls for each job was taking time.
In this patch : remove ATS calls as now the view will store correct status in 
the view tables itself


Diffs
-

  
contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/jobs/Aggregator.java
 99faeca 

Diff: https://reviews.apache.org/r/54305/diff/


Testing
---

manual testing done.


Thanks,

Nitiraj Rathore

Re: Review Request 54303: Ambari-server restart command not working. if any parameters are changed and reset to default.

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54303/#review157718
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Dec. 2, 2016, 11:26 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54303/
> ---
> 
> (Updated Dec. 2, 2016, 11:26 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-19061
> https://issues.apache.org/jira/browse/AMBARI-19061
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Steps to reproduce:**  
> 1\. Change any of the parameters that hamper the ambari-server to restart
> successfully.(password.dat file)  
> 2\. Restart ambari-server, ambari-server restart will fail saying (This is
> expected as we have changed the parameter.)
> 
> 
> 
> 
> Unable to determine server PID. Retrying...
> DB configs consistency check: no errors and warnings were found.
> ERROR: Exiting with exit code -1. 
> 
> 
> 3\. Now set the Parameter to its actual value.  
> 4.Restart ambari-server, ambari-server restart will fail saying
> 
> 
> 
> 
> Using python  /usr/bin/python
> Restarting ambari-server
> WARNING: '' is incorrect PID value. 
> /var/run/ambari-server/ambari-server.pid is corrupt. Removing
> Ambari Server 'restart' completed with warnings.
> 
> 
> 5.If you Restart the Ambari server again. It gets restarted successfully.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/utils.py 62c93ae 
> 
> Diff: https://reviews.apache.org/r/54303/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 54303: Ambari-server restart command not working. if any parameters are changed and reset to default.

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54303/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-19061
https://issues.apache.org/jira/browse/AMBARI-19061


Repository: ambari


Description
---

**Steps to reproduce:**  
1\. Change any of the parameters that hamper the ambari-server to restart
successfully.(password.dat file)  
2\. Restart ambari-server, ambari-server restart will fail saying (This is
expected as we have changed the parameter.)




Unable to determine server PID. Retrying...
DB configs consistency check: no errors and warnings were found.
ERROR: Exiting with exit code -1. 


3\. Now set the Parameter to its actual value.  
4.Restart ambari-server, ambari-server restart will fail saying




Using python  /usr/bin/python
Restarting ambari-server
WARNING: '' is incorrect PID value. 
/var/run/ambari-server/ambari-server.pid is corrupt. Removing
Ambari Server 'restart' completed with warnings.


5.If you Restart the Ambari server again. It gets restarted successfully.


Diffs
-

  ambari-server/src/main/python/ambari_server/utils.py 62c93ae 

Diff: https://reviews.apache.org/r/54303/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk