Re: Review Request 58228: Perf: Refactor ambari db-cleanup to include all big tables

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58228/#review171188
---



I still need to go through the rest of the code review. Please wait until 
others have also given feedback.


ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
Lines 234 (patched)


Let's call this checkForLargeTables



ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
Lines 247 (patched)


Does this work on all DB types that we support?
SQLServer and SQLAnywhere?



ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
Lines 257 (patched)


We should find out which type the DB is and only add the query that is 
needed.



ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
Lines 280 (patched)


we should also log what the limit is.



ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
Lines 283 (patched)


Use String.format



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExecutionCommandDAO.java
Lines 64 (patched)


Does this need a limit with a WHERE clause?


- Alejandro Fernandez


On April 6, 2017, 12:45 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58228/
> ---
> 
> (Updated April 6, 2017, 12:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20687
> https://issues.apache.org/jira/browse/AMBARI-20687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add check for large tables into db consistency check.
> Add code to cleanup these tables into db-cleanup code.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  e7e9433 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java 
> c132a3d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 
> 1dd3b54 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExecutionCommandDAO.java
>  7a3bc01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  79b8bc9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
> 2696f66 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
> c2919b2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExecutionCommandEntity.java
>  85f3a25 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
>  a809295 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
>  f19aa72 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java
>  ff14e3a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestResourceFilterEntity.java
>  8ee41d2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RoleSuccessCriteriaEntity.java
>  3386c24 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostRequestEntity.java
>  b90e192 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyLogicalTaskEntity.java
>  c71d4e4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  7421ca1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeItemEntity.java
>  560970a 
>   ambari-server/src/main/python/ambari-server.py 87cc6c2 
>   ambari-server/src/main/python/ambari_server/dbCleanup.py abc8267 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
>  7d8ba50 
> 
> 
> Diff: https://reviews.apache.org/r/58228/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 58198: Repetitive operation 'Link' in hdfs.py

[zhangxiaolu zhangxiaolu]

> On 四月 5, 2017, 12:16 p.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
> > Line 132 (original)
> > 
> >
> > these symlinks are for different instruction sets. Could you please 
> > elaborate why this change is required?

Yes, you're right. so_target_dir_x86 is for 'linux-is386-32' and so 
target_dir_x64 is for 'linux-amd64-64'.Soory, I'm wrong.I will revoke this 
patch.Sorry.


- zhangxiaolu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58198/#review171116
---


On 四月 5, 2017, 5:52 a.m., zhangxiaolu zhangxiaolu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58198/
> ---
> 
> (Updated 四月 5, 2017, 5:52 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Dmytro 
> Sen, Jaimin Jetly, Nate Cole, Srimanth Gunturi, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20675
> https://issues.apache.org/jira/browse/AMBARI-20675
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> the method of install_snappy is repetitive in hdfs.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  1264284 
> 
> 
> Diff: https://reviews.apache.org/r/58198/diff/1/
> 
> 
> Testing
> ---
> 
> done it
> 
> 
> Thanks,
> 
> zhangxiaolu zhangxiaolu
> 
>

Re: Review Request 58228: Perf: Refactor ambari db-cleanup to include all big tables

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58228/
---

(Updated Квітень 6, 2017, 12:45 до полудня)


Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-20687
https://issues.apache.org/jira/browse/AMBARI-20687


Repository: ambari


Description
---

Add check for large tables into db consistency check.
Add code to cleanup these tables into db-cleanup code.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 e7e9433 
  ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java 
c132a3d 
  ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 
1dd3b54 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExecutionCommandDAO.java
 7a3bc01 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 79b8bc9 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
2696f66 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
c2919b2 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExecutionCommandEntity.java
 85f3a25 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
 a809295 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
 f19aa72 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java
 ff14e3a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestResourceFilterEntity.java
 8ee41d2 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RoleSuccessCriteriaEntity.java
 3386c24 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 49c1594 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostRequestEntity.java
 b90e192 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyLogicalTaskEntity.java
 c71d4e4 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 7421ca1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeItemEntity.java
 560970a 
  ambari-server/src/main/python/ambari-server.py 87cc6c2 
  ambari-server/src/main/python/ambari_server/dbCleanup.py abc8267 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
 7d8ba50 


Diff: https://reviews.apache.org/r/58228/diff/2/

Changes: https://reviews.apache.org/r/58228/diff/1-2/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Review Request 58228: Perf: Refactor ambari db-cleanup to include all big tables

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58228/
---

Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-20687
https://issues.apache.org/jira/browse/AMBARI-20687


Repository: ambari


Description
---

Add check for large tables into db consistency check.
Add code to cleanup these tables into db-cleanup code.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 e7e9433 
  ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java 
c132a3d 
  ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 
1dd3b54 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExecutionCommandDAO.java
 7a3bc01 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 79b8bc9 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
2696f66 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
c2919b2 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExecutionCommandEntity.java
 85f3a25 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
 a809295 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
 f19aa72 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java
 ff14e3a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestResourceFilterEntity.java
 8ee41d2 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RoleSuccessCriteriaEntity.java
 3386c24 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 49c1594 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostRequestEntity.java
 b90e192 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyLogicalTaskEntity.java
 c71d4e4 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 7421ca1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeItemEntity.java
 560970a 
  ambari-server/src/main/python/ambari-server.py 87cc6c2 
  ambari-server/src/main/python/ambari_server/dbCleanup.py abc8267 
  ambari-server/src/main/resources/stacks/HDP/2.3.ECS/repos/repoinfo.xml 
329539e 
  ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/repos/repoinfo.xml 
ad79215 
  ambari-server/src/main/resources/stacks/HDP/2.6/repos/repoinfo.xml 81a70a5 
  ambari-server/src/main/resources/stacks/HDP/3.0/repos/repoinfo.xml 5145064 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
 7d8ba50 


Diff: https://reviews.apache.org/r/58228/diff/1/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/#review171173
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 5, 2017, 11:20 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58218/
> ---
> 
> (Updated April 5, 2017, 11:20 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20685
> https://issues.apache.org/jira/browse/AMBARI-20685
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
> massive request object object with 10's of 1000's of stages and tasks. The 
> web client executes a {{GET}} command against the API to retrieve upgrade 
> groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
> code to run out of stack frame space while recursively building a query. To 
> work around this, the stack frame size per thread can be increased using 
> {{-Xss}} along with a value like 32M.
> 
> {code}
> http://localhost:8080/api/v1/clusters/c1/upgrades/12?
> upgrade_groups/UpgradeGroup/status!=PENDING&
> fields=
>   Upgrade/progress_percent,
>   Upgrade/request_context,
>   Upgrade/request_status,
>   Upgrade/direction,
>   Upgrade/downgrade_allowed,
>   upgrade_groups/UpgradeGroup,
>   Upgrade/*,
>   upgrade_groups/upgrade_items/UpgradeItem/status,
>   minimal_response=true&_=1489680258782
> {code}
> 
> This call gets turn into a LOT of queries, but some seem to be based on the 
> number of stages. For example, this one on a very small upgrade produces very 
> poor SQL...
> 
> {code}
> SELECT
>   stage_id,
>   cluster_host_info,
>   cluster_id,
>   command_execution_type,
>   command_params,
>   display_status,
>   host_params,
>   log_info,
>   request_context,
>   request_id,
>   skippable,
>   status,
>   supports_auto_skip_failure
> FROM stage
> WHERE ((request_id = ?)
> AND stage_id = ?)
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?)))
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  8759844 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
>  bf0fa33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  34d175c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> eee913a 
> 
> 
> Diff: https://reviews.apache.org/r/58218/diff/2/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 17:46 min
> [INFO] Finished at: 2017-04-05T17:30:09-04:00
> [INFO] Final Memory: 67M/861M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade

[Amruta Borkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58122/
---

(Updated April 5, 2017, 11:45 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Nettleton.


Changes
---

Modified to address the feedback comments.


Bugs: AMBARI-20551
https://issues.apache.org/jira/browse/AMBARI-20551


Repository: ambari


Description
---

Following error is thrown while exporting a blueprint from cluster.
 ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request:
java.lang.IllegalArgumentException: Specified configuration type is not 
associated with any service: storm-site
at 
org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416)
at 
org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186)
at 
org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141)
at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839)
at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223)
at 
org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77)
at 
org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 db1aa074d4 


Diff: https://reviews.apache.org/r/58122/diff/4/

Changes: https://reviews.apache.org/r/58122/diff/3-4/


Testing
---

Tested manually. Log shows warning about unmapped config-type but blueprint 
gets exported.


Thanks,

Amruta Borkar

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/
---

(Updated April 5, 2017, 7:20 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
Sid Wagle.


Bugs: AMBARI-20685
https://issues.apache.org/jira/browse/AMBARI-20685


Repository: ambari


Description
---

Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
massive request object object with 10's of 1000's of stages and tasks. The web 
client executes a {{GET}} command against the API to retrieve upgrade 
groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
code to run out of stack frame space while recursively building a query. To 
work around this, the stack frame size per thread can be increased using 
{{-Xss}} along with a value like 32M.

{code}
http://localhost:8080/api/v1/clusters/c1/upgrades/12?
upgrade_groups/UpgradeGroup/status!=PENDING&
fields=
  Upgrade/progress_percent,
  Upgrade/request_context,
  Upgrade/request_status,
  Upgrade/direction,
  Upgrade/downgrade_allowed,
  upgrade_groups/UpgradeGroup,
  Upgrade/*,
  upgrade_groups/upgrade_items/UpgradeItem/status,
  minimal_response=true&_=1489680258782
{code}

This call gets turn into a LOT of queries, but some seem to be based on the 
number of stages. For example, this one on a very small upgrade produces very 
poor SQL...

{code}
SELECT
  stage_id,
  cluster_host_info,
  cluster_id,
  command_execution_type,
  command_params,
  display_status,
  host_params,
  log_info,
  request_context,
  request_id,
  skippable,
  status,
  supports_auto_skip_failure
FROM stage
WHERE ((request_id = ?)
AND stage_id = ?)
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?)))
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
 8759844 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
 bf0fa33 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 49c1594 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
 34d175c 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
eee913a 


Diff: https://reviews.apache.org/r/58218/diff/2/


Testing (updated)
---

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 17:46 min
[INFO] Finished at: 2017-04-05T17:30:09-04:00
[INFO] Final Memory: 67M/861M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/
---

(Updated April 5, 2017, 7:19 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
Sid Wagle.


Bugs: AMBARI-20685
https://issues.apache.org/jira/browse/AMBARI-20685


Repository: ambari


Description
---

Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
massive request object object with 10's of 1000's of stages and tasks. The web 
client executes a {{GET}} command against the API to retrieve upgrade 
groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
code to run out of stack frame space while recursively building a query. To 
work around this, the stack frame size per thread can be increased using 
{{-Xss}} along with a value like 32M.

{code}
http://localhost:8080/api/v1/clusters/c1/upgrades/12?
upgrade_groups/UpgradeGroup/status!=PENDING&
fields=
  Upgrade/progress_percent,
  Upgrade/request_context,
  Upgrade/request_status,
  Upgrade/direction,
  Upgrade/downgrade_allowed,
  upgrade_groups/UpgradeGroup,
  Upgrade/*,
  upgrade_groups/upgrade_items/UpgradeItem/status,
  minimal_response=true&_=1489680258782
{code}

This call gets turn into a LOT of queries, but some seem to be based on the 
number of stages. For example, this one on a very small upgrade produces very 
poor SQL...

{code}
SELECT
  stage_id,
  cluster_host_info,
  cluster_id,
  command_execution_type,
  command_params,
  display_status,
  host_params,
  log_info,
  request_context,
  request_id,
  skippable,
  status,
  supports_auto_skip_failure
FROM stage
WHERE ((request_id = ?)
AND stage_id = ?)
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?)))
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
 8759844 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
 bf0fa33 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 49c1594 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
 34d175c 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
eee913a 


Diff: https://reviews.apache.org/r/58218/diff/2/

Changes: https://reviews.apache.org/r/58218/diff/1-2/


Testing
---

PENDING


Thanks,

Jonathan Hurley

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Jonathan Hurley]

> On April 5, 2017, 5:16 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
> > Lines 264 (patched)
> > 
> >
> > Let's remove dead code

Ugh - I did remove it, but I didn't stage the change before creating the patch. 
Thanks!


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/#review171167
---


On April 5, 2017, 4:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58218/
> ---
> 
> (Updated April 5, 2017, 4:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20685
> https://issues.apache.org/jira/browse/AMBARI-20685
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
> massive request object object with 10's of 1000's of stages and tasks. The 
> web client executes a {{GET}} command against the API to retrieve upgrade 
> groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
> code to run out of stack frame space while recursively building a query. To 
> work around this, the stack frame size per thread can be increased using 
> {{-Xss}} along with a value like 32M.
> 
> {code}
> http://localhost:8080/api/v1/clusters/c1/upgrades/12?
> upgrade_groups/UpgradeGroup/status!=PENDING&
> fields=
>   Upgrade/progress_percent,
>   Upgrade/request_context,
>   Upgrade/request_status,
>   Upgrade/direction,
>   Upgrade/downgrade_allowed,
>   upgrade_groups/UpgradeGroup,
>   Upgrade/*,
>   upgrade_groups/upgrade_items/UpgradeItem/status,
>   minimal_response=true&_=1489680258782
> {code}
> 
> This call gets turn into a LOT of queries, but some seem to be based on the 
> number of stages. For example, this one on a very small upgrade produces very 
> poor SQL...
> 
> {code}
> SELECT
>   stage_id,
>   cluster_host_info,
>   cluster_id,
>   command_execution_type,
>   command_params,
>   display_status,
>   host_params,
>   log_info,
>   request_context,
>   request_id,
>   skippable,
>   status,
>   supports_auto_skip_failure
> FROM stage
> WHERE ((request_id = ?)
> AND stage_id = ?)
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?)))
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  8759844 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
>  bf0fa33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  34d175c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> eee913a 
> 
> 
> Diff: https://reviews.apache.org/r/58218/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57945: Log Search Configuration API

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57945/
---

(Updated April 5, 2017, 11:11 p.m.)


Review request for Ambari, Oliver Szabo and Robert Nettleton.


Changes
---

fix python unit tests, and swagger docs


Bugs: AMBARI-20578
https://issues.apache.org/jira/browse/AMBARI-20578


Repository: ambari


Description
---

Log Search should store it's configurations (inputs, filters) by using an API, 
which the user may implement as well to use their own way of storing the 
configurations. By default for now we offer to store everything in ZooKeeper.

Also separated Log Feeder config handling from the main class.


Diffs (updated)
-

  ambari-logsearch/ambari-logsearch-config-api/.gitignore PRE-CREATION 
  ambari-logsearch/ambari-logsearch-config-api/pom.xml PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/InputConfigMonitor.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactoryTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/NonLogSearchConfigClass.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-config-api/src/test/resources/log4j.xml 
PRE-CREATION 
  ambari-logsearch/ambari-logsearch-config-zookeeper/.gitignore PRE-CREATION 
  ambari-logsearch/ambari-logsearch-config-zookeeper/pom.xml PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 25e4306 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
 a47c71f 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
 8e70850 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
 f93 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputManager.java
 3c80e50 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputS3File.java
 26f1ddb 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java
 73cf449 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java
 80b34e0 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml 
7ef967c 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputFileTest.java
 08aa564 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputManagerTest.java
 368a930 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputManagerTest.java
 0a0a195 
  ambari-logsearch/ambari-logsearch-server/pom.xml 52bda8d 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 73a43ad 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 984e834 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/ShipperConfigResource.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-web/.gitignore PRE-CREATION 
  ambari-logsearch/docker/test-config/logfeeder/logfeeder.properties 068bc3a 
  ambari-logsearch/docker/test-config/logsearch/logsearch.properties cfa985d 
  ambari-logsearch/pom.xml 1e63ced 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
 d9b9b57 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/conf

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/#review171167
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
Lines 264 (patched)


Let's remove dead code


- Alejandro Fernandez


On April 5, 2017, 8:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58218/
> ---
> 
> (Updated April 5, 2017, 8:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20685
> https://issues.apache.org/jira/browse/AMBARI-20685
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
> massive request object object with 10's of 1000's of stages and tasks. The 
> web client executes a {{GET}} command against the API to retrieve upgrade 
> groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
> code to run out of stack frame space while recursively building a query. To 
> work around this, the stack frame size per thread can be increased using 
> {{-Xss}} along with a value like 32M.
> 
> {code}
> http://localhost:8080/api/v1/clusters/c1/upgrades/12?
> upgrade_groups/UpgradeGroup/status!=PENDING&
> fields=
>   Upgrade/progress_percent,
>   Upgrade/request_context,
>   Upgrade/request_status,
>   Upgrade/direction,
>   Upgrade/downgrade_allowed,
>   upgrade_groups/UpgradeGroup,
>   Upgrade/*,
>   upgrade_groups/upgrade_items/UpgradeItem/status,
>   minimal_response=true&_=1489680258782
> {code}
> 
> This call gets turn into a LOT of queries, but some seem to be based on the 
> number of stages. For example, this one on a very small upgrade produces very 
> poor SQL...
> 
> {code}
> SELECT
>   stage_id,
>   cluster_host_info,
>   cluster_id,
>   command_execution_type,
>   command_params,
>   display_status,
>   host_params,
>   log_info,
>   request_context,
>   request_id,
>   skippable,
>   status,
>   supports_auto_skip_failure
> FROM stage
> WHERE ((request_id = ?)
> AND stage_id = ?)
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?)))
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  8759844 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
>  bf0fa33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  34d175c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> eee913a 
> 
> 
> Diff: https://reviews.apache.org/r/58218/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/#review171165
---


Ship it!




Ship It!

- Nate Cole


On April 5, 2017, 4:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58218/
> ---
> 
> (Updated April 5, 2017, 4:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20685
> https://issues.apache.org/jira/browse/AMBARI-20685
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
> massive request object object with 10's of 1000's of stages and tasks. The 
> web client executes a {{GET}} command against the API to retrieve upgrade 
> groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
> code to run out of stack frame space while recursively building a query. To 
> work around this, the stack frame size per thread can be increased using 
> {{-Xss}} along with a value like 32M.
> 
> {code}
> http://localhost:8080/api/v1/clusters/c1/upgrades/12?
> upgrade_groups/UpgradeGroup/status!=PENDING&
> fields=
>   Upgrade/progress_percent,
>   Upgrade/request_context,
>   Upgrade/request_status,
>   Upgrade/direction,
>   Upgrade/downgrade_allowed,
>   upgrade_groups/UpgradeGroup,
>   Upgrade/*,
>   upgrade_groups/upgrade_items/UpgradeItem/status,
>   minimal_response=true&_=1489680258782
> {code}
> 
> This call gets turn into a LOT of queries, but some seem to be based on the 
> number of stages. For example, this one on a very small upgrade produces very 
> poor SQL...
> 
> {code}
> SELECT
>   stage_id,
>   cluster_host_info,
>   cluster_id,
>   command_execution_type,
>   command_params,
>   display_status,
>   host_params,
>   log_info,
>   request_context,
>   request_id,
>   skippable,
>   status,
>   supports_auto_skip_failure
> FROM stage
> WHERE ((request_id = ?)
> AND stage_id = ?)
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?)))
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  8759844 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
>  bf0fa33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  34d175c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> eee913a 
> 
> 
> Diff: https://reviews.apache.org/r/58218/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/#review171164
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
Lines 239-241 (original), 238-245 (patched)


This may look horrible, but it's actually incredible efficient b/c of the 
PK lookup:

Name: org.apache.ambari.server.orm.entities.StageEntity-findByPrimaryKey

Cache Hits: 156500224
DB Hits: 7360



ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java
Lines 670-714 (original)


This was the culprit; this recursion breaks JPA in a lot of ways. More 
importantly, it doesn't let us get the StageEntity from the cache.


- Jonathan Hurley


On April 5, 2017, 4:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58218/
> ---
> 
> (Updated April 5, 2017, 4:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-20685
> https://issues.apache.org/jira/browse/AMBARI-20685
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
> massive request object object with 10's of 1000's of stages and tasks. The 
> web client executes a {{GET}} command against the API to retrieve upgrade 
> groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
> code to run out of stack frame space while recursively building a query. To 
> work around this, the stack frame size per thread can be increased using 
> {{-Xss}} along with a value like 32M.
> 
> {code}
> http://localhost:8080/api/v1/clusters/c1/upgrades/12?
> upgrade_groups/UpgradeGroup/status!=PENDING&
> fields=
>   Upgrade/progress_percent,
>   Upgrade/request_context,
>   Upgrade/request_status,
>   Upgrade/direction,
>   Upgrade/downgrade_allowed,
>   upgrade_groups/UpgradeGroup,
>   Upgrade/*,
>   upgrade_groups/upgrade_items/UpgradeItem/status,
>   minimal_response=true&_=1489680258782
> {code}
> 
> This call gets turn into a LOT of queries, but some seem to be based on the 
> number of stages. For example, this one on a very small upgrade produces very 
> poor SQL...
> 
> {code}
> SELECT
>   stage_id,
>   cluster_host_info,
>   cluster_id,
>   command_execution_type,
>   command_params,
>   display_status,
>   host_params,
>   log_info,
>   request_context,
>   request_id,
>   skippable,
>   status,
>   supports_auto_skip_failure
> FROM stage
> WHERE ((request_id = ?)
> AND stage_id = ?)
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?))
> OR (stage_id = ?)))
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
>  8759844 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
>  bf0fa33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  49c1594 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  34d175c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> eee913a 
> 
> 
> Diff: https://reviews.apache.org/r/58218/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 58218: Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58218/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas, and 
Sid Wagle.


Bugs: AMBARI-20685
https://issues.apache.org/jira/browse/AMBARI-20685


Repository: ambari


Description
---

Large rolling upgrades (1000 hosts with 10,000 host components) creates a 
massive request object object with 10's of 1000's of stages and tasks. The web 
client executes a {{GET}} command against the API to retrieve upgrade 
groups/items. On a large upgrade, this causes the EclipseLink SQL generation 
code to run out of stack frame space while recursively building a query. To 
work around this, the stack frame size per thread can be increased using 
{{-Xss}} along with a value like 32M.

{code}
http://localhost:8080/api/v1/clusters/c1/upgrades/12?
upgrade_groups/UpgradeGroup/status!=PENDING&
fields=
  Upgrade/progress_percent,
  Upgrade/request_context,
  Upgrade/request_status,
  Upgrade/direction,
  Upgrade/downgrade_allowed,
  upgrade_groups/UpgradeGroup,
  Upgrade/*,
  upgrade_groups/upgrade_items/UpgradeItem/status,
  minimal_response=true&_=1489680258782
{code}

This call gets turn into a LOT of queries, but some seem to be based on the 
number of stages. For example, this one on a very small upgrade produces very 
poor SQL...

{code}
SELECT
  stage_id,
  cluster_host_info,
  cluster_id,
  command_execution_type,
  command_params,
  display_status,
  host_params,
  log_info,
  request_context,
  request_id,
  skippable,
  status,
  supports_auto_skip_failure
FROM stage
WHERE ((request_id = ?)
AND stage_id = ?)
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?)))
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
 8759844 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java
 bf0fa33 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 49c1594 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
 34d175c 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
eee913a 


Diff: https://reviews.apache.org/r/58218/diff/1/


Testing
---

PENDING


Thanks,

Jonathan Hurley

Re: Review Request 58211: AMBARI-20674 Able to hide the Delete menu item from UI for a given service

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58211/
---

(Updated April 5, 2017, 6:18 p.m.)


Review request for Ambari, Sangeeta Ravindran and Tim Thorpe.


Summary (updated)
-

AMBARI-20674 Able to hide the Delete menu item from UI for a given service


Bugs: AMBARI-20674
https://issues.apache.org/jira/browse/AMBARI-20674


Repository: ambari


Description
---

Stack driven approach to have UI show/hide Delete service menu item for a given 
service.
A service's metainfo.xml can have
false set at the service level to 
indicate UI should hide the menu item. Directly going thru REST API calls are 
always supported and non-restricted.
If the section is missing, it will be treated as "support" UI delete service 
operation. In other words, UI only hides the menu item if metainfo.xml has 
explicitly set the flag to false.
Example.
"" +
" 2.0" +
" " +
" " +
" HDFS" +
" HDFS" +
" false" +
" " +
" " +
"";


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
 cbff300 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
 a30d783 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
fd65268 
  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
0d0b78b 
  ambari-server/src/main/resources/properties.json 04d32ea 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceInfoTest.java 
1b9296e 
  ambari-web/app/app.js 9c7d874 
  ambari-web/app/mappers/stack_service_mapper.js 4bda89d 
  ambari-web/app/models/host_component.js 3950f97 
  ambari-web/app/models/stack_service.js 4f21288 
  ambari-web/test/mappers/stack_service_mapper_test.js 9da8b24 
  ambari-web/test/views/main/service/item_test.js 4b2e6f9 


Diff: https://reviews.apache.org/r/58211/diff/1/


Testing
---

unit testing
Install a trunk cluster with HDP 2.6 stack.Patch it with the code change both 
ambari server and UI change, then update metainfo.xml for hbase in common 
services (to test inheritance) verify UI does not show Delete service menu item 
for HBase after restart Ambari server.


Thanks,

Di Li

Re: Review Request 58211: AMBARI-20674 About to hide the Delete menu item from UI for a given service

[Sangeeta Ravindran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58211/#review171157
---


Ship it!




Ship It!

- Sangeeta Ravindran


On April 5, 2017, 3:37 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58211/
> ---
> 
> (Updated April 5, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Sangeeta Ravindran and Tim Thorpe.
> 
> 
> Bugs: AMBARI-20674
> https://issues.apache.org/jira/browse/AMBARI-20674
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stack driven approach to have UI show/hide Delete service menu item for a 
> given service.
> A service's metainfo.xml can have
> false set at the service level to 
> indicate UI should hide the menu item. Directly going thru REST API calls are 
> always supported and non-restricted.
> If the section is missing, it will be treated as "support" UI delete service 
> operation. In other words, UI only hides the menu item if metainfo.xml has 
> explicitly set the flag to false.
> Example.
> "" +
> " 2.0" +
> " " +
> " " +
> " HDFS" +
> " HDFS" +
> " false" +
> " " +
> " " +
> "";
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
>  cbff300 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
>  a30d783 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> fd65268 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 0d0b78b 
>   ambari-server/src/main/resources/properties.json 04d32ea 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceInfoTest.java
>  1b9296e 
>   ambari-web/app/app.js 9c7d874 
>   ambari-web/app/mappers/stack_service_mapper.js 4bda89d 
>   ambari-web/app/models/host_component.js 3950f97 
>   ambari-web/app/models/stack_service.js 4f21288 
>   ambari-web/test/mappers/stack_service_mapper_test.js 9da8b24 
>   ambari-web/test/views/main/service/item_test.js 4b2e6f9 
> 
> 
> Diff: https://reviews.apache.org/r/58211/diff/1/
> 
> 
> Testing
> ---
> 
> unit testing
> Install a trunk cluster with HDP 2.6 stack.Patch it with the code change both 
> ambari server and UI change, then update metainfo.xml for hbase in common 
> services (to test inheritance) verify UI does not show Delete service menu 
> item for HBase after restart Ambari server.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 58210: Implement a websocket adapter for stomp.py

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58210/#review171153
---


Ship it!




Ship It!

- Sid Wagle


On April 5, 2017, 1:37 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58210/
> ---
> 
> (Updated April 5, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
> 
> 
> Bugs: AMBARI-20684
> https://issues.apache.org/jira/browse/AMBARI-20684
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This is needed for stomp to work via websocket instead of tcp
> 
> 
> Diffs
> -
> 
>   LICENSE.txt f05016f 
>   NOTICE.txt 50f982c 
>   ambari-agent/conf/unix/install-helper.sh 35aec15 
>   ambari-agent/pom.xml 4807a35 
>   ambari-agent/src/main/python/ambari_agent/client_example.py PRE-CREATION 
>   ambari-agent/src/packages/tarball/all.xml a22f0bb 
>   ambari-common/src/main/python/ambari_stomp/adapter/websocket.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/__init__.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/client/__init__.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/client/geventclient.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/client/threadedclient.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/client/tornadoclient.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/compat.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/exc.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/framing.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/manager.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/messaging.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/server/__init__.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/server/cherrypyserver.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/server/geventserver.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/server/wsgirefserver.py 
> PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/server/wsgiutils.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/streaming.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/utf8validator.py PRE-CREATION 
>   ambari-common/src/main/python/ambari_ws4py/websocket.py PRE-CREATION 
>   ambari-project/pom.xml 98da9f4 
>   ambari-server/pom.xml 8b4c8d6 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml bdbf0de 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/AgentResourceTest.java
>  5feb3cc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/AmbariErrorHandlerTest.java
>  239bfcb 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java
>  f64a97f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariSessionManagerTest.java
>  23f07c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  1c45589 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java
>  d3c8acf 
>   pom.xml f192895 
> 
> 
> Diff: https://reviews.apache.org/r/58210/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58122/#review171149
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
Lines 525 (patched)


Missing spaces after the periods


- Alejandro Fernandez


On April 5, 2017, 4:58 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58122/
> ---
> 
> (Updated April 5, 2017, 4:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-20551
> https://issues.apache.org/jira/browse/AMBARI-20551
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Following error is thrown while exporting a blueprint from cluster.
>  ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request:
> java.lang.IllegalArgumentException: Specified configuration type is not 
> associated with any service: storm-site
> at 
> org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494)
> at 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946)
> at 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939)
> at 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439)
> at 
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416)
> at 
> org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186)
> at 
> org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141)
> at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839)
> at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223)
> at 
> org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77)
> at 
> org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  db1aa074d4 
> 
> 
> Diff: https://reviews.apache.org/r/58122/diff/3/
> 
> 
> Testing
> ---
> 
> Tested manually. Log shows warning about unmapped config-type but blueprint 
> gets exported.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/#review171147
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
Line 113 (original), 114 (patched)


Should we catch a specific exception here that means it was deregistered? 
Catching any seems too wide a net.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
Line 117 (original), 126 (patched)


Technically, this function could also be called outside of an upgrade, so 
the message shouldn't really know it's "upgrade".

Does it make sense to also call this function during any restart command?


- Alejandro Fernandez


On April 5, 2017, 12:27 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58208/
> ---
> 
> (Updated April 5, 2017, 12:27 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-20682
> https://issues.apache.org/jira/browse/AMBARI-20682
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade (especially on a large, heavily used cluster), the 
> DataNodes do not shutdown immediately. However, they do de-register from the 
> NameNode which tricks Ambari into thinking that they are down.
> 
> Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
> DataNode back up before the daemon has shutdown:
> 
> {code}
> 2017-03-14 05:00:25,602 - 
> call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs hdfs://c1ha 
> -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 'hdfs'}
> 2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
> datanode 0.0.0.0:8010')
> 2017-03-14 05:00:28,438 - 
> Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
> hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
> ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
> {'tries': 1, 'user': 'hdfs'}
> 2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
> {code}
> 
> Even though ~ 6 seconds have passed, the daemon is still running as it 
> drains. Therefore, we attempt to start it which causes a NOOP.
> 
> Instead, we should also monitor for the PID.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  b55237d 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 
> 
> 
> Diff: https://reviews.apache.org/r/58208/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 58198: Repetitive operation 'Link' in hdfs.py

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58198/#review171145
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
Line 132 (original)


If it does have to be removed, same should be done fo r HDFS 3.0


- Alejandro Fernandez


On April 5, 2017, 5:52 a.m., zhangxiaolu zhangxiaolu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58198/
> ---
> 
> (Updated April 5, 2017, 5:52 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Dmytro 
> Sen, Jaimin Jetly, Nate Cole, Srimanth Gunturi, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20675
> https://issues.apache.org/jira/browse/AMBARI-20675
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> the method of install_snappy is repetitive in hdfs.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  1264284 
> 
> 
> Diff: https://reviews.apache.org/r/58198/diff/1/
> 
> 
> Testing
> ---
> 
> done it
> 
> 
> Thanks,
> 
> zhangxiaolu zhangxiaolu
> 
>

Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade

[Amruta Borkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58122/
---

(Updated April 5, 2017, 4:58 p.m.)


Review request for Ambari, Di Li and Robert Nettleton.


Changes
---

Updated patch to address the comments


Bugs: AMBARI-20551
https://issues.apache.org/jira/browse/AMBARI-20551


Repository: ambari


Description
---

Following error is thrown while exporting a blueprint from cluster.
 ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request:
java.lang.IllegalArgumentException: Specified configuration type is not 
associated with any service: storm-site
at 
org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439)
at 
org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416)
at 
org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186)
at 
org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141)
at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839)
at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223)
at 
org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77)
at 
org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 db1aa074d4 


Diff: https://reviews.apache.org/r/58122/diff/3/

Changes: https://reviews.apache.org/r/58122/diff/2-3/


Testing
---

Tested manually. Log shows warning about unmapped config-type but blueprint 
gets exported.


Thanks,

Amruta Borkar

Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint

[Amruta Borkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57610/
---

(Updated April 5, 2017, 4:56 p.m.)


Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor 
Magyari.


Changes
---

Updated patch to filter out auth_to_local properties.


Bugs: AMBARI-20366
https://issues.apache.org/jira/browse/AMBARI-20366


Repository: ambari


Description
---

If blueprint is exported from a kerberos enabled cluster Kerberos rules export 
principal names which contain cluster name and Realm, this exports existing 
cluster name and realm name as tokens and replaces those tokens with new values 
of cluster name and realm during successive cluster deployments.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 db1aa074d4 
  
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java
 13db5f8b56 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 dba40437ae 


Diff: https://reviews.apache.org/r/57610/diff/2/

Changes: https://reviews.apache.org/r/57610/diff/1-2/


Testing
---

Tested manually.
Modified test cases.


Thanks,

Amruta Borkar

Re: Review Request 58211: AMBARI-20674 About to hide the Delete menu item from UI for a given service

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58211/#review171139
---


Ship it!




Ship It!

- Tim Thorpe


On April 5, 2017, 3:37 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58211/
> ---
> 
> (Updated April 5, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Sangeeta Ravindran and Tim Thorpe.
> 
> 
> Bugs: AMBARI-20674
> https://issues.apache.org/jira/browse/AMBARI-20674
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stack driven approach to have UI show/hide Delete service menu item for a 
> given service.
> A service's metainfo.xml can have
> false set at the service level to 
> indicate UI should hide the menu item. Directly going thru REST API calls are 
> always supported and non-restricted.
> If the section is missing, it will be treated as "support" UI delete service 
> operation. In other words, UI only hides the menu item if metainfo.xml has 
> explicitly set the flag to false.
> Example.
> "" +
> " 2.0" +
> " " +
> " " +
> " HDFS" +
> " HDFS" +
> " false" +
> " " +
> " " +
> "";
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
>  cbff300 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
>  a30d783 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> fd65268 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 0d0b78b 
>   ambari-server/src/main/resources/properties.json 04d32ea 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceInfoTest.java
>  1b9296e 
>   ambari-web/app/app.js 9c7d874 
>   ambari-web/app/mappers/stack_service_mapper.js 4bda89d 
>   ambari-web/app/models/host_component.js 3950f97 
>   ambari-web/app/models/stack_service.js 4f21288 
>   ambari-web/test/mappers/stack_service_mapper_test.js 9da8b24 
>   ambari-web/test/views/main/service/item_test.js 4b2e6f9 
> 
> 
> Diff: https://reviews.apache.org/r/58211/diff/1/
> 
> 
> Testing
> ---
> 
> unit testing
> Install a trunk cluster with HDP 2.6 stack.Patch it with the code change both 
> ambari server and UI change, then update metainfo.xml for hbase in common 
> services (to test inheritance) verify UI does not show Delete service menu 
> item for HBase after restart Ambari server.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 58206: Complete node name is not shown when node name is larger than 17 characters

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58206/#review171138
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On April 5, 2017, 10:05 a.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58206/
> ---
> 
> (Updated April 5, 2017, 10:05 a.m.)
> 
> 
> Review request for Ambari, Abhishek Kumar, DIPAYAN BHOWMICK, Gaurav Nagar, 
> Nitiraj Rathore, Rohit Choudhary, and venkat sairam.
> 
> 
> Bugs: AMBARI-20678
> https://issues.apache.org/jira/browse/AMBARI-20678
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Introduced ellipsis css class and showing complete name as tooltip.
> 
> 
> Diffs
> -
> 
>   contrib/views/hive20/src/main/resources/ui/app/styles/app.scss f4b63c5 
>   
> contrib/views/hive20/src/main/resources/ui/app/utils/hive-explainer/renderer.js
>  1cfcb15 
> 
> 
> Diff: https://reviews.apache.org/r/58206/diff/1/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>

Re: Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Dmitro Lisnichenko]

> On April 5, 2017, 4:10 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
> > Lines 51-59 (original), 51-59 (patched)
> > 
> >
> > Let's say that this fails to stop gracefully on the de-register. The 
> > code which calls this is looking for a boolean to be returned:
> > 
> > ```
> >   stopped = 
> > datanode_upgrade.pre_rolling_upgrade_shutdown(hdfs_binary)
> >   if not stopped:
> > datanode(action="stop")
> > ```
> > 
> > Should we try/catch `_check_datanode_shutdown` and return False if it 
> > fails?
> 
> Dmitro Lisnichenko wrote:
> this try-catch whould shadow a failure that is covered by 2 our tests

* would


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/#review171118
---


On April 5, 2017, 3:27 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58208/
> ---
> 
> (Updated April 5, 2017, 3:27 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-20682
> https://issues.apache.org/jira/browse/AMBARI-20682
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade (especially on a large, heavily used cluster), the 
> DataNodes do not shutdown immediately. However, they do de-register from the 
> NameNode which tricks Ambari into thinking that they are down.
> 
> Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
> DataNode back up before the daemon has shutdown:
> 
> {code}
> 2017-03-14 05:00:25,602 - 
> call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs hdfs://c1ha 
> -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 'hdfs'}
> 2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
> datanode 0.0.0.0:8010')
> 2017-03-14 05:00:28,438 - 
> Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
> hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
> ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
> {'tries': 1, 'user': 'hdfs'}
> 2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
> {code}
> 
> Even though ~ 6 seconds have passed, the daemon is still running as it 
> drains. Therefore, we attempt to start it which causes a NOOP.
> 
> Instead, we should also monitor for the PID.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  b55237d 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 
> 
> 
> Diff: https://reviews.apache.org/r/58208/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Dmitro Lisnichenko]

> On April 5, 2017, 4:10 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
> > Lines 51-59 (original), 51-59 (patched)
> > 
> >
> > Let's say that this fails to stop gracefully on the de-register. The 
> > code which calls this is looking for a boolean to be returned:
> > 
> > ```
> >   stopped = 
> > datanode_upgrade.pre_rolling_upgrade_shutdown(hdfs_binary)
> >   if not stopped:
> > datanode(action="stop")
> > ```
> > 
> > Should we try/catch `_check_datanode_shutdown` and return False if it 
> > fails?

this try-catch whould shadow a failure that is covered by 2 our tests


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/#review171118
---


On April 5, 2017, 3:27 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58208/
> ---
> 
> (Updated April 5, 2017, 3:27 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-20682
> https://issues.apache.org/jira/browse/AMBARI-20682
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade (especially on a large, heavily used cluster), the 
> DataNodes do not shutdown immediately. However, they do de-register from the 
> NameNode which tricks Ambari into thinking that they are down.
> 
> Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
> DataNode back up before the daemon has shutdown:
> 
> {code}
> 2017-03-14 05:00:25,602 - 
> call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs hdfs://c1ha 
> -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 'hdfs'}
> 2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
> datanode 0.0.0.0:8010')
> 2017-03-14 05:00:28,438 - 
> Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
> hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
> ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
> {'tries': 1, 'user': 'hdfs'}
> 2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
> {code}
> 
> Even though ~ 6 seconds have passed, the daemon is still running as it 
> drains. Therefore, we attempt to start it which causes a NOOP.
> 
> Instead, we should also monitor for the PID.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  b55237d 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 
> 
> 
> Diff: https://reviews.apache.org/r/58208/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 58198: Repetitive operation 'Link' in hdfs.py

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58198/#review171129
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
Lines 129-131 (original), 129-131 (patched)


There should be a test to cover this case.


- Nate Cole


On April 5, 2017, 1:52 a.m., zhangxiaolu zhangxiaolu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58198/
> ---
> 
> (Updated April 5, 2017, 1:52 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Dmytro 
> Sen, Jaimin Jetly, Nate Cole, Srimanth Gunturi, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20675
> https://issues.apache.org/jira/browse/AMBARI-20675
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> the method of install_snappy is repetitive in hdfs.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  1264284 
> 
> 
> Diff: https://reviews.apache.org/r/58198/diff/1/
> 
> 
> Testing
> ---
> 
> done it
> 
> 
> Thanks,
> 
> zhangxiaolu zhangxiaolu
> 
>

Re: Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/#review171128
---


Ship it!




Ship It!

- Nate Cole


On April 5, 2017, 8:27 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58208/
> ---
> 
> (Updated April 5, 2017, 8:27 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-20682
> https://issues.apache.org/jira/browse/AMBARI-20682
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade (especially on a large, heavily used cluster), the 
> DataNodes do not shutdown immediately. However, they do de-register from the 
> NameNode which tricks Ambari into thinking that they are down.
> 
> Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
> DataNode back up before the daemon has shutdown:
> 
> {code}
> 2017-03-14 05:00:25,602 - 
> call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs hdfs://c1ha 
> -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 'hdfs'}
> 2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
> datanode 0.0.0.0:8010')
> 2017-03-14 05:00:28,438 - 
> Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
> hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
> ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
> {'tries': 1, 'user': 'hdfs'}
> 2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
> {code}
> 
> Even though ~ 6 seconds have passed, the daemon is still running as it 
> drains. Therefore, we attempt to start it which causes a NOOP.
> 
> Instead, we should also monitor for the PID.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  b55237d 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 
> 
> 
> Diff: https://reviews.apache.org/r/58208/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 58211: AMBARI-20674 About to hide the Delete menu item from UI for a given service

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58211/
---

Review request for Ambari, Sangeeta Ravindran and Tim Thorpe.


Bugs: AMBARI-20674
https://issues.apache.org/jira/browse/AMBARI-20674


Repository: ambari


Description
---

Stack driven approach to have UI show/hide Delete service menu item for a given 
service.
A service's metainfo.xml can have
false set at the service level to 
indicate UI should hide the menu item. Directly going thru REST API calls are 
always supported and non-restricted.
If the section is missing, it will be treated as "support" UI delete service 
operation. In other words, UI only hides the menu item if metainfo.xml has 
explicitly set the flag to false.
Example.
"" +
" 2.0" +
" " +
" " +
" HDFS" +
" HDFS" +
" false" +
" " +
" " +
"";


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
 cbff300 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
 a30d783 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
fd65268 
  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
0d0b78b 
  ambari-server/src/main/resources/properties.json 04d32ea 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceInfoTest.java 
1b9296e 
  ambari-web/app/app.js 9c7d874 
  ambari-web/app/mappers/stack_service_mapper.js 4bda89d 
  ambari-web/app/models/host_component.js 3950f97 
  ambari-web/app/models/stack_service.js 4f21288 
  ambari-web/test/mappers/stack_service_mapper_test.js 9da8b24 
  ambari-web/test/views/main/service/item_test.js 4b2e6f9 


Diff: https://reviews.apache.org/r/58211/diff/1/


Testing
---

unit testing
Install a trunk cluster with HDP 2.6 stack.Patch it with the code change both 
ambari server and UI change, then update metainfo.xml for hbase in common 
services (to test inheritance) verify UI does not show Delete service menu item 
for HBase after restart Ambari server.


Thanks,

Di Li

Review Request 58210: Implement a websocket adapter for stomp.py

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58210/
---

Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.


Bugs: AMBARI-20684
https://issues.apache.org/jira/browse/AMBARI-20684


Repository: ambari


Description
---

This is needed for stomp to work via websocket instead of tcp


Diffs
-

  LICENSE.txt f05016f 
  NOTICE.txt 50f982c 
  ambari-agent/conf/unix/install-helper.sh 35aec15 
  ambari-agent/pom.xml 4807a35 
  ambari-agent/src/main/python/ambari_agent/client_example.py PRE-CREATION 
  ambari-agent/src/packages/tarball/all.xml a22f0bb 
  ambari-common/src/main/python/ambari_stomp/adapter/websocket.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/__init__.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/client/__init__.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/client/geventclient.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/client/threadedclient.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/client/tornadoclient.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/compat.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/exc.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/framing.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/manager.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/messaging.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/server/__init__.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/server/cherrypyserver.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/server/geventserver.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/server/wsgirefserver.py 
PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/server/wsgiutils.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/streaming.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/utf8validator.py PRE-CREATION 
  ambari-common/src/main/python/ambari_ws4py/websocket.py PRE-CREATION 
  ambari-project/pom.xml 98da9f4 
  ambari-server/pom.xml 8b4c8d6 
  ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml bdbf0de 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/AgentResourceTest.java
 5feb3cc 
  
ambari-server/src/test/java/org/apache/ambari/server/api/AmbariErrorHandlerTest.java
 239bfcb 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java
 f64a97f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariSessionManagerTest.java
 23f07c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 1c45589 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java
 d3c8acf 
  pom.xml f192895 


Diff: https://reviews.apache.org/r/58210/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/#review171118
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
Lines 51-59 (original), 51-59 (patched)


Let's say that this fails to stop gracefully on the de-register. The code 
which calls this is looking for a boolean to be returned:

```
  stopped = datanode_upgrade.pre_rolling_upgrade_shutdown(hdfs_binary)
  if not stopped:
datanode(action="stop")
```

Should we try/catch `_check_datanode_shutdown` and return False if it fails?


- Jonathan Hurley


On April 5, 2017, 8:27 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58208/
> ---
> 
> (Updated April 5, 2017, 8:27 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-20682
> https://issues.apache.org/jira/browse/AMBARI-20682
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade (especially on a large, heavily used cluster), the 
> DataNodes do not shutdown immediately. However, they do de-register from the 
> NameNode which tricks Ambari into thinking that they are down.
> 
> Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
> DataNode back up before the daemon has shutdown:
> 
> {code}
> 2017-03-14 05:00:25,602 - 
> call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs hdfs://c1ha 
> -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 'hdfs'}
> 2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
> datanode 0.0.0.0:8010')
> 2017-03-14 05:00:28,438 - 
> Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
> hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
> ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
> {'tries': 1, 'user': 'hdfs'}
> 2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
> {code}
> 
> Even though ~ 6 seconds have passed, the daemon is still running as it 
> drains. Therefore, we attempt to start it which causes a NOOP.
> 
> Instead, we should also monitor for the PID.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  b55237d 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 
> 
> 
> Diff: https://reviews.apache.org/r/58208/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 58208: Wait For DataNodes To Shutdown During a Rolling Upgrade

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58208/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-20682
https://issues.apache.org/jira/browse/AMBARI-20682


Repository: ambari


Description
---

During a rolling upgrade (especially on a large, heavily used cluster), the 
DataNodes do not shutdown immediately. However, they do de-register from the 
NameNode which tricks Ambari into thinking that they are down.

Since the rolling upgrade uses a {{RESTART}} command, we attempt to start the 
DataNode back up before the daemon has shutdown:

{code}
2017-03-14 05:00:25,602 - call['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs 
dfsadmin -fs hdfs://c1ha -shutdownDatanode 0.0.0.0:8010 upgrade'] {'user': 
'hdfs'}
2017-03-14 05:00:28,438 - call returned (0, 'Submitted a shutdown request to 
datanode 0.0.0.0:8010')
2017-03-14 05:00:28,438 - 
Execute['/usr/hdp/current/hadoop-hdfs-datanode/bin/hdfs dfsadmin -fs 
hdfs://c1ha -D ipc.client.connect.max.retries=5 -D 
ipc.client.connect.retry.interval=1000 -getDatanodeInfo 0.0.0.0:8010'] 
{'tries': 1, 'user': 'hdfs'}
2017-03-14 05:00:35,976 - DataNode has successfully shutdown for upgrade.
{code}

Even though ~ 6 seconds have passed, the daemon is still running as it drains. 
Therefore, we attempt to start it which causes a NOOP.

Instead, we should also monitor for the PID.


Diffs
-

  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
 b55237d 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 1c3c5b7 


Diff: https://reviews.apache.org/r/58208/diff/1/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 58198: Repetitive operation 'Link' in hdfs.py

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58198/#review171116
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
Line 132 (original)


these symlinks are for different instruction sets. Could you please 
elaborate why this change is required?


- Dmitro Lisnichenko


On April 5, 2017, 8:52 a.m., zhangxiaolu zhangxiaolu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58198/
> ---
> 
> (Updated April 5, 2017, 8:52 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Dmytro 
> Sen, Jaimin Jetly, Nate Cole, Srimanth Gunturi, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20675
> https://issues.apache.org/jira/browse/AMBARI-20675
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> the method of install_snappy is repetitive in hdfs.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  1264284 
> 
> 
> Diff: https://reviews.apache.org/r/58198/diff/1/
> 
> 
> Testing
> ---
> 
> done it
> 
> 
> Thanks,
> 
> zhangxiaolu zhangxiaolu
> 
>

Re: Review Request 58207: Select Version step of installer: repo URL validation message issues

[Oleg Nechiporenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58207/#review171115
---


Ship it!




Ship It!

- Oleg Nechiporenko


On April 5, 2017, 12:06 p.m., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58207/
> ---
> 
> (Updated April 5, 2017, 12:06 p.m.)
> 
> 
> Review request for Ambari and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-20681
> https://issues.apache.org/jira/browse/AMBARI-20681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Error message that contains long repository URL overflows the popover 
> block.
> 2. Popover has no title.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/views/wizard/step1_view.js 0161985 
> 
> 
> Diff: https://reviews.apache.org/r/58207/diff/1/
> 
> 
> Testing
> ---
> 
> 20674 passing (19s)
>   128 pending
> 
> 
> Thanks,
> 
> Andrii Babiichuk
> 
>

Review Request 58207: Select Version step of installer: repo URL validation message issues

[Andrii Babiichuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58207/
---

Review request for Ambari and Oleg Nechiporenko.


Bugs: AMBARI-20681
https://issues.apache.org/jira/browse/AMBARI-20681


Repository: ambari


Description
---

1. Error message that contains long repository URL overflows the popover block.
2. Popover has no title.


Diffs
-

  ambari-web/app/views/wizard/step1_view.js 0161985 


Diff: https://reviews.apache.org/r/58207/diff/1/


Testing
---

20674 passing (19s)
  128 pending


Thanks,

Andrii Babiichuk

Review Request 58206: Complete node name is not shown when node name is larger than 17 characters

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58206/
---

Review request for Ambari, Abhishek Kumar, DIPAYAN BHOWMICK, Gaurav Nagar, 
Nitiraj Rathore, Rohit Choudhary, and venkat sairam.


Bugs: AMBARI-20678
https://issues.apache.org/jira/browse/AMBARI-20678


Repository: ambari


Description
---

Introduced ellipsis css class and showing complete name as tooltip.


Diffs
-

  contrib/views/hive20/src/main/resources/ui/app/styles/app.scss f4b63c5 
  
contrib/views/hive20/src/main/resources/ui/app/utils/hive-explainer/renderer.js 
1cfcb15 


Diff: https://reviews.apache.org/r/58206/diff/1/


Testing
---

manually tested.


Thanks,

Pallav Kulshreshtha

Re: Review Request 58205: Centering workflows for zoom breaks when multiple tabs exists

[Padma Priya N]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58205/#review171110
---


Ship it!




Ship It!

- Padma Priya N


On April 5, 2017, 9:38 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58205/
> ---
> 
> (Updated April 5, 2017, 9:38 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-20677
> https://issues.apache.org/jira/browse/AMBARI-20677
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Made the workflow to center while switching between the tabs for the first 
> time
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  980904f 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> fa7c861 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/cytoscape-flow-renderer.js
>  af84f86 
> 
> 
> Diff: https://reviews.apache.org/r/58205/diff/1/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Re: Review Request 58204: User should be able to visualize inherited properties while submitting the workflow

[belliraj hb]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58204/#review171109
---


Ship it!




Ship It!

- belliraj hb


On April 5, 2017, 9:22 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58204/
> ---
> 
> (Updated April 5, 2017, 9:22 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-20676
> https://issues.apache.org/jira/browse/AMBARI-20676
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added the inherited properies such as resourceManager, nameNode to 
> job.properties while submitting the workflow.
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
> 6aed9da 
>   contrib/views/wfmanager/src/main/resources/ui/app/routes/index.js 6d94dfe 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/services/workflow-manager-configs.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/tests/unit/services/workflow-manager-configs-test.js
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58204/diff/1/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Re: Review Request 58205: Centering workflows for zoom breaks when multiple tabs exists

[belliraj hb]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58205/#review171108
---


Ship it!




Ship It!

- belliraj hb


On April 5, 2017, 9:38 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58205/
> ---
> 
> (Updated April 5, 2017, 9:38 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-20677
> https://issues.apache.org/jira/browse/AMBARI-20677
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Made the workflow to center while switching between the tabs for the first 
> time
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  980904f 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> fa7c861 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/cytoscape-flow-renderer.js
>  af84f86 
> 
> 
> Diff: https://reviews.apache.org/r/58205/diff/1/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Review Request 58205: Centering workflows for zoom breaks when multiple tabs exists

[Madhan Reddy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58205/
---

Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and Pallav 
Kulshreshtha.


Bugs: AMBARI-20677
https://issues.apache.org/jira/browse/AMBARI-20677


Repository: ambari


Description
---

Made the workflow to center while switching between the tabs for the first time


Diffs
-

  
contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
 980904f 
  contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
fa7c861 
  
contrib/views/wfmanager/src/main/resources/ui/app/domain/cytoscape-flow-renderer.js
 af84f86 


Diff: https://reviews.apache.org/r/58205/diff/1/


Testing
---

Manual


Thanks,

Madhan Reddy

Re: Review Request 58204: User should be able to visualize inherited properties while submitting the workflow

[Padma Priya N]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58204/#review171107
---


Ship it!




Ship It!

- Padma Priya N


On April 5, 2017, 9:22 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58204/
> ---
> 
> (Updated April 5, 2017, 9:22 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-20676
> https://issues.apache.org/jira/browse/AMBARI-20676
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added the inherited properies such as resourceManager, nameNode to 
> job.properties while submitting the workflow.
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
> 6aed9da 
>   contrib/views/wfmanager/src/main/resources/ui/app/routes/index.js 6d94dfe 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/services/workflow-manager-configs.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/tests/unit/services/workflow-manager-configs-test.js
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58204/diff/1/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Review Request 58204: User should be able to visualize inherited properties while submitting the workflow

[Madhan Reddy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58204/
---

Review request for Ambari, belliraj hb, Gaurav Nagar, Padma Priya N, and Pallav 
Kulshreshtha.


Bugs: AMBARI-20676
https://issues.apache.org/jira/browse/AMBARI-20676


Repository: ambari


Description
---

Added the inherited properies such as resourceManager, nameNode to 
job.properties while submitting the workflow.


Diffs
-

  contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
6aed9da 
  contrib/views/wfmanager/src/main/resources/ui/app/routes/index.js 6d94dfe 
  
contrib/views/wfmanager/src/main/resources/ui/app/services/workflow-manager-configs.js
 PRE-CREATION 
  
contrib/views/wfmanager/src/main/resources/ui/tests/unit/services/workflow-manager-configs-test.js
 PRE-CREATION 


Diff: https://reviews.apache.org/r/58204/diff/1/


Testing
---

Manual


Thanks,

Madhan Reddy

Re: Review Request 58179: For sort/partition operator, if there is only 1 reducer, display just "sort" rather than "sort/partition"

[Padma Priya N]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58179/#review171102
---


Ship it!




Ship It!

- Padma Priya N


On April 4, 2017, 5:08 p.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58179/
> ---
> 
> (Updated April 4, 2017, 5:08 p.m.)
> 
> 
> Review request for Ambari, Abhishek Kumar, Gaurav Nagar, Nitiraj Rathore, 
> Padma Priya N, Pallav Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-20673
> https://issues.apache.org/jira/browse/AMBARI-20673
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now when we have only one reducer, the label for aort/partition will be 
> displayed as partition.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive20/src/main/resources/ui/app/utils/hive-explainer/renderer.js
>  1cfcb15 
> 
> 
> Diff: https://reviews.apache.org/r/58179/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> venkat sairam
> 
>