Re: Review Request 56199: HDP 3.0 TP - Support changed configs and scripts for HDFS

2017-02-02 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56199/#review164086
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 3, 2017, 2:26 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56199/
> ---
> 
> (Updated Feb. 3, 2017, 2:26 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18930
> https://issues.apache.org/jira/browse/AMBARI-18930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HDP 3.0, there are expected changes to configs and the startup scripts for 
> HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hdfs-site.xml
>  689b6d08 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
>  f7aa4c9 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py
>  48f5a1f 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/metainfo.xml 
> 28dfb37 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
> 051cad1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml 
> accbedd 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 
> 85b669e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 4230dd4 
>   ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 
> fb97612 
>   ambari-server/src/test/python/unitTests.py 7941ed3 
> 
> Diff: https://reviews.apache.org/r/56199/diff/
> 
> 
> Testing
> ---
> 
> Verified on live cluster.
> 
> Unit tests passed,
> 
> --
> Total run:1182
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 56261: Perf: start/stop all actions works much slower after few days of testing

2017-02-02 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56261/#review164083
---


Ship it!




Ship It!

- Sid Wagle


On Feb. 3, 2017, 12:24 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56261/
> ---
> 
> (Updated Feb. 3, 2017, 12:24 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-19856
> https://issues.apache.org/jira/browse/AMBARI-19856
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add changes to gce script, to deploy perf cluster with all options needed. 
> Try to find more code to optimize.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  c4df65f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 95436c5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  fd92bed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  95de4e8 
>   contrib/utils/perf/deploy-gce-perf-cluster.py 7e84c40 
> 
> Diff: https://reviews.apache.org/r/56261/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 56266: The user must be clearly communicated about YARN pre-emption requirements when Hive LLAP is enabled

2017-02-02 Thread Vivek Ratnavel Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56266/
---

(Updated Feb. 3, 2017, 3:44 a.m.)


Review request for Ambari, Jaimin Jetly and Yusaku Sako.


Changes
---

Removed the bold formatting in "Note:" and made a minor correction to the text.


Bugs: AMBARI-19859
https://issues.apache.org/jira/browse/AMBARI-19859


Repository: ambari


Description
---

Capacity Scheduler preemption is not enabled by default in HDP. Enabling it 
affects the entire cluster. But it is strongly recommended to enable YARN 
pre-emption before enabling Hive LLAP. The user needs to be warned about this 
before enabling Interactive Query in Hive configs page.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 89eccc6 
  
ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml
 787dcb1 

Diff: https://reviews.apache.org/r/56266/diff/


Testing
---

Verified Manually.


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 56266: The user must be clearly communicated about YARN pre-emption requirements when Hive LLAP is enabled

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56266/#review164079
---


Ship it!




Ship It!

- Alejandro Fernandez


On Feb. 3, 2017, 3:15 a.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56266/
> ---
> 
> (Updated Feb. 3, 2017, 3:15 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: AMBARI-19859
> https://issues.apache.org/jira/browse/AMBARI-19859
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Capacity Scheduler preemption is not enabled by default in HDP. Enabling it 
> affects the entire cluster. But it is strongly recommended to enable YARN 
> pre-emption before enabling Hive LLAP. The user needs to be warned about this 
> before enabling Interactive Query in Hive configs page.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  89eccc6 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml
>  787dcb1 
> 
> Diff: https://reviews.apache.org/r/56266/diff/
> 
> 
> Testing
> ---
> 
> Verified Manually.
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Review Request 56264: Add "live_hosts" metric in AMS for apps

2017-02-02 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56264/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-19858
https://issues.apache.org/jira/browse/AMBARI-19858


Repository: ambari


Description
---

{quote}
live_hosts=> # count of total hosts reporting 
metrics
live_hosts & appId =  => # count of hosts hosting this appId
{quote}

Where appId = { namenode, datanode, hbase ... etc }


Diffs
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 8d567ce 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAppAggregator.java
 d7b0d55 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 6f3d8bc 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java
 7a74e24 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
 58d908a 

Diff: https://reviews.apache.org/r/56264/diff/


Testing
---

mvn test passed for ambari-metrics.


Thanks,

Sid Wagle

Review Request 56266: The user must be clearly communicated about YARN pre-emption requirements when Hive LLAP is enabled

2017-02-02 Thread Vivek Ratnavel Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56266/
---

Review request for Ambari, Jaimin Jetly and Yusaku Sako.


Bugs: AMBARI-19859
https://issues.apache.org/jira/browse/AMBARI-19859


Repository: ambari


Description
---

Capacity Scheduler preemption is not enabled by default in HDP. Enabling it 
affects the entire cluster. But it is strongly recommended to enable YARN 
pre-emption before enabling Hive LLAP. The user needs to be warned about this 
before enabling Interactive Query in Hive configs page.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 89eccc6 
  
ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml
 787dcb1 

Diff: https://reviews.apache.org/r/56266/diff/


Testing
---

Verified Manually.


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 56199: HDP 3.0 TP - Support changed configs and scripts for HDFS

2017-02-02 Thread Alejandro Fernandez 


> On Feb. 3, 2017, 2:31 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml, 
> > line 20
> > 
> >
> > Why was this change needed? Is it just to make it explicit?

I didn't want to have to parse yet another file (which is the service 
metainfo.xml) whenever the service doesn't have the  line.
Most services do have that, so I don't mind checking the service metainfo.xml 
file.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56199/#review164072
---


On Feb. 3, 2017, 2:26 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56199/
> ---
> 
> (Updated Feb. 3, 2017, 2:26 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18930
> https://issues.apache.org/jira/browse/AMBARI-18930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HDP 3.0, there are expected changes to configs and the startup scripts for 
> HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hdfs-site.xml
>  689b6d08 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
>  f7aa4c9 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py
>  48f5a1f 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/metainfo.xml 
> 28dfb37 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
> 051cad1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml 
> accbedd 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 
> 85b669e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 4230dd4 
>   ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 
> fb97612 
>   ambari-server/src/test/python/unitTests.py 7941ed3 
> 
> Diff: https://reviews.apache.org/r/56199/diff/
> 
> 
> Testing
> ---
> 
> Verified on live cluster.
> 
> Unit tests passed,
> 
> --
> Total run:1182
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 56199: HDP 3.0 TP - Support changed configs and scripts for HDFS

2017-02-02 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56199/#review164072
---




ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
(line 20)


Why was this change needed? Is it just to make it explicit?


- Sumit Mohanty


On Feb. 3, 2017, 2:26 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56199/
> ---
> 
> (Updated Feb. 3, 2017, 2:26 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18930
> https://issues.apache.org/jira/browse/AMBARI-18930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HDP 3.0, there are expected changes to configs and the startup scripts for 
> HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hdfs-site.xml
>  689b6d08 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
>  f7aa4c9 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py
>  48f5a1f 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/metainfo.xml 
> 28dfb37 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
> 051cad1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml 
> accbedd 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 
> 85b669e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 4230dd4 
>   ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 
> fb97612 
>   ambari-server/src/test/python/unitTests.py 7941ed3 
> 
> Diff: https://reviews.apache.org/r/56199/diff/
> 
> 
> Testing
> ---
> 
> Verified on live cluster.
> 
> Unit tests passed,
> 
> --
> Total run:1182
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 56199: HDP 3.0 TP - Support changed configs and scripts for HDFS

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56199/#review164071
---




ambari-server/src/test/python/unitTests.py 


This used to blindly import all versions found in common-services, which is 
obviously problematic when the unit tests for HDFS are for stack 2.0.6 and 
common-services 2.1.0.2.0, but end up importing modules from common services 
3.0.0.3.0


- Alejandro Fernandez


On Feb. 3, 2017, 2:26 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56199/
> ---
> 
> (Updated Feb. 3, 2017, 2:26 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18930
> https://issues.apache.org/jira/browse/AMBARI-18930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HDP 3.0, there are expected changes to configs and the startup scripts for 
> HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hdfs-site.xml
>  689b6d08 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
>  f7aa4c9 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py
>  48f5a1f 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/metainfo.xml 
> 28dfb37 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
> 051cad1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml 
> accbedd 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 
> 85b669e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 4230dd4 
>   ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 
> fb97612 
>   ambari-server/src/test/python/unitTests.py 7941ed3 
> 
> Diff: https://reviews.apache.org/r/56199/diff/
> 
> 
> Testing
> ---
> 
> Verified on live cluster.
> 
> Unit tests passed,
> 
> --
> Total run:1182
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 56199: HDP 3.0 TP - Support changed configs and scripts for HDFS

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56199/
---

(Updated Feb. 3, 2017, 2:26 a.m.)


Review request for Ambari, Dmytro Sen, Sid Wagle, and Vitalyi Brodetskyi.


Changes
---

Fixed unit tests.


Bugs: AMBARI-18930
https://issues.apache.org/jira/browse/AMBARI-18930


Repository: ambari


Description
---

In HDP 3.0, there are expected changes to configs and the startup scripts for 
HDFS.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hdfs-site.xml
 689b6d08 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
 f7aa4c9 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py
 48f5a1f 
  ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/metainfo.xml 
28dfb37 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml 
051cad1 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml 
accbedd 
  ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 
85b669e 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
4230dd4 
  ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py fb97612 
  ambari-server/src/test/python/unitTests.py 7941ed3 

Diff: https://reviews.apache.org/r/56199/diff/


Testing (updated)
---

Verified on live cluster.

Unit tests passed,

--
Total run:1182
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez

Re: Review Request 56261: Perf: start/stop all actions works much slower after few days of testing

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56261/#review164064
---


Ship it!




Ship It!

- Alejandro Fernandez


On Feb. 3, 2017, 12:24 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56261/
> ---
> 
> (Updated Feb. 3, 2017, 12:24 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-19856
> https://issues.apache.org/jira/browse/AMBARI-19856
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add changes to gce script, to deploy perf cluster with all options needed. 
> Try to find more code to optimize.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  c4df65f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 95436c5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  fd92bed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  95de4e8 
>   contrib/utils/perf/deploy-gce-perf-cluster.py 7e84c40 
> 
> Diff: https://reviews.apache.org/r/56261/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 56261: Perf: start/stop all actions works much slower after few days of testing

2017-02-02 Thread Vitalyi Brodetskyi 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56261/
---

Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-19856
https://issues.apache.org/jira/browse/AMBARI-19856


Repository: ambari


Description
---

Add changes to gce script, to deploy perf cluster with all options needed. Try 
to find more code to optimize.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 c4df65f 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
95436c5 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
 fd92bed 
  
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
 95de4e8 
  contrib/utils/perf/deploy-gce-perf-cluster.py 7e84c40 

Diff: https://reviews.apache.org/r/56261/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 56230: Stack advisor issues encountered

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56230/#review164039
---




ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py (line 
144)


The same changes needs to be made in service_advisor for YARN in 3.0.0.3.0


- Alejandro Fernandez


On Feb. 2, 2017, 6:08 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56230/
> ---
> 
> (Updated Feb. 2, 2017, 6:08 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-19855
> https://issues.apache.org/jira/browse/AMBARI-19855
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. A default setting that did not hold so many cores in reserve. 20% seems 
> very high. 
> 2. 2. This setting should be validated to flag when an impossible setting has 
> been configured, specifically when the number of configured virtual cores is 
> greater than actual number of cores it should be flagged.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> cba611c 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/stack_advisor.py 
> b72f046 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> a26b661 
> 
> Diff: https://reviews.apache.org/r/56230/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 56230: Stack advisor issues encountered

2017-02-02 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56230/#review164034
---


Ship it!




Ship It!

- Sid Wagle


On Feb. 2, 2017, 6:08 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56230/
> ---
> 
> (Updated Feb. 2, 2017, 6:08 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-19855
> https://issues.apache.org/jira/browse/AMBARI-19855
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. A default setting that did not hold so many cores in reserve. 20% seems 
> very high. 
> 2. 2. This setting should be validated to flag when an impossible setting has 
> been configured, specifically when the number of configured virtual cores is 
> greater than actual number of cores it should be flagged.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> cba611c 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/stack_advisor.py 
> b72f046 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> a26b661 
> 
> Diff: https://reviews.apache.org/r/56230/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 56232: AMBARI-19810. Remove upgrade logic in UpdateCatalog250 for tez-interactive-site's 'tez.runtime.io.sort.mb' and 'tez.runtime.unordered.output.buffer.size-mb'.

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56232/#review164036
---


Ship it!




Ship It!

- Alejandro Fernandez


On Feb. 2, 2017, 10:11 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56232/
> ---
> 
> (Updated Feb. 2, 2017, 10:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19810
> https://issues.apache.org/jira/browse/AMBARI-19810
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-19810. Remove upgrade logic in UpdateCatalog250 for 
> tez-interactive-site's 'tez.runtime.io.sort.mb' and 
> 'tez.runtime.unordered.output.buffer.size-mb'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  6b9076c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  1c455be 
> 
> Diff: https://reviews.apache.org/r/56232/diff/
> 
> 
> Testing
> ---
> 
> Build successful.
> 
> Test cases updated.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Review Request 56232: AMBARI-19810. Remove upgrade logic in UpdateCatalog250 for tez-interactive-site's 'tez.runtime.io.sort.mb' and 'tez.runtime.unordered.output.buffer.size-mb'.

2017-02-02 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56232/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-19810
https://issues.apache.org/jira/browse/AMBARI-19810


Repository: ambari


Description
---

AMBARI-19810. Remove upgrade logic in UpdateCatalog250 for 
tez-interactive-site's 'tez.runtime.io.sort.mb' and 
'tez.runtime.unordered.output.buffer.size-mb'.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 6b9076c 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 1c455be 

Diff: https://reviews.apache.org/r/56232/diff/


Testing
---

Build successful.

Test cases updated.


Thanks,

Swapan Shridhar

Re: Review Request 53686: Stage and Request status should be persisted in the database

2017-02-02 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53686/#review164025
---



Have you tried this patch on a large cluster with success?


ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
 (line 520)


This shouldn't be a static helper call, it should just be on the status 
itself:

"existingTaskStatus.isValidTransition()"
or rather just "!existingTaskStatus.isCompletedState()"



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
 (line 186)


See other comment, either make this non-static on the status enum or just 
use !isCompletedState() directly.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
 (line 310)


nit: Can remove redundant types here, and a lot of other places in this 
patch:

Map> counters = new HashMap<>();
List list = new ArrayList<>

etc etc



ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
 (lines 215 - 216)


Potentially doing db work to multiple stages.  What are the implications of 
failure here?  Should it be @Transactional (if yes, remember the method cannot 
be public).

Also, if some of them can be updated, and others not, can they be 
reconciled to "try again"?



ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
 (lines 260 - 262)


Same as other comment.  Transactional?  Partial updates?



ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskEventPublisher.java
 (lines 35 - 38)


This means that task events will happen on the same thread they came in on. 
 Is that ok?  We can still make this an async bus with one thread so it won't 
hold up processing of large amounts of Stages and HostRoleCommands.



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 (lines 647 - 650)


Since you're deferring to mergeWithoutPublishEvent() that is marked 
@Transactional already, this method may not need those annotations.

@Jonathan for input if it's required for both methods.


- Nate Cole


On Jan. 18, 2017, 5:33 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53686/
> ---
> 
> (Updated Jan. 18, 2017, 5:33 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-18868
> https://issues.apache.org/jira/browse/AMBARI-18868
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stage and Request status should be persisted in the database.
> 
> upgrading to ambari-3.0.0 should add status for all present stages and 
> request for the cluster.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  7837a7b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  dabcb98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
>  3656bfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java
>  31e11c1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> 4a05b32 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3c415df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskCreateEvent.java
>  PRE-CREATION 
>   ambari-server/src/main/java/org/apache/ambari/server/events/TaskEvent.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskUpdateEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskEventPublisher.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  02c4091 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
> 1c4d0a3 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
> d2f899f 
>   
> ambari-server/src/main/java/org/apache/ambari/serve

Re: Review Request 56059: Preview: Package Installation fails due to error in Berkeley DB library

2017-02-02 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164024
---


Ship it!




Ship It!

- Jonathan Hurley


On Feb. 2, 2017, 11:16 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> ---
> 
> (Updated Feb. 2, 2017, 11:16 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the 
> jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on 
> Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
> 
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = 
> self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not 
> verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py",
>  line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = 
> rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = 
> function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, 
> try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = 
> _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise 
> ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - 
> run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check 
> dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 
> 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 
> error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run 
> database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages index using db5 -  (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 469, in 
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - 
> run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_managem

Re: Review Request 56059: Preview: Package Installation fails due to error in Berkeley DB library

2017-02-02 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164021
---


Ship it!




Ship It!

- Nate Cole


On Feb. 2, 2017, 11:16 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> ---
> 
> (Updated Feb. 2, 2017, 11:16 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the 
> jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on 
> Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
> 
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = 
> self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not 
> verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py",
>  line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = 
> rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = 
> function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, 
> try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = 
> _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise 
> ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - 
> run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check 
> dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 
> 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 
> error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run 
> database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages index using db5 -  (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 469, in 
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - 
> run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/li

Re: Review Request 56235: findLatestServiceConfigsByStack query returns deleted config group

2017-02-02 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56235/#review164018
---


Ship it!




Ship It!

- Nate Cole


On Feb. 2, 2017, 3:01 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56235/
> ---
> 
> (Updated Feb. 2, 2017, 3:01 p.m.)
> 
> 
> Review request for Ambari, Di Li and Nate Cole.
> 
> 
> Bugs: AMBARI-19813
> https://issues.apache.org/jira/browse/AMBARI-19813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Query returns deleted config groups as latest active config group.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java
>  8a1b316 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java
>  5890c35 
> 
> Diff: https://reviews.apache.org/r/56235/diff/
> 
> 
> Testing
> ---
> 
> Modified test cases.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 56233: Increase default timeout and threadpool size for the external script to work on slower machines

2017-02-02 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56233/#review164014
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 2, 2017, 7:50 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56233/
> ---
> 
> (Updated Feb. 2, 2017, 7:50 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-19433
> https://issues.apache.org/jira/browse/AMBARI-19433
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase default timeout and threadpool size for the external script to work 
> on slower machines
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  7f047c8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
>  8a35c98 
> 
> Diff: https://reviews.apache.org/r/56233/diff/
> 
> 
> Testing
> ---
> 
> Tested on slower machines.
> Hadoop QA job on the jira stated no existing test failures
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Review Request 56235: findLatestServiceConfigsByStack query returns deleted config group

2017-02-02 Thread Amruta Borkar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56235/
---

Review request for Ambari, Di Li and Nate Cole.


Bugs: AMBARI-19813
https://issues.apache.org/jira/browse/AMBARI-19813


Repository: ambari


Description
---

Query returns deleted config groups as latest active config group.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java
 8a1b316 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java
 5890c35 

Diff: https://reviews.apache.org/r/56235/diff/


Testing
---

Modified test cases.


Thanks,

Amruta Borkar

Re: Review Request 56224: Improve Log Feeder simulation to help scale testing

2017-02-02 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56224/#review164012
---


Ship it!




Ship It!

- Robert Nettleton


On Feb. 2, 2017, 1 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56224/
> ---
> 
> (Updated Feb. 2, 2017, 1 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-19847
> https://issues.apache.org/jira/browse/AMBARI-19847
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Simulated Log Feeder input writes almost identical log messages, which is not 
> really simulates real logs.
> 
> Instead of the logfeeder.simulate.log_message_size property there will be 3 
> new properties (the simulated Log Feeders would enforce a minimum, and a 
> maximum value, and also set a default for them, if they are not specified):
> 
> logfeeder.simulate.number_of_words - the number of different words that may 
> appear in the simulated logs (50 / 100, default 1000)
> logfeeder.simulate.min_log_words - the minimum number of words in a simulated 
> log message (1 / 10, default 5)
> logfeeder.simulate.max_log_words - the maximum number of words in a log 
> message (10 / 20, default 10)
> 
> After the parameters are set each logfeeder will start to load messages like 
> this. In each message all the words are different:
> 
> Word000912 Word000903 Word000940 Word000760 Word000564 Word000762 Word000955 
> Word000553
> Word000320 Word000218 Word000872 Word000204 Word000120 Word000298 Word000727 
> Word000720 Word000475
> Word000413 Word000642 Word000872 Word000464 Word000746 Word000250
> 
> The rest of the properties remain the same:
> 
> logfeeder.simulate.input_number - number of paralell inputs (threads) loading 
> the logs, if it is set and not 0 then all the rest of the configured inputs 
> are ignored (running in simulation mode)!!
> logfeeder.simulate.log_ids - comma separated list of the log ids to propagate 
> at random, if not set by default all the available logs are propagated at 
> random, example: storm_drpc,storm_logviewer,storm_nimbus
> logfeeder.simulate.log_level - the level of the simulated log messages, by 
> default WARN
> logfeeder.simulate.sleep_milliseconds - the time interval at which each 
> simulated inputs writes one log message at random
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
>  743be69 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java
>  5bf600e 
> 
> Diff: https://reviews.apache.org/r/56224/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/#review164011
---


Ship it!




Ship It!

- Robert Nettleton


On Feb. 2, 2017, 4:23 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56179/
> ---
> 
> (Updated Feb. 2, 2017, 4:23 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-19822
> https://issues.apache.org/jira/browse/AMBARI-19822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> If an ambari cluster is secured and kerberos authentication is used for Solr, 
> we need (default) authorizations as well to make sure only the specific 
> service users (ranger, atlas, logsearch) can access their collections (and 
> solr user as well)
> 
> Solution:
> Although RuleBasedAuthorizationPlugin seems to be a good solution here, to 
> map default users to default permissions, unfortunately, permissions and 
> roles using principal name for mapping (not username) from the authentication 
> tokens. Also Solr name rules applied on the username and not on the 
> principal, therefore we need the fully qualified hostname as well in the 
> role-permission mapping. In order to avoid that issue, I added an own plugin 
> (org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
> users with @ format.
> 
> to problem is in here in RuleBasedAuthorizationPlugin.java:
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153
> 
> notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
> (InfraUserRolesLookupStrategy class which I added and included in the new 
> plugin class)
> 
> In case of we need strict host validations i added 2 new json properties for 
> that:
> 1. { "user-host" : {"" : []} }
> 2. {"user-host-regex" : {"" : "hostname-regex"} }
> 
> {{user-host-regex}} has higher precedence then {{user-host}}
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraKerberosHostValidator.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraKerberosHostValidatorTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
>   ambari-logsearch/pom.xml 7aeb4a7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
>  526baea 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
>  d8aea24 
> 
> Diff: https://reviews.apache.org/r/56179/diff/
> 
> 
> Testing
> ---
> 
> unit tests done, behavior validated with unit tests. FT: validated with 
> logsearch and atlas as well.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Review Request 56233: Increase default timeout and threadpool size for the external script to work on slower machines

2017-02-02 Thread Jaimin Jetly 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56233/
---

Review request for Ambari, Sid Wagle and Yusaku Sako.


Bugs: AMBARI-19433
https://issues.apache.org/jira/browse/AMBARI-19433


Repository: ambari


Description
---

Increase default timeout and threadpool size for the external script to work on 
slower machines


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 7f047c8 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
 8a35c98 

Diff: https://reviews.apache.org/r/56233/diff/


Testing
---

Tested on slower machines.
Hadoop QA job on the jira stated no existing test failures


Thanks,

Jaimin Jetly

Re: Review Request 55099: Disable auto start before RU/EU and enable during finalization phase

2017-02-02 Thread Andrii Tkach 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55099/
---

(Updated Feb. 2, 2017, 7:15 p.m.)


Review request for Ambari and Aleksandr Kovalenko.


Bugs: AMBARI-19319
https://issues.apache.org/jira/browse/AMBARI-19319


Repository: ambari


Description
---

Add the API calls to disable/enable auto start and assign it to the UI team to 
make the changes as part of RU/EU orchestration.
Lets see if we can also design an alert that WARNs when autostart is disabled - 
especially if it was disabled due to EU/RU.


Diffs (updated)
-

  ambari-web/app/messages.js 6f8f435 
  ambari-web/app/templates/main/admin/stack_upgrade/stack_upgrade_wizard.hbs 
8bb0904 

Diff: https://reviews.apache.org/r/55099/diff/


Testing
---

19845 passing (24s)
153 pending


Thanks,

Andrii Tkach

Review Request 56231: AMBARI-19829: Several HDFS/YARN widgets on Heatmaps show N/A

2017-02-02 Thread Qin Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56231/
---

Review request for Ambari.


Bugs: AMBARI-19829
https://issues.apache.org/jira/browse/AMBARI-19829


Repository: ambari


Description
---

The following HDFS/YARN widgets on Heatmaps show N/A:
1. HDFS - HDFS Bytes Written, HDFS Bytes Read, DataNode Process Disk I/O 
Utilization, and DataNode Process Network I/O Utilization
2. YARN - Container Failures

The issue was introduced by AMBARI-15835 "Rate metrics requesting widgets do 
not show up on Ambari" which introduced rate metrics and applied rate metrics 
to several HBASE/HDFS/YARN widgets on Summary pages as well as Heatmap pages. 
Rate metrics work fine on Summary pages but they don't work on Heatmap pages 
because current Heatmap design can only show point-in-time metrics and rate 
metrics need a time range.


Diffs
-

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/widgets.json 
bcfb2cc 
  ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/widgets.json 
4a645b0 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/YARN_widgets.json
 4b76a17 
  
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/YARN_widgets.json
 782f21d 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/widgets.json 
4a645b0 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/YARN_widgets.json 
782f21d 

Diff: https://reviews.apache.org/r/56231/diff/


Testing
---

Manually tested in UI.


Thanks,

Qin Liu

Re: Review Request 56059: Preview: Package Installation fails due to error in Berkeley DB library

2017-02-02 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164007
---


Ship it!




Ship It!

- Alejandro Fernandez


On Feb. 2, 2017, 4:16 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> ---
> 
> (Updated Feb. 2, 2017, 4:16 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the 
> jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on 
> Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
> 
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = 
> self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not 
> verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py",
>  line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = 
> rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = 
> function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, 
> try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = 
> _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise 
> ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - 
> run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check 
> dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 
> 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 
> error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run 
> database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages index using db5 -  (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 469, in 
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - 
> run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_manag

Re: Review Request 56196: UI changes to resolve discrepancies between what the stack vs Ambari reports as "live" for NodeManagers

2017-02-02 Thread Yusaku Sako 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56196/#review164005
---


Ship it!




Ship It!

- Yusaku Sako


On Feb. 1, 2017, 11:56 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56196/
> ---
> 
> (Updated Feb. 1, 2017, 11:56 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: AMBARI-19828
> https://issues.apache.org/jira/browse/AMBARI-19828
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Looks like the changes in https://issues.apache.org/jira/browse/AMBARI-9418 
> was meant to be applied to YARN summary / NodeManagers as well, but that is 
> not happening.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/messages.js 33a8289 
>   ambari-web/app/templates/main/service/services/yarn.hbs 138742d 
>   ambari-web/app/views/main/service/services/hdfs.js 40fb761 
>   ambari-web/app/views/main/service/services/yarn.js cf8adeb 
>   ambari-web/test/views/main/service/services/hdfs_test.js 06d0e01 
>   ambari-web/test/views/main/service/services/yarn_test.js 9c1cf7a 
> 
> Diff: https://reviews.apache.org/r/56196/diff/
> 
> 
> Testing
> ---
> 
> Verified Manually.
> Ambari-web unit tests pass.
> 30313 passing (23s)
> 157 pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Review Request 56230: Stack advisor issues encountered

2017-02-02 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56230/
---

Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-19855
https://issues.apache.org/jira/browse/AMBARI-19855


Repository: ambari


Description
---

1. A default setting that did not hold so many cores in reserve. 20% seems very 
high. 
2. 2. This setting should be validated to flag when an impossible setting has 
been configured, specifically when the number of configured virtual cores is 
greater than actual number of cores it should be flagged.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
cba611c 
  ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/stack_advisor.py 
b72f046 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py a26b661 

Diff: https://reviews.apache.org/r/56230/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 56227: ambari-server start failed with exit code 1.

2017-02-02 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56227/#review164001
---


Ship it!




Ship It!

- Jayush Luniya


On Feb. 2, 2017, 2:19 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56227/
> ---
> 
> (Updated Feb. 2, 2017, 2:19 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-19851
> https://issues.apache.org/jira/browse/AMBARI-19851
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Found this issue in ST.  
> Trying to provide setup ambari-server and
> 
> 
> 
> [root@ctr-e126-1485243696039-17812-01-04 ~]# ambari-server setup -s
> Using python  /usr/bin/python
> Setup ambari-server
> Nothing was done. Ambari Setup already performed and cannot re-run setup 
> in silent mode. Use "ambari-server setup" command without -s option to change 
> Ambari setup.
> 
> 
> Trying to start ambari-server
> 
> 
> 
> [root@ctr-e126-1485243696039-17812-01-04 ~]# ambari-server start
> Using python  /usr/bin/python
> Starting ambari-server
> ERROR: Exiting with exit code 1. 
> REASON: Unable to detect a system user for Ambari Server.
> - If this is a new setup, then run the "ambari-server setup" command to 
> create the user
> - If this is an upgrade of an existing setup, run the "ambari-server 
> upgrade" command.
> Refer to the Ambari documentation for more information on setup and 
> upgrade.
> 
> 
> Ambari-server restart in debug log
> 
> 
> 
> 2017-02-01 04:36:34,630 DEBUG 
> com.hw.ambari.ui.util.cluster_managers.CommandExecutor.executeCommandSequence():
>  Sending command [ambari-server restart --debug]
> 2017-02-01 04:36:35,331 DEBUG 
> com.hw.ambari.ui.util.cluster_managers.CommandExecutor.executeCommandSequence():
>  [OUTPUT STREAM]
> Using python  /usr/bin/python
> Restarting ambari-server
> Ambari Server is not running
> ERROR: Exiting with exit code 1. 
> REASON: Unable to detect a system user for Ambari Server.
> - If this is a new setup, then run the "ambari-server setup" command to 
> create the user
> - If this is an upgrade of an existing setup, run the "ambari-server 
> upgrade" command.
> Refer to the Ambari documentation for more information on setup and 
> upgrade.
> 
> **ambari-server setup** without -s setup ambari successful and ambari-server 
> start passed.
> 
> artifacts:  /test-logs/ambari-setup-hw/artifacts/screenshots/com.hw.ambari.ui.tests.consol
> e.TestInstallSeparateSSLCertificate/testA_InstallSeparateSSLCertificate/_1_4_4
> 0_52_Element_has_not_been_found_within_60_seconds__/>  
> cluster: 172.27.30.144
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 78f68ef 
>   ambari-server/src/main/python/ambari_server/serverSetup.py 0658d18 
> 
> Diff: https://reviews.apache.org/r/56227/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 56224: Improve Log Feeder simulation to help scale testing

2017-02-02 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56224/#review164000
---


Ship it!




Ship It!

- Oliver Szabo


On Feb. 2, 2017, 1 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56224/
> ---
> 
> (Updated Feb. 2, 2017, 1 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-19847
> https://issues.apache.org/jira/browse/AMBARI-19847
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Simulated Log Feeder input writes almost identical log messages, which is not 
> really simulates real logs.
> 
> Instead of the logfeeder.simulate.log_message_size property there will be 3 
> new properties (the simulated Log Feeders would enforce a minimum, and a 
> maximum value, and also set a default for them, if they are not specified):
> 
> logfeeder.simulate.number_of_words - the number of different words that may 
> appear in the simulated logs (50 / 100, default 1000)
> logfeeder.simulate.min_log_words - the minimum number of words in a simulated 
> log message (1 / 10, default 5)
> logfeeder.simulate.max_log_words - the maximum number of words in a log 
> message (10 / 20, default 10)
> 
> After the parameters are set each logfeeder will start to load messages like 
> this. In each message all the words are different:
> 
> Word000912 Word000903 Word000940 Word000760 Word000564 Word000762 Word000955 
> Word000553
> Word000320 Word000218 Word000872 Word000204 Word000120 Word000298 Word000727 
> Word000720 Word000475
> Word000413 Word000642 Word000872 Word000464 Word000746 Word000250
> 
> The rest of the properties remain the same:
> 
> logfeeder.simulate.input_number - number of paralell inputs (threads) loading 
> the logs, if it is set and not 0 then all the rest of the configured inputs 
> are ignored (running in simulation mode)!!
> logfeeder.simulate.log_ids - comma separated list of the log ids to propagate 
> at random, if not set by default all the available logs are propagated at 
> random, example: storm_drpc,storm_logviewer,storm_nimbus
> logfeeder.simulate.log_level - the level of the simulated log messages, by 
> default WARN
> logfeeder.simulate.sleep_milliseconds - the time interval at which each 
> simulated inputs writes one log message at random
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
>  743be69 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java
>  5bf600e 
> 
> Diff: https://reviews.apache.org/r/56224/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 56228: Recent workflows in Workflow designer should be in descending order of time of updation

2017-02-02 Thread Nitiraj Rathore 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56228/#review163999
---


Ship it!




Ship It!

- Nitiraj Rathore


On Feb. 2, 2017, 3:10 p.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56228/
> ---
> 
> (Updated Feb. 2, 2017, 3:10 p.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19849
> https://issues.apache.org/jira/browse/AMBARI-19849
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recent workflows in Workflow designer should be in descending order of 
> updated-at value both in recent list as well as Recent Projects screen.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  16fca55 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/drafts-wf.js 
> 02483d4 
> 
> Diff: https://reviews.apache.org/r/56228/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> venkat sairam
> 
>

Re: Review Request 56229: Workflow name mandatory for save in Coordinator and Bundle

2017-02-02 Thread Nitiraj Rathore 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56229/#review163998
---


Ship it!




Ship It!

- Nitiraj Rathore


On Feb. 2, 2017, 4:37 p.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56229/
> ---
> 
> (Updated Feb. 2, 2017, 4:37 p.m.)
> 
> 
> Review request for Ambari, belliraj hb, DIPAYAN BHOWMICK, Gaurav Nagar, 
> Nitiraj Rathore, and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19852
> https://issues.apache.org/jira/browse/AMBARI-19852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Workflow name mandatory for save also in Coordinator and Bundle. This way we 
> can show the workflow name in the recent workflows list.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
> 7b24d34 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
> 57dbeb0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> 4618ab6 
> 
> Diff: https://reviews.apache.org/r/56229/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> venkat sairam
> 
>

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Jonathan Hurley 


> On Feb. 1, 2017, 1:08 p.m., Sandor Magyari wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > ["AUTO_INSTALL_START", "AUTO_START", "FULL"]
> > I think we may also check for other options as well.
> 
> Jonathan Hurley wrote:
> Are the other options valid / used? Why are they not enums - does that 
> mean that people can somehow define their own on the fly?
> 
> Nate Cole wrote:
> You told me AUTO_START was the only supported option.  I won't make that 
> change until we make start type an enum.  String checking is bad.
> 
> Nate Cole wrote:
> Sorry, you didn't, another engineer did.  But my comment stands - until 
> that's an enum, this will be the only check made in this class.
> 
> Sandor Magyari wrote:
> Not sure if we are supporting these options yet officially, but I saw 
> this in agent code, so agent is able to handle these, but it's true that 
> there's no enum on server side.

This should stay a string until there's an enum - why isn't there an enum! :)


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163841
---


On Jan. 31, 2017, 9:32 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Jan. 31, 2017, 9:32 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Dmitro Lisnichenko 


> On Feb. 2, 2017, 4:19 p.m., Dmitro Lisnichenko wrote:
> > ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py, 
> > line 324
> > 
> >
> > I think this assertion makes no sence since we would not get here in 
> > case of any (unhandled) exception
> 
> Attila Doroszlai wrote:
> The point is that the exception is handled (see `except KeyError` in 
> original `CustomServiceOrchestrator`), but the traceback remains "active" 
> (ie. `format_exc()` returns it) later in the same thread.
> 
> However, you are right that this assertion does not test what I intended: 
> the tested code and the assertion do not run in the same thread, so the 
> exception info is not shared.  I'm still trying to figure out why.  Any ideas?

my guess would be that it only works within the except statement (probably that 
is a context management related thing)


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163971
---


On Feb. 2, 2017, 3:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 3:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 56229: Workflow name mandatory for save in Coordinator and Bundle

2017-02-02 Thread belliraj hb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56229/#review163994
---


Ship it!




Ship It!

- belliraj hb


On Feb. 2, 2017, 4:37 p.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56229/
> ---
> 
> (Updated Feb. 2, 2017, 4:37 p.m.)
> 
> 
> Review request for Ambari, belliraj hb, DIPAYAN BHOWMICK, Gaurav Nagar, 
> Nitiraj Rathore, and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19852
> https://issues.apache.org/jira/browse/AMBARI-19852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Workflow name mandatory for save also in Coordinator and Bundle. This way we 
> can show the workflow name in the recent workflows list.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
> 7b24d34 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
> 57dbeb0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> 4618ab6 
> 
> Diff: https://reviews.apache.org/r/56229/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> venkat sairam
> 
>

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/#review163992
---


Ship it!




Ship It!

- Sebastian Toader


On Feb. 2, 2017, 5:23 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56179/
> ---
> 
> (Updated Feb. 2, 2017, 5:23 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-19822
> https://issues.apache.org/jira/browse/AMBARI-19822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> If an ambari cluster is secured and kerberos authentication is used for Solr, 
> we need (default) authorizations as well to make sure only the specific 
> service users (ranger, atlas, logsearch) can access their collections (and 
> solr user as well)
> 
> Solution:
> Although RuleBasedAuthorizationPlugin seems to be a good solution here, to 
> map default users to default permissions, unfortunately, permissions and 
> roles using principal name for mapping (not username) from the authentication 
> tokens. Also Solr name rules applied on the username and not on the 
> principal, therefore we need the fully qualified hostname as well in the 
> role-permission mapping. In order to avoid that issue, I added an own plugin 
> (org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
> users with @ format.
> 
> to problem is in here in RuleBasedAuthorizationPlugin.java:
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153
> 
> notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
> (InfraUserRolesLookupStrategy class which I added and included in the new 
> plugin class)
> 
> In case of we need strict host validations i added 2 new json properties for 
> that:
> 1. { "user-host" : {"" : []} }
> 2. {"user-host-regex" : {"" : "hostname-regex"} }
> 
> {{user-host-regex}} has higher precedence then {{user-host}}
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraKerberosHostValidator.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraKerberosHostValidatorTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
>   ambari-logsearch/pom.xml 7aeb4a7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
>  526baea 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
>  d8aea24 
> 
> Diff: https://reviews.apache.org/r/56179/diff/
> 
> 
> Testing
> ---
> 
> unit tests done, behavior validated with unit tests. FT: validated with 
> logsearch and atlas as well.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Attila Doroszlai 


> On Feb. 2, 2017, 3:19 p.m., Dmitro Lisnichenko wrote:
> > ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py, 
> > line 324
> > 
> >
> > I think this assertion makes no sence since we would not get here in 
> > case of any (unhandled) exception

The point is that the exception is handled (see `except KeyError` in original 
`CustomServiceOrchestrator`), but the traceback remains "active" (ie. 
`format_exc()` returns it) later in the same thread.

However, you are right that this assertion does not test what I intended: the 
tested code and the assertion do not run in the same thread, so the exception 
info is not shared.  I'm still trying to figure out why.  Any ideas?


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163971
---


On Feb. 2, 2017, 2:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 2:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 56229: Workflow name mandatory for save in Coordinator and Bundle

2017-02-02 Thread venkat sairam 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56229/
---

Review request for Ambari, belliraj hb, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj 
Rathore, and Pallav Kulshreshtha.


Bugs: AMBARI-19852
https://issues.apache.org/jira/browse/AMBARI-19852


Repository: ambari


Description
---

Workflow name mandatory for save also in Coordinator and Bundle. This way we 
can show the workflow name in the recent workflows list.


Diffs
-

  contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
7b24d34 
  contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
57dbeb0 
  contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
4618ab6 

Diff: https://reviews.apache.org/r/56229/diff/


Testing
---

Manual testing done


Thanks,

venkat sairam

Re: Review Request 56020: Ambari HDFS Metric alerts turns to UNKNOWN status with error "argument of type 'NoneType' is not iterable"

2017-02-02 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56020/#review163987
---


Ship it!




Ship It!

- Sid Wagle


On Feb. 2, 2017, 4:03 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56020/
> ---
> 
> (Updated Feb. 2, 2017, 4:03 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-19746
> https://issues.apache.org/jira/browse/AMBARI-19746
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Build used :  ambari-2.5.0.0-724
> 
> Test is updating the alert definition minimumValue to "0.0".
> Payload used to update the minimumValue
> 
> /{
>   "AlertDefinition": {
>   "cluster_name": "cl1",
>   "id": 81,
>   "name": "increase_nn_heap_usage_daily",
>   "label": "NameNode Heap Usage (Daily)",
>   "component_name": "NAMENODE",
>   "description": "This service-level alert is triggered if the 
> NameNode heap usage deviation has grown beyond the specified threshold within 
> a day period.",
>   "enabled": true,
>   "ignore_host": false,
>   "interval": 1,
>   "scope": "ANY",
>   "service_name": "HDFS",
>   "source": {
>   "parameters": [{
>   "name": "mergeHaMetrics",
>   "display_name": "Whether active and stanby 
> NameNodes metrics should be merged",
>   "value": "false",
>   "description": "Whether active and stanby 
> NameNodes metrics should be merged.",
>   "type": "STRING",
>   "visibility": "HIDDEN"
>   }, {
>   "name": "interval",
>   "display_name": "Time interval in minutes",
>   "value": "1440.0",
>   "description": "Time interval in minutes.",
>   "type": "NUMERIC",
>   "visibility": "HIDDEN"
>   }, {
>   "name": "appId",
>   "display_name": "AMS application id",
>   "value": "NAMENODE",
>   "description": "The application id used to 
> retrieve the metric.",
>   "type": "STRING",
>   "visibility": "HIDDEN"
>   }, {
>   "name": "metricName",
>   "display_name": "Metric Name",
>   "value": "jvm.JvmMetrics.MemHeapUsedM",
>   "description": "The metric to monitor.",
>   "type": "STRING",
>   "visibility": "HIDDEN"
>   }, {
>   "name": "metric.deviation.warning.threshold",
>   "display_name": "Growth Rate",
>   "value": "20.0",
>   "description": "The percentage of NameNode heap 
> usage growth.",
>   "type": "PERCENT"
>   }, {
>   "name": "metric.deviation.critical.threshold",
>   "display_name": "Growth Rate",
>   "value": "50.0",
>   "description": "The percentage of NameNode heap 
> usage growth.",
>   "type": "PERCENT"
>   }, {
>   "name": "metric.units",
>   "display_name": "Metric Units",
>   "value": "MB",
>   "description": "The units that the metric data 
> points are reported in.",
>   "type": "STRING",
>   "visibility": "HIDDEN"
>   }, {
>   "name": "minimumValue",
>   "display_name": "Minimum Heap",
>   "value": "0.0",
>   "description": "The minimum heap increase in a 
> day.",
>   "type": "NUMERIC"
>   }],
>   "path": 
> "HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py",
>   "type": "SCRIPT"
>   }
>   }
> }
> 
> 
> Alert response :
> 
> 
> 03:14:20 AM

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/
---

(Updated Feb. 2, 2017, 4:23 p.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
Toader.


Changes
---

- its enough to user KerberosName instead of HadoopKerberosName


Bugs: AMBARI-19822
https://issues.apache.org/jira/browse/AMBARI-19822


Repository: ambari


Description
---

Problem:
If an ambari cluster is secured and kerberos authentication is used for Solr, 
we need (default) authorizations as well to make sure only the specific service 
users (ranger, atlas, logsearch) can access their collections (and solr user as 
well)

Solution:
Although RuleBasedAuthorizationPlugin seems to be a good solution here, to map 
default users to default permissions, unfortunately, permissions and roles 
using principal name for mapping (not username) from the authentication tokens. 
Also Solr name rules applied on the username and not on the principal, 
therefore we need the fully qualified hostname as well in the role-permission 
mapping. In order to avoid that issue, I added an own plugin 
(org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
users with @ format.

to problem is in here in RuleBasedAuthorizationPlugin.java:
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153

notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
(InfraUserRolesLookupStrategy class which I added and included in the new 
plugin class)

In case of we need strict host validations i added 2 new json properties for 
that:
1. { "user-host" : {"" : []} }
2. {"user-host-regex" : {"" : "hostname-regex"} }

{{user-host-regex}} has higher precedence then {{user-host}}


Diffs (updated)
-

  ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraKerberosHostValidator.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraKerberosHostValidatorTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
  ambari-logsearch/pom.xml 7aeb4a7 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
 526baea 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
 d8aea24 

Diff: https://reviews.apache.org/r/56179/diff/


Testing
---

unit tests done, behavior validated with unit tests. FT: validated with 
logsearch and atlas as well.


Thanks,

Oliver Szabo

Re: Review Request 53686: Stage and Request status should be persisted in the database

2017-02-02 Thread Jonathan Hurley 


> On Dec. 15, 2016, 11:49 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java,
> >  lines 282-287
> > 
> >
> > Would this count states like FAILED / TIMEDOUT twice?
> 
> Jaimin Jetly wrote:
> yes. It will increment the FAILED counter and also increment COMPLETED 
> counter which is desired.

Why is this desired? Double counting them doesn't seem right.


> On Dec. 15, 2016, 11:49 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java,
> >  lines 445-447
> > 
> >
> > This could be problematic if the merge fails when the transaction 
> > commits. You've already fired the event.
> 
> Jaimin Jetly wrote:
> persistAction method has been annotated with @Transactional and 
> @TransactionalLock.
> I believe if the publisher method is marked to be transactional then the 
> event listener will also fall in that transactional boundary considering that 
> the eventbus being used over here is default which I believe is synchronous. 
> 
> I will verify this myself by checking that publsiher method, eventbus and 
> subscriber method are running on same thread.
> 
> Jaimin Jetly wrote:
> I verified that the publisher and subscriber code runs on same thread and 
> so transactional boundary will extend for the entire logic making sure that 
> tasks and respective stage/request are consistent in their status

Although it may be true that the event bus is synchronous here, the point is 
that the merge() which comes before it doesn't necessarily count until the 
entire transaction is committed. By firing the event in this code, you're 
assuming that the merge() will succeed once the batches operations are flushed 
and the transaction is commited. If the listener modifies any sort of in-memory 
state, then this causes a data integrity problem.


On Dec. 15, 2016, 11:49 a.m., Jaimin Jetly wrote:
> > One thing I don't quite see here (and it could be due to the size of the 
> > patch) is what happens in these two cases:
> > - Something goes wrong when trying to store a task's status. How does the 
> > system recover and mark it completed?
> > - What about waiting until a request is HOLDING and then restarting Ambari 
> > - will the relevent maps get re-populated?
> 
> Jaimin Jetly wrote:
> >> - Something goes wrong when trying to store a task's status. How does 
> the system recover and mark it completed?
> 
> This work only adds logic to add/update stage and request status. The way 
> task status is being updated or the logic for system to recover from anything 
> that goes wrong when storing task status is not changed.
> This work ensures that task status update, respective stage status update 
> and respective request status update happens inside same transactional 
> boundary. Thus all three entities remains consistent in the status they show. 
> This work does not add any recovery logic and piggybacks on existing 
> failure recovery mechanism for updating task status. Thus if something goes 
> wrong storing task status then stage/request status will also be not store 
> and vice-versa. Next time when ambari-agent sends command reports again then 
> task update and respective stage/request status update should also also get 
> updated successfully. 
> 
> >> -  What about waiting until a request is HOLDING and then restarting 
> Ambari - will the relevent maps get re-populated?
> 
> Yes, everytime ambari-server starts, we check for all stages in 
> HostRoleStatus.IN_PROGRESS_STATUSES and publish an event with the tasks. 
> these will repopulate the maps. The patch adds that logic with 
> "publishInProgressTasks(stages)" method in ActionScheduler.java
> 
> I have tested that scenario of restarting ambari-server when a request is 
> ongoing with in progress tasks and validating that stages and requests status 
> is correctly updated
> 
> Sid Wagle wrote:
> The agent re-sending command reports are we generally fault tolerant in 
> that area? Sicne you are probably mostly up-to-date on that part of the code 
> can you shed some light on loss of task status scenario. Do we recover the 
> correct status?
> 
> Jaimin Jetly wrote:
> Yes. Largely. 
> Although There are following specific cases:
> 1) If persisting of "request, it's stages and it's host role commands" 
> for the first time does not happen then the request will be completely missed 
> and ambari-server will never log in the database or track it in anyways.
> 2) If "request, it's stages and it's host role commands" are persisted 
> but subsequent updates and DB merge keeps failing then request will be 
> eventually declared timeout and so will all hrcs
> 
> As far as DB merge

Re: Review Request 56059: Preview: Package Installation fails due to error in Berkeley DB library

2017-02-02 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
---

(Updated Feb. 2, 2017, 6:16 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
Hurley, and Nate Cole.


Changes
---

Fixed existing tests, still adding new tests


Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768


Repository: ambari


Description
---

*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar 
was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari 
server node)
# Register HDP-2.6.0.0-216
# Start package installation

*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - 
run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most 
recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
"/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 
166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = 
self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
"/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 
400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not 
verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py",
 line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = 
rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, 
in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = 
function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, 
in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, 
try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, 
in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = 
_call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, 
in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise 
ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: 
Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: 
rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died 
in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 
error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run 
database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
open Packages index using db5 -  (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - 
run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most 
recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
"/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 
469, in 
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - 
run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File 
"/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 
179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise 
Fail("Failed to distribute repositories/install packages")
{code}


Diffs (updated)
-

  ambar

Re: Review Request 56059: Preview: Package Installation fails due to error in Berkeley DB library

2017-02-02 Thread Dmitro Lisnichenko 


> On Feb. 1, 2017, 11:40 p.m., Nate Cole wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, line 42
> > 
> >
> > Is it possible that these paths are different between redhat6 and 
> > redhat7 (or any of their flavors?)

I think no, /bin and /usr/bin folders are usually dedicated to categories of 
binaries


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163886
---


On Feb. 1, 2017, 9:13 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> ---
> 
> (Updated Feb. 1, 2017, 9:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the 
> jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on 
> Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
> 
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = 
> self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File 
> "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", 
> line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not 
> verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py",
>  line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = 
> rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = 
> function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, 
> try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = 
> _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise 
> ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - 
> run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check 
> dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 
> 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 
> error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run 
> database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages index using db5 -  (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot 
> open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - 
> run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb 
> open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback 
> (most recent call last):
> 2016-12-16

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Sandor Magyari 


> On Feb. 1, 2017, 6:08 p.m., Sandor Magyari wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > ["AUTO_INSTALL_START", "AUTO_START", "FULL"]
> > I think we may also check for other options as well.
> 
> Jonathan Hurley wrote:
> Are the other options valid / used? Why are they not enums - does that 
> mean that people can somehow define their own on the fly?
> 
> Nate Cole wrote:
> You told me AUTO_START was the only supported option.  I won't make that 
> change until we make start type an enum.  String checking is bad.
> 
> Nate Cole wrote:
> Sorry, you didn't, another engineer did.  But my comment stands - until 
> that's an enum, this will be the only check made in this class.

Not sure if we are supporting these options yet officially, but I saw this in 
agent code, so agent is able to handle these, but it's true that there's no 
enum on server side.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163841
---


On Feb. 1, 2017, 2:32 a.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Feb. 1, 2017, 2:32 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Nate Cole 


> On Feb. 1, 2017, 1:43 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > To make this robust against future changes, can we check with not 
> > equals instead?

Once the status becomes an enum, then yes.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163850
---


On Jan. 31, 2017, 9:32 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Jan. 31, 2017, 9:32 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Nate Cole 


> On Feb. 1, 2017, 1:08 p.m., Sandor Magyari wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > ["AUTO_INSTALL_START", "AUTO_START", "FULL"]
> > I think we may also check for other options as well.
> 
> Jonathan Hurley wrote:
> Are the other options valid / used? Why are they not enums - does that 
> mean that people can somehow define their own on the fly?
> 
> Nate Cole wrote:
> You told me AUTO_START was the only supported option.  I won't make that 
> change until we make start type an enum.  String checking is bad.

Sorry, you didn't, another engineer did.  But my comment stands - until that's 
an enum, this will be the only check made in this class.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163841
---


On Jan. 31, 2017, 9:32 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Jan. 31, 2017, 9:32 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Nate Cole 


> On Feb. 1, 2017, 1:08 p.m., Sandor Magyari wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > ["AUTO_INSTALL_START", "AUTO_START", "FULL"]
> > I think we may also check for other options as well.
> 
> Jonathan Hurley wrote:
> Are the other options valid / used? Why are they not enums - does that 
> mean that people can somehow define their own on the fly?

You told me AUTO_START was the only supported option.  I won't make that change 
until we make start type an enum.  String checking is bad.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163841
---


On Jan. 31, 2017, 9:32 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Jan. 31, 2017, 9:32 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56141: Add precheck for Auto-Start being disabled

2017-02-02 Thread Jonathan Hurley 


> On Feb. 1, 2017, 1:08 p.m., Sandor Magyari wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java,
> >  line 67
> > 
> >
> > ["AUTO_INSTALL_START", "AUTO_START", "FULL"]
> > I think we may also check for other options as well.

Are the other options valid / used? Why are they not enums - does that mean 
that people can somehow define their own on the fly?


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56141/#review163841
---


On Jan. 31, 2017, 9:32 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56141/
> ---
> 
> (Updated Jan. 31, 2017, 9:32 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sandor 
> Magyari.
> 
> 
> Bugs: AMBARI-19800
> https://issues.apache.org/jira/browse/AMBARI-19800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A new precheck is required to make sure that auto-start is disabled before 
> starting an upgrade. This JIRA covers the backend requirements. The UI 
> changes will be linked to this one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/AutoStartDisabledCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  a204ada 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/AutoStartDisabledCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/56141/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 40:57.215s
> [INFO] Finished at: Tue Jan 31 18:24:43 EST 2017
> [INFO] Final Memory: 45M/694M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 56020: Ambari HDFS Metric alerts turns to UNKNOWN status with error "argument of type 'NoneType' is not iterable"

2017-02-02 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56020/
---

(Updated Фев. 2, 2017, 4:03 п.п.)


Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-19746
https://issues.apache.org/jira/browse/AMBARI-19746


Repository: ambari


Description
---

Build used :ambari-2.5.0.0-724

Test is updating the alert definition minimumValue to "0.0".
Payload used to update the minimumValue

/{
"AlertDefinition": {
"cluster_name": "cl1",
"id": 81,
"name": "increase_nn_heap_usage_daily",
"label": "NameNode Heap Usage (Daily)",
"component_name": "NAMENODE",
"description": "This service-level alert is triggered if the 
NameNode heap usage deviation has grown beyond the specified threshold within a 
day period.",
"enabled": true,
"ignore_host": false,
"interval": 1,
"scope": "ANY",
"service_name": "HDFS",
"source": {
"parameters": [{
"name": "mergeHaMetrics",
"display_name": "Whether active and stanby 
NameNodes metrics should be merged",
"value": "false",
"description": "Whether active and stanby 
NameNodes metrics should be merged.",
"type": "STRING",
"visibility": "HIDDEN"
}, {
"name": "interval",
"display_name": "Time interval in minutes",
"value": "1440.0",
"description": "Time interval in minutes.",
"type": "NUMERIC",
"visibility": "HIDDEN"
}, {
"name": "appId",
"display_name": "AMS application id",
"value": "NAMENODE",
"description": "The application id used to 
retrieve the metric.",
"type": "STRING",
"visibility": "HIDDEN"
}, {
"name": "metricName",
"display_name": "Metric Name",
"value": "jvm.JvmMetrics.MemHeapUsedM",
"description": "The metric to monitor.",
"type": "STRING",
"visibility": "HIDDEN"
}, {
"name": "metric.deviation.warning.threshold",
"display_name": "Growth Rate",
"value": "20.0",
"description": "The percentage of NameNode heap 
usage growth.",
"type": "PERCENT"
}, {
"name": "metric.deviation.critical.threshold",
"display_name": "Growth Rate",
"value": "50.0",
"description": "The percentage of NameNode heap 
usage growth.",
"type": "PERCENT"
}, {
"name": "metric.units",
"display_name": "Metric Units",
"value": "MB",
"description": "The units that the metric data 
points are reported in.",
"type": "STRING",
"visibility": "HIDDEN"
}, {
"name": "minimumValue",
"display_name": "Minimum Heap",
"value": "0.0",
"description": "The minimum heap increase in a 
day.",
"type": "NUMERIC"
}],
"path": 
"HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py",
"type": "SCRIPT"
}
}
}


Alert response :


03:14:20 AM 01-25-2017 AmbariItems FINE - API call response:{
  "href" : "http://172.27.9.139:8080/api/v1/clusters/cl1/alerts/119";,
  "Alert" : {
"cluster_name" : "cl1",
"component_name" : "NAMENODE",
"definition_id" : 81,
"definition_name" : "increase_nn_heap_usage_daily",
"firmness" : "HARD",
"host_name" : "ctr-e126-1485243696039-2656-01-06.hwx.site",
"id" : 119,
"instance" :

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/
---

(Updated Feb. 2, 2017, 4:02 p.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
Toader.


Changes
---

- add a custom hostname validator (in case of hostnames are strict)
- also keep old RuleBasedAuthorizationPlugin behaviour (to get user roles from 
the userroles map with fully qualified names)


Bugs: AMBARI-19822
https://issues.apache.org/jira/browse/AMBARI-19822


Repository: ambari


Description (updated)
---

Problem:
If an ambari cluster is secured and kerberos authentication is used for Solr, 
we need (default) authorizations as well to make sure only the specific service 
users (ranger, atlas, logsearch) can access their collections (and solr user as 
well)

Solution:
Although RuleBasedAuthorizationPlugin seems to be a good solution here, to map 
default users to default permissions, unfortunately, permissions and roles 
using principal name for mapping (not username) from the authentication tokens. 
Also Solr name rules applied on the username and not on the principal, 
therefore we need the fully qualified hostname as well in the role-permission 
mapping. In order to avoid that issue, I added an own plugin 
(org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
users with @ format.

to problem is in here in RuleBasedAuthorizationPlugin.java:
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153

notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
(InfraUserRolesLookupStrategy class which I added and included in the new 
plugin class)

In case of we need strict host validations i added 2 new json properties for 
that:
1. { "user-host" : {"" : []} }
2. {"user-host-regex" : {"" : "hostname-regex"} }

{{user-host-regex}} has higher precedence then {{user-host}}


Diffs (updated)
-

  ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraKerberosHostValidator.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraKerberosHostValidatorTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
  ambari-logsearch/pom.xml 7aeb4a7 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
 526baea 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
 d8aea24 

Diff: https://reviews.apache.org/r/56179/diff/


Testing (updated)
---

unit tests done, behavior validated with unit tests. FT: validated with 
logsearch and atlas as well.


Thanks,

Oliver Szabo

Re: Review Request 56228: Recent workflows in Workflow designer should be in descending order of time of updation

2017-02-02 Thread belliraj hb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56228/#review163977
---


Ship it!




Ship It!

- belliraj hb


On Feb. 2, 2017, 3:10 p.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56228/
> ---
> 
> (Updated Feb. 2, 2017, 3:10 p.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19849
> https://issues.apache.org/jira/browse/AMBARI-19849
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recent workflows in Workflow designer should be in descending order of 
> updated-at value both in recent list as well as Recent Projects screen.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  16fca55 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/drafts-wf.js 
> 02483d4 
> 
> Diff: https://reviews.apache.org/r/56228/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> venkat sairam
> 
>

Re: Review Request 56189: AMBARI-19827. HiveServer2 Interactive won't start in clusters with less memory.

2017-02-02 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56189/#review163975
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
930)


Lets add a test that goes through this condition. This one has low chance 
of coverage from system tests as hosts used are fairly powerful. So we should 
have a test with hosts in the order of 3-4 GB ram and 1-2 cores and verify the 
output from stack advisor.



ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py (line 
3584)


Why did these values change - were they not being set before?


- Sumit Mohanty


On Feb. 2, 2017, 12:41 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56189/
> ---
> 
> (Updated Feb. 2, 2017, 12:41 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19827
> https://issues.apache.org/jira/browse/AMBARI-19827
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HSI caluclations fails on cluster with small memory. Fixed the calulations 
> for that.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> fe9737a 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> fe58217 
> 
> Diff: https://reviews.apache.org/r/56189/diff/
> 
> 
> Testing
> ---
> 
> Python UT tests fixed and Passes.
> Tested on cluster.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Review Request 56228: Recent workflows in Workflow designer should be in descending order of time of updation

2017-02-02 Thread venkat sairam 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56228/
---

Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
Pallav Kulshreshtha.


Bugs: AMBARI-19849
https://issues.apache.org/jira/browse/AMBARI-19849


Repository: ambari


Description
---

Recent workflows in Workflow designer should be in descending order of 
updated-at value both in recent list as well as Recent Projects screen.


Diffs
-

  
contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
 16fca55 
  contrib/views/wfmanager/src/main/resources/ui/app/components/drafts-wf.js 
02483d4 

Diff: https://reviews.apache.org/r/56228/diff/


Testing
---

Manual testing done


Thanks,

venkat sairam

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163971
---




ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
(line 324)


I think this assertion makes no sence since we would not get here in case 
of any (unhandled) exception


- Dmitro Lisnichenko


On Feb. 2, 2017, 3:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 3:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 56227: ambari-server start failed with exit code 1.

2017-02-02 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56227/
---

Review request for Ambari and Jayush Luniya.


Bugs: AMBARI-19851
https://issues.apache.org/jira/browse/AMBARI-19851


Repository: ambari


Description
---

Found this issue in ST.  
Trying to provide setup ambari-server and



[root@ctr-e126-1485243696039-17812-01-04 ~]# ambari-server setup -s
Using python  /usr/bin/python
Setup ambari-server
Nothing was done. Ambari Setup already performed and cannot re-run setup in 
silent mode. Use "ambari-server setup" command without -s option to change 
Ambari setup.


Trying to start ambari-server



[root@ctr-e126-1485243696039-17812-01-04 ~]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
ERROR: Exiting with exit code 1. 
REASON: Unable to detect a system user for Ambari Server.
- If this is a new setup, then run the "ambari-server setup" command to 
create the user
- If this is an upgrade of an existing setup, run the "ambari-server 
upgrade" command.
Refer to the Ambari documentation for more information on setup and upgrade.


Ambari-server restart in debug log



2017-02-01 04:36:34,630 DEBUG 
com.hw.ambari.ui.util.cluster_managers.CommandExecutor.executeCommandSequence():
 Sending command [ambari-server restart --debug]
2017-02-01 04:36:35,331 DEBUG 
com.hw.ambari.ui.util.cluster_managers.CommandExecutor.executeCommandSequence():
 [OUTPUT STREAM]
Using python  /usr/bin/python
Restarting ambari-server
Ambari Server is not running
ERROR: Exiting with exit code 1. 
REASON: Unable to detect a system user for Ambari Server.
- If this is a new setup, then run the "ambari-server setup" command to 
create the user
- If this is an upgrade of an existing setup, run the "ambari-server 
upgrade" command.
Refer to the Ambari documentation for more information on setup and upgrade.

**ambari-server setup** without -s setup ambari successful and ambari-server 
start passed.

artifacts:   
cluster: 172.27.30.144


Diffs
-

  ambari-server/src/main/python/ambari_server/serverConfiguration.py 78f68ef 
  ambari-server/src/main/python/ambari_server/serverSetup.py 0658d18 

Diff: https://reviews.apache.org/r/56227/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Sandor Magyari 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163970
---


Ship it!




Ship It!

- Sandor Magyari


On Feb. 2, 2017, 1:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 1:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163969
---


Ship it!




Ship It!

- Sebastian Toader


On Feb. 2, 2017, 2:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 2:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Attila Doroszlai 


> On Feb. 2, 2017, 2:21 p.m., Andrew Onischuk wrote:
> > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py, 
> > line 320
> > 
> >
> > Hi Attila.
> > 
> > Checking if the key in dictionary before operation looks less hacky to 
> > me. 
> > 
> > What do you think?

Agree, but you know what they say: "never touch working code"


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163966
---


On Feb. 2, 2017, 2:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 2:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163968
---


Ship it!




Ship It!

- Andrew Onischuk


On Feb. 2, 2017, 1:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 1:36 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Attila Doroszlai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/
---

(Updated Feb. 2, 2017, 2:36 p.m.)


Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor Magyari, 
and Sebastian Toader.


Changes
---

Check dict for key instead of catching exception


Bugs: AMBARI-19846
https://issues.apache.org/jira/browse/AMBARI-19846


Repository: ambari


Description
---

Avoid recording the traceback for this `KeyError`, which is expected for status 
commands.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
9baaf08d6cd1fe38db19a34907d44514bded228d 
  ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 

Diff: https://reviews.apache.org/r/56225/diff/


Testing
---

Manually tested on local cluster.

Unit tests:

```
$ mvn -pl ambari-agent clean test
...
Ran 455 tests in 15.152s

OK
...
[INFO] BUILD SUCCESS
```


Thanks,

Attila Doroszlai

Re: Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/#review163966
---




ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py (line 
320)


Hi Attila.

Checking if the key in dictionary before operation looks less hacky to me. 

What do you think?


- Andrew Onischuk


On Feb. 2, 2017, 1:12 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56225/
> ---
> 
> (Updated Feb. 2, 2017, 1:12 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19846
> https://issues.apache.org/jira/browse/AMBARI-19846
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid recording the traceback for this `KeyError`, which is expected for 
> status commands.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> 9baaf08d6cd1fe38db19a34907d44514bded228d 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
> 3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 
> 
> Diff: https://reviews.apache.org/r/56225/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local cluster.
> 
> Unit tests:
> 
> ```
> $ mvn -pl ambari-agent clean test
> ...
> Ran 455 tests in 15.152s
> 
> OK
> ...
> [INFO] BUILD SUCCESS
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 56225: AMBARI-19846. ambari-agent.out filled with KeyError

2017-02-02 Thread Attila Doroszlai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56225/
---

Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Sandor Magyari, 
and Sebastian Toader.


Bugs: AMBARI-19846
https://issues.apache.org/jira/browse/AMBARI-19846


Repository: ambari


Description
---

Avoid recording the traceback for this `KeyError`, which is expected for status 
commands.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
9baaf08d6cd1fe38db19a34907d44514bded228d 
  ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 
3985c5a21ebcaa5c526b772e8ad13d1bf0427d99 

Diff: https://reviews.apache.org/r/56225/diff/


Testing
---

Manually tested on local cluster.

Unit tests:

```
$ mvn -pl ambari-agent clean test
...
Ran 455 tests in 15.152s

OK
...
[INFO] BUILD SUCCESS
```


Thanks,

Attila Doroszlai

Review Request 56224: Improve Log Feeder simulation to help scale testing

2017-02-02 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56224/
---

Review request for Ambari, Oliver Szabo and Robert Nettleton.


Bugs: AMBARI-19847
https://issues.apache.org/jira/browse/AMBARI-19847


Repository: ambari


Description
---

Simulated Log Feeder input writes almost identical log messages, which is not 
really simulates real logs.

Instead of the logfeeder.simulate.log_message_size property there will be 3 new 
properties (the simulated Log Feeders would enforce a minimum, and a maximum 
value, and also set a default for them, if they are not specified):

logfeeder.simulate.number_of_words - the number of different words that may 
appear in the simulated logs (50 / 100, default 1000)
logfeeder.simulate.min_log_words - the minimum number of words in a simulated 
log message (1 / 10, default 5)
logfeeder.simulate.max_log_words - the maximum number of words in a log message 
(10 / 20, default 10)

After the parameters are set each logfeeder will start to load messages like 
this. In each message all the words are different:

Word000912 Word000903 Word000940 Word000760 Word000564 Word000762 Word000955 
Word000553
Word000320 Word000218 Word000872 Word000204 Word000120 Word000298 Word000727 
Word000720 Word000475
Word000413 Word000642 Word000872 Word000464 Word000746 Word000250

The rest of the properties remain the same:

logfeeder.simulate.input_number - number of paralell inputs (threads) loading 
the logs, if it is set and not 0 then all the rest of the configured inputs are 
ignored (running in simulation mode)!!
logfeeder.simulate.log_ids - comma separated list of the log ids to propagate 
at random, if not set by default all the available logs are propagated at 
random, example: storm_drpc,storm_logviewer,storm_nimbus
logfeeder.simulate.log_level - the level of the simulated log messages, by 
default WARN
logfeeder.simulate.sleep_milliseconds - the time interval at which each 
simulated inputs writes one log message at random


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
 743be69 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java
 5bf600e 

Diff: https://reviews.apache.org/r/56224/diff/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/#review163965
---



In my past experience if a service runs on multiple hosts in kerberised 
environment each service has its own principal. e.g service_name/host1@realm, 
service_name/host2@realm etc. As these are separate principals authorisation 
should be granted for each principal separately even though it's the same 
service. E.g service running on host1 is considered to be secure as host1 is in 
a trusted domain thus for the service which identifies itself as 
service/host1@realm more permissions can be granted. If the service is running 
on host2 which is in an untrusted domain than for the service which identifies 
itself as service/host2@realm only limited permission is given.

Authorisation enforced based only on the service name and not the fully 
qualified principal could lead to security breach.

- Sebastian Toader


On Feb. 1, 2017, 9:46 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56179/
> ---
> 
> (Updated Feb. 1, 2017, 9:46 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-19822
> https://issues.apache.org/jira/browse/AMBARI-19822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> If an ambari cluster is secured and kerberos authentication is used for Solr, 
> we need (default) authorizations as well to make sure only the specific 
> service users (ranger, atlas, logsearch) can access their collections (and 
> solr user as well)
> 
> Solution:
> Although RuleBasedAuthorizationPlugin seems to be a good solution here, to 
> map default users to default permissions, unfortunately, permissions and 
> roles using principal name for mapping (not username) from the authentication 
> tokens. Also Solr name rules applied on the username and not on the 
> principal, therefore we need the fully qualified hostname as well in the 
> role-permission mapping. In order to avoid that issue, I added an own plugin 
> (org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
> users with @ format.
> 
> to problem is in here in RuleBasedAuthorizationPlugin.java:
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153
> 
> notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
> (InfraUserRolesLookupStrategy class which I added and included in the new 
> plugin class)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
>   ambari-logsearch/pom.xml 7aeb4a7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
>  526baea 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
>  d8aea24 
> 
> Diff: https://reviews.apache.org/r/56179/diff/
> 
> 
> Testing
> ---
> 
> unit tests done, behavior validated with unit tests. other tests (FT) with 
> using ranger and atlas are in progress...
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 56179: Add infra-solr-plugin for authorization (with Kerberos)

2017-02-02 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56179/#review163963
---


Ship it!




Ship It!

- Miklos Gergely


On Feb. 1, 2017, 8:46 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56179/
> ---
> 
> (Updated Feb. 1, 2017, 8:46 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-19822
> https://issues.apache.org/jira/browse/AMBARI-19822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> If an ambari cluster is secured and kerberos authentication is used for Solr, 
> we need (default) authorizations as well to make sure only the specific 
> service users (ranger, atlas, logsearch) can access their collections (and 
> solr user as well)
> 
> Solution:
> Although RuleBasedAuthorizationPlugin seems to be a good solution here, to 
> map default users to default permissions, unfortunately, permissions and 
> roles using principal name for mapping (not username) from the authentication 
> tokens. Also Solr name rules applied on the username and not on the 
> principal, therefore we need the fully qualified hostname as well in the 
> role-permission mapping. In order to avoid that issue, I added an own plugin 
> (org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin), to map 
> users with @ format.
> 
> to problem is in here in RuleBasedAuthorizationPlugin.java:
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.2/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L153
> 
> notice that InfraRuleBasedAuthorizationPlugin is only a copy of that file 
> (InfraUserRolesLookupStrategy class which I added and included in the new 
> plugin class)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-infra-solr-plugin/pom.xml PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraUserRolesLookupStrategyTest.java
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml c486050 
>   ambari-logsearch/pom.xml 7aeb4a7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
>  526baea 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2
>  d8aea24 
> 
> Diff: https://reviews.apache.org/r/56179/diff/
> 
> 
> Testing
> ---
> 
> unit tests done, behavior validated with unit tests. other tests (FT) with 
> using ranger and atlas are in progress...
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 56205: AMBARI-19842:Custom Action should be created without prompting for the action type

2017-02-02 Thread Gaurav Nagar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56205/#review163961
---


Ship it!




Ship It!

- Gaurav Nagar


On Feb. 2, 2017, 7:01 a.m., Padma Priya N wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56205/
> ---
> 
> (Updated Feb. 2, 2017, 7:01 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19842
> https://issues.apache.org/jira/browse/AMBARI-19842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed the prompting to provide the action type while creating a custom 
> action. Added ability to change the custom action type in the action editor.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> ad5f3f8 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-action-editor.js
>  43eeb5b 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-importer.js 
> ff75e7e 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/templates/components/flow-designer.hbs
>  df0a9ba 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/templates/components/workflow-action-editor.hbs
>  0c569ec 
>   contrib/views/wfmanager/src/main/resources/ui/app/utils/constants.js 
> 1dd1c31 
> 
> Diff: https://reviews.apache.org/r/56205/diff/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Padma Priya N
> 
>

Re: Review Request 56207: AMBARI-19843. Publish asset has issues when different users logins

2017-02-02 Thread Gaurav Nagar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56207/#review163962
---


Ship it!




Ship It!

- Gaurav Nagar


On Feb. 2, 2017, 9:30 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56207/
> ---
> 
> (Updated Feb. 2, 2017, 9:30 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19843
> https://issues.apache.org/jira/browse/AMBARI-19843
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Appended the temp workflow file name with random number for dry run. And also 
> Deleting the temp workflow file after dry run.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetResource.java
>  af86810 
> 
> Diff: https://reviews.apache.org/r/56207/diff/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Re: Review Request 56022: Content of yarn-env.sh on host is not same as in the downloaded config file from Ambari UI

2017-02-02 Thread Attila Magyar 


> On Jan. 31, 2017, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-env.xml,
> >  line 21
> > 
> >
> > Does this file differ from the one used for HDP 3.0? If so, they need 
> > to be identical for now, until we figure out what will change in HDP 3.0.

The yarn-env sh content is the same. The content of the whole file is not 
because the yarn-env in HDP2.6 is inherited from other templates. I checked the 
inherited properties and they all have a counterpart in HDP3.0 yarn-env.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56022/#review163694
---


On Jan. 31, 2017, 1:43 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56022/
> ---
> 
> (Updated Jan. 31, 2017, 1:43 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Robert Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19683
> https://issues.apache.org/jira/browse/AMBARI-19683
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Exported YARN configuration was different than the stored configuration. I 
> replaced the Python logic that assembled yarn security opt with jinja 
> template logic.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  aed8abc 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/configuration/yarn-env.xml
>  d8531b1 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py
>  4d47925 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-env.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/metainfo.xml 
> ad457b2 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 
> 62a4d46 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 
> 774f3c6 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 0eb5561 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 
> 5ebfb45 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py d4341e1 
> 
> Diff: https://reviews.apache.org/r/56022/diff/
> 
> 
> Testing
> ---
> 
> Tested manually:
>  - created a cluster with YARN
>  - enabled kerberos
>  - exported configuration
>  - checked that yarn security opts are presented in the exported config
>  - disablde kerberos
>  - checked that yarn security opts are not presented in the exported config
> 
> existing unittests ran successfully
> one irrelevant failure:
> 
> java.lang.AssertionError
> org.apache.ambari.server.serveraction.kerberos.UpdateKerberosConfigsServerActionTest.testUpdateConfigForceSecurityEnabled(UpdateKerberosC
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 56207: AMBARI-19843. Publish asset has issues when different users logins

2017-02-02 Thread belliraj hb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56207/#review163948
---


Ship it!




Ship It!

- belliraj hb


On Feb. 2, 2017, 9:30 a.m., Madhan Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56207/
> ---
> 
> (Updated Feb. 2, 2017, 9:30 a.m.)
> 
> 
> Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
> Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19843
> https://issues.apache.org/jira/browse/AMBARI-19843
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Appended the temp workflow file name with random number for dry run. And also 
> Deleting the temp workflow file after dry run.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetResource.java
>  af86810 
> 
> Diff: https://reviews.apache.org/r/56207/diff/
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Madhan Reddy
> 
>

Review Request 56207: AMBARI-19843. Publish asset has issues when different users logins

2017-02-02 Thread Madhan Reddy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56207/
---

Review request for Ambari, belliraj hb, Gaurav Nagar, Nitiraj Rathore, and 
Pallav Kulshreshtha.


Bugs: AMBARI-19843
https://issues.apache.org/jira/browse/AMBARI-19843


Repository: ambari


Description
---

Appended the temp workflow file name with random number for dry run. And also 
Deleting the temp workflow file after dry run.


Diffs
-

  
contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetResource.java
 af86810 

Diff: https://reviews.apache.org/r/56207/diff/


Testing
---

Manual


Thanks,

Madhan Reddy