date:20160517


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47525/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, and Pallav 
Kulshreshtha.


Bugs: AMBARI-16726
https://issues.apache.org/jira/browse/AMBARI-16726


Repository: ambari


Description
---

Earlier : The UI does not get cleared properly after the upload. It still shows 
file name.

In this patch : the UI will no longer show the old file name if the upload is 
successfull.


Diffs
-

  
contrib/views/hive/src/main/resources/ui/hive-web/app/components/file-upload.js 
1cd05ae 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/upload-table.js
 361de7b 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/upload-table.hbs
 eb95292 

Diff: https://reviews.apache.org/r/47525/diff/


Testing
---

manual testing done.


Thanks,

Nitiraj Rathore

Review Request 47524: Hive view : For Upload Table, 'default' DB should be selected by default


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47524/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, and Pallav 
Kulshreshtha.


Bugs: AMBARI-16429
https://issues.apache.org/jira/browse/AMBARI-16429


Repository: ambari


Description
---

Earlier : first db in the list was selected or was kept empty.

In this patch : 'default' hive db will be selected.


Diffs
-

  
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/upload-table.js
 361de7b 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/upload-table.hbs
 eb95292 

Diff: https://reviews.apache.org/r/47524/diff/


Testing
---

Manual testing done.


Thanks,

Nitiraj Rathore

Review Request 47523: AMBARI-16725 Alert type = RECOVERY does not have connection timeout


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47523/
---

Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Richard Zang, Xi 
Wang, and Yusaku Sako.


Bugs: AMBARI-16725
https://issues.apache.org/jira/browse/AMBARI-16725


Repository: ambari


Description
---

RECOVERY alert types do not have a connection timeout, therefore, this should 
not be shown in the UI (and not have an error because it's blank).


Diffs
-

  ambari-web/app/controllers/main/alerts/definition_configs_controller.js 
617f7cc 

Diff: https://reviews.apache.org/r/47523/diff/


Testing
---

Local ambari-web test passed.
27819 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 47517: AMBARI-16724 Allow customizing Connection Timeout for METRIC Alerts


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47517/
---

(Updated May 18, 2016, 6:09 a.m.)


Review request for Ambari, Jaimin Jetly, Richard Zang, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-16724
https://issues.apache.org/jira/browse/AMBARI-16724


Repository: ambari


Description
---

METRIC alert types have a connection timeout in their definition but that value 
is not exposed in the Ambari Web UI (like WEB alerts are).
 "uri" : {
"http" : "{{ams-hbase-site/hbase.master.info.port}}",
"default_port" : 61310.0,
"connection_timeout" : 5.0
  }
 
Include exposing the Connection Timeout critical threshold in Ambari Web for 
METRIC alert types.


Diffs
-

  ambari-web/app/controllers/main/alerts/definition_configs_controller.js 
617f7cc 
  
ambari-web/test/controllers/main/alerts/definitions_configs_controller_test.js 
f3cf28c 

Diff: https://reviews.apache.org/r/47517/diff/


Testing
---

Modified unit test.
Local ambari-web test passed.
27819 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Review Request 47517: AMBARI-16724 Allow customizing Connection Timeout for METRIC Alerts


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47517/
---

Review request for Ambari, Jaimin Jetly, Richard Zang, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-16724
https://issues.apache.org/jira/browse/AMBARI-16724


Repository: ambari


Description
---

METRIC alert types have a connection timeout in their definition but that value 
is not exposed in the Ambari Web UI (like WEB alerts are).
 "uri" : {
"http" : "{{ams-hbase-site/hbase.master.info.port}}",
"default_port" : 61310.0,
"connection_timeout" : 5.0
  }
 
Include exposing the Connection Timeout critical threshold in Ambari Web for 
METRIC alert types.


Diffs
-

  ambari-web/app/controllers/main/alerts/definition_configs_controller.js 
617f7cc 
  
ambari-web/test/controllers/main/alerts/definitions_configs_controller_test.js 
f3cf28c 

Diff: https://reviews.apache.org/r/47517/diff/


Testing
---

Modified unit test.
Local ambari-web test passed.
27819 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Review Request 47514: Kerberos wizard gets reset and does not remember selections/data entered in previous step when you click on Back

2016-05-17 Thread Anita Jebaraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47514/
---

Review request for Ambari, Alexandr Antonenko and Di Li.


Bugs: AMBARI-15951
https://issues.apache.org/jira/browse/AMBARI-15951


Repository: ambari


Description
---

1. Navigate to Admin->Kerberos. Click on Enable Kerberos.
2. In the wizard, select Existing Active Directory for e.g.
3. Check all the pre-requisites.
4. Click on Next to get to the Configure Kerberos page.
5. Click on Back.
6. The type of KDC radio button gets reset to Existing MIT KDC.
7. Same thing happens if you Navigate to Install and Test Kerberos Client page 
and navigate back to the second or first page of the wizard.

Expected results:
Like other wizards such as Add Service Wizard, the wizard should remember 
selections/data when navigating back in the wizard.


Diffs
-

  ambari-web/app/controllers/main/admin/kerberos/step1_controller.js 7f561dd 
  ambari-web/app/views/main/admin/kerberos/step1_view.js 2e7c4cf 
  ambari-web/test/controllers/main/admin/kerberos/step1_controller_test.js 
280f923 

Diff: https://reviews.apache.org/r/47514/diff/


Testing
---

Ran mvn test
27719 tests complete (32 seconds)
154 tests pending


Thanks,

Anita Jebaraj

Review Request 47513: AMBARI-16723 Stack id needs to be handled differently according to its source


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47513/
---

Review request for Ambari, Jaimin Jetly, Nate Cole, Richard Zang, Xi Wang, and 
Yusaku Sako.


Bugs: AMBARI-16723
https://issues.apache.org/jira/browse/AMBARI-16723


Repository: ambari


Description
---

For default stacks, stack id contains stack name and stack version, while for 
others, it should be "stack + "-" + stack_version + "-" + vdf_version"


Diffs
-

  ambari-web/app/controllers/installer.js 421f2dc 
  ambari-web/app/mappers/stack_mapper.js 473c466 

Diff: https://reviews.apache.org/r/47513/diff/


Testing
---

Local ambari-web test passed.
27819 tests complete (23 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 47508: AMBARI-16721 - Host Filters : 'Alerts' value for 'HOST STATUS' filter is absent


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47508/#review133698
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On May 18, 2016, 1:31 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47508/
> ---
> 
> (Updated May 18, 2016, 1:31 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Zhe (Joe) Wang.
> 
> 
> Bugs: AMBARI-16721
> https://issues.apache.org/jira/browse/AMBARI-16721
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "Alert" option back. Refactor existing filter logic. Move specific filter 
> logic out of common mixin.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/combo_search_box.js 2fa4479 
>   ambari-web/app/data/host/categories.js 03e2fee 
>   ambari-web/app/mixins/common/table_server_view_mixin.js 66f886a 
>   ambari-web/app/views/main/host/combo_search_box.js caf200f 
> 
> Diff: https://reviews.apache.org/r/47508/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit tests passed.
>   27819 tests complete (33 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 47506: AMBARI-16720. Update calculation logic for LLAP configs. AMBARI-16722. Change 'Number of LLAP Daemons', 'In-Memory Cache per Daemon', 'Maximum CPUs per Daemon' to be a 'Text

2016-05-17 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47506/#review133692
---


Ship it!




We have gone through the code change in person.

- Sumit Mohanty


On May 18, 2016, 1:57 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47506/
> ---
> 
> (Updated May 18, 2016, 1:57 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16720 and AMBARI-16722
> https://issues.apache.org/jira/browse/AMBARI-16720
> https://issues.apache.org/jira/browse/AMBARI-16722
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - AMBARI-16720. Update calculation logic for LLAP configs. 
> - AMBARI-16722. Change 'Number of LLAP Daemons', 'In-Memory Cache per 
> Daemon', 'Maximum CPUs per Daemon' to be a 'Text Box' and 'YARN Memory per 
> Daemon' to be a 'Label' instead of 'Slider'.
> 
> 
> AMBARI-16720 - Following properties can/will get updated based on what 
> triggered the calculation.
> 
> 
> - LLAP concurrency - 
> hive-interactive-site/hive.server2.tez.sessions.per.default.queue
> - Number of LLAP nodes used - hive-interactive-env/num_llap_nodes
> - LLAP YARN container Size (in MB) - 
> hive-interactive-site/hive.llap.daemon.yarn.container.mb
> - Number of Executors in container - 
> hive-interactive-site/hive.llap.daemon.num.executors
> - Cache Per node - hive-interactive-site/hive.llap.io.memory.size
> - Property related to cache calculation above - 
> hive-interactive-site/hive.llap.io.enabled
> - LLAP Heap size per node - hive-interactive-env/llap_heap_size
> - Slider App Master Container Size - 
> hive-interactive-env/slider_am_container_size
> 
> 
> The trigger point for updating LLAP configs (mentioned above) is change in 
> values of any of the following:
> 
> (1). 'enable_hive_interactive' set to 'true' 
> (2). 'llap_queue_capacity' 
> (3). 'hive.server2.tez.sessions.per.default.queue'
> (4). 'llap' named queue getting selected for config 
> 'hive.llap.daemon.queue.name' and only 2 queues exist ('llap' and 'default') 
> at root level.
> 
> - If change in value for 'llap_queue_capacity' or 
> 'hive.server2.tez.sessions.per.default.queue' is detected, that config
> value is not calulated, but read and use in calculation for dependent configs.
>  
> - If at any point of time, calulation cant be done (mostly because dependent 
> config can't be retrieved or LLAP queue capacity is not good enough),
> the value for configs are set to their minimum or default values. A user can 
> retrigger the calulation by sliding the "% of cluster capacity" slider.
> 
> 
> Further, as part of 'AMBARI-16722', following configs have been made a Text 
> box instead of Slider earlier:
> 
> - hive-interactive-env/num_llap_nodes
> - hive-interactive-site/hive.llap.io.memory.size
> - hive-interactive-site/hive.llap.daemon.num.executors
> 
> Following configs has been converted to a Label instead of a Slider earlier:
> 
> - hive-interactive-site/hive.llap.daemon.yarn.container.mb
> 
> 
> ---
> Brief on calculations is as follows :
> ---
> 
> 
> LLAP concurrency: 
> 
> Calulated if this is not the driver for calculation, otherwise current value 
> is read.
> 
> = 25% of LLAP queue size / 'Tez AM container Size' based on YARN min 
> container size 
> 
> 
> Slider App Master Container Size:
> 
> 
> = Caluclated value lies b/w 256-1024 MB based on it's own value and YARN min 
> container size. 
> 
> 
> total_am_capacity_required = normalized value of 'tez_am_container_size' 
> based on YARN min container size * llap_concurrency + Slider App Master 
> Container Size
> cap_available_for_daemons = total_llap_queue_size - total_am_capacity_required
> 
> 
> Number of LLAP nodes used:
> -
> 
> = Normalized value of 'cap_available_for_daemons / yarn_nm_mem_in_mb' based 
> on YARN min container size.
> 
> 
> LLAP YARN container Size:
> 
> 
> if Number of LLAP nodes used < 1.00: 
>   = Normalized value of 'cap_available_for_daemons' based on YARN min 
> container size.
> else
>   = Normalized value of 'yarn_nm_mem_in_mb' based on YARN min container size.
>   
> 
> Number of Executors in container:
> 
> 
> = minimum (LLAP YARN container Size / hive_tez_container_size, Number of 
> CPU's per Node Manager Host).
> 
> 
> total Mem for Executors = Number of Executors in container * Read value of 
> hive_tez_container_size
> 
> 
> Cache Per node :
> --
>

Re: Review Request 47428: Changes to Phoenix QueryServer Kerberos configuration

2016-05-17 Thread Josh Elser


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47428/
---

(Updated May 18, 2016, 3:09 a.m.)


Review request for Ambari and Robert Levas.


Changes
---

New patch which successfully does upgrades from 2.2.2.0 to 2.4.0.0-SNAPSHOT. 
Wouldn't have been possible without lots of great help from Robert Levas 
(thanks again!).


Bugs: AMBARI-16171
https://issues.apache.org/jira/browse/AMBARI-16171


Repository: ambari


Description
---

The up-coming version of Phoenix will contain some new functionality to support 
Kerberos authentication of clients via SPNEGO with the Phoenix Query Server 
(PQS).

Presently, Ambari will configure PQS to use the hbase service keytab which will 
result in the SPNEGO authentication failing as the RFC requires that the 
"primary" component of the Kerberos principal for the server is "HTTP". Thus, 
we need to ensure that we switch PQS over to use the spnego.service.keytab as 
the keytab and "HTTP/_HOST@REALM" as the principal.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 2e857ed 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 41f538e 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
c9536f8 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 20fa50f 

Diff: https://reviews.apache.org/r/47428/diff/


Testing
---

Unit testing, still working through a "real" Ambari upgrade


Thanks,

Josh Elser

Re: Review Request 47506: AMBARI-16720. Update calculation logic for LLAP configs. AMBARI-16722. Change 'Number of LLAP Daemons', 'In-Memory Cache per Daemon', 'Maximum CPUs per Daemon' to be a 'Text

2016-05-17 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47506/
---

(Updated May 18, 2016, 1:57 a.m.)

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.

Changes
---

Added Screenshot.

Bugs: AMBARI-16720 and AMBARI-16722
https://issues.apache.org/jira/browse/AMBARI-16720
https://issues.apache.org/jira/browse/AMBARI-16722

Repository: ambari

Description
---

- AMBARI-16720. Update calculation logic for LLAP configs.
- AMBARI-16722. Change 'Number of LLAP Daemons', 'In-Memory Cache per Daemon',
'Maximum CPUs per Daemon' to be a 'Text Box' and 'YARN Memory per Daemon' to be
a 'Label' instead of 'Slider'.

AMBARI-16720 - Following properties can/will get updated based on what
triggered the calculation.

- LLAP concurrency -
hive-interactive-site/hive.server2.tez.sessions.per.default.queue
- Number of LLAP nodes used - hive-interactive-env/num_llap_nodes
- LLAP YARN container Size (in MB) -
hive-interactive-site/hive.llap.daemon.yarn.container.mb
- Number of Executors in container -
hive-interactive-site/hive.llap.daemon.num.executors
- Cache Per node - hive-interactive-site/hive.llap.io.memory.size
- Property related to cache calculation above -
hive-interactive-site/hive.llap.io.enabled
- LLAP Heap size per node - hive-interactive-env/llap_heap_size
- Slider App Master Container Size -
hive-interactive-env/slider_am_container_size

The trigger point for updating LLAP configs (mentioned above) is change in
values of any of the following:

(1). 'enable_hive_interactive' set to 'true'
(2). 'llap_queue_capacity'
(3). 'hive.server2.tez.sessions.per.default.queue'
(4). 'llap' named queue getting selected for config
'hive.llap.daemon.queue.name' and only 2 queues exist ('llap' and 'default') at
root level.

- If change in value for 'llap_queue_capacity' or
'hive.server2.tez.sessions.per.default.queue' is detected, that config
value is not calulated, but read and use in calculation for dependent configs.

- If at any point of time, calulation cant be done (mostly because dependent
config can't be retrieved or LLAP queue capacity is not good enough),
the value for configs are set to their minimum or default values. A user can
retrigger the calulation by sliding the "% of cluster capacity" slider.

Further, as part of 'AMBARI-16722', following configs have been made a Text box
instead of Slider earlier:

- hive-interactive-env/num_llap_nodes
- hive-interactive-site/hive.llap.io.memory.size
- hive-interactive-site/hive.llap.daemon.num.executors

Following configs has been converted to a Label instead of a Slider earlier:

- hive-interactive-site/hive.llap.daemon.yarn.container.mb

---
Brief on calculations is as follows :
---

LLAP concurrency:

Calulated if this is not the driver for calculation, otherwise current value is
read.

= 25% of LLAP queue size / 'Tez AM container Size' based on YARN min container
size

Slider App Master Container Size:

= Caluclated value lies b/w 256-1024 MB based on it's own value and YARN min
container size.

total_am_capacity_required = normalized value of 'tez_am_container_size' based
on YARN min container size * llap_concurrency + Slider App Master Container Size
cap_available_for_daemons = total_llap_queue_size - total_am_capacity_required

Number of LLAP nodes used:
-

= Normalized value of 'cap_available_for_daemons / yarn_nm_mem_in_mb' based on
YARN min container size.

LLAP YARN container Size:

if Number of LLAP nodes used < 1.00:
= Normalized value of 'cap_available_for_daemons' based on YARN min container
size.
else
= Normalized value of 'yarn_nm_mem_in_mb' based on YARN min container size.

Number of Executors in container:

= minimum (LLAP YARN container Size / hive_tez_container_size, Number of CPU's
per Node Manager Host).

total Mem for Executors = Number of Executors in container * Read value of
hive_tez_container_size

Cache Per node :
--

= LLAP YARN container Size - total Mem for Executors.

hive-interactive-site/hive.llap.io.enabled set to 'false' if 'Cache Per node'
< 64, else 'true'.

LLAP Heap size per node:
---

= maximum (total Mem for Executors * 0.8, total Mem for Executors - 1024)

Diffs
-

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
fffcd03

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
10518dc

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/the

Review Request 47506: AMBARI-16720. Update calculation logic for LLAP configs. AMBARI-16722. Change 'Number of LLAP Daemons', 'In-Memory Cache per Daemon', 'Maximum CPUs per Daemon' to be a 'Text Box'

2016-05-17 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47506/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.

Bugs: AMBARI-16720 and AMBARI-16722
https://issues.apache.org/jira/browse/AMBARI-16720
https://issues.apache.org/jira/browse/AMBARI-16722

Repository: ambari

Description
---

AMBARI-16720 - Following properties can/will get updated based on what
triggered the calculation.

The trigger point for updating LLAP configs (mentioned above) is change in
values of any of the following:

- If change in value for 'llap_queue_capacity' or
'hive.server2.tez.sessions.per.default.queue' is detected, that config
value is not calulated, but read and use in calculation for dependent configs.

Further, as part of 'AMBARI-16722', following configs have been made a Text box
instead of Slider earlier:

- hive-interactive-env/num_llap_nodes
- hive-interactive-site/hive.llap.io.memory.size
- hive-interactive-site/hive.llap.daemon.num.executors

Following configs has been converted to a Label instead of a Slider earlier:

- hive-interactive-site/hive.llap.daemon.yarn.container.mb

---
Brief on calculations is as follows :
---

LLAP concurrency:

Calulated if this is not the driver for calculation, otherwise current value is
read.

= 25% of LLAP queue size / 'Tez AM container Size' based on YARN min container
size

Slider App Master Container Size:

= Caluclated value lies b/w 256-1024 MB based on it's own value and YARN min
container size.

Number of LLAP nodes used:
-

= Normalized value of 'cap_available_for_daemons / yarn_nm_mem_in_mb' based on
YARN min container size.

LLAP YARN container Size:

Number of Executors in container:

= minimum (LLAP YARN container Size / hive_tez_container_size, Number of CPU's
per Node Manager Host).

total Mem for Executors = Number of Executors in container * Read value of
hive_tez_container_size

Cache Per node :
--

= LLAP YARN container Size - total Mem for Executors.

hive-interactive-site/hive.llap.io.enabled set to 'false' if 'Cache Per node'
< 64, else 'true'.

LLAP Heap size per node:
---

= maximum (total Mem for Executors * 0.8, total Mem for Executors - 1024)

Diffs
-

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
fffcd03

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
10518dc

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
90b89f0
ambari-server/src/main/resources/stacks/HDP/2.5/servi

Review Request 47508: AMBARI-16721 - Host Filters : 'Alerts' value for 'HOST STATUS' filter is absent

2016-05-17 Thread Richard Zang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47508/
---

Review request for Ambari, Jaimin Jetly and Zhe (Joe) Wang.


Bugs: AMBARI-16721
https://issues.apache.org/jira/browse/AMBARI-16721


Repository: ambari


Description
---

Add "Alert" option back. Refactor existing filter logic. Move specific filter 
logic out of common mixin.


Diffs
-

  ambari-web/app/controllers/main/host/combo_search_box.js 2fa4479 
  ambari-web/app/data/host/categories.js 03e2fee 
  ambari-web/app/mixins/common/table_server_view_mixin.js 66f886a 
  ambari-web/app/views/main/host/combo_search_box.js caf200f 

Diff: https://reviews.apache.org/r/47508/diff/


Testing
---

Manually tested on live cluster.
All unit tests passed.
  27819 tests complete (33 seconds)
  154 tests pending


Thanks,

Richard Zang

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services

2016-05-17 Thread Nate Cole



> On May 17, 2016, 2:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml, 
> > line 166
> > 
> >
> > Does the group name have to be unique now or is this only for the 
> > purposes of making debugging easier?
> 
> Tim Thorpe wrote:
> In order to be able to add a custom component into the priority list of a 
> particular service check step, there would need to be something distinct 
> about them, either the name or the title.
> 
> Tim Thorpe wrote:
> Currently the group ordering logic is counting on the group name to be 
> unique.  It could use the group title.

The title is used directly by the UI and should not be a reliable indicator of 
group.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133587
---


On May 16, 2016, 2:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 2:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts

2016-05-17 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47438/#review133662
---


Ship it!




Ship It!

- Sumit Mohanty


On May 17, 2016, 12:51 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47438/
> ---
> 
> (Updated May 17, 2016, 12:51 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth 
> Gunturi.
> 
> 
> Bugs: AMBARI-16695
> https://issues.apache.org/jira/browse/AMBARI-16695
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are new HDFS alerts that watch growth rates. Some (like RPC) have 
> "minimum" values, meaning we ignore growth until we are past a certain value 
> (like latency in seconds).
> 
> There are a few other alerts that I think also need minimums.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> aedbdfe 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py
>  3296493 
> 
> Diff: https://reviews.apache.org/r/47438/diff/
> 
> 
> Testing
> ---
> 
> Deployed manually to verify nothing breaks.
> 
> --
> Ran 261 tests in 6.682s
> 
> OK
> --
> Total run:1016
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts

2016-05-17 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47438/#review133661
---


Ship it!




Ship It!

- Aravindan Vijayan


On May 17, 2016, 12:51 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47438/
> ---
> 
> (Updated May 17, 2016, 12:51 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth 
> Gunturi.
> 
> 
> Bugs: AMBARI-16695
> https://issues.apache.org/jira/browse/AMBARI-16695
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are new HDFS alerts that watch growth rates. Some (like RPC) have 
> "minimum" values, meaning we ignore growth until we are past a certain value 
> (like latency in seconds).
> 
> There are a few other alerts that I think also need minimums.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> aedbdfe 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py
>  3296493 
> 
> Diff: https://reviews.apache.org/r/47438/diff/
> 
> 
> Testing
> ---
> 
> Deployed manually to verify nothing breaks.
> 
> --
> Ran 261 tests in 6.682s
> 
> OK
> --
> Total run:1016
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts

2016-05-17 Thread Sid Wagle



> On May 17, 2016, 1:52 a.m., Sid Wagle wrote:
> > Just realized I am missing UpgradeCatalog changes, will add subsequent 
> > patch.

No need for upgrade steps since the Catalog deletes all standard deviation 
alerts which get added back on server restart.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47438/#review133478
---


On May 17, 2016, 12:51 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47438/
> ---
> 
> (Updated May 17, 2016, 12:51 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth 
> Gunturi.
> 
> 
> Bugs: AMBARI-16695
> https://issues.apache.org/jira/browse/AMBARI-16695
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are new HDFS alerts that watch growth rates. Some (like RPC) have 
> "minimum" values, meaning we ignore growth until we are past a certain value 
> (like latency in seconds).
> 
> There are a few other alerts that I think also need minimums.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> aedbdfe 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py
>  3296493 
> 
> Diff: https://reviews.apache.org/r/47438/diff/
> 
> 
> Testing
> ---
> 
> Deployed manually to verify nothing breaks.
> 
> --
> Ran 261 tests in 6.682s
> 
> OK
> --
> Total run:1016
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 47498: Knox Gateway Uses Wrong Keystore After Upgrade

2016-05-17 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47498/#review133651
---




ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server_upgrade.py
 (line 87)


This was a good time to also cleanup some technical debt. AMBARI-15419 
seeds conf directories now, so I was able to remove this code for Falcon, 
Oozie, Knox, and Flume



ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
 (line 121)


Making a backup here is just courtesy, but it's good to have backups of the 
JKS in case something goes very wrong.



ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
 (lines 127 - 129)


On upgrade, seed data directory now. This will ensure it's only done on the 
right stack versions.



ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
 (lines 86 - 91)


This only does work if Knox supports versioned directories in the target 
version. 

If upgrading from HDP 2.2 to 2.3.0.0, then nothing has to happen since the 
data directory doesn't change.


- Jonathan Hurley


On May 17, 2016, 6:35 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47498/
> ---
> 
> (Updated May 17, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-16717
> https://issues.apache.org/jira/browse/AMBARI-16717
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When upgrading Knox, the {{data}} directory and its security artifacts are 
> not copied over to the "versioned" data directory. This causes the 
> {{gateway.jks}} keystore to be automatically re-generated. If the 
> installation was using a custom keystore/certificate, then this will cause 
> connections to be rejected after a successful startup. 
> 
> {code:title=Knox 2.2 -> 2.3.0.0}
> /usr/hdp/current/knox-server/data -> /var/lib/knox/data
> {code}
> 
> {code:title=Knox 2.3.2.0+}
> /usr/hdp/current/knox-server/data -> /var/lib/knox/data-2.3.2.0-1234
> {code}
> 
> As a result, after upgrading the {{/var/lib/knox/data-2.3.2.0-1234}} does not 
> contain any of the security artifacts from the prior version.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server_upgrade.py
>  8eca96c 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py
>  4d72463 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_upgrade.py
>  64c0032 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
>  e5e4103 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
>  63949f8 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
>  fcce418 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
>  28d2991 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
>  5f0c6aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
>  cf79368 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
>  fab12c8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
>  1887414 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
>  469f2e7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  56cd6d0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
>  29ebeff 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  7d67f8e 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  5616cb4 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> d755516 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 
> ef97d20 
>   ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 
> 6dfb609 
>   amb

Review Request 47498: Knox Gateway Uses Wrong Keystore After Upgrade

2016-05-17 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47498/
---

Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-16717
https://issues.apache.org/jira/browse/AMBARI-16717


Repository: ambari


Description
---

When upgrading Knox, the {{data}} directory and its security artifacts are not 
copied over to the "versioned" data directory. This causes the {{gateway.jks}} 
keystore to be automatically re-generated. If the installation was using a 
custom keystore/certificate, then this will cause connections to be rejected 
after a successful startup. 

{code:title=Knox 2.2 -> 2.3.0.0}
/usr/hdp/current/knox-server/data -> /var/lib/knox/data
{code}

{code:title=Knox 2.3.2.0+}
/usr/hdp/current/knox-server/data -> /var/lib/knox/data-2.3.2.0-1234
{code}

As a result, after upgrading the {{/var/lib/knox/data-2.3.2.0-1234}} does not 
contain any of the security artifacts from the prior version.


Diffs
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server_upgrade.py
 8eca96c 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py
 4d72463 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_upgrade.py
 64c0032 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
 e5e4103 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
 63949f8 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
 fcce418 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 28d2991 
  
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
 5f0c6aa 
  
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
 cf79368 
  
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
 fab12c8 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
 1887414 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
 469f2e7 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 56cd6d0 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
6b74af0 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
 29ebeff 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 7d67f8e 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
e3bc7a3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 5616cb4 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
d755516 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py ef97d20 
  ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 6dfb609 
  ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py fbc55ca 

Diff: https://reviews.apache.org/r/47498/diff/


Testing
---

mvn clean test

Verified that an upgrade copies the data directory and that Knox starts up on 
the custom keystore.


Thanks,

Jonathan Hurley

Review Request 47497: Atlas Integration : Extend Ambari with settings for Atlas environment configuration

2016-05-17 Thread Tom Beerbower


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47497/
---

Review request for Ambari.


Bugs: AMBARI-16691
https://issues.apache.org/jira/browse/AMBARI-16691


Repository: ambari


Description
---

In ATLAS-616, we uncovered problems with Atlas web service when testing queries 
at scale. The investigation revealed a fix possible with tuning some GC 
parameters. ATLAS-616 suggests the GC values in documentation.

Default these settings in Ambari.

The values we have recommended are all to be set in atlas-env.sh config file. 
They are:

Common for all JDKs:

#export ATLAS_SERVER_OPTS="-server -XX:SoftRefLRUPolicyMSPerMB=0 
-XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC 
-XX:+CMSParallelRemarkEnabled -XX:+PrintTenuringDistribution 
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dumps/atlas_server.hprof 
-Xloggc:logs/gc-worker.log -verbose:gc -XX:+UseGCLogFileRotation 
-XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1m -XX:+PrintGCDetails 
-XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps"


For JDK7:

#export ATLAS_SERVER_HEAP="-Xms15360m -Xmx15360m -XX:MaxNewSize=3072m 
-XX:PermSize=100M -XX:MaxPermSize=512m"


For JDK8:

#export ATLAS_SERVER_HEAP="-Xms15360m -Xmx15360m -XX:MaxNewSize=5120m 
-XX:MetaspaceSize=100M -XX:MaxMetaspaceSize=512m"


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 b33a956 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml
 bae4de3 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
689e1fd 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py cf82a9c 

Diff: https://reviews.apache.org/r/47497/diff/


Testing
---

manual test

updated unit test

mvn clean test


Thanks,

Tom Beerbower

Re: Review Request 47455: AMBARI-16702 Zeppelin cluster deployment fails due to unavailability of zeppelin service check script


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47455/#review133636
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/service_check.py
 (line 29)


print ()?


- Jayush Luniya


On May 17, 2016, 1:50 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47455/
> ---
> 
> (Updated May 17, 2016, 1:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Rohit 
> Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16702
> https://issues.apache.org/jira/browse/AMBARI-16702
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - add service check script for zeppelin
> 
> error log in jira issue
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  866d746 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/service_check.py
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47455/diff/
> 
> 
> Testing
> ---
> 
> manually tested on centos 6.4, Ambari 2.4.0.0
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 47456: Takeover config merge should handle AMS hbase configs

2016-05-17 Thread Srimanth Gunturi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47456/#review133634
---




ambari-server/src/main/resources/scripts/takeover_files_mapping.json (line 2)


If we are creating a mapping file for file/path=>config-type, we should add 
other commonly used paths and map them to config-types. Files like 
'log4j.properties' which maps to hdfs-log4j, yarn-log4j, etc...

This file by default should have all the most common paths, and their 
correct config-types.


- Srimanth Gunturi


On May 17, 2016, 2:16 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47456/
> ---
> 
> (Updated May 17, 2016, 2:16 p.m.)
> 
> 
> Review request for Ambari, Sid Wagle and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16703
> https://issues.apache.org/jira/browse/AMBARI-16703
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hbase-site conflicts between cluster and AMS.
> 
> Hard code the config path as /etc/ams-hbase/conf to be AMS HBase configs.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/scripts/takeover_config_merge.py f940f27 
>   ambari-server/src/main/resources/scripts/takeover_files_mapping.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47456/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47456: Takeover config merge should handle AMS hbase configs

2016-05-17 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47456/#review133631
---




ambari-server/src/main/resources/scripts/takeover_files_mapping.json (line 2)


Lets add this for ams-hbase-env, ams-hbase-log4j,


- Sid Wagle


On May 17, 2016, 2:16 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47456/
> ---
> 
> (Updated May 17, 2016, 2:16 p.m.)
> 
> 
> Review request for Ambari, Sid Wagle and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16703
> https://issues.apache.org/jira/browse/AMBARI-16703
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hbase-site conflicts between cluster and AMS.
> 
> Hard code the config path as /etc/ams-hbase/conf to be AMS HBase configs.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/scripts/takeover_config_merge.py f940f27 
>   ambari-server/src/main/resources/scripts/takeover_files_mapping.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47456/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47434: Remove unused parameters from hawq-site.xml

2016-05-17 Thread Matt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47434/#review133630
---


Ship it!




Ship It!

- Matt


On May 16, 2016, 5:14 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47434/
> ---
> 
> (Updated May 16, 2016, 5:14 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr 
> Diachenko.
> 
> 
> Bugs: AMBARI-16694
> https://issues.apache.org/jira/browse/AMBARI-16694
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove the below unused parameters from hawq-site.xml
> hawq_re_cgroup_hierarchy_name
> hawq_re_cgroup_mount_point
> hawq_re_cpu_enable
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  170e8cf 
> 
> Diff: https://reviews.apache.org/r/47434/diff/
> 
> 
> Testing
> ---
> 
> yes.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 5:39 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 829
> > 
> >
> > What about RestartGrouping, ColocatedGrouping, StartGrouping, 
> > StopGrouping, UpdateStackGrouping? How about parent.merge(iterator) instead?

Implemented parent.merge(iterator), RestartGrouping, ColocatedGrouping, 
StartGrouping, StopGrouping, UpdateStackGrouping just all use the default 
behavior like what was happening before with mergeRegularGrouping.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133556
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 2:30 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  lines 822-831
> > 
> >
> > Visitor or abstract method?

Added the mergeRegularGroupingByName to the Grouping class as 
merge(Iterator) throws AmbariException
The other methods were added to ClusterGrouping and ServiceCheckGrouping 
respectively, overridding the merge method from the Grouping class.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133533
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 47428: Changes to Phoenix QueryServer Kerberos configuration

2016-05-17 Thread Josh Elser


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47428/
---

(Updated May 17, 2016, 7:58 p.m.)


Review request for Ambari and Robert Levas.


Bugs: AMBARI-16171
https://issues.apache.org/jira/browse/AMBARI-16171


Repository: ambari


Description
---

The up-coming version of Phoenix will contain some new functionality to support 
Kerberos authentication of clients via SPNEGO with the Phoenix Query Server 
(PQS).

Presently, Ambari will configure PQS to use the hbase service keytab which will 
result in the SPNEGO authentication failing as the RFC requires that the 
"primary" component of the Kerberos principal for the server is "HTTP". Thus, 
we need to ensure that we switch PQS over to use the spnego.service.keytab as 
the keytab and "HTTP/_HOST@REALM" as the principal.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 1f3b1d3 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
c9536f8 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 f36e640 

Diff: https://reviews.apache.org/r/47428/diff/


Testing
---

Unit testing, still working through a "real" Ambari upgrade


Thanks,

Josh Elser

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 5:39 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 844
> > 
> >
> > The after tag is overloaded. Meaning it could mean insert this service 
> > after another service in some place and in some places it could mean insert 
> > this  group after another group. I am wondering if there would be a case 
> > where we would need a combination (example: Create a new group that is not 
> > in the stack upgrade pack and add 2 new services to this group in a 
> > specific order). 
> > 
> > Instead of leaving the  tag to interpretation, why not be 
> > explicit as ,  etc. That way we can 
> > support combinations
> 
> Jayush Luniya wrote:
> Example:
> 
> 
>   CORE_MASTERS
>   
> MY_MASTER_1
>   
> 
> 
> 
> 
>   CORE_MASTERS
>   MY_SERVICE_1  
>   
> MY_MASTER_2
>   
> 
>  
>  Alterative
> 
>
>   CORE_MASTERS
>   
> MY_MASTER_1
>   
> 
> 
> 
> 
>   CORE_MASTERS
>   
> MY_SERVICE_1  
> MY_MASTER_2
>   
> 

With custom services they should just be able to create two new groups rather 
than try to combine them into one.  If there really is a need for the a new 
group that spans multiple services, then it is probably a candidate for 
inclusion in the main upgrade xml.  You can even include an empty group in the 
main upgrade xml like:




- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133556
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual t

Re: Review Request 46738: Converting cluster to Kerberos the views should automatically adjust and work prior to the kerberization.

2016-05-17 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46738/#review133610
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On May 13, 2016, 7:09 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46738/
> ---
> 
> (Updated May 13, 2016, 7:09 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16138
> https://issues.apache.org/jira/browse/AMBARI-16138
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If view is managed by ambari, then fetching the webhdfsurl, webhdfs auth and 
> hive auth based on the hive and hdfs service config.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  3547ad3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  f36e640 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/HelpService.java 
> 05b55d2 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
>  1442748 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/jobs/atsJobs/ATSParserFactory.java
>  f5e9bcf 
>   contrib/views/hive/src/main/resources/view.xml 8f8a470 
>   
> contrib/views/hive/src/test/java/org/apache/ambari/view/hive/BaseHiveTest.java
>  25db721 
>   contrib/views/pig/src/main/resources/view.xml 9df91f8 
>   contrib/views/tez/readme.md fdb9459 
>   
> contrib/views/tez/src/main/java/org/apache/ambari/view/tez/ViewController.java
>  440ac65 
>   contrib/views/tez/src/main/resources/view.xml d8105f1 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/Services.java
>  1bace94 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/AuthConfigurationBuilder.java
>  4911455 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
>  8b45cd6 
>   
> contrib/views/utils/src/test/java/org/apache/ambari/view/utils/ambari/ServicesTest.java
>  c7c0990 
> 
> Diff: https://reviews.apache.org/r/46738/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested on local Vm.
> Tested Hive, Pig, Hdfs and Tez views with HDFS HA and kerberos.
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Review Request 47488: Ambari copies jdbc driver jar into hadoop/lib and haoop-yarn/lib

2016-05-17 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47488/
---

Review request for Ambari, Andrew Onischuk, Mugdha Varadkar, and Sumit Mohanty.


Bugs: AMBARI-16716
https://issues.apache.org/jira/browse/AMBARI-16716


Repository: ambari


Description
---

PROBLEM:
Ambari copies the jdbc driver (be it mysql or oracle) into 
/usr/hdp/xxx/hadoop/lib and /usr/hdp/xxx/hadoop-yarn/lib, this can cause 
problem for hive and other components, as user updates the jar inside 
/usr/hdp/xx/hive/lib, but those on the hadoop and yarn lib are also in the 
classpath.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
 3eb8591 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
 d653000 
  ambari-server/conf/unix/ambari.properties 239a731 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 bf11bf6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 bb58670 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 5e51f5f 
  ambari-server/src/main/python/ambari_server/serverSetup.py 217d988 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 32a36a7 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/setup_ranger_hbase.py
 1e860d9 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 784da9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
 f660562 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 30ffc8f 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py
 d0dd9bb 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py
 daf0301 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 9f5d799 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_ranger_hive.py
 7515e9b 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 37bd77c 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/setup_ranger_kafka.py
 3a3ecfe 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 d1268a1 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/setup_ranger_knox.py
 64e2060 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
 d69339e 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
 3060353 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
 75924c5 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
 799447b 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 9fc0971 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_ranger_storm.py
 1dd85e9 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 3306cf2 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/setup_ranger_yarn.py
 1f61d41 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
 8fc1a64 

Diff: https://reviews.apache.org/r/47488/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml, 
> > line 166
> > 
> >
> > Does the group name have to be unique now or is this only for the 
> > purposes of making debugging easier?
> 
> Tim Thorpe wrote:
> In order to be able to add a custom component into the priority list of a 
> particular service check step, there would need to be something distinct 
> about them, either the name or the title.

Currently the group ordering logic is counting on the group name to be unique.  
It could use the group title.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133587
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java,
> >  line 485
> > 
> >
> > May want to check that upgradeFile is not null before calling 
> > getAbsolutePath() on it, or return null if upgradeFile is null at the start.

This should never be null because of how the method is called.  A list of files 
in the upgrade folder is processed and the one that matches config-upgrade.xml 
is parsed.  But I added the extra check just in case.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133587
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 2:30 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java,
> >  lines 112-113
> > 
> >
> > With no other exceptions, can just use "regular" logging syntax:  
> > LOG.debug("Service upgrades folder {} for service {} in stack {} is 
> > empty.", absUpgradesDir, serviceDir.getName(), stackId)

This code was copied and pasted from the same method which also deals with the 
package dir.


> On May 17, 2016, 2:30 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java,
> >  lines 116-117
> > 
> >
> > Maybe LOG.info() here.  If nothing happens we'll have no record unless 
> > DEBUG is enabled (which is, admittedly, an awful idea)

It is normal for a service to not have an upgrades folder.  The upgrade folder 
is only required IF a service wants to extend the upgrade xml with its own 
specific logic.  There is no reason to litter the logs with this.

This code was copied and pasted from the same method which also deals with the 
package dir.


> On May 17, 2016, 2:30 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml, 
> > line 166
> > 
> >
> > Changing this name may (or may not) impact UI.  The group name is not 
> > guaranteed unique, and some additional information may be displayed based 
> > on it's name (unconfirmed, but that's the intent).  Also, make consistent 
> > using an '_': SERVICE_CHECK_1

In order to be able to add a custom component into a specific service check 
group, something about them needs to be unique.  So either the name or the 
title.  I could change the title from "All Service Checks after core masters" 
or something like that.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133533
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 2:30 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  lines 890-892
> > 
> >
> > Really need to throw here or just log it and continue; ?

When the StackManager encounters errors while reading the stack, it almost 
always ends up throwing an exception and ambari-server fails to start.  I'm ok 
with just logging the issue for this and other errors but then you could think 
that everything is fine with the service specific upgrade xml unless you dig 
through the log.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133533
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:33 a.m., Jayush Luniya wrote:
> > Can you add unit test coverage?
> 
> Jayush Luniya wrote:
> We should have unit tests in particular to validate incorrectly authored 
> service upgrade packs. What happens if we add a circular dependency (example: 
> KAFKA is marked with KNOX and KNOX is marked with 
> KAFKA)
> 
> Jayush Luniya wrote:
> Also we should document any unsupported use cases.

Added unit test for both a possible case and for the circular dependency.  
Really the circular dependency case also covers when you have FOO after BAR but 
BAR is not included.  Where should the documentation go?


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133499
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml, 
> > line 166
> > 
> >
> > Does the group name have to be unique now or is this only for the 
> > purposes of making debugging easier?

In order to be able to add a custom component into the priority list of a 
particular service check step, there would need to be something distinct about 
them, either the name or the title.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133587
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services

2016-05-17 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133587
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
(line 473)


May want to check that upgradeFile is not null before calling 
getAbsolutePath() on it, or return null if upgradeFile is null at the start.



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 702)


Add some java doc to these new methods.



ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml (line 
166)


Does the group name have to be unique now or is this only for the purposes 
of making debugging easier?


- Alejandro Fernandez


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Review Request 47480: AMBARI-16714: Pull 'Groups' txt shown on Alert details filters from the messages.js file

2016-05-17 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47480/
---

Review request for Ambari and Alexandr Antonenko.


Bugs: AMBARI-16714
https://issues.apache.org/jira/browse/AMBARI-16714


Repository: ambari


Description
---

Pull 'Groups' txt shown on Alert details filters from the messages.js file, 
it's currently a hardcoded label in the view code.


Diffs
-

  ambari-web/app/views/main/alert_definitions_view.js b65b8cf 

Diff: https://reviews.apache.org/r/47480/diff/


Testing
---

Manually patch a trunk cluster with UI change, verify the label is displayed 
correctly.


Thanks,

Di Li

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 5:39 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 844
> > 
> >
> > The after tag is overloaded. Meaning it could mean insert this service 
> > after another service in some place and in some places it could mean insert 
> > this  group after another group. I am wondering if there would be a case 
> > where we would need a combination (example: Create a new group that is not 
> > in the stack upgrade pack and add 2 new services to this group in a 
> > specific order). 
> > 
> > Instead of leaving the  tag to interpretation, why not be 
> > explicit as ,  etc. That way we can 
> > support combinations

Example:


  CORE_MASTERS
  
MY_MASTER_1
  




  CORE_MASTERS
  MY_SERVICE_1  
  
MY_MASTER_2
  

 
 Alterative

   
  CORE_MASTERS
  
MY_MASTER_1
  




  CORE_MASTERS
  
MY_SERVICE_1  
MY_MASTER_2
  



- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133556
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133556
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 674)


new ArrayList<>()



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 705)


Use constant instead
See StackDefinitionDirectory::CONFIG_UPGRADE_XML_FILENAME_PREFIX



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 734)


new ArrayList<>()



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 745)


Validate that all sub-groups in the list are of the same type.



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 753)


Might not necessarily be original if this is a new group for the service. 
(Line#745).



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 829)


What about RestartGrouping, ColocatedGrouping, StartGrouping, StopGrouping, 
UpdateStackGrouping? How about parent.merge(iterator) instead?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 837)


nit: child instead of next?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 844)


The after tag is overloaded. Meaning it could mean insert this service 
after another service in some place and in some places it could mean insert 
this  group after another group. I am wondering if there would be a case where 
we would need a combination (example: Create a new group that is not in the 
stack upgrade pack and add 2 new services to this group in a specific order). 

Instead of leaving the  tag to interpretation, why not be explicit 
as ,  etc. That way we can support 
combinations



ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml (line 
158)


SERVICE_CHECK_1 instead


- Jayush Luniya


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422

Re: Review Request 47452: Do not use implicit routing by default in solr-client

2016-05-17 Thread Robert Nettleton


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/#review133578
---


Ship it!




Ship It!

- Robert Nettleton


On May 17, 2016, 1:29 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47452/
> ---
> 
> (Updated May 17, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16701
> https://issues.apache.org/jira/browse/AMBARI-16701
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default ambari should not use implicit routing for creating collections
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  d661288 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  21e7fee 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
>  c888269 
> 
> Diff: https://reviews.apache.org/r/47452/diff/
> 
> 
> Testing
> ---
> 
> ambari logsearch solr-client tests:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 47439: Enabling/Disabling interactive query should sustain browser refreshes


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47439/#review133575
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On May 17, 2016, 12:57 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47439/
> ---
> 
> (Updated May 17, 2016, 12:57 a.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang, Richard Zang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-16696
> https://issues.apache.org/jira/browse/AMBARI-16696
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> #STR:#
> 1 Enable interactive query and save the hive configs
> 2 This will create requests that can be tracked in background ops
> 3 While a task "Restart LLAP" or "Install Components"  is ongoing, Refresh 
> the browser 
> 
> #Expected Result:# The ongoing request should be completed and subsequent 
> requests ("Start HiveServer2 Interactive") should be triggered
> #Actual Result:# The ongoing request is completed but the subsequent request 
> is not triggered.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/details.js 374b077 
>   ambari-web/app/mixins/main/service/configs/component_actions_by_configs.js 
> 5e1eb50 
>   ambari-web/app/utils/ajax/ajax.js e7059e7 
>   ambari-web/app/utils/batch_scheduled_requests.js 7274fdc 
> 
> Diff: https://reviews.apache.org/r/47439/diff/
> 
> 
> Testing
> ---
> 
> Verified manually that the patch addresses the issue on a cluster
> Verified that all ambari-web unit tests works:
> 
>   27817 tests complete (31 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 47379: PXF operations considers the agent status

2016-05-17 Thread Matt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47379/#review133561
---


Ship it!




Ship It!

- Matt


On May 13, 2016, 5:27 p.m., Goutam Tadi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47379/
> ---
> 
> (Updated May 13, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, 
> Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-16670
> https://issues.apache.org/jira/browse/AMBARI-16670
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF operations considers the agent status
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/service/item.js 9250bfc 
>   ambari-web/app/models/host_component.js 569879f 
>   ambari-web/test/controllers/main/service/item_test.js 383aa2f 
> 
> Diff: https://reviews.apache.org/r/47379/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> User should be able to do the following ops on PXF:
> Start PXF slaves if one or more slaves are down
> Stop PXF slaves if one or more slaves are up
> Start disabled if all agents are up
> Stop disabled if all agents are down
> Service Check disabled if one or more slaves are down
> Service Check enabled if all agents are up
> 
> 
> Thanks,
> 
> Goutam Tadi
> 
>

Re: Review Request 47467: Hive View does not work with multi-bytes characters


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47467/
---

(Updated May 17, 2016, 3:27 p.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-16713
https://issues.apache.org/jira/browse/AMBARI-16713


Repository: ambari


Description
---

Writing bytes instead of String to stream.


Diffs
-

  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java
 bd01ead 

Diff: https://reviews.apache.org/r/47467/diff/


Testing (updated)
---

Manually Tested


Thanks,

Gaurav Nagar

Review Request 47467: Hive View does not work with multi-bytes characters


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47467/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-16713
https://issues.apache.org/jira/browse/AMBARI-16713


Repository: ambari


Description
---

Writing bytes instead of String to stream.


Diffs
-

  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java
 bd01ead 

Diff: https://reviews.apache.org/r/47467/diff/


Testing
---

Manuall Tested


Thanks,

Gaurav Nagar

Re: Review Request 47460: Explain script on pig views encountered an IOExecption

2016-05-17 Thread Rohit Choudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47460/#review133547
---


Ship it!




Ship It!

- Rohit Choudhary


On May 17, 2016, 3 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47460/
> ---
> 
> (Updated May 17, 2016, 3 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16707
> https://issues.apache.org/jira/browse/AMBARI-16707
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed url formation for explian script
> 
> 
> Diffs
> -
> 
>   
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/jobs/JobResourceManager.java
>  2bd5745 
> 
> Diff: https://reviews.apache.org/r/47460/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 47465: hostname start to have localhost value in ambari-agent.ini file after upgrade from 2.2.0.0 to 2.4.0.0

2016-05-17 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47465/#review133546
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On May 17, 2016, 6:01 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47465/
> ---
> 
> (Updated May 17, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-16711
> https://issues.apache.org/jira/browse/AMBARI-16711
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:  
> 1) Install old version  
> 2) Make ambari only upgrade
> 
> Actual result:  
> hostname start to have localhost value in ambari-agent.ini file after upgrade
> from 2.2.0.0 to 2.4.0.0  
> But before upgrade value was correct.
> 
> 
> 
> 
> [server]
> hostname=localhost
> 
> 
> 
> 
> 
> INFO 2016-05-17 09:58:56,204 NetUtil.py:60 - Connecting to 
> https://localhost:8440/ca
> WARNING 2016-05-17 09:58:56,205 NetUtil.py:89 - Failed to connect to 
> https://localhost:8440/ca due to [Errno 111] Connection refused  
> WARNING 2016-05-17 09:58:56,205 NetUtil.py:112 - Server at 
> https://localhost:8440 is not reachable, sleeping for 10 seconds...
> root@os-u14-ddqwts-upg-sanity-u-2200-2:/var/log/ambari-agent# rmp -qa | 
> grep ambari
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/install-helper.sh 8a5e1f1 
> 
> Diff: https://reviews.apache.org/r/47465/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 47465: hostname start to have localhost value in ambari-agent.ini file after upgrade from 2.2.0.0 to 2.4.0.0


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47465/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-16711
https://issues.apache.org/jira/browse/AMBARI-16711


Repository: ambari


Description
---

STR:  
1) Install old version  
2) Make ambari only upgrade

Actual result:  
hostname start to have localhost value in ambari-agent.ini file after upgrade
from 2.2.0.0 to 2.4.0.0  
But before upgrade value was correct.




[server]
hostname=localhost





INFO 2016-05-17 09:58:56,204 NetUtil.py:60 - Connecting to 
https://localhost:8440/ca
WARNING 2016-05-17 09:58:56,205 NetUtil.py:89 - Failed to connect to 
https://localhost:8440/ca due to [Errno 111] Connection refused  
WARNING 2016-05-17 09:58:56,205 NetUtil.py:112 - Server at 
https://localhost:8440 is not reachable, sleeping for 10 seconds...
root@os-u14-ddqwts-upg-sanity-u-2200-2:/var/log/ambari-agent# rmp -qa | 
grep ambari


Diffs
-

  ambari-agent/conf/unix/install-helper.sh 8a5e1f1 

Diff: https://reviews.apache.org/r/47465/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 47460: Explain script on pig views encountered an IOExecption


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47460/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-16707
https://issues.apache.org/jira/browse/AMBARI-16707


Repository: ambari


Description
---

Changed url formation for explian script


Diffs
-

  
contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/jobs/JobResourceManager.java
 2bd5745 

Diff: https://reviews.apache.org/r/47460/diff/


Testing
---

Manually tested


Thanks,

Gaurav Nagar

Review Request 47459: ADLS as default FS is not supported for any views


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47459/
---

Review request for Ambari, Chris Nauroth, DIPAYAN BHOWMICK, Nitiraj Rathore, 
Pallav Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-16706
https://issues.apache.org/jira/browse/AMBARI-16706


Repository: ambari


Description
---

Added adl related properties


Diffs
-

  contrib/views/utils/pom.xml c15afad 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/ValidatorUtils.java
 c936355 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
 8b45cd6 

Diff: https://reviews.apache.org/r/47459/diff/


Testing
---

Manually Tested


Thanks,

Gaurav Nagar

Re: Review Request 47297: Extend logging for ActionQueue's retry logic

2016-05-17 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47297/#review133542
---


Ship it!




Ship It!

- Sebastian Toader


On May 17, 2016, 9:31 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47297/
> ---
> 
> (Updated May 17, 2016, 9:31 a.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16630
> https://issues.apache.org/jira/browse/AMBARI-16630
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> At info level, commands should be tracked if they are retried on failure or 
> not, and if not, what is the reason for that.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 85389f9 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 7d0efa2 
> 
> Diff: https://reviews.apache.org/r/47297/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:19.911s
> [INFO] Finished at: Thu May 12 10:58:43 CEST 2016
> [INFO] Final Memory: 17M/850M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 47455: AMBARI-16702 Zeppelin cluster deployment fails due to unavailability of zeppelin service check script

2016-05-17 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47455/#review133544
---


Ship it!




Ship It!

- Sumit Mohanty


On May 17, 2016, 1:50 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47455/
> ---
> 
> (Updated May 17, 2016, 1:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Rohit 
> Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16702
> https://issues.apache.org/jira/browse/AMBARI-16702
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - add service check script for zeppelin
> 
> error log in jira issue
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  866d746 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/service_check.py
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47455/diff/
> 
> 
> Testing
> ---
> 
> manually tested on centos 6.4, Ambari 2.4.0.0
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services

2016-05-17 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133533
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 743)


nit: can use newer collections syntax without the generic specifics: 
List list = new ArrayList<>();



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 750)


nit: can use newer collections syntax without the generic specifics: 
Map map = new HashMap<>();



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 764)


could use a MultiMap?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(lines 765 - 766)


An Entry iterator is preferred



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(lines 822 - 831)


Visitor or abstract method?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(lines 836 - 837)


Need an iterator directly?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(lines 890 - 892)


Really need to throw here or just log it and continue; ?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 934)


Some javadoc would be nice



ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 112 - 113)


With no other exceptions, can just use "regular" logging syntax:  
LOG.debug("Service upgrades folder {} for service {} in stack {} is empty.", 
absUpgradesDir, serviceDir.getName(), stackId)



ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 116 - 117)


Maybe LOG.info() here.  If nothing happens we'll have no record unless 
DEBUG is enabled (which is, admittedly, an awful idea)



ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml (line 
166)


Changing this name may (or may not) impact UI.  The group name is not 
guaranteed unique, and some additional information may be displayed based on 
it's name (unconfirmed, but that's the intent).  Also, make consistent using an 
'_': SERVICE_CHECK_1


- Nate Cole


On May 16, 2016, 2:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 2:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java

Re: Review Request 47456: Takeover config merge should handle AMS hbase configs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47456/
---

(Updated May 17, 2016, 2:16 p.m.)


Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16703
https://issues.apache.org/jira/browse/AMBARI-16703


Repository: ambari


Description
---

hbase-site conflicts between cluster and AMS.

Hard code the config path as /etc/ams-hbase/conf to be AMS HBase configs.


Diffs (updated)
-

  ambari-server/src/main/resources/scripts/takeover_config_merge.py f940f27 
  ambari-server/src/main/resources/scripts/takeover_files_mapping.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/47456/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 47457: Hive Service check is failed after Upgrade to 2.4.0.0 :[ core.exceptions.Fail / "error":"No user found]

2016-05-17 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47457/#review133540
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Травень 17, 2016, 1:53 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47457/
> ---
> 
> (Updated Травень 17, 2016, 1:53 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16705
> https://issues.apache.org/jira/browse/AMBARI-16705
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:  
> 1) Install old version (Add ranger)  
> 2) Make ambari only upgrade  
> 3) Run Hive service check
> 
> All logs:  
>  
> Cluster:
> 
> Actual result:  
> Hive Service check is failed after Upgrade to 2.4.0.0 :[ core.exceptions.Fail
> / "error":"No user found]
> 
> 
> 
> 
> stderr:   /var/lib/ambari-agent/data/errors-484.txt
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
>  line 167, in 
> HiveServiceCheck().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 254, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
>  line 95, in service_check
> webhcat_service_check()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py",
>  line 125, in webhcat_service_check
> logoutput=True)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/var/lib/ambari-agent/tmp/templetonSmoke.sh 
> os-s11-3-swuvps-cst-upg-smk-212-4.openstacklocal cstm-smoke 50111 
> idtest.cstm-smoke.1463370007.79.pig no_keytab false kinit no_principal 
> /var/lib/ambari-agent/tmp' returned 1. Templeton Smoke Test (ddl cmd): 
> Failed. : {"error":"No user found.  Missing user.name parameter."}http_code 
> <401>
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
>  3c4a245 
> 
> Diff: https://reviews.apache.org/r/47457/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 47457: Hive Service check is failed after Upgrade to 2.4.0.0 :[ core.exceptions.Fail / "error":"No user found]


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47457/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16705
https://issues.apache.org/jira/browse/AMBARI-16705


Repository: ambari


Description
---

STR:  
1) Install old version (Add ranger)  
2) Make ambari only upgrade  
3) Run Hive service check

All logs:   
Cluster:

Actual result:  
Hive Service check is failed after Upgrade to 2.4.0.0 :[ core.exceptions.Fail
/ "error":"No user found]




stderr:   /var/lib/ambari-agent/data/errors-484.txt

Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
 line 167, in 
HiveServiceCheck().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 254, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
 line 95, in service_check
webhcat_service_check()
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py",
 line 125, in webhcat_service_check
logoutput=True)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 
'/var/lib/ambari-agent/tmp/templetonSmoke.sh 
os-s11-3-swuvps-cst-upg-smk-212-4.openstacklocal cstm-smoke 50111 
idtest.cstm-smoke.1463370007.79.pig no_keytab false kinit no_principal 
/var/lib/ambari-agent/tmp' returned 1. Templeton Smoke Test (ddl cmd): Failed. 
: {"error":"No user found.  Missing user.name parameter."}http_code <401>


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
 3c4a245 

Diff: https://reviews.apache.org/r/47457/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 47306: Ambari Admin Privilege required for Pig and Hive View. 403 error received when opening Pig View by a non-admin user

2016-05-17 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47306/#review133539
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On May 12, 2016, 1:29 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47306/
> ---
> 
> (Updated May 12, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16634
> https://issues.apache.org/jira/browse/AMBARI-16634
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Extended view Cluster interface to return hosts for service component.
> Used this method for geting the hosts for hive server, webhcat. Instead of 
> using Ambari Api in attached mode.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java 
> 529e09a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java
>  f661844 
>   
> ambari-server/src/test/java/org/apache/ambari/server/view/ClusterImplTest.java
>  daf87ec 
>   
> ambari-server/src/test/java/org/apache/ambari/server/view/RemoteAmbariClusterTest.java
>  ce8fe7d 
>   ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java 
> f1b8177 
>   
> contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
>  718c5a5 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
>  1442748 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/AmbariApi.java
>  6072d28 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/Services.java
>  1bace94 
> 
> Diff: https://reviews.apache.org/r/47306/diff/
> 
> 
> Testing
> ---
> 
> Manual Testing.
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 47449: All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in ambari server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47449/
---

(Updated May 17, 2016, 1:56 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-16700
https://issues.apache.org/jira/browse/AMBARI-16700


Repository: ambari


Description (updated)
---

All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in
ambari server log

from Ambari server log




23 Apr 2016 23:16:46,292 ERROR [pool-7-thread-1] BaseProvider:244 - Caught 
exception getting JMX metrics : Connection refused, skipping same exceptions 
for next 5 minutes
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at 
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at 
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at 
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:184)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:182)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

This blocked all Hbase-HA runs deployment


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 55e49d9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 0aa0a1e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
 fc76b1e 

Diff: https://reviews.apache.org/r/47449/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 47456: Takeover config merge should handle AMS hbase configs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47456/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16703
https://issues.apache.org/jira/browse/AMBARI-16703


Repository: ambari


Description
---

hbase-site conflicts between cluster and AMS.

Hard code the config path as /etc/ams-hbase/conf to be AMS HBase configs.


Diffs
-

  ambari-server/src/main/resources/scripts/takeover_config_merge.py f940f27 

Diff: https://reviews.apache.org/r/47456/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 47455: AMBARI-16702 Zeppelin cluster deployment fails due to unavailability of zeppelin service check script

2016-05-17 Thread Renjith Kamath


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47455/
---

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Rohit Choudhary, 
and Sumit Mohanty.


Bugs: AMBARI-16702
https://issues.apache.org/jira/browse/AMBARI-16702


Repository: ambari


Description
---

- add service check script for zeppelin

error log in jira issue


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
 866d746 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/service_check.py
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47455/diff/


Testing
---

manually tested on centos 6.4, Ambari 2.4.0.0


Thanks,

Renjith Kamath

Re: Review Request 45810: zeppelin_log_dir change leads to fail for different Zeppelin Notebook service actions

2016-05-17 Thread Renjith Kamath


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45810/#review133536
---


Ship it!




- Renjith Kamath


On April 6, 2016, 1:10 p.m., Sagar Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45810/
> ---
> 
> (Updated April 6, 2016, 1:10 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Rohit Choudhary, Renjith 
> Kamath, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15732
> https://issues.apache.org/jira/browse/AMBARI-15732
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change zeppelin_log_dir to any new value (/var/log/zeppelin1). Save changes. 
> Try to stop/restart service. You will get an error.
> Error log in https://issues.apache.org/jira/browse/AMBARI-15732
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  c24b3fa 
> 
> Diff: https://reviews.apache.org/r/45810/diff/
> 
> 
> Testing
> ---
> 
> Tested on Centos 6.4
> 
> 
> Thanks,
> 
> Sagar Kulkarni
> 
>

Re: Review Request 47452: Do not use implicit routing by default in solr-client


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/
---

(Updated May 17, 2016, 1:29 p.m.)


Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, and 
Sumit Mohanty.


Changes
---

fixed compile error


Bugs: AMBARI-16701
https://issues.apache.org/jira/browse/AMBARI-16701


Repository: ambari


Description
---

By default ambari should not use implicit routing for creating collections


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 d661288 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
 21e7fee 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
 c888269 

Diff: https://reviews.apache.org/r/47452/diff/


Testing
---

ambari logsearch solr-client tests:
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Oliver Szabo

Re: Review Request 47427: Atlas Server script error during upgrade.

2016-05-17 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47427/#review133528
---


Ship it!




Ship It!

- Robert Levas


On May 16, 2016, 5:53 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47427/
> ---
> 
> (Updated May 16, 2016, 5:53 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Robert Levas.
> 
> 
> Bugs: AMBARI-16693
> https://issues.apache.org/jira/browse/AMBARI-16693
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The error 'kafka_broker_hosts' was not found in configurations dictionary! is 
> seen during upgrade ...
> 
> "stderr" : "Traceback (most recent call last):\n  File 
> \"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py\",
>  line 165, in \nMetadataServer().execute()\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
>  line 254, in execute\nmethod(env)\n  File 
> \"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py\",
>  line 82, in stop\nimport params\n  File 
> \"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py\",
>  line 139, in \nif not len(kafka_broker_hosts) == 0:\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py\",
>  line 73, in __getattr__\nraise Fail(\"Configuration parameter '\" + 
> self.name + \"' was not found in configurations 
> dictionary!\")\nresource_management.core.exceptions.Fail: Configuration 
> parameter 'kafka_broker_hosts' was not found in c
 onfigurations dictionary!",
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  fb4a55f 
> 
> Diff: https://reviews.apache.org/r/47427/diff/
> 
> 
> Testing
> ---
> 
> manual test install
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>

Review Request 47452: Do not use implicit routing by default in solr-client


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/
---

Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, and 
Sumit Mohanty.


Bugs: AMBARI-16701
https://issues.apache.org/jira/browse/AMBARI-16701


Repository: ambari


Description
---

By default ambari should not use implicit routing for creating collections


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 d661288 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
 21e7fee 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
 c888269 

Diff: https://reviews.apache.org/r/47452/diff/


Testing
---

ambari logsearch solr-client tests:
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Oliver Szabo

Re: Review Request 47397: Consistent logsearch/logfeeder property names


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47397/#review133526
---


Ship it!




Ship It!

- Oliver Szabo


On May 17, 2016, 12:28 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47397/
> ---
> 
> (Updated May 17, 2016, 12:28 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16676
> https://issues.apache.org/jira/browse/AMBARI-16676
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the stack definition logsearch/logfeeder.properties looks like this: 
> "logsearch."/ "logfeeder."
> Use the same names in logsearch module too
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
>  f4ca51b 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
>  8cc5b6a 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
> 1c5ee8d 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  91f4ffb 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
>  d0aea47 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  42d836c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  d144172 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  b5c042d 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
>  b5efd24 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
>  411b954 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigMgr.java
>  7a6e3e9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
>  2f47ec5 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrMetricsLoader.java
>  0bd66d2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/ConfigUtil.java
>  36dcc96 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties
>  8400cad 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  fa8451c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
>  82457b7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8bc82e9 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
>  e34b267 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  c6d73ea 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  4d248b4 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/output.config.json.j2
>  7d0319d 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json 63d16e3 
> 
> Diff: https://reviews.apache.org/r/47397/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> ambari-logsearch-logfeeder
> Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> ambari-server (python only)
> Total run:1014
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 47397: Consistent logsearch/logfeeder property names

2016-05-17 Thread Miklos Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47397/
---

(Updated May 17, 2016, 12:28 p.m.)


Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Changes
---

Updated default property names too to follow the convention. Also added a small 
fix for ambari server logs - in some log files the milliseconds are separated 
with a '.', not a ','


Bugs: AMBARI-16676
https://issues.apache.org/jira/browse/AMBARI-16676


Repository: ambari


Description
---

In the stack definition logsearch/logfeeder.properties looks like this: 
"logsearch."/ "logfeeder."
Use the same names in logsearch module too


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
 f4ca51b 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 8cc5b6a 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
1c5ee8d 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
 91f4ffb 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
 d0aea47 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
 42d836c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
 d144172 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 b5c042d 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
 b5efd24 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
 411b954 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigMgr.java
 7a6e3e9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
 2f47ec5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrMetricsLoader.java
 0bd66d2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/ConfigUtil.java
 36dcc96 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
8400cad 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
 fa8451c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
 82457b7 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 8bc82e9 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
 e34b267 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
 c6d73ea 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
 4d248b4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/output.config.json.j2
 7d0319d 
  ambari-server/src/test/python/stacks/2.4/configs/default.json 63d16e3 

Diff: https://reviews.apache.org/r/47397/diff/


Testing
---

Tested on local cluster.

ambari-logsearch-logfeeder
Tests run: 35, Failures: 0, Errors: 0, Skipped: 0


ambari-server (python only)
Total run:1014
Total errors:0
Total failures:0


Thanks,

Miklos Gergely

Re: Review Request 47449: All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in ambari server log

2016-05-17 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47449/#review133525
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On May 17, 2016, 1:24 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47449/
> ---
> 
> (Updated May 17, 2016, 1:24 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-16700
> https://issues.apache.org/jira/browse/AMBARI-16700
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in
> ambari server log
> 
> from Ambari server log
> 
> 
> 
> 
> 23 Apr 2016 23:16:46,292 ERROR [pool-7-thread-1] BaseProvider:244 - 
> Caught exception getting JMX metrics : Connection refused, skipping same 
> exceptions for next 5 minutes
> java.net.ConnectException: Connection refused
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>   at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>   at 
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>   at java.net.Socket.connect(Socket.java:589)
>   at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
>   at sun.net.www.http.HttpClient.(HttpClient.java:211)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:308)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:326)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
>   at 
> java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
>   at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
>   at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
>   at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
>   at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:184)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:182)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> 
> 
>  -ruobns-hbase-ha-2/artifacts/screenshots/com.hw.ambari.ui.tests.installer.Inst
> allHadoop/install/_23_23_38_5_Element_has_not_been_found_within_60_seconds__/t
> est_logs/>
> 
> This blocked all Hbase-HA runs deployment
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  55e49d9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
>  0aa0a1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
>  fc76b1e 
> 
> Diff: https://reviews.apache.org/r/47449/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47446: Logsearch color codes works only for upper case log levels.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47446/#review133524
---


Ship it!




Ship It!

- Oliver Szabo


On May 17, 2016, 8:35 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47446/
> ---
> 
> (Updated May 17, 2016, 8:35 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16698
> https://issues.apache.org/jira/browse/AMBARI-16698
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains:
> Fix for log level color code doesn't show up, if log levels are not in upper 
> case.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/ViewUtils.js
>  ba21b65 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/ComparisonView.js
>  8ad0c4b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/LogFileView.js
>  2d19d78 
> 
> Diff: https://reviews.apache.org/r/47446/diff/
> 
> 
> Testing
> ---
> 
> Setup logsearch on 3 node cluster and tested the above features.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:41 a.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 685
> > 
> >
> > UGM?

This was just for testing purposes.  I'll remove the unnecessary logging.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133501
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 47417: Replace '*' to hdp version in lzo packages

2016-05-17 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47417/
---

(Updated Травень 17, 2016, 10:24 до полудня)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sumit 
Mohanty.


Bugs: AMBARI-16683
https://issues.apache.org/jira/browse/AMBARI-16683


Repository: ambari


Description
---

As i understood(according to discussion with Andrew O) we had some jira for '*' 
to hdp version replacement in all packages. But here it was not replaced:
{code}
def get_lzo_packages(stack_version_unformatted):
  lzo_packages = []
 
  if OSCheck.is_redhat_family() or OSCheck.is_suse_family():
lzo_packages += ["lzo", "hadoop-lzo-native"]
  elif OSCheck.is_ubuntu_family():
lzo_packages += ["liblzo2-2"]

  if stack_version_unformatted and 
check_stack_feature(StackFeature.ROLLING_UPGRADE, stack_version_unformatted):
lzo_packages += ["hadooplzo_*"]
  else:
lzo_packages += ["hadoop-lzo"]

  return lzo_packages
{code}


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
 e189d62 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
 fe488c3 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 784da9c 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
 75924c5 
  ambari-server/src/test/python/stacks/2.0.6/configs/client-upgrade.json 
47ab0a3 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json f0e5208 
  ambari-server/src/test/python/stacks/2.0.6/configs/nn_ru_lzo.json d44b002 
  ambari-server/src/test/python/stacks/2.0.6/configs/ranger-namenode-start.json 
b163a61 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 6c7fe18 
  ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py c36b428 
  
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
 7b07d87 
  ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
cf84bb7 
  ambari-server/src/test/python/stacks/2.2/configs/oozie-downgrade.json 7e5346c 
  ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 1c75d65 

Diff: https://reviews.apache.org/r/47417/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Review Request 47449: All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in ambari server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47449/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-16700
https://issues.apache.org/jira/browse/AMBARI-16700


Repository: ambari


Description
---

All HBase-HA deployment fail with 'Caught exception getting JMX metrics' in
ambari server log

from Ambari server log




23 Apr 2016 23:16:46,292 ERROR [pool-7-thread-1] BaseProvider:244 - Caught 
exception getting JMX metrics : Connection refused, skipping same exceptions 
for next 5 minutes
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at 
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at 
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at 
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:184)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:182)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)




This blocked all Hbase-HA runs deployment


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 55e49d9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 0aa0a1e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
 fc76b1e 

Diff: https://reviews.apache.org/r/47449/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 46962: Hive View and Pig View : one user overriding job details of other user in database

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46962/
---

(Updated May 17, 2016, 10:10 a.m.)

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar,
Rohit Choudhary, and Ashwin Rajeev.

Changes
---

did the correction suggested in the code reviews.

Bugs: AMBARI-16242
https://issues.apache.org/jira/browse/AMBARI-16242

Repository: ambari

Description
---

Earlier :
the id for dynamic entity was created in the application with the use of
instance data for that view. This data is dependent on users and created
separate sequences for each user. So the id for one user was also generated for
other user. Hence the data in db of one user was getting overriden by other
user.

In this patch :
Now the id is generated using table_sequence techique of JPA. table for
sequence is ambari_sequence. UpgradeCatalog240.java will add the current
sequence number for existing tables and for new table the sequence name will be
added automatically. As sequence size of 50 is provided for better performances

Diffs (updated)
-

ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
1f3b1d3

ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java
0ed260d

contrib/views/hive/src/main/java/org/apache/ambari/view/hive/persistence/DataStoreStorage.java
1e8f07f

contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/jobs/JobService.java
f7f883b

contrib/views/pig/src/main/java/org/apache/ambari/view/pig/persistence/DataStoreStorage.java
7ae7721

Diff: https://reviews.apache.org/r/46962/diff/

Testing
---

Following manual testing has been done.
1. upgrade script creates correct current sequences. Done by manually upgrading
old ambari with existing views. Next job will allocate 50 more ids
2. for new tables the entry in ambari_sequence is automatically added and the
ids will start from 1.

Thanks,

Nitiraj Rathore

Re: Review Request 46962: Hive View and Pig View : one user overriding job details of other user in database



> On May 16, 2016, 4:48 p.m., Ajit Kumar wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 320
> > 
> >
> > Is maxId == 0 valid?

yes it is valid. the id of first entity will be 1. verified.


- Nitiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46962/#review133376
---


On May 16, 2016, 1:59 p.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46962/
> ---
> 
> (Updated May 16, 2016, 1:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16242
> https://issues.apache.org/jira/browse/AMBARI-16242
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier : 
> the id for dynamic entity was created in the application with the use of 
> instance data for that view. This data is dependent on users and created 
> separate sequences for each user. So the id for one user was also generated 
> for other user. Hence the data in db of one user was getting overriden by 
> other user. 
> 
> In this patch : 
> Now the id is generated using table_sequence techique of JPA. table for 
> sequence is ambari_sequence. UpgradeCatalog240.java will add the current 
> sequence number for existing tables and for new table the sequence name will 
> be added automatically. As sequence size of 50 is provided for better 
> performances
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  dc8d9b7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java
>  0ed260d 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/persistence/DataStoreStorage.java
>  1e8f07f 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/jobs/JobService.java
>  f7f883b 
>   
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/persistence/DataStoreStorage.java
>  7ae7721 
> 
> Diff: https://reviews.apache.org/r/46962/diff/
> 
> 
> Testing
> ---
> 
> Following manual testing has been done.
> 1. upgrade script creates correct current sequences. Done by manually 
> upgrading old ambari with existing views. Next job will allocate 50 more ids
> 2. for new tables the entry in ambari_sequence is automatically added and the 
> ids will start from 1.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Re: Review Request 47421: Configuration Tasks Are Being Skipped During Upgrade

2016-05-17 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47421/#review133516
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On May 16, 2016, 9:50 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47421/
> ---
> 
> (Updated May 16, 2016, 9:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-16687
> https://issues.apache.org/jira/browse/AMBARI-16687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade from HDP 2.x to 2.y, all of the configuration tasks are 
> being shown as skipped. This is due to AMBARI-15222 where the configuration 
> packs were being calculated incorrectly resulting in empty maps.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  b069862 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
>  f2e00fe 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml
>  564b5bd 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml 
> 257893f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
>  1fa4d00 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> bf041de 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 45e2f94 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  7873853 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> f71ef1a 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> a7dbba3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  2e5c002 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
>  771f830 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradeValidityTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> cb034d0 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 623b45c 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_partial.xml
>  4932e92 
> 
> Diff: https://reviews.apache.org/r/47421/diff/
> 
> 
> Testing
> ---
> 
> Wrote a new unit test which validates all upgrade XML files which have 
> configuration tasks.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 47446: Logsearch color codes works only for upper case log levels.

2016-05-17 Thread Dharmesh Makwana


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47446/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16698
https://issues.apache.org/jira/browse/AMBARI-16698


Repository: ambari


Description
---

Patch contains:
Fix for log level color code doesn't show up, if log levels are not in upper 
case.


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/ViewUtils.js
 ba21b65 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/ComparisonView.js
 8ad0c4b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/LogFileView.js
 2d19d78 

Diff: https://reviews.apache.org/r/47446/diff/


Testing
---

Setup logsearch on 3 node cluster and tested the above features.


Thanks,

Dharmesh Makwana

Re: Review Request 47300: Included few more steps in "Take a tour" feature and few more enhancements

2016-05-17 Thread Don Bosco Durai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47300/#review133506
---


Ship it!




Tested creating 3 node cluster after applying this.

- Don Bosco Durai


On May 12, 2016, 11:06 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47300/
> ---
> 
> (Updated May 12, 2016, 11:06 a.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Jaimin Jetly, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16632
> https://issues.apache.org/jira/browse/AMBARI-16632
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains
> 1) Added few steps more in "Take a tour" option.
> 2) When url contains "host_name" and "component_name" params than separate 
> tab should be open for that component.
> 3) Comparison tab clean-up, default pageSize changed to 25 and css changes.
> 4) In histogram graph, time gap is shown above the graph.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Intro.js
>  6abccf6 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Utils.js
>  5d55689 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/ViewUtils.js
>  ba21b65 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/common/DatePickerLayout.js
>  0849820 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
>  09bbacc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
>  805b507 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/ComparisonView.js
>  8ad0c4b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
>  cb4228c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/LogFileView.js
>  2d19d78 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/troubleshoot/TroubleShootLayoutView.js
>  e1bc90a 
>   ambari-logsearch/ambari-logsearch-portal/src/main/webapp/styles/style.css 
> 05fd491 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/audit/AuditTabLayoutView_tmpl.html
>  2f0a596 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/graphs/GraphLayoutView_tmpl.html
>  fb4ee3c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
>  4564562 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/troubleshoot/TroubleShootLayoutView_tmpl.html
>  b3eeaf5 
> 
> Diff: https://reviews.apache.org/r/47300/diff/
> 
> 
> Testing
> ---
> 
> Tested in local Ambari environment.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 47297: Extend logging for ActionQueue's retry logic

2016-05-17 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47297/
---

(Updated máj. 17, 2016, 7:31 de)


Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Oliver Szabo, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Changes
---

TaskId is added to the log message


Bugs: AMBARI-16630
https://issues.apache.org/jira/browse/AMBARI-16630


Repository: ambari


Description
---

At info level, commands should be tracked if they are retried on failure or 
not, and if not, what is the reason for that.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 85389f9 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 7d0efa2 

Diff: https://reviews.apache.org/r/47297/diff/


Testing
---

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:19.911s
[INFO] Finished at: Thu May 12 10:58:43 CEST 2016
[INFO] Final Memory: 17M/850M
[INFO] 


Thanks,

Daniel Gergely

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services



> On May 17, 2016, 6:33 a.m., Jayush Luniya wrote:
> > Can you add unit test coverage?
> 
> Jayush Luniya wrote:
> We should have unit tests in particular to validate incorrectly authored 
> service upgrade packs. What happens if we add a circular dependency (example: 
> KAFKA is marked with KNOX and KNOX is marked with 
> KAFKA)

Also we should document any unsupported use cases.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review133499
---


On May 16, 2016, 6:50 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 16, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1cd2ffa 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 9c6a02d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 1745de8 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
> Testing
> ---
> 
> Manual testing so far.  I have the code read the upgrade xml and all of its 
> service specific xml files, built the upgrade pack and then write the full 
> upgrade xml to disk and then compare the results to the original upgrade xml.
> 
> This review is mostly for the design doc which is attached to the JIRA.  Not 
> sure how to create a review board with a design doc instead of a patch file.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services