Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Mugdha Varadkar]

> On Sept. 14, 2016, 5:58 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py,
> >  line 694
> > 
> >
> > Why using stack_supports_ranger_kerberos? It sounds like a workaround 
> > for >= HDP-2.5+.
> 
> Mugdha Varadkar wrote:
> Yes, using stack_supports_ranger_kerberos feature here only for stack 2.5 
> onwards, since this feature is also introduced from stack 2.5
> Do you want me to add new feature here ?

Added feature in updated patch


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/#review148939
---


On Sept. 15, 2016, 5:26 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51890/
> ---
> 
> (Updated Sept. 15, 2016, 5:26 a.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18386
> https://issues.apache.org/jira/browse/AMBARI-18386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
> right jdbc url for repo configs that are set for creating a repository in 
> Ranger for Secure and simple install of hive. 
> However in HiverServer2 HA env, jdbc url needs to change in repo config with 
> zk url.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  169fbff 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  5e3e233 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8c838db 
> 
> Diff: https://reviews.apache.org/r/51890/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/
---

(Updated Sept. 15, 2016, 5:26 a.m.)


Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
Velmurugan Periasamy.


Changes
---

Added stack feature in updated patch

Ran 268 tests in 6.913s

OK
--
Total run:1126
Total errors:0
Total failures:0


Bugs: AMBARI-18386
https://issues.apache.org/jira/browse/AMBARI-18386


Repository: ambari


Description
---

>From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
>right jdbc url for repo configs that are set for creating a repository in 
>Ranger for Secure and simple install of hive. 
However in HiverServer2 HA env, jdbc url needs to change in repo config with zk 
url.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 169fbff 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 5e3e233 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 8c838db 

Diff: https://reviews.apache.org/r/51890/diff/


Testing
---

Tested Ranger hive plugin with Ranger install on centos6


Thanks,

Mugdha Varadkar

Re: Review Request 51903: [PREVIEW] Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez]

> On Sept. 15, 2016, 2:49 a.m., Jonathan Hurley wrote:
> > I think we can do better. Why not just use `os.path.exists` to check the 
> > `current_dir` structure. In the case of atlas, the psuedo code woudl read:
> > 
> > if "/etc/atlas/conf" is a directory and if "/usr/hdp/current/atlas-client" 
> > is a valid link
> >   then seed stuff
> >   
> > Basically we just want to seed IFF both the `conf_dir` is a phyiscal 
> > directory and the `current_dir` is valid (indicates installed)

The seeding happens when we install the new HDP 2.5 bits and before Atlas RPM 
is installed, so /etc/atlas/conf will not exist and 
/usr/hdp/current/atlas-client will not be a valid link (still points to 
/usr/hdp/2.4/atlas-client which DNE) .
Line 361 is needed because we cannot create symlink /etc/atlas/conf -> 
/usr/hdp/current/atlas-client/conf since the source will be needed once the 
Atlas RPM is installed.

Line 618 is needed because Atlas RPM brings in artifacts not known to Ambari 
that are first stored in /etc/atlas/conf, which then gets copied to 
/etc/atlas/conf.backup because we will need to make /etc/atlas/conf be a 
symlink to /usr/hdp/current/atlas-client/conf


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51903/#review149019
---


On Sept. 15, 2016, 1:57 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51903/
> ---
> 
> (Updated Sept. 15, 2016, 1:57 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18368
> https://issues.apache.org/jira/browse/AMBARI-18368
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> 
> * Install Ambari 2.2.2 with HDP 2.4 with HBase, Solr, and Hive (this is 
> important)
> * Perform EU/RU to HDP 2.5 
> * Add Atlas Service
> 
> Atlas Server log contains,
> 
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index  
> 
> Fix:
> The Hive RPM installs /usr/$stack/$version/atlas with some partial packages 
> that contain Hive hooks, while the Atlas RPM is responsible for installing 
> the full content.
> If the user does not have Atlas currently installed on their stack, then 
> /usr/$stack/current/atlas-client will be a broken symlink, and we should not 
> create the symlink /etc/atlas/conf -> /usr/$stack/current/atlas-client/conf .
> If we mistakenly create this symlink, then when the user performs an EU/RU 
> and then adds Atlas service then the Atlas RPM will not be able to copy its 
> artifacts into /etc/atlas/conf directory and therefore prevent Ambari from by 
> copying those unmanaged contents into /etc/atlas/$version/0
> 
> Further, when installing Atlas service, we must copy the artifacts from 
> /etc/atlas/conf.backup/* to /etc/atlas/conf (which is now a symlink to 
> /usr/hdp/current/atlas-client/conf/) with the no-clobber flag.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c60b324 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  06a366e 
> 
> Diff: https://reviews.apache.org/r/51903/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> 
> 
> Need to perform more tests on a live cluster.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 51903: [PREVIEW] Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51903/#review149019
---



I think we can do better. Why not just use `os.path.exists` to check the 
`current_dir` structure. In the case of atlas, the psuedo code woudl read:

if "/etc/atlas/conf" is a directory and if "/usr/hdp/current/atlas-client" is a 
valid link
  then seed stuff
  
Basically we just want to seed IFF both the `conf_dir` is a phyiscal directory 
and the `current_dir` is valid (indicates installed)

- Jonathan Hurley


On Sept. 14, 2016, 9:57 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51903/
> ---
> 
> (Updated Sept. 14, 2016, 9:57 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18368
> https://issues.apache.org/jira/browse/AMBARI-18368
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> 
> * Install Ambari 2.2.2 with HDP 2.4 with HBase, Solr, and Hive (this is 
> important)
> * Perform EU/RU to HDP 2.5 
> * Add Atlas Service
> 
> Atlas Server log contains,
> 
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index  
> 
> Fix:
> The Hive RPM installs /usr/$stack/$version/atlas with some partial packages 
> that contain Hive hooks, while the Atlas RPM is responsible for installing 
> the full content.
> If the user does not have Atlas currently installed on their stack, then 
> /usr/$stack/current/atlas-client will be a broken symlink, and we should not 
> create the symlink /etc/atlas/conf -> /usr/$stack/current/atlas-client/conf .
> If we mistakenly create this symlink, then when the user performs an EU/RU 
> and then adds Atlas service then the Atlas RPM will not be able to copy its 
> artifacts into /etc/atlas/conf directory and therefore prevent Ambari from by 
> copying those unmanaged contents into /etc/atlas/$version/0
> 
> Further, when installing Atlas service, we must copy the artifacts from 
> /etc/atlas/conf.backup/* to /etc/atlas/conf (which is now a symlink to 
> /usr/hdp/current/atlas-client/conf/) with the no-clobber flag.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c60b324 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  06a366e 
> 
> Diff: https://reviews.apache.org/r/51903/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> 
> 
> Need to perform more tests on a live cluster.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 51903: [PREVIEW] Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51903/#review149018
---




ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 (line 346)


Any concern if ATLAS was installed before the upgrade. In that case, 
/etc/atlas/conf is probably pointing to a valid folder. If we have not seen 
this scenario fail in our tests then my comment can be ignored.


- Sumit Mohanty


On Sept. 15, 2016, 1:57 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51903/
> ---
> 
> (Updated Sept. 15, 2016, 1:57 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18368
> https://issues.apache.org/jira/browse/AMBARI-18368
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> 
> * Install Ambari 2.2.2 with HDP 2.4 with HBase, Solr, and Hive (this is 
> important)
> * Perform EU/RU to HDP 2.5 
> * Add Atlas Service
> 
> Atlas Server log contains,
> 
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index  
> 
> Fix:
> The Hive RPM installs /usr/$stack/$version/atlas with some partial packages 
> that contain Hive hooks, while the Atlas RPM is responsible for installing 
> the full content.
> If the user does not have Atlas currently installed on their stack, then 
> /usr/$stack/current/atlas-client will be a broken symlink, and we should not 
> create the symlink /etc/atlas/conf -> /usr/$stack/current/atlas-client/conf .
> If we mistakenly create this symlink, then when the user performs an EU/RU 
> and then adds Atlas service then the Atlas RPM will not be able to copy its 
> artifacts into /etc/atlas/conf directory and therefore prevent Ambari from by 
> copying those unmanaged contents into /etc/atlas/$version/0
> 
> Further, when installing Atlas service, we must copy the artifacts from 
> /etc/atlas/conf.backup/* to /etc/atlas/conf (which is now a symlink to 
> /usr/hdp/current/atlas-client/conf/) with the no-clobber flag.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c60b324 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  06a366e 
> 
> Diff: https://reviews.apache.org/r/51903/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> 
> 
> Need to perform more tests on a live cluster.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 51903: [PREVIEW] Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51903/
---

Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
Hurley, and Nate Cole.


Bugs: AMBARI-18368
https://issues.apache.org/jira/browse/AMBARI-18368


Repository: ambari


Description
---

Steps to Reproduce:

* Install Ambari 2.2.2 with HDP 2.4 with HBase, Solr, and Hive (this is 
important)
* Perform EU/RU to HDP 2.5 
* Add Atlas Service

Atlas Server log contains,

Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not find 
the specified config set: vertex_index  

Fix:
The Hive RPM installs /usr/$stack/$version/atlas with some partial packages 
that contain Hive hooks, while the Atlas RPM is responsible for installing the 
full content.
If the user does not have Atlas currently installed on their stack, then 
/usr/$stack/current/atlas-client will be a broken symlink, and we should not 
create the symlink /etc/atlas/conf -> /usr/$stack/current/atlas-client/conf .
If we mistakenly create this symlink, then when the user performs an EU/RU and 
then adds Atlas service then the Atlas RPM will not be able to copy its 
artifacts into /etc/atlas/conf directory and therefore prevent Ambari from by 
copying those unmanaged contents into /etc/atlas/$version/0

Further, when installing Atlas service, we must copy the artifacts from 
/etc/atlas/conf.backup/* to /etc/atlas/conf (which is now a symlink to 
/usr/hdp/current/atlas-client/conf/) with the no-clobber flag.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 c60b324 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 06a366e 

Diff: https://reviews.apache.org/r/51903/diff/


Testing
---

--
Total run:1125
Total errors:0
Total failures:0
OK


Need to perform more tests on a live cluster.


Thanks,

Alejandro Fernandez

Re: Review Request 51901: AMBARI-18393. Hive Server Interactive (HSI) fails to start with 'Permission denied' for User Hive, if HSI starts before HS2.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51901/#review148982
---


Ship it!




Ship It!

- Sumit Mohanty


On Sept. 14, 2016, 10:23 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51901/
> ---
> 
> (Updated Sept. 14, 2016, 10:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18393
> https://issues.apache.org/jira/browse/AMBARI-18393
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:**
> 
> - Install Cluster using Blueprint including HiveServerInteractive
> - Start services fail at Hive interactive start, with below error:
> 
> ==
> --
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
>  line 535, in 
> HiveServerInteractive().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
>  line 115, in start
> self.setup_security()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
>  line 335, in setup_security
> Execute(slider_keytab_install_cmd, user=params.hive_user)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 71, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'slider install-keytab 
> --keytab /etc/security/keytabs/hive.llap.zk.sm.keytab --folder hive 
> --overwrite' returned 56. 2016-08-15 23:47:57,518 [main] INFO  
> tools.SliderUtils - JVM initialized into secure mode with kerberos realm 
> HWQE.HORTONWORKS.COM
> 2016-08-15 23:47:59,108 [main] INFO  impl.TimelineClientImpl - Timeline 
> service address: 
> http://nat-s11-4-lkws-stackdeploy-3.openstacklocal:8188/ws/v1/timeline/
> 2016-08-15 23:48:01,584 [main] WARN  shortcircuit.DomainSocketFactory - The 
> short-circuit local reads feature cannot be used because libhadoop cannot be 
> loaded.
> 2016-08-15 23:48:01,633 [main] INFO  client.RMProxy - Connecting to 
> ResourceManager at 
> nat-s11-4-lkws-stackdeploy-5.openstacklocal/172.22.71.181:8050
> 2016-08-15 23:48:01,983 [main] INFO  client.AHSProxy - Connecting to 
> Application History server at 
> nat-s11-4-lkws-stackdeploy-3.openstacklocal/172.22.71.168:10200
> 2016-08-15 23:48:03,297 [main] WARN  client.SliderClient - The 
> 'install-keytab' option has been deprecated.  Please use 'keytab --install'.
> 2016-08-15 23:48:03,440 [main] WARN  retry.RetryInvocationHandler - Exception 
> while invoking ClientNamenodeProtocolTranslatorPB.mkdirs over null. Not 
> retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
>  Permission denied: user=hive, access=WRITE, 
> inode="/user/hive/.slider/keytabs/hive":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> 

Re: Review Request 51901: AMBARI-18393. Hive Server Interactive (HSI) fails to start with 'Permission denied' for User Hive, if HSI starts before HS2.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51901/
---

(Updated Sept. 14, 2016, 10:23 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Updated testing Section.


Bugs: AMBARI-18393
https://issues.apache.org/jira/browse/AMBARI-18393


Repository: ambari


Description
---

**Issue:**

- Install Cluster using Blueprint including HiveServerInteractive
- Start services fail at Hive interactive start, with below error:

==
--
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 535, in 
HiveServerInteractive().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 115, in start
self.setup_security()
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 335, in setup_security
Execute(slider_keytab_install_cmd, user=params.hive_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 71, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 93, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 141, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 294, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'slider install-keytab 
--keytab /etc/security/keytabs/hive.llap.zk.sm.keytab --folder hive 
--overwrite' returned 56. 2016-08-15 23:47:57,518 [main] INFO  
tools.SliderUtils - JVM initialized into secure mode with kerberos realm 
HWQE.HORTONWORKS.COM
2016-08-15 23:47:59,108 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://nat-s11-4-lkws-stackdeploy-3.openstacklocal:8188/ws/v1/timeline/
2016-08-15 23:48:01,584 [main] WARN  shortcircuit.DomainSocketFactory - The 
short-circuit local reads feature cannot be used because libhadoop cannot be 
loaded.
2016-08-15 23:48:01,633 [main] INFO  client.RMProxy - Connecting to 
ResourceManager at 
nat-s11-4-lkws-stackdeploy-5.openstacklocal/172.22.71.181:8050
2016-08-15 23:48:01,983 [main] INFO  client.AHSProxy - Connecting to 
Application History server at 
nat-s11-4-lkws-stackdeploy-3.openstacklocal/172.22.71.168:10200
2016-08-15 23:48:03,297 [main] WARN  client.SliderClient - The 'install-keytab' 
option has been deprecated.  Please use 'keytab --install'.
2016-08-15 23:48:03,440 [main] WARN  retry.RetryInvocationHandler - Exception 
while invoking ClientNamenodeProtocolTranslatorPB.mkdirs over null. Not 
retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
 Permission denied: user=hive, access=WRITE, 
inode="/user/hive/.slider/keytabs/hive":hdfs:hdfs:drwxr-xr-x
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1811)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1794)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4011)
at 

Review Request 51901: AMBARI-18393. Hive Server Interactive (HSI) fails to start with 'Permission denied' for User Hive, if HSI starts before HS2.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51901/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-18393
https://issues.apache.org/jira/browse/AMBARI-18393


Repository: ambari


Description
---

**Issue:**

- Install Cluster using Blueprint including HiveServerInteractive
- Start services fail at Hive interactive start, with below error:

==
--
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 535, in 
HiveServerInteractive().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 115, in start
self.setup_security()
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py",
 line 335, in setup_security
Execute(slider_keytab_install_cmd, user=params.hive_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 71, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 93, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 141, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 294, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'slider install-keytab 
--keytab /etc/security/keytabs/hive.llap.zk.sm.keytab --folder hive 
--overwrite' returned 56. 2016-08-15 23:47:57,518 [main] INFO  
tools.SliderUtils - JVM initialized into secure mode with kerberos realm 
HWQE.HORTONWORKS.COM
2016-08-15 23:47:59,108 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://nat-s11-4-lkws-stackdeploy-3.openstacklocal:8188/ws/v1/timeline/
2016-08-15 23:48:01,584 [main] WARN  shortcircuit.DomainSocketFactory - The 
short-circuit local reads feature cannot be used because libhadoop cannot be 
loaded.
2016-08-15 23:48:01,633 [main] INFO  client.RMProxy - Connecting to 
ResourceManager at 
nat-s11-4-lkws-stackdeploy-5.openstacklocal/172.22.71.181:8050
2016-08-15 23:48:01,983 [main] INFO  client.AHSProxy - Connecting to 
Application History server at 
nat-s11-4-lkws-stackdeploy-3.openstacklocal/172.22.71.168:10200
2016-08-15 23:48:03,297 [main] WARN  client.SliderClient - The 'install-keytab' 
option has been deprecated.  Please use 'keytab --install'.
2016-08-15 23:48:03,440 [main] WARN  retry.RetryInvocationHandler - Exception 
while invoking ClientNamenodeProtocolTranslatorPB.mkdirs over null. Not 
retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
 Permission denied: user=hive, access=WRITE, 
inode="/user/hive/.slider/keytabs/hive":hdfs:hdfs:drwxr-xr-x
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1811)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1794)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4011)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1102)
 

Re: Review Request 51851: AMBARI-18332: Blueprints: API should make available "setting" property from blueprint

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51851/#review148974
---


Ship it!




Ship It!

- Sumit Mohanty


On Sept. 13, 2016, 6:28 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51851/
> ---
> 
> (Updated Sept. 13, 2016, 6:28 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18332
> https://issues.apache.org/jira/browse/AMBARI-18332
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18332: Blueprints: API should make available "setting" property from 
> blueprint
> 
> ** Issue: **
> A new section *setting* was added to blueprints to support auto-start. 
> However, when querying the blueprint that used to deploy a cluster, the 
> *setting* section is absent.
> 
> ** Fix: **
> Add the *setting* section to the blueprint response that is returned during a 
> http://:/api/v1/blueprints/
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
>  775827b4425a6668fd11d435ca7e4175d1f75d16 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java
>  75eff7e1d1d5b8db232f109eb08b6cb57ce0a105 
> 
> Diff: https://reviews.apache.org/r/51851/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [11.486s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.041s]
> [INFO] Ambari Web  SUCCESS [1:11.461s]
> [INFO] Ambari Views .. SUCCESS [1.161s]
> [INFO] Ambari Admin View . SUCCESS [8.439s]
> [INFO] ambari-metrics  SUCCESS [0.681s]
> [INFO] Ambari Metrics Common . SUCCESS [4.584s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.078s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.289s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.410s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [4.003s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.559s]
> [INFO] Ambari Metrics Collector .. SUCCESS [10.463s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.870s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.009s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:19.907s]
> [INFO] Ambari Server . SUCCESS [3:03.887s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.286s]
> [INFO] Ambari Agent .. SUCCESS [26.876s]
> [INFO] Ambari Client . SUCCESS [0.048s]
> [INFO] Ambari Python Client .. SUCCESS [0.940s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.030s]
> [INFO] Ambari Shell .. SUCCESS [0.047s]
> [INFO] Ambari Python Shell ... SUCCESS [0.672s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.994s]
> [INFO] ambari-logsearch .. SUCCESS [0.302s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.225s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.219s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [6.735s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.926s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.074s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 7:13.702s
> [INFO] Finished at: Tue Sep 13 11:00:07 PDT 2016
> [INFO] Final Memory: 302M/1191M
> [INFO] 
> 
> 
> ** 2. mvn test -DskipPythonTests -Dtest=Blueprint*Test **
> 
> ** Added a new test BlueprintResourceProviderTest::testPopulateSettingList() 
> **
> * This test creates the setting JSON and adds it to the blueprint response *
> 
> Results :
> 
> Tests run: 247, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] 
> 

Re: Review Request 51897: Config compare shows incorrect value after enabling kerberos

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51897/#review148963
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On Sept. 14, 2016, 8:40 p.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51897/
> ---
> 
> (Updated Sept. 14, 2016, 8:40 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Jaimin Jetly, Zhe (Joe) Wang, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-18389
> https://issues.apache.org/jira/browse/AMBARI-18389
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> "placeholder" only make sense when config is editable.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/configs/objects/service_config_property.js 0c7673d 
>   ambari-web/test/models/configs/objects/service_config_property_test.js 
> a6772d3 
> 
> Diff: https://reviews.apache.org/r/51897/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster, all unit tests passed.
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 51897: Config compare shows incorrect value after enabling kerberos

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51897/
---

(Updated Sept. 14, 2016, 8:40 p.m.)


Review request for Ambari, Andrii Tkach, Jaimin Jetly, Zhe (Joe) Wang, and 
Yusaku Sako.


Changes
---

Add all related fields to dependency.


Bugs: AMBARI-18389
https://issues.apache.org/jira/browse/AMBARI-18389


Repository: ambari


Description
---

"placeholder" only make sense when config is editable.


Diffs (updated)
-

  ambari-web/app/models/configs/objects/service_config_property.js 0c7673d 
  ambari-web/test/models/configs/objects/service_config_property_test.js 
a6772d3 

Diff: https://reviews.apache.org/r/51897/diff/


Testing
---

Manually tested on live cluster, all unit tests passed.


Thanks,

Richard Zang

Review Request 51898: AMBARI-18392 : Cluster failure due to APP_TIMELINE_SERVER and RESOURCEMANAGER failed to start

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51898/
---

Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-18392
https://issues.apache.org/jira/browse/AMBARI-18392


Repository: ambari


Description
---

Exception Trace
{code}
 Error accessing file:/usr/hdp/2.5.1.0-28/oozie/share/lib/spark
azure: java.io.IOException: Error accessing 
file:/usr/hdp/2.5.1.0-28/oozie/share/lib/spark
azure: at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1536)
azure: at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1574)
azure: at 
org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:676)
azure: at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:357)
azure: at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:359)
azure: at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:359)
azure: at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:341)
azure: at 
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1986)
azure: at 
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1954)
azure: at 
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1919)
azure: at 
org.apache.ambari.fast_hdfs_resource.Resource.createResource(Resource.java:216)
azure: at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:83)
azure: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
azure: at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
azure: at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
azure: at java.lang.reflect.Method.invoke(Method.java:498)
azure: at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
azure: at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
{code}

RC
Not all the required directories were created on the DFS as part of the sys 
prep thus both Application Timeline server and Resource Manager are failing.


Diffs
-

  ambari-server/src/main/resources/scripts/Ambaripreupload.py 09dcd7e 

Diff: https://reviews.apache.org/r/51898/diff/


Testing
---

Manual testing.


Thanks,

Aravindan Vijayan

Re: Review Request 51897: Config compare shows incorrect value after enabling kerberos

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51897/#review148956
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On Sept. 14, 2016, 7:27 p.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51897/
> ---
> 
> (Updated Sept. 14, 2016, 7:27 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Jaimin Jetly, Zhe (Joe) Wang, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-18389
> https://issues.apache.org/jira/browse/AMBARI-18389
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> "placeholder" only make sense when config is editable.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/configs/objects/service_config_property.js 0c7673d 
>   ambari-web/test/models/configs/objects/service_config_property_test.js 
> a6772d3 
> 
> Diff: https://reviews.apache.org/r/51897/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster, all unit tests passed.
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Review Request 51897: Config compare shows incorrect value after enabling kerberos

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51897/
---

Review request for Ambari, Andrii Tkach, Jaimin Jetly, Zhe (Joe) Wang, and 
Yusaku Sako.


Bugs: AMBARI-18389
https://issues.apache.org/jira/browse/AMBARI-18389


Repository: ambari


Description
---

"placeholder" only make sense when config is editable.


Diffs
-

  ambari-web/app/models/configs/objects/service_config_property.js 0c7673d 
  ambari-web/test/models/configs/objects/service_config_property_test.js 
a6772d3 

Diff: https://reviews.apache.org/r/51897/diff/


Testing
---

Manually tested on live cluster, all unit tests passed.


Thanks,

Richard Zang

Re: Review Request 51894: View instance cloning functionality

[Sangeeta Ravindran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51894/
---

(Updated Sept. 14, 2016, 6:55 p.m.)


Review request for Ambari, Alexandr Antonenko and Di Li.


Changes
---

Updated the patch and diff to ensure the right view version is displayed when 
cloning instance. Also added back the empty first column (which was removed in 
the first patch) in the view instance list.


Bugs: AMBARI-18378
https://issues.apache.org/jira/browse/AMBARI-18378


Repository: ambari


Description
---

This task is for providing an option to clone / copy a view instance.
A new "copy" icon will appear next to a view instance. Clicking on this opens 
up the existing view with instance name and display name of the view appended 
with _Copy.
This will be useful when creating multiple view instances with mild 
configuration differences between them.


Diffs (updated)
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
 20bd25c 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
0aefaba 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/routes.js 239e39e 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html 
924885f 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/listTable.html
 bc441cd 

Diff: https://reviews.apache.org/r/51894/diff/


Testing
---

Manual testing with different types of view instances.
Currently there are no automated tests for ambari-admin.


File Attachments (updated)


AMBARI-18378.patch
  
https://reviews.apache.org/media/uploaded/files/2016/09/14/f1299304-2bad-42d2-8509-831aba536e9a__AMBARI-18378.patch


Thanks,

Sangeeta Ravindran

Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Amruta Borkar]

> On Sept. 14, 2016, 5:11 p.m., Di Li wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java,
> >  line 35
> > 
> >
> > should configType, property, value be mapped to xml elements explicitly 
> > ?

Hi Di,
In this case mapping should not be necessary as properties in class have same 
names as defined in metainfo.xml and DependencyConditions class is mapped to 
xml element 'condition' in DependencyInfo class.


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/#review148930
---


On Sept. 13, 2016, 9:28 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51815/
> ---
> 
> (Updated Sept. 13, 2016, 9:28 p.m.)
> 
> 
> Review request for Ambari, Shantanu Mundkur, Di Li, Juanjo  Marron, Laszlo 
> Puskas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18355
> https://issues.apache.org/jira/browse/AMBARI-18355
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. Please refer to document attached to Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
>  e3db662 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  a5f33ff 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 65d166a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  b1de8ef 
> 
> Diff: https://reviews.apache.org/r/51815/diff/
> 
> 
> Testing
> ---
> 
> Written Junit test cases. Perfomred manual testing to check the Namenode HA 
> component dependency. Was able to proceed with the installation for valid 
> blueprint. and got validation error message while registering blueprint when 
> the Blueprint did not satisfy the conditional dependencies.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/#review148939
---




ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 (line 694)


Why using stack_supports_ranger_kerberos? It sounds like a workaround for 
>= HDP-2.5+.


- Jayush Luniya


On Sept. 14, 2016, 4:32 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51890/
> ---
> 
> (Updated Sept. 14, 2016, 4:32 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18386
> https://issues.apache.org/jira/browse/AMBARI-18386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
> right jdbc url for repo configs that are set for creating a repository in 
> Ranger for Secure and simple install of hive. 
> However in HiverServer2 HA env, jdbc url needs to change in repo config with 
> zk url.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  5e3e233 
> 
> Diff: https://reviews.apache.org/r/51890/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49802: Host Namenode Rebalance HDFS failed

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49802/#review148934
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 (line 283)


Why would the json data be None?


- Alejandro Fernandez


On Sept. 14, 2016, 8:07 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49802/
> ---
> 
> (Updated Sept. 14, 2016, 8:07 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, JaimejucoummaUW 
> JaimejucoummaUW, Zhe (Joe) Wang, Kuldeep Mishra, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17546
> https://issues.apache.org/jira/browse/AMBARI-17546
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> the host view ,excute namenode rebalance HDFS, json none lead fail
> 
> Rebalance HDFS in the host web view , it will not show the Rebalance HDFS 
> dialog to fill the disk capacity value, so rebalance hdfs wil fail. It would 
> be difficult to change from fore-end, moreover, the change may not fit into 
> fore-end's design ,and I sense the bug is a serious issue, so I think it is 
> better to give it the default value in the namenode.py .
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  df02311 
> 
> Diff: https://reviews.apache.org/r/49802/diff/
> 
> 
> Testing
> ---
> 
> done
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/#review148930
---




ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
 (line 35)


should configType, property, value be mapped to xml elements explicitly ?


- Di Li


On Sept. 13, 2016, 9:28 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51815/
> ---
> 
> (Updated Sept. 13, 2016, 9:28 p.m.)
> 
> 
> Review request for Ambari, Shantanu Mundkur, Di Li, Juanjo  Marron, Laszlo 
> Puskas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18355
> https://issues.apache.org/jira/browse/AMBARI-18355
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. Please refer to document attached to Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
>  e3db662 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  a5f33ff 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 65d166a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  b1de8ef 
> 
> Diff: https://reviews.apache.org/r/51815/diff/
> 
> 
> Testing
> ---
> 
> Written Junit test cases. Perfomred manual testing to check the Namenode HA 
> component dependency. Was able to proceed with the installation for valid 
> blueprint. and got validation error message while registering blueprint when 
> the Blueprint did not satisfy the conditional dependencies.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Review Request 51894: View instance cloning functionality

[Sangeeta Ravindran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51894/
---

Review request for Ambari, Alexandr Antonenko and Di Li.


Bugs: AMBARI-18378
https://issues.apache.org/jira/browse/AMBARI-18378


Repository: ambari


Description
---

This task is for providing an option to clone / copy a view instance.
A new "copy" icon will appear next to a view instance. Clicking on this opens 
up the existing view with instance name and display name of the view appended 
with _Copy.
This will be useful when creating multiple view instances with mild 
configuration differences between them.


Diffs
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
 20bd25c 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
0aefaba 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/routes.js 239e39e 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html 
924885f 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/listTable.html
 bc441cd 

Diff: https://reviews.apache.org/r/51894/diff/


Testing
---

Manual testing with different types of view instances.
Currently there are no automated tests for ambari-admin.


Thanks,

Sangeeta Ravindran

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/#review148923
---



I agree with Nate in that there's a much larger problem looming here; all of 
the command JSON files stored for every agent has the passwords in plaintext. I 
think you should take time to design how the fix for that would work and then 
integrate it with the work you're doing here for alerts.


ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py (lines 138 - 
139)


`_mask_passwords_in_configurations` is still changing the values directly 
in the original `__configurations` dictionary. It should be making a copy to 
dump.



ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py (lines 169 - 
172)


This function doesn't do much anymore aside from just take params and pass 
them down the line. 

It can be removed in favor of just using the `_replace_passwords` directly



ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
 (lines 59 - 60)


Documentation around what this set is supposed to actually be



ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 (lines 91 - 92)


Documentation around what this set is supposed to actually be



ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
(lines 227 - 246)


This method is still inefficient in that it repeats the same work over and 
over for every command. It can simply cache the results of properties which are 
PASSWORD type and return that.


- Jonathan Hurley


On Sept. 12, 2016, 6:48 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51705/
> ---
> 
> (Updated Sept. 12, 2016, 6:48 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18334
> https://issues.apache.org/jira/browse/AMBARI-18334
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The configurations.json file loaded in the ambari-agent cache located at 
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.
> 
> Approach:
> 
> The password_config_type is included in the heartbeat response for alert 
> definition command and execution command, for which the values are dumped 
> into the json file. The password_config_type contains the information on 
> which properties in the configurations has the propertyType password. Based 
> on the response, the json is parsed and the password values are masked before 
> dumping it into the configurations.json file.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py 72b87be 
>   ambari-agent/src/test/python/ambari_agent/TestAlerts.py 2bddc43 
>   ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py 
> a418f6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
>  0562c15 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
>  4d2e048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  29737ee 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 70c24f9 
> 
> Diff: https://reviews.apache.org/r/51705/diff/
> 
> 
> Testing
> ---
> 
> Updated the test cases.
> Ran mvn test.
> 
> Manually tested by setting up a cluster, the password fields in the 
> configurations.json is masked. During testing, everytime the ambari agent is 
> restarted, it registers with the server and the memory cache and file cache 
> are updated, the alerts in turn uses the value from the memory cache.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/#review148924
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Sept. 14, 2016, 4:32 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51890/
> ---
> 
> (Updated Sept. 14, 2016, 4:32 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18386
> https://issues.apache.org/jira/browse/AMBARI-18386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
> right jdbc url for repo configs that are set for creating a repository in 
> Ranger for Secure and simple install of hive. 
> However in HiverServer2 HA env, jdbc url needs to change in repo config with 
> zk url.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  5e3e233 
> 
> Diff: https://reviews.apache.org/r/51890/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Mugdha Varadkar]

> On Sept. 14, 2016, 3:25 p.m., Velmurugan Periasamy wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py,
> >  line 696
> > 
> >
> > Should we use zookeeper based URL even if HA is not enabled? That would 
> > cover any additional parameters that need to be added for Hive (like http 
> > transport mode etc)

Updated in latest patch


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/#review148907
---


On Sept. 14, 2016, 4:32 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51890/
> ---
> 
> (Updated Sept. 14, 2016, 4:32 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18386
> https://issues.apache.org/jira/browse/AMBARI-18386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
> right jdbc url for repo configs that are set for creating a repository in 
> Ranger for Secure and simple install of hive. 
> However in HiverServer2 HA env, jdbc url needs to change in repo config with 
> zk url.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  5e3e233 
> 
> Diff: https://reviews.apache.org/r/51890/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/
---

(Updated Sept. 14, 2016, 4:32 p.m.)


Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
Velmurugan Periasamy.


Changes
---

Updated patch removed hive-server2 HA check. Updating the jdbc url config from 
2.5 stack onwards


Summary (updated)
-

AMBARI-18386 Handle Ranger hive repo config jdbc url change for stack 2.5


Bugs: AMBARI-18386
https://issues.apache.org/jira/browse/AMBARI-18386


Repository: ambari


Description
---

>From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
>right jdbc url for repo configs that are set for creating a repository in 
>Ranger for Secure and simple install of hive. 
However in HiverServer2 HA env, jdbc url needs to change in repo config with zk 
url.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 5e3e233 

Diff: https://reviews.apache.org/r/51890/diff/


Testing
---

Tested Ranger hive plugin with Ranger install on centos6


Thanks,

Mugdha Varadkar

Re: Review Request 51892: Create authentication filter to encapsulate the various Ambari authentication methods

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51892/#review148910
---




ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 


This is now handled by Spring via the `spring-security.xml` file.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 


This is now handled by Spring via the `spring-security.xml` file.



ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilter.java
 


This class was renamed to `AmbariBasicAuthenticationFilter` since 
`AmbariAuthenticationFilter` made for a better interface name for 
Ambari-specific authentication filter implementations.



ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
 (line 52)


This was basically renamed from `AmbariAuthenticationFilter` since 
`AmbariAuthenticationFilter` made for a better interface name.



ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
 (line 51)


`AmbariJWTAuthenticationFilter` was created fir symmetrical reasons and to 
augment the behavior of `JwtAuthenticationFilter` to capture inforamtion for 
the audit loggger.  It works like the `AmbariBasicAuthenticationFilter` in this 
respect.



ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml (line 64)


This sets up the `AmbariJWTAuthenticationFilter` and negates the need to 
inject members. Therefore the code to invoke injection is removed from the 
`AmbariServer` class.



ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml (line 72)


This sets up the `AmbariAuthorizationFilter` and negates the need to inject 
members. Therefore the code to invoke injection is removed from the 
`AmbariServer` class.


- Robert Levas


On Sept. 14, 2016, 11:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51892/
> ---
> 
> (Updated Sept. 14, 2016, 11:42 a.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18367
> https://issues.apache.org/jira/browse/AMBARI-18367
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Create a Spring authentication filter to encapsulate the various Ambari 
> authentication methods since the Spring filter chain allows for a single 
> authentication filter and Ambari needs to allow for multiple, optional, 
> authentication filters to handle one of (but not limited to) the following 
> authentication methods:
> 
> - Basic Auth 
> - SSO (JWT)
> - Kerberos token
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  deac313 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  bd34c25 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilter.java
>  0a312f3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariDelegatingAuthenticationFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
>  2f676b4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/AuthenticationJwtUserNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java
>  760621e 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml e1697e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilterTest.java
>  0f2b104 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilterTest.java
>  PRE-CREATION 
>   
> 

Review Request 51892: Create authentication filter to encapsulate the various Ambari authentication methods

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51892/
---

Review request for Ambari, Daniel Gergely, Jonathan Hurley, Myroslav 
Papirkovskyy, Nate Cole, and Sebastian Toader.


Bugs: AMBARI-18367
https://issues.apache.org/jira/browse/AMBARI-18367


Repository: ambari


Description
---

Create a Spring authentication filter to encapsulate the various Ambari 
authentication methods since the Spring filter chain allows for a single 
authentication filter and Ambari needs to allow for multiple, optional, 
authentication filters to handle one of (but not limited to) the following 
authentication methods:

- Basic Auth 
- SSO (JWT)
- Kerberos token


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 deac313 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 bd34c25 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilter.java
 0a312f3 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariDelegatingAuthenticationFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
 2f676b4 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/AuthenticationJwtUserNotFoundException.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java
 760621e 
  ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml e1697e2 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilterTest.java
 0f2b104 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilterTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariDelegatingAuthenticationFilterTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilterTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
 1d71fe6 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilterTest.java
 b08d1ac 

Diff: https://reviews.apache.org/r/51892/diff/


Testing
---

Manually tested

#Local test results:

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:40:31.332s
[INFO] Finished at: Wed Sep 14 10:18:07 EDT 2016
[INFO] Final Memory: 67M/693M
[INFO] 


# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url, when hiveserver2 HA is enabled

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/#review148907
---




ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 (line 696)


Should we use zookeeper based URL even if HA is not enabled? That would 
cover any additional parameters that need to be added for Hive (like http 
transport mode etc)


- Velmurugan Periasamy


On Sept. 14, 2016, 11:49 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51890/
> ---
> 
> (Updated Sept. 14, 2016, 11:49 a.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18386
> https://issues.apache.org/jira/browse/AMBARI-18386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
> right jdbc url for repo configs that are set for creating a repository in 
> Ranger for Secure and simple install of hive. 
> However in HiverServer2 HA env, jdbc url needs to change in repo config with 
> zk url.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  5e3e233 
> 
> Diff: https://reviews.apache.org/r/51890/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 51846: AMBARI-18375 Ranger Plugin configs is not generated for Hive interactive

[Mugdha Varadkar]

> On Sept. 13, 2016, 8:14 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_ranger_hive_interactive.py,
> >  line 76
> > 
> >
> > What was the diff?

There is a indentation diff at line 56, 57

< 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51846/
> ---
> 
> (Updated Sept. 13, 2016, 1:39 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18375
> https://issues.apache.org/jira/browse/AMBARI-18375
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If audit to hdfs is OFF for ranger hive plugin, configs related to plugins 
> are not generated for Hive interactive
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_ranger_hive_interactive.py
>  0b5d5db 
> 
> Diff: https://reviews.apache.org/r/51846/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger hive plugin and hive interactive with Ranger install on centos6
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Review Request 51890: AMBARI-18386 Handle Ranger hive repo config jdbc url, when hiveserver2 HA is enabled

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51890/
---

Review request for Ambari, Gautam Borad, Jayush Luniya, Sumit Mohanty, and 
Velmurugan Periasamy.


Bugs: AMBARI-18386
https://issues.apache.org/jira/browse/AMBARI-18386


Repository: ambari


Description
---

>From ambari stacks for Ranger: enable ranger hive plugin takes care of using 
>right jdbc url for repo configs that are set for creating a repository in 
>Ranger for Secure and simple install of hive. 
However in HiverServer2 HA env, jdbc url needs to change in repo config with zk 
url.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 5e3e233 

Diff: https://reviews.apache.org/r/51890/diff/


Testing
---

Tested Ranger hive plugin with Ranger install on centos6


Thanks,

Mugdha Varadkar

Re: Review Request 51838: Do not execute stack-selector-tool after HDP component install on sysprepped hosts.

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51838/
---

(Updated Sept. 14, 2016, 1:47 p.m.)


Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Jayush Luniya, and 
Sumit Mohanty.


Bugs: AMBARI-18370
https://issues.apache.org/jira/browse/AMBARI-18370


Repository: ambari


Description
---

1. Modified Ambaripreupload.py to invoke 'hdp-select set all' to ensure that 
when preparing a sysprepped host the stack selector tool was executed for the 
selected HDP components
2. The shared initialisation script that is executed once after each HDP 
component install as AFTER-install hook skips executing 'hdp-select set all' if 
it's a sysprepped host


Diffs (updated)
-

  ambari-server/src/main/resources/scripts/Ambaripreupload.py 9492e17 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
 819d8f7 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
 9982dc6 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 06a366e 

Diff: https://reviews.apache.org/r/51838/diff/


Testing
---

Manual testing.

Python unit tests:
--
Total run:1125
Total errors:0
Total failures:0
OK
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:43.849s
[INFO] Finished at: Tue Sep 13 09:56:49 CEST 2016
[INFO] Final Memory: 73M/799M
[INFO] 


Thanks,

Sebastian Toader

Re: Review Request 51838: Do not execute stack-selector-tool after HDP component install on sysprepped hosts.

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51838/
---

(Updated Sept. 14, 2016, 1:43 p.m.)


Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Jayush Luniya, and 
Sumit Mohanty.


Changes
---

Add new line to the end of Ambaripreupload.py


Bugs: AMBARI-18370
https://issues.apache.org/jira/browse/AMBARI-18370


Repository: ambari


Description
---

1. Modified Ambaripreupload.py to invoke 'hdp-select set all' to ensure that 
when preparing a sysprepped host the stack selector tool was executed for the 
selected HDP components
2. The shared initialisation script that is executed once after each HDP 
component install as AFTER-install hook skips executing 'hdp-select set all' if 
it's a sysprepped host


Diffs (updated)
-

  ambari-server/src/main/resources/scripts/Ambaripreupload.py 9492e17 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
 819d8f7 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
 9982dc6 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 06a366e 

Diff: https://reviews.apache.org/r/51838/diff/


Testing
---

Manual testing.

Python unit tests:
--
Total run:1125
Total errors:0
Total failures:0
OK
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:43.849s
[INFO] Finished at: Tue Sep 13 09:56:49 CEST 2016
[INFO] Final Memory: 73M/799M
[INFO] 


Thanks,

Sebastian Toader

Re: Review Request 51843: rolling restart datanode cluster name in audit log show null

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51843/#review148894
---


Ship it!




Ship It!

- Oliver Szabo


On Sept. 14, 2016, 8:31 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51843/
> ---
> 
> (Updated Sept. 14, 2016, 8:31 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, Robert Nettleton, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18374
> https://issues.apache.org/jira/browse/AMBARI-18374
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when rolling restart services ( eg:DataNode ) ,cluster name in audit log show 
> null
> 
> 2016-09-13T17:29:22.033+0800, User(admin), RemoteIp(127.0.0.1), 
> Operation(Request from server), RequestType(POST), 
> url(http://localhost:8080/api/v1/clusters/amabri/requests), ResultStatus(202 
> Accepted), Command(RESTART), Cluster name(null)
> 
> 2016-09-13T17:29:22.042+0800, User(admin), 
> Operation(PARSE.ROLLING-RESTART.DATANODE.1.1), Status(IN_PROGRESS), 
> RequestId(15)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RequestEventCreator.java
>  075e328 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/RequestEventCreatorTest.java
>  1074f0e 
> 
> Diff: https://reviews.apache.org/r/51843/diff/
> 
> 
> Testing
> ---
> 
> done!
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 51838: Do not execute stack-selector-tool after HDP component install on sysprepped hosts.

[Sebastian Toader]

> On Sept. 13, 2016, 8:42 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py,
> >  line 49
> > 
> >
> > Has this been tested on a sysprepped cluster that is deployed and then 
> > undergoes either one of these 2 operations:
> > * add another service
> > * perform an EU/RU (this may not be supported yet, but we need to be 
> > aware that we're using the host_sysprepped_flag to indicate a fresh install 
> > but the code still runs after an EU/RU).
> > 
> > If those ops are not supported in sysprepped clusters, then close this 
> > issue.
> 
> Jayush Luniya wrote:
> Wouldnt we be calling install_packages() during RU/EU? The after install 
> hook wouldnt be called during RU/EU.

Currently on sys_prepped cluster RU/EU is not supported. The after install hook 
is not invoked during RU/EU.


- Sebastian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51838/#review148757
---


On Sept. 14, 2016, 1:27 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51838/
> ---
> 
> (Updated Sept. 14, 2016, 1:27 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Jayush Luniya, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18370
> https://issues.apache.org/jira/browse/AMBARI-18370
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Modified Ambaripreupload.py to invoke 'hdp-select set all' to ensure that 
> when preparing a sysprepped host the stack selector tool was executed for the 
> selected HDP components
> 2. The shared initialisation script that is executed once after each HDP 
> component install as AFTER-install hook skips executing 'hdp-select set all' 
> if it's a sysprepped host
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/scripts/Ambaripreupload.py 9492e17 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
>  819d8f7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
>  9982dc6 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  06a366e 
> 
> Diff: https://reviews.apache.org/r/51838/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Python unit tests:
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:43.849s
> [INFO] Finished at: Tue Sep 13 09:56:49 CEST 2016
> [INFO] Final Memory: 73M/799M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 51838: Do not execute stack-selector-tool after HDP component install on sysprepped hosts.

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51838/
---

(Updated Sept. 14, 2016, 1:27 p.m.)


Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Jayush Luniya, and 
Sumit Mohanty.


Changes
---

Log warning message about potential side effects of skipping hdp-stack-selector 
tool on component intsall


Bugs: AMBARI-18370
https://issues.apache.org/jira/browse/AMBARI-18370


Repository: ambari


Description
---

1. Modified Ambaripreupload.py to invoke 'hdp-select set all' to ensure that 
when preparing a sysprepped host the stack selector tool was executed for the 
selected HDP components
2. The shared initialisation script that is executed once after each HDP 
component install as AFTER-install hook skips executing 'hdp-select set all' if 
it's a sysprepped host


Diffs (updated)
-

  ambari-server/src/main/resources/scripts/Ambaripreupload.py 9492e17 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
 819d8f7 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
 9982dc6 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 06a366e 

Diff: https://reviews.apache.org/r/51838/diff/


Testing
---

Manual testing.

Python unit tests:
--
Total run:1125
Total errors:0
Total failures:0
OK
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:43.849s
[INFO] Finished at: Tue Sep 13 09:56:49 CEST 2016
[INFO] Final Memory: 73M/799M
[INFO] 


Thanks,

Sebastian Toader

Re: Review Request 51843: rolling restart datanode cluster name in audit log show null

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51843/#review148872
---


Ship it!




Ship It!

- Sebastian Toader


On Sept. 14, 2016, 8:04 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51843/
> ---
> 
> (Updated Sept. 14, 2016, 8:04 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18374
> https://issues.apache.org/jira/browse/AMBARI-18374
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when rolling restart services ( eg:DataNode ) ,cluster name in audit log show 
> null
> 
> 2016-09-13T17:29:22.033+0800, User(admin), RemoteIp(127.0.0.1), 
> Operation(Request from server), RequestType(POST), 
> url(http://localhost:8080/api/v1/clusters/amabri/requests), ResultStatus(202 
> Accepted), Command(RESTART), Cluster name(null)
> 
> 2016-09-13T17:29:22.042+0800, User(admin), 
> Operation(PARSE.ROLLING-RESTART.DATANODE.1.1), Status(IN_PROGRESS), 
> RequestId(15)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RequestEventCreator.java
>  075e328 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/RequestEventCreatorTest.java
>  1074f0e 
> 
> Diff: https://reviews.apache.org/r/51843/diff/
> 
> 
> Testing
> ---
> 
> done!
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 49802: Host Namenode Rebalance HDFS failed

[wang yaoxin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49802/
---

(Updated 九月 14, 2016, 8:07 a.m.)


Review request for Ambari, Alejandro Fernandez, JaimejucoummaUW 
JaimejucoummaUW, Zhe (Joe) Wang, Kuldeep Mishra, and Yusaku Sako.


Changes
---

It would be difficult to change from fore-end, moreover, the change may not fit 
into fore-end's design ,and I sense the bug is a serious issue


Bugs: AMBARI-17546
https://issues.apache.org/jira/browse/AMBARI-17546


Repository: ambari


Description (updated)
---

the host view ,excute namenode rebalance HDFS, json none lead fail

Rebalance HDFS in the host web view , it will not show the Rebalance HDFS 
dialog to fill the disk capacity value, so rebalance hdfs wil fail. It would be 
difficult to change from fore-end, moreover, the change may not fit into 
fore-end's design ,and I sense the bug is a serious issue, so I think it is 
better to give it the default value in the namenode.py .


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 df02311 

Diff: https://reviews.apache.org/r/49802/diff/


Testing
---

done


Thanks,

wang yaoxin