Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178330
---


Fix it, then Ship it!





ambari-server/src/main/python/ambari_server/serverUpgrade.py
Lines 569 (patched)


Add a comment for what this does


- Alejandro Fernandez


On June 19, 2017, 10:38 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated June 19, 2017, 10:38 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py 160c91d 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178324
---


Ship it!




Ship It!

- Sumit Mohanty


On June 19, 2017, 10:38 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated June 19, 2017, 10:38 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py 160c91d 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178323
---


Ship it!




Ship It!

- Jayush Luniya


On June 19, 2017, 10:38 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated June 19, 2017, 10:38 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py 160c91d 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Vitalyi Brodetskyi]

> On Червень 19, 2017, 3:49 після полудня, Jayush Luniya wrote:
> > ambari-server/src/main/python/ambari_server/serverUpgrade.py
> > Line 428 (original), 428 (patched)
> > 
> >
> > Any unit tests to cover this?

Will add tomorrow.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178253
---


On Червень 19, 2017, 10:38 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated Червень 19, 2017, 10:38 після полудня)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py 160c91d 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/
---

(Updated Червень 19, 2017, 10:38 після полудня)


Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.


Bugs: AMBARI-21263
https://issues.apache.org/jira/browse/AMBARI-21263


Repository: ambari


Description
---

{code}
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
Traceback (most recent call last):
  File "/usr/sbin/ambari-server.py", line 941, in 
mainBody()
  File "/usr/sbin/ambari-server.py", line 911, in mainBody
main(options, args, parser)
  File "/usr/sbin/ambari-server.py", line 863, in main
action_obj.execute()
  File "/usr/sbin/ambari-server.py", line 78, in execute
self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", line 
363, in upgrade
replay_mpack_logs()
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
983, in replay_mpack_logs
install_mpack(replay_options, replay_mode=True)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
896, in install_mpack
(mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
mpack_archive_path) = _install_mpack(options, replay_mode)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
792, in _install_mpack
process_service_definitions_artifact(artifact, artifact_source_dir, options)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
515, in process_service_definitions_artifact
create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
file, options.force)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
235, in create_symlink
create_symlink_using_path(src_path, dest_link, force)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
247, in create_symlink_using_path
sudo.symlink(src_path, dest_link)
  File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
line 123, in symlink
os.symlink(source, link_name)
OSError: [Errno 17] File exists
{code}


Diffs (updated)
-

  ambari-server/src/main/python/ambari_server/serverUpgrade.py 160c91d 


Diff: https://reviews.apache.org/r/60126/diff/2/

Changes: https://reviews.apache.org/r/60126/diff/1-2/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 60206: AMBARI-21281 Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60206/#review178317
---


Ship it!




Ship It!

- Tim Thorpe


On June 19, 2017, 8:37 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60206/
> ---
> 
> (Updated June 19, 2017, 8:37 p.m.)
> 
> 
> Review request for Ambari and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21281
> https://issues.apache.org/jira/browse/AMBARI-21281
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  2f6ffc1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
>  8e59602 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java
>  960a04c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  51bd168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
>  288f98a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  014ab42 
> 
> 
> Diff: https://reviews.apache.org/r/60206/diff/1/
> 
> 
> Testing
> ---
> 
> existing unit tests.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178310
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> ---
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan 
> Hurley.
> 
> 
> Bugs: AMBARI-21190
> https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to 
> hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of 
> ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it 
> does not harm to have all of the fs values in this properly it will be good 
> to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml
>  d98bc21 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml
>  439c495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml
>  2c62851 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 1610bb5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
>  1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
> 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 60152: Add "zeppelin.server.ssl.port" in Zeppelin's setting

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60152/#review178309
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 19, 2017, 7:10 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60152/
> ---
> 
> (Updated June 19, 2017, 7:10 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21271
> https://issues.apache.org/jira/browse/AMBARI-21271
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "zeppelin.server.ssl.port" in Zeppelin's setting. This is in associated 
> with the changes made in https://issues.apache.org/jira/browse/ZEPPELIN-1321.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-config.xml
>  85f1859509 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 564dead112 
>   ambari-server/src/test/python/stacks/2.6/configs/default.json 2570657b70 
> 
> 
> Diff: https://reviews.apache.org/r/60152/diff/2/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6.4
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60205: Add support for consecutive login failure accounting

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60205/#review178308
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 19, 2017, 8:26 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60205/
> ---
> 
> (Updated June 19, 2017, 8:26 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21216
> https://issues.apache.org/jira/browse/AMBARI-21216
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add support for consecutive login failure accounting where as log-in failures 
> should increment the {{users.consecutive_failures}} and successful 
> authentication attempts should reset the value to 0.
> 
> Concurrency should be kept in mind to handle simultaneous authentication 
> attempts for the same user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  01920f86d6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.java 
> 0e28e50709 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java
>  66e9003873 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandler.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandlerImpl.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
>  ac3e15fa0f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
>  fca8b29fe2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/UserNotFoundException.java
>  f6c4bcf2a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilter.java
>  1b001ec020 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java
>  6137b68e99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthoritiesPopulator.java
>  5c482a1a13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProvider.java
>  517efe49b8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/InvalidUsernamePasswordCombinationException.java
>  db8238184b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  35eb255fcb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/internal/AmbariInternalAuthenticationProvider.java
>  c57bdf1a99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java
>  3c3a446a61 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 0c86591010 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 83b1f48049 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 215c01d7b7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 40ba709d1d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> 35951f142b 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b7244abee2 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 
> bdbf0de3fa 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilterTest.java
>  18c4ccee77 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilterTest.java
>  961e65dfbb 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilterTest.java
>  5503fac6ac 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationProviderDisableUserTest.java
>  33100dd33b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
>  1bf122e0a1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
>  d9eb3350fe 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProviderTest.java
>  65a5400dc7 
> 
> 
> 

Re: Review Request 60206: AMBARI-21281 Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60206/#review178307
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 19, 2017, 8:37 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60206/
> ---
> 
> (Updated June 19, 2017, 8:37 p.m.)
> 
> 
> Review request for Ambari and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21281
> https://issues.apache.org/jira/browse/AMBARI-21281
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  2f6ffc1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
>  8e59602 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java
>  960a04c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  51bd168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
>  288f98a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  014ab42 
> 
> 
> Diff: https://reviews.apache.org/r/60206/diff/1/
> 
> 
> Testing
> ---
> 
> existing unit tests.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 60206: AMBARI-21281 Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60206/
---

Review request for Ambari and Tim Thorpe.


Bugs: AMBARI-21281
https://issues.apache.org/jira/browse/AMBARI-21281


Repository: ambari


Description
---

Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 2f6ffc1 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
 8e59602 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java
 960a04c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
 51bd168 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
 288f98a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 014ab42 


Diff: https://reviews.apache.org/r/60206/diff/1/


Testing
---

existing unit tests.


Thanks,

Di Li

Re: Review Request 60205: Add support for consecutive login failure accounting

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60205/#review178292
---




ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
Lines 1286-1288 (patched)


The exception thrown may not directly be a 
`javax.persistence.OptimisticLockException`, so we need to look at the cause(s) 
to determine if an `javax.persistence.OptimisticLockException` is at play.



ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
Lines 1295-1304 (patched)


The `UserEntity` object at this point may be detached, so we need to get an 
_attached_ one and ensure that it contains the latest data... so a _refresh_ is 
needed.


- Robert Levas


On June 19, 2017, 4:26 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60205/
> ---
> 
> (Updated June 19, 2017, 4:26 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21216
> https://issues.apache.org/jira/browse/AMBARI-21216
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add support for consecutive login failure accounting where as log-in failures 
> should increment the {{users.consecutive_failures}} and successful 
> authentication attempts should reset the value to 0.
> 
> Concurrency should be kept in mind to handle simultaneous authentication 
> attempts for the same user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  01920f86d6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.java 
> 0e28e50709 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java
>  66e9003873 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandler.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandlerImpl.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
>  ac3e15fa0f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
>  fca8b29fe2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/UserNotFoundException.java
>  f6c4bcf2a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilter.java
>  1b001ec020 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java
>  6137b68e99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthoritiesPopulator.java
>  5c482a1a13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProvider.java
>  517efe49b8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/InvalidUsernamePasswordCombinationException.java
>  db8238184b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  35eb255fcb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/internal/AmbariInternalAuthenticationProvider.java
>  c57bdf1a99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java
>  3c3a446a61 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 0c86591010 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 83b1f48049 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 215c01d7b7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 40ba709d1d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> 35951f142b 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b7244abee2 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 
> bdbf0de3fa 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilterTest.java
>  18c4ccee77 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilterTest.java
>  961e65dfbb 
>   
> 

Review Request 60205: Add support for consecutive login failure accounting

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60205/
---

Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene Chekanskiy, 
Jonathan Hurley, Laszlo Puskas, and Sebastian Toader.


Bugs: AMBARI-21216
https://issues.apache.org/jira/browse/AMBARI-21216


Repository: ambari


Description
---

Add support for consecutive login failure accounting where as log-in failures 
should increment the {{users.consecutive_failures}} and successful 
authentication attempts should reset the value to 0.

Concurrency should be kept in mind to handle simultaneous authentication 
attempts for the same user.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 01920f86d6 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.java 
0e28e50709 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java
 66e9003873 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandler.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandlerImpl.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationException.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java
 ac3e15fa0f 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java
 fca8b29fe2 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/UserNotFoundException.java
 f6c4bcf2a7 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilter.java
 1b001ec020 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java
 6137b68e99 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthoritiesPopulator.java
 5c482a1a13 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProvider.java
 517efe49b8 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/InvalidUsernamePasswordCombinationException.java
 db8238184b 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
 35eb255fcb 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/internal/AmbariInternalAuthenticationProvider.java
 c57bdf1a99 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java
 3c3a446a61 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 0c86591010 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 83b1f48049 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 215c01d7b7 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 40ba709d1d 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 35951f142b 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b7244abee2 
  ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 
bdbf0de3fa 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilterTest.java
 18c4ccee77 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilterTest.java
 961e65dfbb 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilterTest.java
 5503fac6ac 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationProviderDisableUserTest.java
 33100dd33b 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
 1bf122e0a1 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
 d9eb3350fe 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProviderTest.java
 65a5400dc7 


Diff: https://reviews.apache.org/r/60205/diff/1/


Testing
---

Manually tested in cluster

#Local test results: 
```
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 30:58 min
[INFO] Finished at: 2017-06-19T15:16:18-04:00
[INFO] Final Memory: 108M/1760M
[INFO] 
```

#Jasper test results: PENDING


Thanks,

Robert Levas

Re: Review Request 60118: AMBARI-21259 Need to use provided password when usersync sync source and authentication method are different

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60118/#review178287
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 15, 2017, 1:20 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60118/
> ---
> 
> (Updated June 15, 2017, 1:20 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jayush 
> Luniya.
> 
> 
> Bugs: AMBARI-21259
> https://issues.apache.org/jira/browse/AMBARI-21259
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari for Ranger, the password for user authentication is taken from 
> Ranger User Info Tab for LDAP/AD when sync-source and authentication method 
> is LDAP/AD.
> 
> For the scenario where sync-source and authentication method are different or 
> when credentials for sync-source and authentication are different will need 
> to take password from advanced section for LDAP/AD.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  094d239 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  8f89416 
> 
> 
> Diff: https://reviews.apache.org/r/60118/diff/1/
> 
> 
> Testing
> ---
> 
> Unit test report:
> 
> Ran 272 tests in 9.308s
> 
> OK
> --
> Total run:1160
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Review Request 60201: AMBARI-17382 : Migrate AMS queries to use ROW_TIMESTAMP instead of native timerange hint

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60201/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17382
https://issues.apache.org/jira/browse/AMBARI-17382


Repository: ambari


Description
---

With PHOENIX-914Link, there is a change in implementation , As earlier, 
timestamp range was passed as a hint to the query to get advantage of native 
timerange optimization in hbase but with new implementation we can mark the 
timestamp column in the schema as a ROW_TIMESTAMP and pass timestamp range with 
“where” clause only to achieve equivalent performance and better accuracy.

For ex:-

With earlier implementation , AMS forms query like this:-
SELECT /*+ NATIVE_TIME_RANGE(1448029523000) */ METRIC_NAME, APP_ID, 
INSTANCE_ID, SERVER_TIME, UNITS, METRIC_SUM, HOSTS_COUNT, METRIC_MAX, 
METRIC_MIN FROM METRIC_AGGREGATE WHERE (METRIC_NAME IN 
('regionserver.Server.totalRequestCount', 
'regionserver.Server.blockCacheCountHitPercent', 
'regionserver.Server.regionCount', 'regionserver.Server.compactionQueueLength', 
'regionserver.Server.storeFileCount', 'master.Server.averageLoad')) AND APP_ID 
= 'ams-hbase' AND SERVER_TIME >= 1448029643000 AND SERVER_TIME < 1448033243 
ORDER BY METRIC_NAME, SERVER_TIME LIMIT 11520


WORK DONE
1. Removed START_TIME column from AMS schema and access layer.
2. Removed timestamp field from TimelineMetric object. 
3. From now on, StartTime field in TimelineMetric object maps to 'SERVER_TIME' 
in metric tables which are of type 'ROW_TIMESTAMP'


Diffs
-

  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/SingleValuedTimelineMetric.java
 4bb9355fa9408ac19899cc2c507f639c53951d30 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java
 3d3b19ce6ef09b647125ee00c9e8d1c87d76c3cc 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetrics.java
 383079a63cb2f89d6e71554000439f5b17aa253b 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsEhCacheSizeOfEngine.java
 0e23e17039359f73c40d2880465200be1dd5642f 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 a290cedfc1307c38a340658e5302c095da0e36be 
  ambari-metrics/ambari-metrics-timelineservice/pom.xml 
34794e6f6b033400fc69bd87b3ba739bee2fb002 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 7ad88a133b91dfc88434c80e05e50a1f2214d991 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
 aa53430f9e784c8c5a5cf40de789e94319995518 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 83f2392d9f03fb534c4be0bd21511d4e28711a4b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregator.java
 0f6dd7904c74a5f97878c9015dadb3ccb2d2e679 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 cae72632024117f972f2a3b1232073595a8ea9df 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostAggregator.java
 8f941e1dfd708f1d54340d85c8f8bf7057f885da 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java
 c8b5728288d350d093254e310c7f012c6d2dff59 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
 c7b605fc2e0f150c505aef207e99b3707042d832 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
 57a3034d064f338aebb0096f3d0faca4a6e4390d 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
 51c96c682f97b1ceb84d7dfe83f39d79680a4cf1 
  

Re: Review Request 59877: Custom log4j.properties of Flume

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59877/#review178286
---



Pushed to trunk, commit df21f9c4b2577a1d801145b131b8a065c8d02a7e

- Alejandro Fernandez


On June 19, 2017, 8:36 a.m., wang jianfei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59877/
> ---
> 
> (Updated June 19, 2017, 8:36 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jaimin Jetly, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-21191
> https://issues.apache.org/jira/browse/AMBARI-21191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently,if you want to config log4j.properties of Flume,you must change the 
> log4j.properties.j2 in the ambari-agent-host installed flume-agent.
> So,we make it configurable by adding log4j.properties.template.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/configuration/flume-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
>  6b1d286 
> 
> 
> Diff: https://reviews.apache.org/r/59877/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> wang jianfei
> 
>

Re: Review Request 59877: Custom log4j.properties of Flume

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59877/#review178284
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 19, 2017, 8:36 a.m., wang jianfei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59877/
> ---
> 
> (Updated June 19, 2017, 8:36 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jaimin Jetly, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-21191
> https://issues.apache.org/jira/browse/AMBARI-21191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently,if you want to config log4j.properties of Flume,you must change the 
> log4j.properties.j2 in the ambari-agent-host installed flume-agent.
> So,we make it configurable by adding log4j.properties.template.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/configuration/flume-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
>  6b1d286 
> 
> 
> Diff: https://reviews.apache.org/r/59877/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> wang jianfei
> 
>

Re: Review Request 60156: AMBARI-21158. Eliminate Maven warnings

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60156/#review178280
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 16, 2017, 4:17 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60156/
> ---
> 
> (Updated June 16, 2017, 4:17 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-21158
> https://issues.apache.org/jira/browse/AMBARI-21158
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * specified versions where missing
> * moved multiple `execution` occurrences under the same `executions`
> * removed duplicates
> 
> 
> Diffs
> -
> 
>   ambari-funtest/pom.xml 3738106fcc48f2bb739dcfc741b93e96be970911 
>   ambari-metrics/ambari-metrics-host-monitoring/pom.xml 
> d6c1fab924a63de260840f6b208f1c18ccd148e1 
>   ambari-metrics/pom.xml b5859164f95fc6496fa78fd9cc005797223b140e 
>   ambari-project/pom.xml 7e1b813957e7c154ee09b7ca57c490ced185dbcb 
>   ambari-server/pom.xml 91c35e5b50f7db8340d83a0d3cba953b9174fd82 
>   ambari-web/pom.xml d7a1d186ccb95ca1eef11c59eae16c7836842285 
> 
> 
> Diff: https://reviews.apache.org/r/60156/diff/1/
> 
> 
> Testing
> ---
> 
> ```
> mvn -DskipTests -fae clean package
> ```
> 
> Only `python-client` fails, due to missing `setuptools`
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60156: AMBARI-21158. Eliminate Maven warnings

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60156/#review178271
---


Ship it!




Ship It!

- Laszlo Puskas


On June 16, 2017, 4:17 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60156/
> ---
> 
> (Updated June 16, 2017, 4:17 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-21158
> https://issues.apache.org/jira/browse/AMBARI-21158
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * specified versions where missing
> * moved multiple `execution` occurrences under the same `executions`
> * removed duplicates
> 
> 
> Diffs
> -
> 
>   ambari-funtest/pom.xml 3738106fcc48f2bb739dcfc741b93e96be970911 
>   ambari-metrics/ambari-metrics-host-monitoring/pom.xml 
> d6c1fab924a63de260840f6b208f1c18ccd148e1 
>   ambari-metrics/pom.xml b5859164f95fc6496fa78fd9cc005797223b140e 
>   ambari-project/pom.xml 7e1b813957e7c154ee09b7ca57c490ced185dbcb 
>   ambari-server/pom.xml 91c35e5b50f7db8340d83a0d3cba953b9174fd82 
>   ambari-web/pom.xml d7a1d186ccb95ca1eef11c59eae16c7836842285 
> 
> 
> Diff: https://reviews.apache.org/r/60156/diff/1/
> 
> 
> Testing
> ---
> 
> ```
> mvn -DskipTests -fae clean package
> ```
> 
> Only `python-client` fails, due to missing `setuptools`
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60156: AMBARI-21158. Eliminate Maven warnings

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60156/#review178270
---


Ship it!




Ship It!

- Nate Cole


On June 16, 2017, 12:17 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60156/
> ---
> 
> (Updated June 16, 2017, 12:17 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-21158
> https://issues.apache.org/jira/browse/AMBARI-21158
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * specified versions where missing
> * moved multiple `execution` occurrences under the same `executions`
> * removed duplicates
> 
> 
> Diffs
> -
> 
>   ambari-funtest/pom.xml 3738106fcc48f2bb739dcfc741b93e96be970911 
>   ambari-metrics/ambari-metrics-host-monitoring/pom.xml 
> d6c1fab924a63de260840f6b208f1c18ccd148e1 
>   ambari-metrics/pom.xml b5859164f95fc6496fa78fd9cc005797223b140e 
>   ambari-project/pom.xml 7e1b813957e7c154ee09b7ca57c490ced185dbcb 
>   ambari-server/pom.xml 91c35e5b50f7db8340d83a0d3cba953b9174fd82 
>   ambari-web/pom.xml d7a1d186ccb95ca1eef11c59eae16c7836842285 
> 
> 
> Diff: https://reviews.apache.org/r/60156/diff/1/
> 
> 
> Testing
> ---
> 
> ```
> mvn -DskipTests -fae clean package
> ```
> 
> Only `python-client` fails, due to missing `setuptools`
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60198: Handle scenario when host in-memory aggregation is not working

[Dmytro Sen]

> On Июнь 19, 2017, 4:52 п.п., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostConditionalAggregator.java
> > Lines 62 (patched)
> > 
> >
> > With changes to UUID (metric, app, instance) + UUID (host) coming up. 
> > How will this work?
> > 
> > As long as we can do UUID -> host this should still be ok, however need 
> > to measure performance of this query on a large dataset.

This will need to be reworked after UUID commit, current patch is just to show 
the general idea


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60198/#review178267
---


On Июнь 19, 2017, 1:56 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60198/
> ---
> 
> (Updated Июнь 19, 2017, 1:56 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-21279
> https://issues.apache.org/jira/browse/AMBARI-21279
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If monitor aggregation is enabled we do not do collector side 5 minutes 
> aggregation. If a monitor is down we do not get metrics from that host. 
> Metrics collector should check that metrics from a host are available per 
> minute but aggregates are not.
> This could be based on liveliness check of a monitor. More like a heartbeat 
> ping. When we loose ping for > 1 min < 5 min we can aggregate metrics for 
> that time period and hostId.
> Sinks and monitor should switch back to collector reporting state if local 
> aggregation app is not available.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  f984253 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
>  2eb3553 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostConditionalAggregator.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostConditionalAggregator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60198/diff/1/
> 
> 
> Testing
> ---
> 
> In progress
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 60198: Handle scenario when host in-memory aggregation is not working

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60198/#review178267
---


Ship it!





ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostConditionalAggregator.java
Lines 62 (patched)


With changes to UUID (metric, app, instance) + UUID (host) coming up. How 
will this work?

As long as we can do UUID -> host this should still be ok, however need to 
measure performance of this query on a large dataset.


- Sid Wagle


On June 19, 2017, 1:56 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60198/
> ---
> 
> (Updated June 19, 2017, 1:56 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-21279
> https://issues.apache.org/jira/browse/AMBARI-21279
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If monitor aggregation is enabled we do not do collector side 5 minutes 
> aggregation. If a monitor is down we do not get metrics from that host. 
> Metrics collector should check that metrics from a host are available per 
> minute but aggregates are not.
> This could be based on liveliness check of a monitor. More like a heartbeat 
> ping. When we loose ping for > 1 min < 5 min we can aggregate metrics for 
> that time period and hostId.
> Sinks and monitor should switch back to collector reporting state if local 
> aggregation app is not available.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  f984253 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
>  2eb3553 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostConditionalAggregator.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostConditionalAggregator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60198/diff/1/
> 
> 
> Testing
> ---
> 
> In progress
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review178255
---


Ship it!




Ship It!

- Di Li


On June 13, 2017, 9:56 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 13, 2017, 9:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/4/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178253
---




ambari-server/src/main/python/ambari_server/serverUpgrade.py
Line 428 (original), 428 (patched)


Any unit tests to cover this?


- Jayush Luniya


On June 15, 2017, 5:11 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated June 15, 2017, 5:11 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py b488ca6 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60126: Ambari 2.5.1 upgrade fails with HDPSearch mpack installed

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60126/#review178251
---




ambari-server/src/main/python/ambari_server/serverUpgrade.py
Lines 430 (patched)


We should not just check one version directory. It is possible that 
common-services/HDFS/2.1.0 is included in Ambari codebase and hence not a 
symlink but common-services/HDFS/3.0.0 is brought in via mpack and hence is a 
symlink.


- Jayush Luniya


On June 15, 2017, 5:11 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60126/
> ---
> 
> (Updated June 15, 2017, 5:11 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21263
> https://issues.apache.org/jira/browse/AMBARI-21263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
> Traceback (most recent call last):
>   File "/usr/sbin/ambari-server.py", line 941, in 
> mainBody()
>   File "/usr/sbin/ambari-server.py", line 911, in mainBody
> main(options, args, parser)
>   File "/usr/sbin/ambari-server.py", line 863, in main
> action_obj.execute()
>   File "/usr/sbin/ambari-server.py", line 78, in execute
> self.fn(*self.args, **self.kwargs)
>   File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", 
> line 363, in upgrade
> replay_mpack_logs()
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 983, in replay_mpack_logs
> install_mpack(replay_options, replay_mode=True)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 896, in install_mpack
> (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, 
> mpack_archive_path) = _install_mpack(options, replay_mode)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 792, in _install_mpack
> process_service_definitions_artifact(artifact, artifact_source_dir, 
> options)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 515, in process_service_definitions_artifact
> create_symlink(src_service_definitions_dir, dest_service_definitions_dir, 
> file, options.force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 235, in create_symlink
> create_symlink_using_path(src_path, dest_link, force)
>   File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 
> 247, in create_symlink_using_path
> sudo.symlink(src_path, dest_link)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", 
> line 123, in symlink
> os.symlink(source, link_name)
> OSError: [Errno 17] File exists
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py b488ca6 
> 
> 
> Diff: https://reviews.apache.org/r/60126/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60196/#review178245
---


Ship it!




Ship It!

- Robert Nettleton


On June 19, 2017, 11:52 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60196/
> ---
> 
> (Updated June 19, 2017, 11:52 a.m.)
> 
> 
> Review request for Ambari, Miklos Gergely and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21277
> https://issues.apache.org/jira/browse/AMBARI-21277
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> remvoe commons-httpclient dependency from logsearch/logfeeder
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  df00c15 
> 
> 
> Diff: https://reviews.apache.org/r/60196/diff/2/
> 
> 
> Testing
> ---
> 
> done. manually checked the applications.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 60113: Exception needs to be handled properly for mail alert

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60113/#review178243
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Червень 19, 2017, 9:10 до полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60113/
> ---
> 
> (Updated Червень 19, 2017, 9:10 до полудня)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-21248
> https://issues.apache.org/jira/browse/AMBARI-21248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In org.apache.ambari.server.state.services.AlertNoticeDispatchService, we 
> have below code snippet:
> 
> 
> ... 
> protected void runOneIteration() throws Exception { 
> … 
> String targetType = target.getNotificationType(); 
> NotificationDispatcher dispatcher = 
> m_dispatchFactory.getDispatcher(targetType); 
> … 
> if (dispatcher.isDigestSupported()) { 
> ...
> {code}
> If the database has a change to AlertTargetEntity's targetType column 
> (unlikely but let's assume it happened), then dispatcher can be null and the 
> subsequent if clause will throw an exception. In 
> https://github.com/google/guava/blob/master/guava/src/com/google/common/util/concurrent/AbstractScheduledService.java,
>  we see that the exception may happen silently since shutDown() and 
> notifyFailed() does not log anything. 
> ... 
> {code}
> class Task implements Runnable {
> @Override
> public void run() {
> lock.lock();
> try {
> if (runningTask.isCancelled()) { // task may have been cancelled 
> while blocked on the lock. return; }
> 
> AbstractScheduledService.this.runOneIteration();
> } catch (Throwable t) {
> try {
> shutDown();
> } catch (Exception ignored) {
> logger.log(Level.WARNING, "Error while attempting to shut 
> down the service after failure.", ignored);
> }
> 
> notifyFailed(t);
> runningTask.cancel(false); // prevent future invocations. 
> } finally {
> lock.unlock();
> }
> 
> }
> }
> }
> 
> 
> So, runOneIteration will shutdown the scheduler if it hits an uncaught 
> exception. We should wrap a try/catch.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  2972f51 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/services/AlertNoticeDispatchServiceTest.java
>  42ee366 
> 
> 
> Diff: https://reviews.apache.org/r/60113/diff/3/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 60156: AMBARI-21158. Eliminate Maven warnings

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60156/#review178241
---


Ship it!




Ship It!

- Jonathan Hurley


On June 16, 2017, 12:17 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60156/
> ---
> 
> (Updated June 16, 2017, 12:17 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-21158
> https://issues.apache.org/jira/browse/AMBARI-21158
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * specified versions where missing
> * moved multiple `execution` occurrences under the same `executions`
> * removed duplicates
> 
> 
> Diffs
> -
> 
>   ambari-funtest/pom.xml 3738106fcc48f2bb739dcfc741b93e96be970911 
>   ambari-metrics/ambari-metrics-host-monitoring/pom.xml 
> d6c1fab924a63de260840f6b208f1c18ccd148e1 
>   ambari-metrics/pom.xml b5859164f95fc6496fa78fd9cc005797223b140e 
>   ambari-project/pom.xml 7e1b813957e7c154ee09b7ca57c490ced185dbcb 
>   ambari-server/pom.xml 91c35e5b50f7db8340d83a0d3cba953b9174fd82 
>   ambari-web/pom.xml d7a1d186ccb95ca1eef11c59eae16c7836842285 
> 
> 
> Diff: https://reviews.apache.org/r/60156/diff/1/
> 
> 
> Testing
> ---
> 
> ```
> mvn -DskipTests -fae clean package
> ```
> 
> Only `python-client` fails, due to missing `setuptools`
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 60198: Handle scenario when host in-memory aggregation is not working

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60198/
---

Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-21279
https://issues.apache.org/jira/browse/AMBARI-21279


Repository: ambari


Description
---

If monitor aggregation is enabled we do not do collector side 5 minutes 
aggregation. If a monitor is down we do not get metrics from that host. Metrics 
collector should check that metrics from a host are available per minute but 
aggregates are not.
This could be based on liveliness check of a monitor. More like a heartbeat 
ping. When we loose ping for > 1 min < 5 min we can aggregate metrics for that 
time period and hostId.
Sinks and monitor should switch back to collector reporting state if local 
aggregation app is not available.


Diffs
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 f984253 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
 2eb3553 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostConditionalAggregator.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostConditionalAggregator.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60198/diff/1/


Testing
---

In progress


Thanks,

Dmytro Sen

Re: Review Request 60011: Remove Zookeeper as a required service from YARN

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60011/#review178230
---


Ship it!




Ship It!

- Nate Cole


On June 12, 2017, 1:44 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60011/
> ---
> 
> (Updated June 12, 2017, 1:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, and 
> Nate Cole.
> 
> 
> Bugs: AMBARI-21206
> https://issues.apache.org/jira/browse/AMBARI-21206
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an 
> explicit dependency because HDFS had one.
> 
> Enabling the following settings in YARN requires ZOOKEEPER:
> 
> yarn.resourcemanager.recovery.enabled
> yarn.nodemanager.recovery.enabled
> yarn.resourcemanager.ha.enabled
> hadoop.registry.rm.enabled
> 
> In addition, these properties are related to ZOOKEEPER and should be empty if 
> ZOOKEEPER is not installed:
> 
> yarn.resourcemanager.zk-address
> hadoop.registry.zk.quorum
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 0769ae8 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/metainfo.xml 
> 3e1a7ae 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/configuration/yarn-site.xml
>  64e0bcb 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml 
> a802795 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py
>  0fb538d 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 726514b 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 571ff26 
>   ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 
> d4d28c9 
> 
> 
> Diff: https://reviews.apache.org/r/60011/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test -DskipSurefireTests
> mvn clean test -DskipSurefireTests -Dpython.test.mask=test_stack_advisor.py
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59095/#review178229
---


Ship it!




Ship It!

- Nate Cole


On June 6, 2017, 4:16 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59095/
> ---
> 
> (Updated June 6, 2017, 4:16 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, 
> Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20967
> https://issues.apache.org/jira/browse/AMBARI-20967
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS has a static dependency on Zookeeper even though it is only needed when 
> NameNode HA is enabled. The check for Zookeeper should be done before 
> enabling HA rather than enforcing it at installation time.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 0769ae8 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/metainfo.xml 
> 3e1a7ae 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
> b72f309 
>   
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml 
> a802795 
> 
> 
> Diff: https://reviews.apache.org/r/59095/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> Manual testing - installed HDFS only cluster and attempted to enable NameNode 
> HA.  This is prevented with the same error you'd see if ZooKeeper was enabled 
> but you had fewer than 3 ZooKeeper servers configured.
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 60118: AMBARI-21259 Need to use provided password when usersync sync source and authentication method are different

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60118/#review178226
---


Ship it!




Ship It!

- Gautam Borad


On June 15, 2017, 1:20 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60118/
> ---
> 
> (Updated June 15, 2017, 1:20 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jayush 
> Luniya.
> 
> 
> Bugs: AMBARI-21259
> https://issues.apache.org/jira/browse/AMBARI-21259
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari for Ranger, the password for user authentication is taken from 
> Ranger User Info Tab for LDAP/AD when sync-source and authentication method 
> is LDAP/AD.
> 
> For the scenario where sync-source and authentication method are different or 
> when credentials for sync-source and authentication are different will need 
> to take password from advanced section for LDAP/AD.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  094d239 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  8f89416 
> 
> 
> Diff: https://reviews.apache.org/r/60118/diff/1/
> 
> 
> Testing
> ---
> 
> Unit test report:
> 
> Ran 272 tests in 9.308s
> 
> OK
> --
> Total run:1160
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60196/#review178224
---


Ship it!




Ship It!

- Miklos Gergely


On June 19, 2017, 11:52 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60196/
> ---
> 
> (Updated June 19, 2017, 11:52 a.m.)
> 
> 
> Review request for Ambari, Miklos Gergely and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21277
> https://issues.apache.org/jira/browse/AMBARI-21277
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> remvoe commons-httpclient dependency from logsearch/logfeeder
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  df00c15 
> 
> 
> Diff: https://reviews.apache.org/r/60196/diff/2/
> 
> 
> Testing
> ---
> 
> done. manually checked the applications.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60196/
---

(Updated June 19, 2017, 11:52 a.m.)


Review request for Ambari, Miklos Gergely and Robert Nettleton.


Changes
---

remove job-history file


Bugs: AMBARI-21277
https://issues.apache.org/jira/browse/AMBARI-21277


Repository: ambari


Description
---

remvoe commons-httpclient dependency from logsearch/logfeeder


Diffs (updated)
-

  ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 
  ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 df00c15 


Diff: https://reviews.apache.org/r/60196/diff/2/

Changes: https://reviews.apache.org/r/60196/diff/1-2/


Testing
---

done. manually checked the applications.


Thanks,

Oliver Szabo

Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60196/
---

Review request for Ambari, Miklos Gergely and Robert Nettleton.


Bugs: AMBARI-21277
https://issues.apache.org/jira/browse/AMBARI-21277


Repository: ambari


Description
---

remvoe commons-httpclient dependency from logsearch/logfeeder


Diffs
-

  ambari-infra/job-repository.db PRE-CREATION 
  ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 
  ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 df00c15 


Diff: https://reviews.apache.org/r/60196/diff/1/


Testing
---

done. manually checked the applications.


Thanks,

Oliver Szabo

Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

[Mugdha Varadkar]

> On June 16, 2017, 11:20 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> > Lines 61 (patched)
> > 
> >
> > If an existing cluster with Hive and Ranger is Kerberized, this logic 
> > may also need to exist in Stack Advisor so that config changes cause it to 
> > be saved.

Hi Alejandro,

This property is used for Hive when Hive is configured to use different 
file-system schemes supported by HDFS, it is also independent of whether the 
cluster is kerberized or not. It is a new property introduced recently.
For existing cluster, stack advisor logic will not be needed to recommend. If 
user want this property, they can add it via custom-config or it will be added 
during minor upgrade of 2.6.x to 2.6.y


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178136
---


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> ---
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan 
> Hurley.
> 
> 
> Bugs: AMBARI-21190
> https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to 
> hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of 
> ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it 
> does not harm to have all of the fs values in this properly it will be good 
> to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml
>  d98bc21 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml
>  439c495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml
>  2c62851 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 1610bb5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
>  1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 
> 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 59828: Ability to anonymize data during log processing

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59828/
---

(Updated June 19, 2017, 10:34 a.m.)


Review request for Ambari, Oliver Szabo and Robert Nettleton.


Changes
---

fix description


Bugs: AMBARI-21181
https://issues.apache.org/jira/browse/AMBARI-21181


Repository: ambari


Description (updated)
---

Have a new mapper called map_anonymize which accepts a pattern containing 
 elements. After the mapping if the pattern is found in the field, than 
the parts matching the  elements are replaced with a configurable hiding 
character ( default '*' ).

Example:
pattern: the secret is  /  which is confidential
log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential 
bla bla
after the mapping: Bla bla bla the secret is *** / *** which is 
confidential bla bla

This patch also fixes the checkpoint file writing, which wasn't atomic. The 
checkpoint files are now written under a temporary name first, than atomically 
renamed to the proper name, thus making it impossible to have a half written 
file.


Diffs
-

  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java
 f88435f 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java
 db086c5 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java
 cf37e62 
  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java
 32aded8 
  ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java
 2bc4e68 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
 fba596d 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
 8c76785 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java
 4bf162b 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java
 ffd6cec 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json
 e2ed625 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml 
e641018 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java
 dcb0393 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java
 b18439c 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java
 18744e2 
  
ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json
 4ccef74 
  
ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json
 d3685a4 


Diff: https://reviews.apache.org/r/59828/diff/2/


Testing
---

Tested on local cluster, and by the new unit tests added.


Thanks,

Miklos Gergely

Re: Review Request 59828: Ability to anonymize data during log processing

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59828/
---

(Updated June 19, 2017, 10:31 a.m.)


Review request for Ambari, Oliver Szabo and Robert Nettleton.


Changes
---

Use * instead of X by default to hide sensitive log message parts


Bugs: AMBARI-21181
https://issues.apache.org/jira/browse/AMBARI-21181


Repository: ambari


Description (updated)
---

Have a new mapper called map_anonymize which accepts a pattern containing 
 elements. After the mapping if the pattern is found in the field, than 
the parts matching the  elements are replaced with a configurable hiding 
character ( default '*' ).

Example:
pattern: the secret is  /  which is confidential
log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential 
bla bla
after the mapping: Bla bla bla the secret is *** / *** which is 
confidential bla bla

This patch also fixes the checkpoint file writing, which wasn't atomic. The 
checkpoint files are now written under a temporary name first, than atomically 
renamed to the proper name, thus making it impossible to have a half written 
file.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java
 f88435f 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java
 db086c5 
  
ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java
 cf37e62 
  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java
 32aded8 
  ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java
 2bc4e68 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
 fba596d 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
 8c76785 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java
 4bf162b 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java
 ffd6cec 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json
 e2ed625 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml 
e641018 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java
 dcb0393 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java
 b18439c 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java
 18744e2 
  
ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json
 4ccef74 
  
ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json
 d3685a4 


Diff: https://reviews.apache.org/r/59828/diff/2/

Changes: https://reviews.apache.org/r/59828/diff/1-2/


Testing
---

Tested on local cluster, and by the new unit tests added.


Thanks,

Miklos Gergely

Re: Review Request 60113: Exception needs to be handled properly for mail alert

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60113/
---

(Updated Июнь 19, 2017, 9:10 д.п.)


Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-21248
https://issues.apache.org/jira/browse/AMBARI-21248


Repository: ambari


Description
---

In org.apache.ambari.server.state.services.AlertNoticeDispatchService, we have 
below code snippet:


... 
protected void runOneIteration() throws Exception { 
… 
String targetType = target.getNotificationType(); 
NotificationDispatcher dispatcher = 
m_dispatchFactory.getDispatcher(targetType); 
… 
if (dispatcher.isDigestSupported()) { 
...
{code}
If the database has a change to AlertTargetEntity's targetType column (unlikely 
but let's assume it happened), then dispatcher can be null and the subsequent 
if clause will throw an exception. In 
https://github.com/google/guava/blob/master/guava/src/com/google/common/util/concurrent/AbstractScheduledService.java,
 we see that the exception may happen silently since shutDown() and 
notifyFailed() does not log anything. 
... 
{code}
class Task implements Runnable {
@Override
public void run() {
lock.lock();
try {
if (runningTask.isCancelled()) { // task may have been cancelled 
while blocked on the lock. return; }

AbstractScheduledService.this.runOneIteration();
} catch (Throwable t) {
try {
shutDown();
} catch (Exception ignored) {
logger.log(Level.WARNING, "Error while attempting to shut 
down the service after failure.", ignored);
}

notifyFailed(t);
runningTask.cancel(false); // prevent future invocations. 
} finally {
lock.unlock();
}

}
}
}


So, runOneIteration will shutdown the scheduler if it hits an uncaught 
exception. We should wrap a try/catch.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
 2972f51 
  
ambari-server/src/test/java/org/apache/ambari/server/state/services/AlertNoticeDispatchServiceTest.java
 42ee366 


Diff: https://reviews.apache.org/r/60113/diff/3/

Changes: https://reviews.apache.org/r/60113/diff/2-3/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 59877: Custom log4j.properties of Flume

[wang jianfei]

> On 六月 16, 2017, 11:54 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/configuration/flume-log4j.xml
> > Lines 95 (patched)
> > 
> >
> > I didn't catch this earlier, is this really ment to be added when 
> > Ambari is upgraded regardless of the stack version?
> > 
> > What should happen during a stack upgrade?

Thank you for your advice.I take a look at this parameter and find that when 
ambari is going to be upgraded,the properties of new_configs  which contains 
the "  will be added in new Stack while the old 
configs doesn't contains those prpoerties.So,I think it's better to add this.


- wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59877/#review178140
---


On 六月 7, 2017, 10:46 a.m., wang jianfei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59877/
> ---
> 
> (Updated 六月 7, 2017, 10:46 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jaimin Jetly, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-21191
> https://issues.apache.org/jira/browse/AMBARI-21191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently,if you want to config log4j.properties of Flume,you must change the 
> log4j.properties.j2 in the ambari-agent-host installed flume-agent.
> So,we make it configurable by adding log4j.properties.template.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/configuration/flume-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
>  6b1d286 
> 
> 
> Diff: https://reviews.apache.org/r/59877/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> wang jianfei
> 
>

Re: Review Request 60152: Add "zeppelin.server.ssl.port" in Zeppelin's setting

[Prabhjyot Singh]

> On June 17, 2017, 3:10 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-config.xml
> > Lines 40 (patched)
> > 
> >
> > Is this really needed on an Ambari upgrade?
> > What about during stack upgrades.

Yes, this requires Ambari upgrade.
Have made relavent changes for stack upgrade, which only applies when the stack 
is upgraded from 2.5 to 2.6.


- Prabhjyot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60152/#review178128
---


On June 19, 2017, 12:40 p.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60152/
> ---
> 
> (Updated June 19, 2017, 12:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21271
> https://issues.apache.org/jira/browse/AMBARI-21271
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "zeppelin.server.ssl.port" in Zeppelin's setting. This is in associated 
> with the changes made in https://issues.apache.org/jira/browse/ZEPPELIN-1321.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-config.xml
>  85f1859509 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 564dead112 
>   ambari-server/src/test/python/stacks/2.6/configs/default.json 2570657b70 
> 
> 
> Diff: https://reviews.apache.org/r/60152/diff/2/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6.4
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60152: Add "zeppelin.server.ssl.port" in Zeppelin's setting

[Prabhjyot Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60152/
---

(Updated June 19, 2017, 12:40 p.m.)


Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
venkat sairam.


Bugs: AMBARI-21271
https://issues.apache.org/jira/browse/AMBARI-21271


Repository: ambari


Description
---

Add "zeppelin.server.ssl.port" in Zeppelin's setting. This is in associated 
with the changes made in https://issues.apache.org/jira/browse/ZEPPELIN-1321.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-config.xml
 85f1859509 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
564dead112 
  ambari-server/src/test/python/stacks/2.6/configs/default.json 2570657b70 


Diff: https://reviews.apache.org/r/60152/diff/2/

Changes: https://reviews.apache.org/r/60152/diff/1-2/


Testing
---

Manually on CentOS 6.4


Thanks,

Prabhjyot Singh