[jira] [Commented] (AMBARI-12440) RU: hdp-select set all should be called before finalize
[ https://issues.apache.org/jira/browse/AMBARI-12440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631406#comment-14631406 ] Hudson commented on AMBARI-12440: - ABORTED: Integrated in Ambari-trunk-Commit #3130 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3130/]) AMBARI-12440. RU: hdp-select set all should be called before finalize (ncole) (ncole: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=ee08a07c2f49fa23966edf7d7869f593182af96b) * ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java * ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml * ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml * ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_direction.xml * ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java * ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml RU: hdp-select set all should be called before finalize --- Key: AMBARI-12440 URL: https://issues.apache.org/jira/browse/AMBARI-12440 Project: Ambari Issue Type: Bug Components: ambari-server Reporter: Nate Cole Assignee: Nate Cole Fix For: 2.1.1 Attachments: AMBARI-12440.patch Add a cluster group to handle the hdp-select set all task -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12448) Non-Root: Ranger Admin install fails with permission denied
[ https://issues.apache.org/jira/browse/AMBARI-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631407#comment-14631407 ] Hudson commented on AMBARI-12448: - ABORTED: Integrated in Ambari-trunk-Commit #3130 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3130/]) AMBARI-12448. Non-Root: Ranger Admin install fails with permission denied (aonishuk) (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=306f28f550dcac464e25255ac5e5be2bf476f996) * ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py * ambari-common/src/main/python/resource_management/core/providers/system.py * ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py * ambari-common/src/main/python/resource_management/core/sudo.py Non-Root: Ranger Admin install fails with permission denied --- Key: AMBARI-12448 URL: https://issues.apache.org/jira/browse/AMBARI-12448 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12446) Starting ranger enabled services fails if folder to copy JDBC driver does not exist
[
https://issues.apache.org/jira/browse/AMBARI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631244#comment-14631244
]
Hadoop QA commented on AMBARI-12446:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745799/AMBARI-12446.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3417//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3417//console
This message is automatically generated.
Starting ranger enabled services fails if folder to copy JDBC driver does not
exist
---
Key: AMBARI-12446
URL: https://issues.apache.org/jira/browse/AMBARI-12446
Project: Ambari
Issue Type: Bug
Affects Versions: 2.0.0, 2.1.0
Reporter: Velmurugan Periasamy
Assignee: Gautam Borad
Priority: Blocker
Fix For: 2.1.0
Attachments: AMBARI-12446.patch
STR with namenode
-
While installing a cluster, Make sure namenode host does not have
'/usr/share/java' directory .
-- Add Ranger service
-- Enable NameNode HA
Workaround:
--
Make sure '/usr/share/java' exists on the namenode host or any host that has
ranger enabled component.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
[ https://issues.apache.org/jira/browse/AMBARI-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631249#comment-14631249 ] Hudson commented on AMBARI-12447: - SUCCESS: Integrated in Ambari-trunk-Commit #3129 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3129/]) AMBARI-12447.Warn the user if the user is about to delete a host with slaves that are not in decommissioned state (akovalenko) (akovalenko: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=002ea2f5deaf85d05d272c49ea3b1077549961e2) * ambari-web/app/messages.js * ambari-web/app/controllers/main/host/details.js * ambari-web/app/views/main/host/summary.js * ambari-web/app/templates/main/host/details/doDeleteHostPopup.hbs * ambari-web/app/templates/main/host/summary.hbs * ambari-web/app/views/main/host/details/host_component_view.js * ambari-web/test/controllers/main/host/details_test.js Warn the user if the user is about to delete a host with slaves that are not in decommissioned state Key: AMBARI-12447 URL: https://issues.apache.org/jira/browse/AMBARI-12447 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.2.0 Reporter: Aleksandr Kovalenko Assignee: Aleksandr Kovalenko Fix For: 2.2.0 Attachments: AMBARI-12447.patch Should be implemented in Hosts - Host Details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AMBARI-12448) Non-Root: Ranger Admin install fails with permission denied
[ https://issues.apache.org/jira/browse/AMBARI-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Onischuk resolved AMBARI-12448. -- Resolution: Fixed Committed to trunk and branch-2.1 Non-Root: Ranger Admin install fails with permission denied --- Key: AMBARI-12448 URL: https://issues.apache.org/jira/browse/AMBARI-12448 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36568: Non-Root: Ranger Admin install fails with permission denied
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36568/#review92072 --- Ship it! Ship It! - Vitalyi Brodetskyi On Липень 17, 2015, 12:16 після полудня, Andrew Onischuk wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36568/ --- (Updated Липень 17, 2015, 12:16 після полудня) Review request for Ambari and Vitalyi Brodetskyi. Bugs: AMBARI-12448 https://issues.apache.org/jira/browse/AMBARI-12448 Repository: ambari Description --- When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' Diffs - ambari-common/src/main/python/resource_management/core/providers/system.py db29e2c ambari-common/src/main/python/resource_management/core/sudo.py 740eea3 ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 3556ed1 ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py dd41370 Diff: https://reviews.apache.org/r/36568/diff/ Testing --- mvn clean test Thanks, Andrew Onischuk
Review Request 36568: Non-Root: Ranger Admin install fails with permission denied
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36568/ --- Review request for Ambari and Vitalyi Brodetskyi. Bugs: AMBARI-12448 https://issues.apache.org/jira/browse/AMBARI-12448 Repository: ambari Description --- When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' Diffs - ambari-common/src/main/python/resource_management/core/providers/system.py db29e2c ambari-common/src/main/python/resource_management/core/sudo.py 740eea3 ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 3556ed1 ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py dd41370 Diff: https://reviews.apache.org/r/36568/diff/ Testing --- mvn clean test Thanks, Andrew Onischuk
[jira] [Created] (AMBARI-12448) Non-Root: Ranger Admin install fails with permission denied
Andrew Onischuk created AMBARI-12448: Summary: Non-Root: Ranger Admin install fails with permission denied Key: AMBARI-12448 URL: https://issues.apache.org/jira/browse/AMBARI-12448 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-7656) Add support for Hbase REST API
[ https://issues.apache.org/jira/browse/AMBARI-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631379#comment-14631379 ] Rick Kellogg commented on AMBARI-7656: -- Support for automatic startup of the HBase StarGate REST API Gateway would be superb. Without it manual work arounds are required. See the following for details on the startup script: https://wiki.apache.org/hadoop/Hbase/Stargate Add support for Hbase REST API -- Key: AMBARI-7656 URL: https://issues.apache.org/jira/browse/AMBARI-7656 Project: Ambari Issue Type: New Feature Reporter: Marcus Young There is currently no first-class Hbase REST API support out of the box for Ambari. It would be nice to have it install as part of the setup, so that it can be managed via Ganglia, and propagate to every node in a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12448) Non-Root: Ranger Admin install fails with permission denied
[ https://issues.apache.org/jira/browse/AMBARI-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631384#comment-14631384 ] Hudson commented on AMBARI-12448: - SUCCESS: Integrated in Ambari-branch-2.1 #235 (See [https://builds.apache.org/job/Ambari-branch-2.1/235/]) AMBARI-12448. Non-Root: Ranger Admin install fails with permission denied (aonishuk) (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=4c0dcf62e8147a5db0f1a531b700276a8efc296b) * ambari-common/src/main/python/resource_management/core/sudo.py * ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py * ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py * ambari-common/src/main/python/resource_management/core/providers/system.py Non-Root: Ranger Admin install fails with permission denied --- Key: AMBARI-12448 URL: https://issues.apache.org/jira/browse/AMBARI-12448 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 When installing Ranger on Ambari 2.1.0-1426 with 2.3.0.0-2553 with non-root server/agent, umask 027 the Ranger Admin Install fails with the following error: resource_management.core.exceptions.Fail: Execution of 'python /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. Traceback (most recent call last): File /usr/hdp/current/ranger-admin/dba_script.py, line 1412, in module main(sys.argv) File /usr/hdp/current/ranger-admin/dba_script.py, line 1104, in main populate_global_dict() File /usr/hdp/current/ranger-admin/dba_script.py, line 63, in populate_global_dict read_config_file = open(os.path.join(RANGER_ADMIN_HOME,'install.properties')) IOError: [Errno 13] Permission denied: '/usr/hdp/current/ranger-admin/install.properties' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 36569: Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36569/ --- Review request for Ambari and Dmitro Lisnichenko. Bugs: AMBARI-12449 https://issues.apache.org/jira/browse/AMBARI-12449 Repository: ambari Description --- The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide Diffs - ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py 2d3e42c ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py 0169a9b Diff: https://reviews.apache.org/r/36569/diff/ Testing --- mvn clean test Thanks, Andrew Onischuk
Re: Review Request 36569: Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36569/#review92077 --- Ship it! Ship It! - Dmitro Lisnichenko On July 17, 2015, 1:29 p.m., Andrew Onischuk wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36569/ --- (Updated July 17, 2015, 1:29 p.m.) Review request for Ambari and Dmitro Lisnichenko. Bugs: AMBARI-12449 https://issues.apache.org/jira/browse/AMBARI-12449 Repository: ambari Description --- The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide Diffs - ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py 2d3e42c ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py 0169a9b Diff: https://reviews.apache.org/r/36569/diff/ Testing --- mvn clean test Thanks, Andrew Onischuk
[jira] [Created] (AMBARI-12449) Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
Andrew Onischuk created AMBARI-12449: Summary: Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 Key: AMBARI-12449 URL: https://issues.apache.org/jira/browse/AMBARI-12449 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36567: AMBARI-12446 : Starting ranger enabled services fails if folder to copy JDBC driver does not exist
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36567/#review92076 --- Ship it! Ship It! - Sumit Mohanty On July 17, 2015, 10:48 a.m., Gautam Borad wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36567/ --- (Updated July 17, 2015, 10:48 a.m.) Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, Mahadev Konar, Selvamohan Neethiraj, and Velmurugan Periasamy. Bugs: AMBARI-12446 https://issues.apache.org/jira/browse/AMBARI-12446 Repository: ambari Description --- If /usr/share/java/ directory doesn't exist then enable plugin fails to copy db connector file. Diffs - ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py a46448c Diff: https://reviews.apache.org/r/36567/diff/ Testing --- Tested on 3 node isntance by moving /usr/share/java folder and trying to enable Ranger plugin, the dir got created successfully. -- Ran 231 tests in 8.430s OK -- Total run:772 Total errors:0 Total failures:0 OK Thanks, Gautam Borad
[jira] [Resolved] (AMBARI-12449) Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
[ https://issues.apache.org/jira/browse/AMBARI-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Onischuk resolved AMBARI-12449. -- Resolution: Fixed Committed to trunk and branch-2.1 Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 --- Key: AMBARI-12449 URL: https://issues.apache.org/jira/browse/AMBARI-12449 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12429) Deleting a host using the API causes NPE
[ https://issues.apache.org/jira/browse/AMBARI-12429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated AMBARI-12429: - Attachment: (was: AMBARI-12429.patch) Deleting a host using the API causes NPE Key: AMBARI-12429 URL: https://issues.apache.org/jira/browse/AMBARI-12429 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Priority: Blocker Fix For: 2.1.1 Attachments: ambari-server.log There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. ambari-server-2.1.0-2480.x86_64 ambari-server --hash 7d0fd97cc697b22357f4c499b80b8baf9a1c1dbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12429) Deleting a host using the API causes NPE
[ https://issues.apache.org/jira/browse/AMBARI-12429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated AMBARI-12429: - Attachment: (was: AMBARI-12429.branch-2.1.1.patch) Deleting a host using the API causes NPE Key: AMBARI-12429 URL: https://issues.apache.org/jira/browse/AMBARI-12429 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Priority: Blocker Fix For: 2.1.1 Attachments: ambari-server.log There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. ambari-server-2.1.0-2480.x86_64 ambari-server --hash 7d0fd97cc697b22357f4c499b80b8baf9a1c1dbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36564: Deleting a host using the API causes NPE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36564/ --- (Updated July 17, 2015, 6:09 a.m.) Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, John Speidel, Nate Cole, Sumit Mohanty, and Sid Wagle. Changes --- Checked for == null instead of catching NPE exception Bugs: AMBARI-12429 https://issues.apache.org/jira/browse/AMBARI-12429 Repository: ambari Description --- There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java de9ae52 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java 4c14426 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestOperationLevel.java c7c0160 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java fa49d7f ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestOperationLevelDAO.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java 2c11e55 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 665dd56 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 9f25ad7 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestOperationLevelTest.java bd4ad90 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java ed8336e Diff: https://reviews.apache.org/r/36564/diff/ Testing --- System tests passed, see test matrix in comments of AMBARI-12429. Waiting for unit test results. Thanks, Alejandro Fernandez
[jira] [Updated] (AMBARI-12429) Deleting a host using the API causes NPE
[ https://issues.apache.org/jira/browse/AMBARI-12429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated AMBARI-12429: - Attachment: AMBARI-12429.branch-2.1.1.patch AMBARI-12429.patch Deleting a host using the API causes NPE Key: AMBARI-12429 URL: https://issues.apache.org/jira/browse/AMBARI-12429 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Priority: Blocker Fix For: 2.1.1 Attachments: AMBARI-12429.branch-2.1.1.patch, AMBARI-12429.patch, ambari-server.log There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. ambari-server-2.1.0-2480.x86_64 ambari-server --hash 7d0fd97cc697b22357f4c499b80b8baf9a1c1dbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36564: Deleting a host using the API causes NPE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36564/#review92030 --- Ship it! Ship It! - Sid Wagle On July 17, 2015, 6:09 a.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36564/ --- (Updated July 17, 2015, 6:09 a.m.) Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, John Speidel, Nate Cole, Sumit Mohanty, and Sid Wagle. Bugs: AMBARI-12429 https://issues.apache.org/jira/browse/AMBARI-12429 Repository: ambari Description --- There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java de9ae52 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java 4c14426 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestOperationLevel.java c7c0160 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java fa49d7f ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestOperationLevelDAO.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java 2c11e55 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 665dd56 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 9f25ad7 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestOperationLevelTest.java bd4ad90 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java ed8336e Diff: https://reviews.apache.org/r/36564/diff/ Testing --- System tests passed, see test matrix in comments of AMBARI-12429. Waiting for unit test results. Thanks, Alejandro Fernandez
[jira] [Commented] (AMBARI-12429) Deleting a host using the API causes NPE
[ https://issues.apache.org/jira/browse/AMBARI-12429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630951#comment-14630951 ] Hudson commented on AMBARI-12429: - SUCCESS: Integrated in Ambari-trunk-Commit #3128 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3128/]) AMBARI-12429. Deleting a host using the API causes NPE (alejandro) (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=358dc1d8eb361480932380d8546d58b264a25c38) * ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java * ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java * ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestOperationLevelDAO.java * ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java * ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestOperationLevelTest.java * ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java * ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestOperationLevel.java * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java Deleting a host using the API causes NPE Key: AMBARI-12429 URL: https://issues.apache.org/jira/browse/AMBARI-12429 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Priority: Blocker Fix For: 2.1.1 Attachments: AMBARI-12429.branch-2.1.1.patch, AMBARI-12429.patch, ambari-server.log There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. ambari-server-2.1.0-2480.x86_64 ambari-server --hash 7d0fd97cc697b22357f4c499b80b8baf9a1c1dbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12429) Deleting a host using the API causes NPE
[ https://issues.apache.org/jira/browse/AMBARI-12429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630943#comment-14630943 ] Hudson commented on AMBARI-12429: - SUCCESS: Integrated in Ambari-branch-2.1 #234 (See [https://builds.apache.org/job/Ambari-branch-2.1/234/]) AMBARI-12429. Deleting a host using the API causes NPE (alejandro) (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=b577a57cc4701fa24fb8b8253b2c44974e929aee) * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java * ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java * ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java * ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java * ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java * ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestOperationLevelDAO.java * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestOperationLevel.java * ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestOperationLevelEntity.java * ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java * ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestOperationLevelTest.java Deleting a host using the API causes NPE Key: AMBARI-12429 URL: https://issues.apache.org/jira/browse/AMBARI-12429 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Priority: Blocker Fix For: 2.1.1 Attachments: AMBARI-12429.branch-2.1.1.patch, AMBARI-12429.patch, ambari-server.log There are 3 issues while deleting hosts. 1. Created a cluster with multiple hosts, then stopped all of the services on 1 host (preferably one with only clients so it has nothing to stop). Then deleted the host using the API. E.g., curl -u admin:admin -H X-Requested-By: ambari -X DELETE http://c6401.ambari.apache.org:8080/api/v1/hosts/c6404.ambari.apache.org This led to Null Pointer Exceptions in ambari-server because the UI is still generating requests to get the ServiceComponentHost response, which isn't locking code, and makes request to get the HostState (this record has been deleted), so a NPE is thrown. This needs to be more robust; adding locks around here may have other repercussions, so I decided to just check for != null. 2. If a Host with DataNode becomes decommissioned, it will have a record in the requestoperationlevel table, whose records are not currently being deleted when a Host is deleted. 3. There are differences between deleting a Host using the /hosts/name and /clusters/name/hosts/name API. In the former, since no cluster is provided, it blindly deletes the host without checking if it has any masters/slaves on it, which need to be stopped and deleted first. ambari-server-2.1.0-2480.x86_64 ambari-server --hash 7d0fd97cc697b22357f4c499b80b8baf9a1c1dbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12450) Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
Robert Levas created AMBARI-12450:
-
Summary: Kerberos: ServiceResourceProvider queries for KDC
connectivity when not needed
Key: AMBARI-12450
URL: https://issues.apache.org/jira/browse/AMBARI-12450
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.0.0, 2.0.1, 2.1.0
Reporter: Robert Levas
Assignee: Robert Levas
Fix For: 2.1.1
When querying for information about services installed in a Kerberized cluster
via the REST API, the ServiceResourceProvider always attempts to contact the
KDC (or Active Directory) if the KERBEROS service is selected within the query.
This can be seen about every 15 seconds, when the UI queries for the state of
the services in a Kerberized cluster using the following query:
{noformat}
GET
/api/v1/clusters/{cluster_name}/services?fields=ServiceInfo/state,ServiceInfo/maintenance_stateminimal_response=true
{noformat}
The result from this query does not contain the KDC connectivity attributes
(which is expected), yet the detail are obtained.
This issue causes excess overhead in Ambari as well as on the relevant KDC or
Active Directory. Also the kdamin.log fills up with messages like:
{noformat:title=/var/log/kadmind.log}
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:31:42 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:32:49 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:34:35 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:35:28 some-host-1 kadmind[2383](info): closing down fd 29
{noformat}
*Solution*
Only query for the KDC attributes when explicitly or implicitly queried. This
can be done by conditionally setting the relevant properties near
{{org/apache/ambari/server/controller/internal/ServiceResourceProvider.java:1394}}
by inspecting the request for relevant identifiers using something like the
following:
{code}
requestedIds.contains(propertyId) || isPropertyCategoryRequested(propertyId,
requestedIds);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AMBARI-12446) Starting ranger enabled services fails if folder to copy JDBC driver does not exist
[ https://issues.apache.org/jira/browse/AMBARI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631510#comment-14631510 ] Hudson commented on AMBARI-12446: - SUCCESS: Integrated in Ambari-branch-2.1 #236 (See [https://builds.apache.org/job/Ambari-branch-2.1/236/]) AMBARI-12446. Starting ranger enabled services fails if folder to copy JDBC driver does not exist (Gautam Borad via smohanty) (smohanty: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=56640af7af755ba2c3b8336a486b005170c9603b) * ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py Starting ranger enabled services fails if folder to copy JDBC driver does not exist --- Key: AMBARI-12446 URL: https://issues.apache.org/jira/browse/AMBARI-12446 Project: Ambari Issue Type: Bug Affects Versions: 2.0.0, 2.1.0 Reporter: Velmurugan Periasamy Assignee: Gautam Borad Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12446.patch STR with namenode - While installing a cluster, Make sure namenode host does not have '/usr/share/java' directory . -- Add Ranger service -- Enable NameNode HA Workaround: -- Make sure '/usr/share/java' exists on the namenode host or any host that has ranger enabled component. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
[ https://issues.apache.org/jira/browse/AMBARI-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631187#comment-14631187 ] Aleksandr Kovalenko commented on AMBARI-12447: -- committed to trunk Warn the user if the user is about to delete a host with slaves that are not in decommissioned state Key: AMBARI-12447 URL: https://issues.apache.org/jira/browse/AMBARI-12447 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.2.0 Reporter: Aleksandr Kovalenko Assignee: Aleksandr Kovalenko Fix For: 2.2.0 Attachments: AMBARI-12447.patch Should be implemented in Hosts - Host Details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36567: AMBARI-12446 : Starting ranger enabled services fails if folder to copy JDBC driver does not exist
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36567/ --- (Updated July 17, 2015, 10:48 a.m.) Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, Mahadev Konar, Selvamohan Neethiraj, and Velmurugan Periasamy. Bugs: AMBARI-12446 https://issues.apache.org/jira/browse/AMBARI-12446 Repository: ambari Description --- If /usr/share/java/ directory doesn't exist then enable plugin fails to copy db connector file. Diffs - ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py a46448c Diff: https://reviews.apache.org/r/36567/diff/ Testing --- Tested on 3 node isntance by moving /usr/share/java folder and trying to enable Ranger plugin, the dir got created successfully. -- Ran 231 tests in 8.430s OK -- Total run:772 Total errors:0 Total failures:0 OK Thanks, Gautam Borad
[jira] [Updated] (AMBARI-12446) Starting ranger enabled services fails if folder to copy JDBC driver does not exist
[ https://issues.apache.org/jira/browse/AMBARI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautam Borad updated AMBARI-12446: -- Attachment: AMBARI-12446.patch Starting ranger enabled services fails if folder to copy JDBC driver does not exist --- Key: AMBARI-12446 URL: https://issues.apache.org/jira/browse/AMBARI-12446 Project: Ambari Issue Type: Bug Affects Versions: 2.0.0, 2.1.0 Reporter: Velmurugan Periasamy Assignee: Gautam Borad Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12446.patch STR with namenode - While installing a cluster, Make sure namenode host does not have '/usr/share/java' directory . -- Add Ranger service -- Enable NameNode HA Workaround: -- Make sure '/usr/share/java' exists on the namenode host or any host that has ranger enabled component. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
Aleksandr Kovalenko created AMBARI-12447: Summary: Warn the user if the user is about to delete a host with slaves that are not in decommissioned state Key: AMBARI-12447 URL: https://issues.apache.org/jira/browse/AMBARI-12447 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.2.0 Reporter: Aleksandr Kovalenko Assignee: Aleksandr Kovalenko Fix For: 2.2.0 Should be implemented in Hosts - Host Details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
[ https://issues.apache.org/jira/browse/AMBARI-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631096#comment-14631096 ] Andrii Babiichuk commented on AMBARI-12447: --- +1 for the patch Warn the user if the user is about to delete a host with slaves that are not in decommissioned state Key: AMBARI-12447 URL: https://issues.apache.org/jira/browse/AMBARI-12447 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.2.0 Reporter: Aleksandr Kovalenko Assignee: Aleksandr Kovalenko Fix For: 2.2.0 Attachments: AMBARI-12447.patch Should be implemented in Hosts - Host Details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
[ https://issues.apache.org/jira/browse/AMBARI-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Kovalenko updated AMBARI-12447: - Attachment: AMBARI-12447.patch Warn the user if the user is about to delete a host with slaves that are not in decommissioned state Key: AMBARI-12447 URL: https://issues.apache.org/jira/browse/AMBARI-12447 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.2.0 Reporter: Aleksandr Kovalenko Assignee: Aleksandr Kovalenko Fix For: 2.2.0 Attachments: AMBARI-12447.patch Should be implemented in Hosts - Host Details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12447) Warn the user if the user is about to delete a host with slaves that are not in decommissioned state
[
https://issues.apache.org/jira/browse/AMBARI-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631176#comment-14631176
]
Hadoop QA commented on AMBARI-12447:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745795/AMBARI-12447.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3416//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3416//console
This message is automatically generated.
Warn the user if the user is about to delete a host with slaves that are not
in decommissioned state
Key: AMBARI-12447
URL: https://issues.apache.org/jira/browse/AMBARI-12447
Project: Ambari
Issue Type: Bug
Components: ambari-web
Affects Versions: 2.2.0
Reporter: Aleksandr Kovalenko
Assignee: Aleksandr Kovalenko
Fix For: 2.2.0
Attachments: AMBARI-12447.patch
Should be implemented in Hosts - Host Details page.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 36567: AMBARI-12446 : Starting ranger enabled services fails if folder to copy JDBC driver does not exist
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36567/#review92068 --- Ship it! Ship It! - Andrew Onischuk On July 17, 2015, 10:48 a.m., Gautam Borad wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36567/ --- (Updated July 17, 2015, 10:48 a.m.) Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, Mahadev Konar, Selvamohan Neethiraj, and Velmurugan Periasamy. Bugs: AMBARI-12446 https://issues.apache.org/jira/browse/AMBARI-12446 Repository: ambari Description --- If /usr/share/java/ directory doesn't exist then enable plugin fails to copy db connector file. Diffs - ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py a46448c Diff: https://reviews.apache.org/r/36567/diff/ Testing --- Tested on 3 node isntance by moving /usr/share/java folder and trying to enable Ranger plugin, the dir got created successfully. -- Ran 231 tests in 8.430s OK -- Total run:772 Total errors:0 Total failures:0 OK Thanks, Gautam Borad
[jira] [Commented] (AMBARI-12446) Starting ranger enabled services fails if folder to copy JDBC driver does not exist
[ https://issues.apache.org/jira/browse/AMBARI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631569#comment-14631569 ] Hudson commented on AMBARI-12446: - SUCCESS: Integrated in Ambari-trunk-Commit #3131 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3131/]) AMBARI-12446. Starting ranger enabled services fails if folder to copy JDBC driver does not exist (Gautam Borad via smohanty) (smohanty: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=1d644b457cb928d2def3dccf018fdb789d3cee21) * ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py Starting ranger enabled services fails if folder to copy JDBC driver does not exist --- Key: AMBARI-12446 URL: https://issues.apache.org/jira/browse/AMBARI-12446 Project: Ambari Issue Type: Bug Affects Versions: 2.0.0, 2.1.0 Reporter: Velmurugan Periasamy Assignee: Gautam Borad Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12446.patch STR with namenode - While installing a cluster, Make sure namenode host does not have '/usr/share/java' directory . -- Add Ranger service -- Enable NameNode HA Workaround: -- Make sure '/usr/share/java' exists on the namenode host or any host that has ranger enabled component. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AMBARI-11868) Hive service check fails when hive.metastore.warehouse.dir property is modified
[
https://issues.apache.org/jira/browse/AMBARI-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayush Luniya resolved AMBARI-11868.
Resolution: Duplicate
Hive service check fails when hive.metastore.warehouse.dir property is
modified
---
Key: AMBARI-11868
URL: https://issues.apache.org/jira/browse/AMBARI-11868
Project: Ambari
Issue Type: Bug
Components: stacks
Affects Versions: 1.7.0
Environment: Ambari 1.7.0 with HDP 2.2.4.2 stack and Ambari 1.7.1
with PHD 3.0 stack
Reporter: Alexander Denissov
Assignee: Jayush Luniya
When Hive is installed as part of the cluster and
hive.metastore.warehouse.dir property in hive-site is modified to
anything other than /apps/hive/warehouse then HIVE Service Check fails with
the message:
Fail: Execution of 'hadoop --config /etc/hadoop/conf fs -test -e
/apps/hive/warehouse/hcatsmokeida8c06502_date351115' returned 1.
The problem is due to the fact that /apps/hive/warehouse is hardcoded in
ambari-server/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hcat_service_check.py
file. When the actual warehouse location changes, the script has to perform
existence check in the new location.
The solution is to use the variable from params.py, i.e. substitute the lines:
old:
output_file = format(/apps/hive/warehouse/hcatsmoke{unique})
new:
output_file = format({hive_apps_whs_dir}/hcatsmoke{unique})
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (AMBARI-11868) Hive service check fails when hive.metastore.warehouse.dir property is modified
[
https://issues.apache.org/jira/browse/AMBARI-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayush Luniya reassigned AMBARI-11868:
--
Assignee: Jayush Luniya
Hive service check fails when hive.metastore.warehouse.dir property is
modified
---
Key: AMBARI-11868
URL: https://issues.apache.org/jira/browse/AMBARI-11868
Project: Ambari
Issue Type: Bug
Components: stacks
Affects Versions: 1.7.0
Environment: Ambari 1.7.0 with HDP 2.2.4.2 stack and Ambari 1.7.1
with PHD 3.0 stack
Reporter: Alexander Denissov
Assignee: Jayush Luniya
When Hive is installed as part of the cluster and
hive.metastore.warehouse.dir property in hive-site is modified to
anything other than /apps/hive/warehouse then HIVE Service Check fails with
the message:
Fail: Execution of 'hadoop --config /etc/hadoop/conf fs -test -e
/apps/hive/warehouse/hcatsmokeida8c06502_date351115' returned 1.
The problem is due to the fact that /apps/hive/warehouse is hardcoded in
ambari-server/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hcat_service_check.py
file. When the actual warehouse location changes, the script has to perform
existence check in the new location.
The solution is to use the variable from params.py, i.e. substitute the lines:
old:
output_file = format(/apps/hive/warehouse/hcatsmoke{unique})
new:
output_file = format({hive_apps_whs_dir}/hcatsmoke{unique})
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AMBARI-11868) Hive service check fails when hive.metastore.warehouse.dir property is modified
[
https://issues.apache.org/jira/browse/AMBARI-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayush Luniya updated AMBARI-11868:
---
Fix Version/s: 2.1.0
Hive service check fails when hive.metastore.warehouse.dir property is
modified
---
Key: AMBARI-11868
URL: https://issues.apache.org/jira/browse/AMBARI-11868
Project: Ambari
Issue Type: Bug
Components: stacks
Affects Versions: 1.7.0
Environment: Ambari 1.7.0 with HDP 2.2.4.2 stack and Ambari 1.7.1
with PHD 3.0 stack
Reporter: Alexander Denissov
Assignee: Jayush Luniya
Fix For: 2.1.0
When Hive is installed as part of the cluster and
hive.metastore.warehouse.dir property in hive-site is modified to
anything other than /apps/hive/warehouse then HIVE Service Check fails with
the message:
Fail: Execution of 'hadoop --config /etc/hadoop/conf fs -test -e
/apps/hive/warehouse/hcatsmokeida8c06502_date351115' returned 1.
The problem is due to the fact that /apps/hive/warehouse is hardcoded in
ambari-server/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hcat_service_check.py
file. When the actual warehouse location changes, the script has to perform
existence check in the new location.
The solution is to use the variable from params.py, i.e. substitute the lines:
old:
output_file = format(/apps/hive/warehouse/hcatsmoke{unique})
new:
output_file = format({hive_apps_whs_dir}/hcatsmoke{unique})
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 35684: Windows unit tests: Agent unit tests: fix the imports failing patches
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35684/#review92103 --- Review#4 I am getting following errors on Mac == ERROR: test_create_directory_failed_no_parent (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1191, in patched arg = patching.__enter__() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1250, in __enter__ self.target = self.getter() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1412, in lambda getter = lambda: _importer(target) File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1096, in _importer thing = __import__(import_path) ImportError: No module named sudo == ERROR: test_create_directory_not_recursive (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1191, in patched arg = patching.__enter__() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1250, in __enter__ self.target = self.getter() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1412, in lambda getter = lambda: _importer(target) File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1096, in _importer thing = __import__(import_path) ImportError: No module named sudo == ERROR: test_create_directory_path_is_file_or_line (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1191, in patched arg = patching.__enter__() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1250, in __enter__ self.target = self.getter() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1412, in lambda getter = lambda: _importer(target) File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1096, in _importer thing = __import__(import_path) ImportError: No module named sudo == ERROR: test_create_directory_recursive (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1191, in patched arg = patching.__enter__() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1250, in __enter__ self.target = self.getter() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1412, in lambda getter = lambda: _importer(target) File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1096, in _importer thing = __import__(import_path) ImportError: No module named sudo == ERROR: test_delete_directory (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1191, in patched arg = patching.__enter__() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1250, in __enter__ self.target = self.getter() File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1412, in lambda getter = lambda: _importer(target) File /Users/jluniya/merge/trunk/ambari/ambari-common/src/test/python/mock/mock.py, line 1096, in _importer thing = __import__(import_path) ImportError: No module named sudo == ERROR: test_delete_directory_with_path_to_file (TestDirectoryResource.TestDirectoryResource) -- Traceback (most recent call last): File
[jira] [Commented] (AMBARI-12449) Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
[ https://issues.apache.org/jira/browse/AMBARI-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631636#comment-14631636 ] Hudson commented on AMBARI-12449: - FAILURE: Integrated in Ambari-branch-2.1 #237 (See [https://builds.apache.org/job/Ambari-branch-2.1/237/]) AMBARI-12449. Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 (aonishuk) (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=4570ca3190f7b5448fa70a1ed31fb3d16e1fdb94) * ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py * ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 --- Key: AMBARI-12449 URL: https://issues.apache.org/jira/browse/AMBARI-12449 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
Myroslav Papirkovskyy created AMBARI-12451: -- Summary: Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Myroslav Papirkovskyy updated AMBARI-12451: --- Attachment: AMBARI-12451.patch Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631775#comment-14631775 ] Mahadev konar commented on AMBARI-12451: +1 for the patch. Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12449) Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027
[ https://issues.apache.org/jira/browse/AMBARI-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631728#comment-14631728 ] Hudson commented on AMBARI-12449: - SUCCESS: Integrated in Ambari-trunk-Commit #3132 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3132/]) AMBARI-12449. Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 (aonishuk) (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=36b89a16a368ff254004529d80bf003ceae523bb) * ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py * ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py Ranger KMS after some time becomes stopped on non-root agent + systemwide umask 027 --- Key: AMBARI-12449 URL: https://issues.apache.org/jira/browse/AMBARI-12449 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 The reason why we didn't see this in QE builds is because seems like QE use ambari-only umask not system-wide -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631780#comment-14631780 ] Myroslav Papirkovskyy commented on AMBARI-12451: pushed to trunk and branch-2.1 Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Myroslav Papirkovskyy resolved AMBARI-12451. Resolution: Fixed Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12443) On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7
[ https://issues.apache.org/jira/browse/AMBARI-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Di Li updated AMBARI-12443: --- Attachment: AMBARI-12443.patch On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7 --- Key: AMBARI-12443 URL: https://issues.apache.org/jira/browse/AMBARI-12443 Project: Ambari Issue Type: Bug Affects Versions: 2.1.0 Reporter: Di Li Fix For: 2.1.1 Attachments: AMBARI-12443.patch, hdfs_property_labels.jpg On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 Review back to step 7 Customize Services For example, the dfs.datanode.http.address is displayed as DataNode See screenshot attached for details. The dfs.datanode.http.address is show with the correct label when navigate from step 6 Assign Slaves and Clients to step 7 Customize Services -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12443) On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7
[ https://issues.apache.org/jira/browse/AMBARI-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Di Li updated AMBARI-12443: --- Description: On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 Review back to step 7 Customize Services For example, the dfs.datanode.http.address is displayed as DataNode See screenshot attached for details. The dfs.datanode.http.address is show with the correct label when navigate from step 6 Assign Slaves and Clients to step 7 Customize Services was: On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7 For example, the dfs.datanode.http.address is displayed as DataNode See screenshot attached for details. On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7 --- Key: AMBARI-12443 URL: https://issues.apache.org/jira/browse/AMBARI-12443 Project: Ambari Issue Type: Bug Affects Versions: 2.1.0 Reporter: Di Li Fix For: 2.1.1 Attachments: hdfs_property_labels.jpg On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 Review back to step 7 Customize Services For example, the dfs.datanode.http.address is displayed as DataNode See screenshot attached for details. The dfs.datanode.http.address is show with the correct label when navigate from step 6 Assign Slaves and Clients to step 7 Customize Services -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
On July 18, 2015, 1:22 a.m., Sumit Mohanty wrote: ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java, line 705 https://reviews.apache.org/r/36587/diff/1/?file=1015041#file1015041line705 Wont it allow multiple threads to get in? Sid Wagle wrote: Not sure I follow the question. Only the lock owner can proceed with initProviderMaps() other would wait on it, all waiting ones will see the first one has set initialized to true and exit after acquiring the lock. Subsequent threads will never as for lock since initialized will be true from then on until next reset. Sumit Mohanty wrote: I thought the read lock may be held simultaneously by multiple reader threads Yes, that is right multiple readers can hOld the lock which still solves deadlock problem But we could have multiple threads updating provider maps. Alternative is to use compareAnsSet on automiC boolean which means only 1 thread performs init however if that init fails other reader might miss a chance, i gUess that might be ok too, thoughts? - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92148 --- On July 17, 2015, 11:45 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 17, 2015, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
On July 18, 2015, 1:22 a.m., Sumit Mohanty wrote: ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java, line 705 https://reviews.apache.org/r/36587/diff/1/?file=1015041#file1015041line705 Wont it allow multiple threads to get in? Sid Wagle wrote: Not sure I follow the question. Only the lock owner can proceed with initProviderMaps() other would wait on it, all waiting ones will see the first one has set initialized to true and exit after acquiring the lock. Subsequent threads will never as for lock since initialized will be true from then on until next reset. Sumit Mohanty wrote: I thought the read lock may be held simultaneously by multiple reader threads Sid Wagle wrote: Yes, that is right multiple readers can hOld the lock which still solves deadlock problem But we could have multiple threads updating provider maps. Alternative is to use compareAnsSet on automiC boolean which means only 1 thread performs init however if that init fails other reader might miss a chance, i gUess that might be ok too, thoughts? Actually, multiple calls to init is OK as they simply overwrite the old value in the maps. Your choice. - Sumit --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92148 --- On July 18, 2015, 4:44 a.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 18, 2015, 4:44 a.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 18, 2015, 5:39 a.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Changes --- Missing initialization code added. Thanks Sumit. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
[jira] [Updated] (AMBARI-12455) RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
[ https://issues.apache.org/jira/browse/AMBARI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated AMBARI-12455: - Attachment: AMBARI-12455.patch RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc --- Key: AMBARI-12455 URL: https://issues.apache.org/jira/browse/AMBARI-12455 Project: Ambari Issue Type: Story Components: ambari-agent Affects Versions: 2.0.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Fix For: 2.1.2 Attachments: AMBARI-12455.patch Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade. https://docs.google.com/document/d/1MkXNbc9R8Rj11R2WyHsrmDQ2sTGhjYrrOoUPbAdOkEU/edit?disco=AQ1qHbU This can be a simple Python/SQL script that * On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services. * If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED. * If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED. * If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED. For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
On July 18, 2015, 1:22 a.m., Sumit Mohanty wrote: ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java, line 705 https://reviews.apache.org/r/36587/diff/1/?file=1015041#file1015041line705 Wont it allow multiple threads to get in? Sid Wagle wrote: Not sure I follow the question. Only the lock owner can proceed with initProviderMaps() other would wait on it, all waiting ones will see the first one has set initialized to true and exit after acquiring the lock. Subsequent threads will never as for lock since initialized will be true from then on until next reset. I thought the read lock may be held simultaneously by multiple reader threads - Sumit --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92148 --- On July 17, 2015, 11:45 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 17, 2015, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
On July 18, 2015, 4:54 a.m., Sumit Mohanty wrote: ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java, line 722 https://reviews.apache.org/r/36587/diff/2/?file=1015164#file1015164line722 Did these get missed in the new code? I did not see the new HashMap calls. Intentionally left out, old was re-initializing for no good reason. There was no point in clear all the maps since we replace the values anyways. Although please make sure you also put it through the scanner. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92165 --- On July 18, 2015, 4:44 a.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 18, 2015, 4:44 a.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
[jira] [Created] (AMBARI-12455) RU - Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
Alejandro Fernandez created AMBARI-12455: Summary: RU - Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc Key: AMBARI-12455 URL: https://issues.apache.org/jira/browse/AMBARI-12455 Project: Ambari Issue Type: Story Components: ambari-agent Affects Versions: 2.0.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Fix For: 2.1.2 Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade. https://docs.google.com/document/d/1MkXNbc9R8Rj11R2WyHsrmDQ2sTGhjYrrOoUPbAdOkEU/edit?disco=AQ1qHbU This can be a simple Python/SQL script that * On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services. * If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED. * If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED. * If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED. For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12455) RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
[
https://issues.apache.org/jira/browse/AMBARI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632243#comment-14632243
]
Hadoop QA commented on AMBARI-12455:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745924/AMBARI-12455.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
ambari-server.
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3422//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3422//console
This message is automatically generated.
RU - Magician Script to correct data inconsistencies, allow retrying repo
installation, force finalize to versions, etc
---
Key: AMBARI-12455
URL: https://issues.apache.org/jira/browse/AMBARI-12455
Project: Ambari
Issue Type: Story
Components: ambari-agent
Affects Versions: 2.0.0
Reporter: Alejandro Fernandez
Assignee: Alejandro Fernandez
Fix For: 2.1.2
Attachments: AMBARI-12455.patch
Support has identified the need to come up with a script to fix any
mismatches in the database, or identify problems during Rolling Upgrade.
This can be a simple Python script that,
* On a newly installed cluster, ensures that there is at least one cluster
version whose state is CURRENT. If not, will advise the user to restart
services.
* If the user has Registered and Installed repos, check that each one has a
unique version and display name. Further, if any are stuck in an INSTALLING
state, will let the user take three potential actions: leave as is, force to
INSTALLED, force to INSTALL_FAILED.
* If the user has Registered and Installed repos, and one cluster_version is
already in an UPGRADING state, perhaps because hdp-select changed the
symlinks and a component was restarted, or the user inadvertently started a
manual upgrade, will allow the user to force it back to INSTALLED.
* If the user in the in the middle of an upgrade, and they want to force one
of the versions are CURRENT, it will update all DB records accordingly, and
set the previously CURRENT version to INSTALLED.
For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 18, 2015, 4:44 a.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Changes --- I just realized a global initialize breaks multi cluster support, of course this is not currently something we support fully but I decided to take into account your suggestion as well, this code shold only execute initilize once, the unit test for changing JMX port works with this patch. Will run full suite on Monday. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92165 --- ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java (line 706) https://reviews.apache.org/r/36587/#comment146160 Did these get missed in the new code? I did not see the new HashMap calls. - Sumit Mohanty On July 18, 2015, 4:44 a.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 18, 2015, 4:44 a.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Review Request 36592: RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36592/ --- Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle. Bugs: AMBARI-12455 https://issues.apache.org/jira/browse/AMBARI-12455 Repository: ambari Description --- Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade. This can be a simple Python script that, - On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services. - If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED. - If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED. - If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED. For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres. Diffs - ambari-server/src/main/resources/scripts/ru_magician.py PRE-CREATION Diff: https://reviews.apache.org/r/36592/diff/ Testing --- Tested on Ambari 2.1.0 with Postgres, and Ambari 2.0.1 on MySQL (using local and external DB), all of the scenarios above. I still have to do more thorough testing in the cases of Finalize, but this is a good preliminary patch to start getting some feedback. I could have used SQLAlchemy as my ORM, but I really wanted a standalone script that could be a quick v1, so I went for raw SQL that works on both Postgres MySQL. Thanks, Alejandro Fernandez
Re: Review Request 36592: RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36592/#review92151 --- ambari-server/src/main/resources/scripts/ru_magician.py (line 225) https://reviews.apache.org/r/36592/#comment146132 License = LGPL ambari-server/src/main/resources/scripts/ru_magician.py (line 231) https://reviews.apache.org/r/36592/#comment146133 License = GPL - Sid Wagle On July 18, 2015, 1:58 a.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36592/ --- (Updated July 18, 2015, 1:58 a.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Sid Wagle, and Yusaku Sako. Bugs: AMBARI-12455 https://issues.apache.org/jira/browse/AMBARI-12455 Repository: ambari Description --- Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade. This can be a simple Python script that, - On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services. - If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED. - If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED. - If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED. For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres. Diffs - ambari-server/src/main/resources/scripts/ru_magician.py PRE-CREATION Diff: https://reviews.apache.org/r/36592/diff/ Testing --- Tested on Ambari 2.1.0 with Postgres, and Ambari 2.0.1 on MySQL (using local and external DB), all of the scenarios above. I still have to do more thorough testing in the cases of Finalize, but this is a good preliminary patch to start getting some feedback. I could have used SQLAlchemy as my ORM, but I really wanted a standalone script that could be a quick v1, so I went for raw SQL that works on both Postgres MySQL. Thanks, Alejandro Fernandez
[jira] [Updated] (AMBARI-12455) RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc
[ https://issues.apache.org/jira/browse/AMBARI-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated AMBARI-12455: - Summary: RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc (was: RU - Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc) RU - Magician Script to correct data inconsistencies, allow retrying repo installation, force finalize to versions, etc --- Key: AMBARI-12455 URL: https://issues.apache.org/jira/browse/AMBARI-12455 Project: Ambari Issue Type: Story Components: ambari-agent Affects Versions: 2.0.0 Reporter: Alejandro Fernandez Assignee: Alejandro Fernandez Fix For: 2.1.2 Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade. https://docs.google.com/document/d/1MkXNbc9R8Rj11R2WyHsrmDQ2sTGhjYrrOoUPbAdOkEU/edit?disco=AQ1qHbU This can be a simple Python/SQL script that * On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services. * If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED. * If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED. * If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED. For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 36581: Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36581/
---
Review request for Ambari, John Speidel, Robert Nettleton, and Tom Beerbower.
Bugs: AMBARI-12450
https://issues.apache.org/jira/browse/AMBARI-12450
Repository: ambari
Description
---
When querying for information about services installed in a Kerberized cluster
via the REST API, the ServiceResourceProvider always attempts to contact the
KDC (or Active Directory) if the KERBEROS service is selected within the query.
This can be seen about every 15 seconds, when the UI queries for the state of
the services in a Kerberized cluster using the following query:
```
GET
/api/v1/clusters/{cluster_name}/services?fields=ServiceInfo/state,ServiceInfo/maintenance_stateminimal_response=true
```
The result from this query does not contain the KDC connectivity attributes
(which is expected), yet the detail are obtained.
This issue causes excess overhead in Ambari as well as on the relevant KDC or
Active Directory. Also the kdamin.log fills up with messages like:
```
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:31:42 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:32:49 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:34:35 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:35:28 some-host-1 kadmind[2383](info): closing down fd 29
```
#Solution
Only query for the KDC attributes when explicitly or implicitly queried. This
can be done by conditionally setting the relevant properties near
`org/apache/ambari/server/controller/internal/ServiceResourceProvider.java:1394`
by inspecting the request for relevant identifiers using something like the
following:
```
requestedIds.contains(propertyId) || isPropertyCategoryRequested(propertyId,
requestedIds);
```
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseProvider.java
ca5e70e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
a13bbd3
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ServiceResourceProviderTest.java
9ec1610
Diff: https://reviews.apache.org/r/36581/diff/
Testing
---
Manually tested using various query strings.
#Local test results:
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 46:16.723s
[INFO] Finished at: Fri Jul 17 15:51:36 EDT 2015
[INFO] Final Memory: 47M/724M
[INFO]
#Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 36581: Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36581/
---
(Updated July 17, 2015, 4:07 p.m.)
Review request for Ambari, John Speidel, Robert Nettleton, and Tom Beerbower.
Bugs: AMBARI-12450
https://issues.apache.org/jira/browse/AMBARI-12450
Repository: ambari
Description (updated)
---
When querying for information about services installed in a Kerberized cluster
via the REST API, the ServiceResourceProvider always attempts to contact the
KDC (or Active Directory) if the KERBEROS service is selected within the query.
This can be seen about every 15 seconds, when the UI queries for the state of
the services in a Kerberized cluster using the following query:
```
GET
/api/v1/clusters/{cluster_name}/services?fields=ServiceInfo/state,ServiceInfo/maintenance_stateminimal_response=true
```
The result from this query does not contain the KDC connectivity attributes
(which is expected), yet the detail are obtained.
This issue causes excess overhead in Ambari as well as on the relevant KDC or
Active Directory. Also the kdamin.log fills up with messages like:
```
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:31:42 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:32:49 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:34:35 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:35:28 some-host-1 kadmind[2383](info): closing down fd 29
```
#Solution
Only query for the KDC attributes when explicitly or implicitly queried. This
can be done by conditionally setting the relevant properties near
`org/apache/ambari/server/controller/internal/ServiceResourceProvider.java:1394`
by inspecting the request for relevant identifiers using something like the
following:
```
requestedIds.contains(propertyId) || isPropertyCategoryRequested(propertyId,
requestedIds) || isPropertyEntryRequested(propertyId, requestedIds)
```
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseProvider.java
ca5e70e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
a13bbd3
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ServiceResourceProviderTest.java
9ec1610
Diff: https://reviews.apache.org/r/36581/diff/
Testing
---
Manually tested using various query strings.
#Local test results:
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 46:16.723s
[INFO] Finished at: Fri Jul 17 15:51:36 EDT 2015
[INFO] Final Memory: 47M/724M
[INFO]
#Jenkins test results: PENDING
Thanks,
Robert Levas
[jira] [Commented] (AMBARI-12453) Ambari server deadlock causing cluster create to be stuck
[ https://issues.apache.org/jira/browse/AMBARI-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632101#comment-14632101 ] Siddharth Wagle commented on AMBARI-12453: -- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Ambari server deadlock causing cluster create to be stuck - Key: AMBARI-12453 URL: https://issues.apache.org/jira/browse/AMBARI-12453 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Siddharth Wagle Assignee: Siddharth Wagle Priority: Critical Fix For: 2.1.1 Deadlock appears due to conflicting locking mechanisms used by the Property Provider module. Attaching jstack for the deadlock. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12453) Ambari server deadlock causing cluster create to be stuck
[ https://issues.apache.org/jira/browse/AMBARI-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle updated AMBARI-12453: - Attachment: jstack.out Ambari server deadlock causing cluster create to be stuck - Key: AMBARI-12453 URL: https://issues.apache.org/jira/browse/AMBARI-12453 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Siddharth Wagle Assignee: Siddharth Wagle Priority: Critical Fix For: 2.1.1 Attachments: jstack.out Deadlock appears due to conflicting locking mechanisms used by the Property Provider module. Attaching jstack for the deadlock. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12452) Template widget type should show n/a if no data available
Xi Wang created AMBARI-12452: Summary: Template widget type should show n/a if no data available Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12454) Metric Monitor Start Failed with error ambari-metrics-monitor already running
Siddharth Wagle created AMBARI-12454:
Summary: Metric Monitor Start Failed with error
ambari-metrics-monitor already running
Key: AMBARI-12454
URL: https://issues.apache.org/jira/browse/AMBARI-12454
Project: Ambari
Issue Type: Bug
Components: ambari-metrics
Affects Versions: 2.1.0
Reporter: Siddharth Wagle
Assignee: Siddharth Wagle
Priority: Critical
Fix For: 2.1.1
Metrics monitor start script should not fail if daemon already running with
correct PID
{code}
Checking for previously running Metric Monitor...
tput: No value for $TERM and no -T specified
ERROR: ambari-metrics-monitor already running
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Review Request 36589: Metric Monitor Start Failed with error ambari-metrics-monitor already running
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36589/ --- Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12454 https://issues.apache.org/jira/browse/AMBARI-12454 Repository: ambari Description --- Metrics monitor start script should not fail if daemon already running with correct PID Checking for previously running Metric Monitor... tput: No value for $TERM and no -T specified ERROR: ambari-metrics-monitor already running Diffs - ambari-metrics/ambari-metrics-host-monitoring/conf/unix/ambari-metrics-monitor 86c2ac7 Diff: https://reviews.apache.org/r/36589/diff/ Testing --- Manually verified: [root@ams-test-1 ~]# su - ams -c '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf start' psutil build directory is not empty, continuing... Verifying Python version compatibility... Using python /usr/bin/python2.6 Checking for previously running Metric Monitor... WARN: ambari-metrics-monitor already running with PID: 6090 Exiting. Thanks, Sid Wagle
[jira] [Commented] (AMBARI-12452) Template widget type should show n/a if no data available
[
https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632132#comment-14632132
]
Hadoop QA commented on AMBARI-12452:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745905/AMBARI-12452.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3420//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3420//console
This message is automatically generated.
Template widget type should show n/a if no data available
-
Key: AMBARI-12452
URL: https://issues.apache.org/jira/browse/AMBARI-12452
Project: Ambari
Issue Type: Bug
Components: ambari-web
Affects Versions: 2.1.1
Reporter: Xi Wang
Assignee: Xi Wang
Fix For: 2.1.1
Attachments: AMBARI-12452.patch, after patch 2.png, after patch.png
If Template consists of an expression and any metric value is not available
then template widget type should show n/a instead of the metric. Currently it
seems to be showing empty string which is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AMBARI-12453) Ambari server deadlock causing cluster create to be stuck
[
https://issues.apache.org/jira/browse/AMBARI-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632134#comment-14632134
]
Hadoop QA commented on AMBARI-12453:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745911/jstack.out
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3421//console
This message is automatically generated.
Ambari server deadlock causing cluster create to be stuck
-
Key: AMBARI-12453
URL: https://issues.apache.org/jira/browse/AMBARI-12453
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.1.0
Reporter: Siddharth Wagle
Assignee: Siddharth Wagle
Priority: Critical
Fix For: 2.1.1
Attachments: jstack.out
Deadlock appears due to conflicting locking mechanisms used by the Property
Provider module.
Attaching jstack for the deadlock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AMBARI-12450) Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
[
https://issues.apache.org/jira/browse/AMBARI-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-12450:
--
Attachment: AMBARI-12450_01.patch
Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
--
Key: AMBARI-12450
URL: https://issues.apache.org/jira/browse/AMBARI-12450
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.0.0, 2.0.1, 2.1.0
Reporter: Robert Levas
Assignee: Robert Levas
Labels: kerberos, rest_api
Fix For: 2.1.1
Attachments: AMBARI-12450_01.patch
When querying for information about services installed in a Kerberized
cluster via the REST API, the ServiceResourceProvider always attempts to
contact the KDC (or Active Directory) if the KERBEROS service is selected
within the query.
This can be seen about every 15 seconds, when the UI queries for the state
of the services in a Kerberized cluster using the following query:
{noformat}
GET
/api/v1/clusters/{cluster_name}/services?fields=ServiceInfo/state,ServiceInfo/maintenance_stateminimal_response=true
{noformat}
The result from this query does not contain the KDC connectivity attributes
(which is expected), yet the detail are obtained.
This issue causes excess overhead in Ambari as well as on the relevant KDC or
Active Directory. Also the kdamin.log fills up with messages like:
{noformat:title=/var/log/kadmind.log}
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:31:42 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:32:49 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:34:35 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:35:28 some-host-1 kadmind[2383](info): closing down fd 29
{noformat}
*Solution*
Only query for the KDC attributes when explicitly or implicitly queried. This
can be done by conditionally setting the relevant properties near
{{org/apache/ambari/server/controller/internal/ServiceResourceProvider.java:1394}}
by inspecting the request for relevant identifiers using something like the
following:
{code}
requestedIds.contains(propertyId) || isPropertyCategoryRequested(propertyId,
requestedIds);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631913#comment-14631913 ] Hudson commented on AMBARI-12451: - SUCCESS: Integrated in Ambari-branch-2.1 #238 (See [https://builds.apache.org/job/Ambari-branch-2.1/238/]) AMBARI-12451. Ranger configurations missing after HDP 2.2-2.3 upgrade. (mpapirkovskyy) (mpapyrkovskyy: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=fc89328b7c7c0af7eb8f87bb5a804a2a9e937a8c) * ambari-server/src/main/resources/upgrade/catalog/UpgradeCatalog_2.2_to_2.3.json * ambari-server/src/main/resources/upgrade/catalog/UpgradeCatalog_2.2_to_2.3_step2.json Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12452) Template widget type should show n/a if no data available
[ https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xi Wang updated AMBARI-12452: - Attachment: after patch 2.png after patch.png Template widget type should show n/a if no data available - Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: after patch 2.png, after patch.png If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12452) Template widget type should show n/a if no data available
[ https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xi Wang updated AMBARI-12452: - Attachment: AMBARI-12452.patch Template widget type should show n/a if no data available - Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: AMBARI-12452.patch, after patch 2.png, after patch.png If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12451) Ranger configurations missing after HDP 2.2-2.3 upgrade
[ https://issues.apache.org/jira/browse/AMBARI-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631897#comment-14631897 ] Hudson commented on AMBARI-12451: - SUCCESS: Integrated in Ambari-trunk-Commit #3133 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3133/]) AMBARI-12451. Ranger configurations missing after HDP 2.2-2.3 upgrade. (mpapirkovskyy) (mpapyrkovskyy: http://git-wip-us.apache.org/repos/asf?p=ambari.gita=commith=7997bf0b53cd1e4e40e73aa443d5fdf80ba34da8) * ambari-server/src/main/resources/upgrade/catalog/UpgradeCatalog_2.2_to_2.3_step2.json * ambari-server/src/main/resources/upgrade/catalog/UpgradeCatalog_2.2_to_2.3.json Ranger configurations missing after HDP 2.2-2.3 upgrade Key: AMBARI-12451 URL: https://issues.apache.org/jira/browse/AMBARI-12451 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Myroslav Papirkovskyy Assignee: Myroslav Papirkovskyy Priority: Blocker Fix For: 2.1.0 Attachments: AMBARI-12451.patch After 2.2-2.3 manual upgrade, 2.3 Advanced properties for Ranger Service and plugin related config on components (hdfs,hive,hbase,knox,storm,yarn,kafka) Ambari UI after successful Set Current HDP Version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12452) Template widget type should show n/a if no data available
[ https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xi Wang updated AMBARI-12452: - Attachment: AMBARI-12452.patch Template widget type should show n/a if no data available - Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: AMBARI-12452.patch, after patch 2.png, after patch.png If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12443) On Install wizard UI - Some advanced HDFS properties show with wrong label when navigate from step 8 back to step 7
[
https://issues.apache.org/jira/browse/AMBARI-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632117#comment-14632117
]
Hadoop QA commented on AMBARI-12443:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745872/AMBARI-12443.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3419//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3419//console
This message is automatically generated.
On Install wizard UI - Some advanced HDFS properties show with wrong label
when navigate from step 8 back to step 7
---
Key: AMBARI-12443
URL: https://issues.apache.org/jira/browse/AMBARI-12443
Project: Ambari
Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Di Li
Assignee: Di Li
Fix For: 2.1.1
Attachments: AMBARI-12443.patch, hdfs_property_labels.jpg
On Install wizard UI - Some advanced HDFS properties show with wrong label
when navigate from step 8 Review back to step 7 Customize Services
For example, the dfs.datanode.http.address is displayed as DataNode
See screenshot attached for details.
The dfs.datanode.http.address is show with the correct label when navigate
from step 6 Assign Slaves and Clients to step 7 Customize Services
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
On July 18, 2015, 1:22 a.m., Sumit Mohanty wrote: ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java, line 705 https://reviews.apache.org/r/36587/diff/1/?file=1015041#file1015041line705 Wont it allow multiple threads to get in? Not sure I follow the question. Only the lock owner can proceed with initProviderMaps() other would wait on it, all waiting ones will see the first one has set initialized to true and exit after acquiring the lock. Subsequent threads will never as for lock since initialized will be true from then on until next reset. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92148 --- On July 17, 2015, 11:45 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 17, 2015, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
[jira] [Commented] (AMBARI-12206) Ambari Metrics refresh and alerts icons are overflowed
[ https://issues.apache.org/jira/browse/AMBARI-12206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631866#comment-14631866 ] Jaimin D Jetly commented on AMBARI-12206: - +1 for the patch. Ambari Metrics refresh and alerts icons are overflowed -- Key: AMBARI-12206 URL: https://issues.apache.org/jira/browse/AMBARI-12206 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.0 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: AMBARI-12206.patch, AMBARI-12206.patch, ambari-metrics-correct-big.png, ambari-metrics-correct-small.png, ambari-metrics-icons-issue.png Original Estimate: 12h Remaining Estimate: 12h On services menu, the refresh icon and alerts icon got overflowed for Ambari Metrics. See attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12452) Template widget type should show n/a if no data available
[ https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xi Wang updated AMBARI-12452: - Attachment: (was: AMBARI-12452.patch) Template widget type should show n/a if no data available - Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: after patch 2.png, after patch.png If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12452) Template widget type should show n/a if no data available
[ https://issues.apache.org/jira/browse/AMBARI-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632032#comment-14632032 ] Xi Wang commented on AMBARI-12452: -- 6475 tests complete (11 seconds) 90 tests pending Template widget type should show n/a if no data available - Key: AMBARI-12452 URL: https://issues.apache.org/jira/browse/AMBARI-12452 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.1 Reporter: Xi Wang Assignee: Xi Wang Fix For: 2.1.1 Attachments: after patch 2.png, after patch.png If Template consists of an expression and any metric value is not available then template widget type should show n/a instead of the metric. Currently it seems to be showing empty string which is confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
[jira] [Commented] (AMBARI-12383) Gauge warning gets overflowed on service summary page
[ https://issues.apache.org/jira/browse/AMBARI-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631863#comment-14631863 ] Jaimin D Jetly commented on AMBARI-12383: - +1 for the patch. Gauge warning gets overflowed on service summary page -- Key: AMBARI-12383 URL: https://issues.apache.org/jira/browse/AMBARI-12383 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 2.1.0 Reporter: Xi Wang Assignee: Xi Wang Priority: Critical Fix For: 2.1.1 Attachments: AMBARI-12383.patch, after-patch.png, overflowed.png -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12453) Ambari server deadlock causing cluster create to be stuck
Siddharth Wagle created AMBARI-12453: Summary: Ambari server deadlock causing cluster create to be stuck Key: AMBARI-12453 URL: https://issues.apache.org/jira/browse/AMBARI-12453 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.1.0 Reporter: Siddharth Wagle Assignee: Siddharth Wagle Priority: Critical Fix For: 2.1.1 Deadlock appears due to conflicting locking mechanisms used by the Property Provider module. Attaching jstack for the deadlock. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12450) Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
[
https://issues.apache.org/jira/browse/AMBARI-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632099#comment-14632099
]
Hadoop QA commented on AMBARI-12450:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745877/AMBARI-12450_01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
ambari-server.
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/3418//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/3418//console
This message is automatically generated.
Kerberos: ServiceResourceProvider queries for KDC connectivity when not needed
--
Key: AMBARI-12450
URL: https://issues.apache.org/jira/browse/AMBARI-12450
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.0.0, 2.0.1, 2.1.0
Reporter: Robert Levas
Assignee: Robert Levas
Labels: kerberos, rest_api
Fix For: 2.1.1
Attachments: AMBARI-12450_01.patch
When querying for information about services installed in a Kerberized
cluster via the REST API, the ServiceResourceProvider always attempts to
contact the KDC (or Active Directory) if the KERBEROS service is selected
within the query.
This can be seen about every 15 seconds, when the UI queries for the state
of the services in a Kerberized cluster using the following query:
{noformat}
GET
/api/v1/clusters/{cluster_name}/services?fields=ServiceInfo/state,ServiceInfo/maintenance_stateminimal_response=true
{noformat}
The result from this query does not contain the KDC connectivity attributes
(which is expected), yet the detail are obtained.
This issue causes excess overhead in Ambari as well as on the relevant KDC or
Active Directory. Also the kdamin.log fills up with messages like:
{noformat:title=/var/log/kadmind.log}
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:31:42 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:31:42 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:32:49 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:32:49 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:34:35 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:34:35 some-host-1 kadmind[2383](info): closing down fd 29
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request: kadm5_init,
admin/ad...@example.com, success, client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128, vers=3, flavor=6
Jun 29 14:35:28 some-host-1 kadmind[2383](Notice): Request:
kadm5_get_principal, admin/ad...@example.com, success,
client=admin/ad...@example.com,
service=kadmin/some-host-1.c.pramod-thangali.inter...@example.com,
addr=10.240.70.128
Jun 29 14:35:28 some-host-1 kadmind[2383](info): closing down fd 29
{noformat}
*Solution*
Only query for the KDC attributes when explicitly or implicitly queried. This
can be done by
Re: Review Request 36589: Metric Monitor Start Failed with error ambari-metrics-monitor already running
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36589/ --- (Updated July 17, 2015, 11:47 p.m.) Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12454 https://issues.apache.org/jira/browse/AMBARI-12454 Repository: ambari Description --- Metrics monitor start script should not fail if daemon already running with correct PID Checking for previously running Metric Monitor... tput: No value for $TERM and no -T specified ERROR: ambari-metrics-monitor already running Diffs - ambari-metrics/ambari-metrics-host-monitoring/conf/unix/ambari-metrics-monitor 86c2ac7 Diff: https://reviews.apache.org/r/36589/diff/ Testing (updated) --- Manually verified: [root@ams-test-1 ~]# su - ams -c '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf start' psutil build directory is not empty, continuing... Verifying Python version compatibility... Using python /usr/bin/python2.6 Checking for previously running Metric Monitor... WARN: ambari-metrics-monitor already running with PID: 6090 Exiting. [root@ams-test-1 ~]# echo $? 0 Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92141 --- Ship it! Ship It! - Alejandro Fernandez On July 17, 2015, 11:45 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 17, 2015, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
Re: Review Request 36589: Metric Monitor Start Failed with error ambari-metrics-monitor already running
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36589/#review92143 --- Ship it! Ship It! - Alejandro Fernandez On July 17, 2015, 11:47 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36589/ --- (Updated July 17, 2015, 11:47 p.m.) Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12454 https://issues.apache.org/jira/browse/AMBARI-12454 Repository: ambari Description --- Metrics monitor start script should not fail if daemon already running with correct PID Checking for previously running Metric Monitor... tput: No value for $TERM and no -T specified ERROR: ambari-metrics-monitor already running Diffs - ambari-metrics/ambari-metrics-host-monitoring/conf/unix/ambari-metrics-monitor 86c2ac7 Diff: https://reviews.apache.org/r/36589/diff/ Testing --- Manually verified: [root@ams-test-1 ~]# su - ams -c '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf start' psutil build directory is not empty, continuing... Verifying Python version compatibility... Using python /usr/bin/python2.6 Checking for previously running Metric Monitor... WARN: ambari-metrics-monitor already running with PID: 6090 Exiting. [root@ams-test-1 ~]# echo $? 0 Thanks, Sid Wagle
Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/#review92148 --- ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java (line 684) https://reviews.apache.org/r/36587/#comment146115 Wont it allow multiple threads to get in? - Sumit Mohanty On July 17, 2015, 11:45 p.m., Sid Wagle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36587/ --- (Updated July 17, 2015, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev Konar, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-12453 https://issues.apache.org/jira/browse/AMBARI-12453 Repository: ambari Description --- The high level picture seems to be: Requests made from the UI for host metrics trying to figure out the actual metrics service and the code that updates in-memory maps which hold information of where that service is and what ports to use to connect to it etc. These property maps are update by Observers on important events like Cluster/Service/Host CRUD by resetting a volatile variable. The contention occurs due the thread that actually enters the monitor protecting the volatile var and asks for another lock on the cluster which is held by some other thread which also needs a value from the in-memory maps and waits on the object monitor that it cannot enter. Note: Web based deployments get away because not a lot of CRUD ops happen in parallel to Reads like the use case of monitoring the Blueprint deploy as the cluster is being provisioned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 380a0fe Diff: https://reviews.apache.org/r/36587/diff/ Testing --- All unit test passed. Thanks, Sid Wagle
