[jira] [Commented] (AMBARI-22635) Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed
[ https://issues.apache.org/jira/browse/AMBARI-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292102#comment-16292102 ] Hudson commented on AMBARI-22635: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8524 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8524/]) AMBARI-22635: Addendum fix Ambari should create a dummy core-site.xml (vishalsuvagia: [http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f1f730229fe559439189726c0cd40cd58cbf0135]) * (edit) ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/setup_ranger_kafka.py > Ambari should create a dummy core-site.xml for Ranger plugins when namenode > is not installed > > > Key: AMBARI-22635 > URL: https://issues.apache.org/jira/browse/AMBARI-22635 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0, 2.6.1 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22635-branch-2.6.1.patch, > AMBARI-22635-branch-2.6.2.patch, AMBARI-22635-branch-2.6.patch, > AMBARI-22635-trunk.1-addendum.patch, AMBARI-22635-trunk.1.patch, > AMBARI-22635-trunk.patch > > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22635) Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed
[ https://issues.apache.org/jira/browse/AMBARI-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292073#comment-16292073 ] Vishal Suvagia commented on AMBARI-22635: - [^AMBARI-22635-trunk.1-addendum.patch] pushed to [trunk|https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=f1f730229fe559439189726c0cd40cd58cbf0135] > Ambari should create a dummy core-site.xml for Ranger plugins when namenode > is not installed > > > Key: AMBARI-22635 > URL: https://issues.apache.org/jira/browse/AMBARI-22635 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0, 2.6.1 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22635-branch-2.6.1.patch, > AMBARI-22635-branch-2.6.2.patch, AMBARI-22635-branch-2.6.patch, > AMBARI-22635-trunk.1-addendum.patch, AMBARI-22635-trunk.1.patch, > AMBARI-22635-trunk.patch > > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22655) Livy/Livy2 Unable To Start Due to Address Already In Use
Jonathan Hurley created AMBARI-22655: Summary: Livy/Livy2 Unable To Start Due to Address Already In Use Key: AMBARI-22655 URL: https://issues.apache.org/jira/browse/AMBARI-22655 Project: Ambari Issue Type: Bug Affects Versions: 2.6.1 Reporter: Jonathan Hurley Assignee: Jonathan Hurley Priority: Critical Fix For: 2.6.2 While restarting Livy and Livy2 on a non-root cluster, the following is seen: {code} 17/12/14 14:36:23 WARN LivyConf: The configuration key livy.repl.enableHiveContext has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.repl.enable-hive-context instead. 17/12/14 14:36:23 WARN LivyConf: The configuration key livy.server.csrf_protection.enabled has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.server.csrf-protection.enabled instead. 17/12/14 14:36:23 INFO AccessManager: AccessControlManager acls disabled;users with view permission: ;users with modify permission: ;users with super permission: cstm-zeppelin;other allowed users: * 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Welcome to 17/12/14 14:36:28 INFO LineBufferedStream: stdout: __ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: / __/__ ___ _/ /__ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: _\ \/ _ \/ _ `/ __/ '_/ 17/12/14 14:36:28 INFO LineBufferedStream: stdout:/___/ .__/\_,_/_/ /_/\_\ version 2.2.0.2.6.4.0-73 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /_/ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Using Scala version 2.11.8, OpenJDK 64-Bit Server VM, 1.8.0_131 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Branch HEAD 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Compiled by user jenkins on 2017-12-13T19:08:32Z 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Revision a24017869f5450397136ee8b11be818e7cd3facb 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Url g...@github.com:hortonworks/spark2.git 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Type --help for more information. 17/12/14 14:36:29 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 17/12/14 14:36:31 INFO AHSProxy: Connecting to Application History server at nat-yc-r7-ovvs-ambari-autostart-4-re-2.openstacklocal/172.22.121.144:10200 17/12/14 14:36:32 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 17/12/14 14:36:33 INFO StateStore$: Using FileSystemStateStore for recovery. 17/12/14 14:36:33 INFO BatchSessionManager: Recovered 0 batch sessions. Next session id: 0 17/12/14 14:36:33 INFO InteractiveSessionManager: Recovered 0 interactive sessions. Next session id: 0 17/12/14 14:36:33 INFO InteractiveSessionManager: Heartbeat watchdog thread started. 17/12/14 14:36:33 INFO LivyServer: SPNEGO auth enabled (principal = HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com) 17/12/14 14:36:33 INFO LivyServer: CSRF protection is enabled. 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Login using keytab /etc/security/keytabs/spnego.service.keytab, for principal HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Map server: nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklocal to principal: [HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com], added = true 17/12/14 14:36:34 WARN AbstractLifeCycle: FAILED ServerConnector@df1cff6{SSL-http/1.1}{0.0.0.0:8999}: java.net.BindException: Address already in use java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.server.Server.doStart(Server.java:366) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.livy.server.WebServer.start(WebServer.scala:92) at org.apache.livy.server.LivyServer.start(LivyServer.scala:259) at org.apache.livy.server.LivyServer$.main(LivyServer.scala:339) at
[jira] [Commented] (AMBARI-22648) zeppelin server keytab missing from zeppelin-site.xml
[ https://issues.apache.org/jira/browse/AMBARI-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291885#comment-16291885 ] Hudson commented on AMBARI-22648: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8523 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8523/]) AMBARI-22648: zeppelin server keytab missing from zeppelin-site.xml (jluniya: [http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=d95d3ebeaf9e5334828ebc5d16fa223895f49f36]) * (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py * (edit) ambari-server/src/test/python/stacks/2.6/configs/default.json > zeppelin server keytab missing from zeppelin-site.xml > - > > Key: AMBARI-22648 > URL: https://issues.apache.org/jira/browse/AMBARI-22648 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: Bikas Saha >Assignee: Bikas Saha >Priority: Critical > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22648.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22635) Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed
[ https://issues.apache.org/jira/browse/AMBARI-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291873#comment-16291873 ] Hadoop QA commented on AMBARI-22635: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12902073/AMBARI-22635-branch-2.6.2.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12846//console This message is automatically generated. > Ambari should create a dummy core-site.xml for Ranger plugins when namenode > is not installed > > > Key: AMBARI-22635 > URL: https://issues.apache.org/jira/browse/AMBARI-22635 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0, 2.6.1 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22635-branch-2.6.1.patch, > AMBARI-22635-branch-2.6.2.patch, AMBARI-22635-branch-2.6.patch, > AMBARI-22635-trunk.1-addendum.patch, AMBARI-22635-trunk.1.patch, > AMBARI-22635-trunk.patch > > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22642) LDAPS sync Connection Refused
[ https://issues.apache.org/jira/browse/AMBARI-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291871#comment-16291871 ] Hadoop QA commented on AMBARI-22642: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12902088/ambari-22642.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 core tests{color}. The patch passed unit tests in ambari-server. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12845//console This message is automatically generated. > LDAPS sync Connection Refused > -- > > Key: AMBARI-22642 > URL: https://issues.apache.org/jira/browse/AMBARI-22642 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 > Environment: java version "1.8.0_121" > Java(TM) SE Runtime Environment (build 1.8.0_121-tdc1-b13) > Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) > AD Domain Controllers > LDAP v.3 > 2012 R2 OS >Reporter: David F. Quiroga >Priority: Minor > Labels: easyfix, patch > Attachments: ambari-22642.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Ambari server configured to use "secure" ldap authentication. > authentication.ldap.primaryUrl=:636 > authentication.ldap.useSSL=true > We call the ldap_sync_events REST endpoint frequently to synchronize > existing groups and a specific list groups. We had no issues with this until > mid-October at which point we began to see: > {code} > "status" : "ERROR", > "status_detail" : "Caught exception running LDAP sync. simple bind > failed: **:636; nested exception is > javax.naming.CommunicationException: simple bind failed: **:636 [Root > exception is java.net.SocketException: Connection reset]", > {code} > Troubleshooting: > * We saw random success and failure when attempting to sync a single group. > * With useSSL=false and an updated port ldap sync was consistently successful. > Cause: > * By default, ldap connection only uses pooled connections when connecting to > a directory server over LDAP. Enabling SSL causes it to disable the pooling, > resulting in poorer performance and failures due to connection resets. > * Around mid-October we increased the number of groups defined on the system > (50+), this pushed us outside the "safe zone". > Fix: > Enable the SSL connections pooling by adding the below argument to startup > options. > -Dcom.sun.jndi.ldap.connect.pool.protocol='plain ssl' > Reference: > [https://confluence.atlassian.com/jirakb/connecting-jira-to-active-directory-over-ldaps-fails-with-connection-reset-763004137.htm] > [https://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html] > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22648) zeppelin server keytab missing from zeppelin-site.xml
[ https://issues.apache.org/jira/browse/AMBARI-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayush Luniya updated AMBARI-22648: --- Affects Version/s: (was: 2.6.1) 2.6.0 > zeppelin server keytab missing from zeppelin-site.xml > - > > Key: AMBARI-22648 > URL: https://issues.apache.org/jira/browse/AMBARI-22648 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: Bikas Saha >Assignee: Bikas Saha >Priority: Critical > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22648.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error
[ https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291848#comment-16291848 ] Hadoop QA commented on AMBARI-22629: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12902106/AMBARI-22629.branch-2.6.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12844//console This message is automatically generated. > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error > - > > Key: AMBARI-22629 > URL: https://issues.apache.org/jira/browse/AMBARI-22629 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0 >Reporter: Robert Levas >Assignee: Eugene Chekanskiy > Labels: blueprints, kerberos > Fix For: 2.6.2 > > Attachments: AMBARI-22629.branch-2.6.patch, > blueprint_single_node_zk.json, cluster_template_single_node_zk.json, > screenshot-error-dialog.png > > > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error: > {noformat} > The data directory has not been set. Generated data can not be stored. > {noformat} > !screenshot-error-dialog.png! > This is caused by an invalid security state set for the installed components > since the appropriate state is not set while enabling Kerberos during the > installation process: > {noformat} > ambari=> select * from hostcomponentstate; > id | cluster_id | component_name | version| current_state | host_id > | service_name | upgrade_state | security_state > ++--+--+---+-+--+---+ > 1 | 2 | KERBEROS_CLIENT | UNKNOWN | INSTALLED | 1 > | KERBEROS | NONE | UNSECURED > 2 | 2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED | 1 > | ZOOKEEPER| NONE | UNSECURED > 3 | 2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED | 1 > | ZOOKEEPER| NONE | UNSECURED > {noformat} > The expected state for each component is {{SECURED}}, not {{UNSECURED}}. > Because of this, Ambari _thinks_ there is no work to be done, causing this > issue. > *Steps to reproduce*: > # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs > are installed on the Ambari server host with the krb5.conf setup properly > # Install Blueprint - [^blueprint_single_node_zk.json] > # Create clister - [^cluster_template_single_node_zk.json] > # When cluster is created, Kerberos should be enabled and all services up > # Disable Kerberos - error occurs during Unkerberize Cluster task. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22648) zeppelin server keytab missing from zeppelin-site.xml
[ https://issues.apache.org/jira/browse/AMBARI-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayush Luniya updated AMBARI-22648: --- Fix Version/s: trunk, 2.6.2 > zeppelin server keytab missing from zeppelin-site.xml > - > > Key: AMBARI-22648 > URL: https://issues.apache.org/jira/browse/AMBARI-22648 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.1 >Reporter: Bikas Saha >Assignee: Bikas Saha >Priority: Critical > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22648.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22648) zeppelin server keytab missing from zeppelin-site.xml
[ https://issues.apache.org/jira/browse/AMBARI-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayush Luniya updated AMBARI-22648: --- Affects Version/s: 2.6.1 > zeppelin server keytab missing from zeppelin-site.xml > - > > Key: AMBARI-22648 > URL: https://issues.apache.org/jira/browse/AMBARI-22648 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.1 >Reporter: Bikas Saha >Assignee: Bikas Saha >Priority: Critical > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22648.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22648) zeppelin server keytab missing from zeppelin-site.xml
[ https://issues.apache.org/jira/browse/AMBARI-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291846#comment-16291846 ] Jayush Luniya commented on AMBARI-22648: Committed to Trunk commit d95d3ebeaf9e5334828ebc5d16fa223895f49f36 Author: Jayush LuniyaDate: Thu Dec 14 16:41:43 2017 -0800 AMBARI-22648: zeppelin server keytab missing from zeppelin-site.xml (Bikas Saha via jluniya) > zeppelin server keytab missing from zeppelin-site.xml > - > > Key: AMBARI-22648 > URL: https://issues.apache.org/jira/browse/AMBARI-22648 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.1 >Reporter: Bikas Saha >Assignee: Bikas Saha >Priority: Critical > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22648.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapan Shridhar updated AMBARI-22649: - Attachment: AMBARI-22649.1.patch > Library for querying cluster_settings and stack_settings in command*.json. > -- > > Key: AMBARI-22649 > URL: https://issues.apache.org/jira/browse/AMBARI-22649 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Reporter: Swapan Shridhar >Assignee: Swapan Shridhar > Fix For: 3.0.0 > > Attachments: AMBARI-22649.1.patch, AMBARI-22649.patch > > > Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced > "cluster settings" in Ambari. > *=* > *Library for querying _clusterSettings_ and _stackSettings_ for its contents > in command\*.json.* > *=* > One should be able to query for a given *clusterSettings* or *stackSettings*: > - by passing in the setting name(one or more) in order to get it back as > key-value map, or > - just get the value back for a passed-in setting. > *Functions for clusterSettings:* > *-* > - get_cluster_setting_entries(setting_names) : > -- Retrieves the passed-in cluster setting entr(y/ies) and their values > as a map. >If 'setting_names' is passed-in as None : all the settings names and > their corresponding values will be returned as map. >If 'setting_names' is passed-in as empty set : None will be returned. > - get_cluster_setting_value(setting_name) : > -- Retrieves the passed-in cluster setting entry's value. > - is_security_enabled() : > -- Retrieves the cluster's security status. > *Functions for stackSettings:* > *-* > Stack settings as of now has 5 settings : stack_name, stack_root, > stack_features, stack_tools, stack_packages. stack_name, stack_root have > string as values, whereas stack_features, stack_tools, stack_packages have > values as JSON. Further there already exists python functions in files : > *stack_features.py*, *stack_tools.py* and *stack_select.py*. >- get_stack_setting_entries(setting_names) : > -- Retrieves the passed-in stack setting entr(y/ies) and their values > as a map. > If 'setting_names' is passed-in as None, all the settings names > and their corresponding values will be returned as map. > If 'setting_names' is passed-in as empty set : None will be > returned. >- get_stack_setting_value(setting_name): > -- Retrieves the passed-in stack setting entry's value. > - get_stack_name(): > -- Retrieves the stack name. > - get_stack_root(): >-- Retrieves the stack root. > > *Modifications in _stack_features.py, stack_tools.py and stack_select.py_ > files:* > *--* > - Given that these already exist and as of now they read the relevant stack > setting from *configurations/cluster_env*. > - Thus, code has been added to try reading from /stackSettings first by > calling the new fn.() get_stack_setting_value(). if setting not found, go for > the fall back *configurations/cluster_env* (which would be removed soon, > when we remove cluster_env). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290572#comment-16290572 ] Swapan Shridhar edited comment on AMBARI-22649 at 12/14/17 9:00 PM: *Testing:* Tested on live cluster *=* *clusterSettings:* *=* *A.* get_cluster_setting_entries(): ** - 1. Retrieve *single* setting : 'recovery_enabled' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled']) *o/p*: {'recovery_enabled': True} - 2. Retrieve *two* settings : 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 3. Retrieve settings where passed in empty set -> Expected nothing returned -- In get_cluster_setting_entries(). Passed-in setting(s) : set([]) *o/p*: None - 4. Retrieve *three* settings : 'smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False, 'smokeuser': 'ambari-qa'} - 5. Retrieve *three* settings where *middle setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'abc', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 6. Retrieve *three* settings where *1st setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['abc', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 7. Retrieve *three* settings where *last setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce', 'abc']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 8. Retrieve passed in setting which is *non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1']) *o/p* : None - 9. Retrieve *two* passed in settings and both are non-existent -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1', 'non-existent2']) *o/p* : None - 10. Retrieve settings where set passed in is None. -> *returns all settings.* -- In get_cluster_setting_entries(). Passed-in setting(s) : None *o/p*: {'security_enabled': 'false', 'namenode_rolling_restart_timeout': '4200', 'enable_external_ranger': 'false', 'override_uid': 'true', 'kerberos_domain': 'EXAMPLE.COM', 'one_dir_per_partition': 'false', 'agent_mounts_ignore_list': '', 'repo_ubuntu_template': '{{package_type}} {{base_url}} {{components}}', 'ignore_groupsusers_create': 'false', 'alerts_repeat_tolerance': '1', 'hide_yarn_memory_widget': 'false', 'fetch_nonlocal_groups': 'true', 'manage_dirs_on_root': 'true', 'recovery_lifetime_max_count': '1024', 'recovery_type': 'AUTO_START', 'ignore_bad_mounts': 'false', 'recovery_window_in_minutes': '60', 'sysprep_skip_copy_tarballs_hdfs': 'false', 'user_group': 'hadoop', 'namenode_rolling_restart_safemode_exit_timeout': '3600', 'recovery_retry_interval': '5', 'sysprep_skip_copy_oozie_share_lib_to_hdfs': 'false', 'sysprep_skip_setup_jce': 'false', 'manage_hive_fsroot': 'true', 'service_check_type': 'full', 'recovery_enabled': 'true', 'recovery_max_count': '6', 'sysprep_skip_create_users_and_groups': 'false', 'smokeuser_keytab': '/etc/security/keytabs/smokeuser.headless.keytab', 'managed_hdfs_resource_property_names': 'false', 'smokeuser': 'ambari-qa', 'sysprep_skip_copy_fast_jar_hdfs': 'false'} - 11. Retrieve settings. Passed is in List (Expected Set) -- In get_cluster_setting_entries(). Passed-in setting(s) : ['recovery_enabled'] 'setting_names' type expected to be a set. Passed-in type is : **o/p**: None *B.* get_cluster_setting_value(): ** - 1. Retrieve cluster_setting : 'hide_yarn_memory_widget' -- In get_cluster_setting_value(). Passed-in setting : hide_yarn_memory_widget *o/p* : false - 2. Retrieve cluster_setting : 'recovery_max_count' -- In get_cluster_setting_value(). Passed-in setting : recovery_max_count *o/p* : 6 - 3. Retrieve cluster_setting where passed in setting is 'non_existing' --> Empty string -- In get_cluster_setting_value(). Passed-in setting : non_existing *o/p* : None - 4. Retrieve cluster_setting setting passed in is "None" -- In get_cluster_setting_value(). Passed-in setting : None *o/p* : None
[jira] [Commented] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
[ https://issues.apache.org/jira/browse/AMBARI-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291460#comment-16291460 ] Maciej Sitarz commented on AMBARI-22654: [~rlevas] I attached screen captures which show the problem ( I forgot to mark the INSTALL steps, but that should be obvious). Also I attached a sample stack to reproduce the problem. Script files contain just a stub, nothing is installed or started, but it's enough to show the problem. One question about stack scripts. MASTER and CLIENT components have INSTALL and CONFIGURE functions. Install is being run, but I was not able to the log from configure function. > Kerberos: Kerberos-related tasks occur after INSTALL stage of services being > added > -- > > Key: AMBARI-22654 > URL: https://issues.apache.org/jira/browse/AMBARI-22654 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.4.3, 2.5.2 >Reporter: Maciej Sitarz >Assignee: Robert Levas >Priority: Critical > Attachments: SAMPLESRV2_v1.tgz, all_hosts.png, host1_master.png, > host2_master.png, host3_slave.png, host4_client.png, host5_slave_client.png > > > Kerberos-related configs (keytab,principal creation) are not applied before > INSTALL command is built on add service. > This occurs when new services and components are added to an existing cluster > where Kerberos is enabled. Due to the order Kerberos-related configurations > (keytabs, principals) will not be available for new services INSTALL step > functions. > This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
[ https://issues.apache.org/jira/browse/AMBARI-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Sitarz updated AMBARI-22654: --- Attachment: host5_slave_client.png host4_client.png host3_slave.png host2_master.png host1_master.png all_hosts.png Screen captures of the "Install, Start and Test" step from "Add Service Wizard" > Kerberos: Kerberos-related tasks occur after INSTALL stage of services being > added > -- > > Key: AMBARI-22654 > URL: https://issues.apache.org/jira/browse/AMBARI-22654 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.4.3, 2.5.2 >Reporter: Maciej Sitarz >Assignee: Robert Levas >Priority: Critical > Attachments: SAMPLESRV2_v1.tgz, all_hosts.png, host1_master.png, > host2_master.png, host3_slave.png, host4_client.png, host5_slave_client.png > > > Kerberos-related configs (keytab,principal creation) are not applied before > INSTALL command is built on add service. > This occurs when new services and components are added to an existing cluster > where Kerberos is enabled. Due to the order Kerberos-related configurations > (keytabs, principals) will not be available for new services INSTALL step > functions. > This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
[ https://issues.apache.org/jira/browse/AMBARI-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Sitarz updated AMBARI-22654: --- Attachment: SAMPLESRV2_v1.tgz Sample stack to reproduce the issue > Kerberos: Kerberos-related tasks occur after INSTALL stage of services being > added > -- > > Key: AMBARI-22654 > URL: https://issues.apache.org/jira/browse/AMBARI-22654 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.4.3, 2.5.2 >Reporter: Maciej Sitarz >Assignee: Robert Levas >Priority: Critical > Attachments: SAMPLESRV2_v1.tgz > > > Kerberos-related configs (keytab,principal creation) are not applied before > INSTALL command is built on add service. > This occurs when new services and components are added to an existing cluster > where Kerberos is enabled. Due to the order Kerberos-related configurations > (keytabs, principals) will not be available for new services INSTALL step > functions. > This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error
[ https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chekanskiy updated AMBARI-22629: --- Attachment: AMBARI-22629.branch-2.6.patch > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error > - > > Key: AMBARI-22629 > URL: https://issues.apache.org/jira/browse/AMBARI-22629 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0 >Reporter: Robert Levas >Assignee: Eugene Chekanskiy > Labels: blueprints, kerberos > Fix For: 2.6.2 > > Attachments: AMBARI-22629.branch-2.6.patch, > blueprint_single_node_zk.json, cluster_template_single_node_zk.json, > screenshot-error-dialog.png > > > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error: > {noformat} > The data directory has not been set. Generated data can not be stored. > {noformat} > !screenshot-error-dialog.png! > This is caused by an invalid security state set for the installed components > since the appropriate state is not set while enabling Kerberos during the > installation process: > {noformat} > ambari=> select * from hostcomponentstate; > id | cluster_id | component_name | version| current_state | host_id > | service_name | upgrade_state | security_state > ++--+--+---+-+--+---+ > 1 | 2 | KERBEROS_CLIENT | UNKNOWN | INSTALLED | 1 > | KERBEROS | NONE | UNSECURED > 2 | 2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED | 1 > | ZOOKEEPER| NONE | UNSECURED > 3 | 2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED | 1 > | ZOOKEEPER| NONE | UNSECURED > {noformat} > The expected state for each component is {{SECURED}}, not {{UNSECURED}}. > Because of this, Ambari _thinks_ there is no work to be done, causing this > issue. > *Steps to reproduce*: > # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs > are installed on the Ambari server host with the krb5.conf setup properly > # Install Blueprint - [^blueprint_single_node_zk.json] > # Create clister - [^cluster_template_single_node_zk.json] > # When cluster is created, Kerberos should be enabled and all services up > # Disable Kerberos - error occurs during Unkerberize Cluster task. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22629) Disabling Kerberos after enabled during Blueprint install fails with missing data directory error
[ https://issues.apache.org/jira/browse/AMBARI-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chekanskiy updated AMBARI-22629: --- Status: Patch Available (was: Open) > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error > - > > Key: AMBARI-22629 > URL: https://issues.apache.org/jira/browse/AMBARI-22629 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0 >Reporter: Robert Levas >Assignee: Eugene Chekanskiy > Labels: blueprints, kerberos > Fix For: 2.6.2 > > Attachments: AMBARI-22629.branch-2.6.patch, > blueprint_single_node_zk.json, cluster_template_single_node_zk.json, > screenshot-error-dialog.png > > > Disabling Kerberos after enabled during Blueprint install fails with missing > data directory error: > {noformat} > The data directory has not been set. Generated data can not be stored. > {noformat} > !screenshot-error-dialog.png! > This is caused by an invalid security state set for the installed components > since the appropriate state is not set while enabling Kerberos during the > installation process: > {noformat} > ambari=> select * from hostcomponentstate; > id | cluster_id | component_name | version| current_state | host_id > | service_name | upgrade_state | security_state > ++--+--+---+-+--+---+ > 1 | 2 | KERBEROS_CLIENT | UNKNOWN | INSTALLED | 1 > | KERBEROS | NONE | UNSECURED > 2 | 2 | ZOOKEEPER_CLIENT | 2.5.0.0-1245 | INSTALLED | 1 > | ZOOKEEPER| NONE | UNSECURED > 3 | 2 | ZOOKEEPER_SERVER | 2.5.0.0-1245 | STARTED | 1 > | ZOOKEEPER| NONE | UNSECURED > {noformat} > The expected state for each component is {{SECURED}}, not {{UNSECURED}}. > Because of this, Ambari _thinks_ there is no work to be done, causing this > issue. > *Steps to reproduce*: > # Setup Ambari, ensure KDC is installed on some host and Kerberos client libs > are installed on the Ambari server host with the krb5.conf setup properly > # Install Blueprint - [^blueprint_single_node_zk.json] > # Create clister - [^cluster_template_single_node_zk.json] > # When cluster is created, Kerberos should be enabled and all services up > # Disable Kerberos - error occurs during Unkerberize Cluster task. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
[ https://issues.apache.org/jira/browse/AMBARI-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Levas reassigned AMBARI-22654: - Assignee: Robert Levas > Kerberos: Kerberos-related tasks occur after INSTALL stage of services being > added > -- > > Key: AMBARI-22654 > URL: https://issues.apache.org/jira/browse/AMBARI-22654 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.4.3, 2.5.2 >Reporter: Maciej Sitarz >Assignee: Robert Levas >Priority: Critical > > Kerberos-related configs (keytab,principal creation) are not applied before > INSTALL command is built on add service. > This occurs when new services and components are added to an existing cluster > where Kerberos is enabled. Due to the order Kerberos-related configurations > (keytabs, principals) will not be available for new services INSTALL step > functions. > This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
[ https://issues.apache.org/jira/browse/AMBARI-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291116#comment-16291116 ] Robert Levas commented on AMBARI-22654: --- [~Maciej.Sitarz] There is a known issue related to Kerberos-related configurations (or existing services) are not be updated when new services (or components) are added to a cluster where Kerberos has been enabled. This is due to an ambiguity issue where some configurations should be updated and some should be left unchanged. Ambari is not yet smart enough to know what which fall under those categories. As for your issue, new services should be properly configured for Kerberos and needed Kerberos identities should be created when transitioning from INIT to INSTALL. I have not come across a scenario where this was not happening for a while... especially Ambari 2.5.2. However there may be a case where a Yarn-related Kerberos identity is not created when installing Hive or the interactive Hive server (LLAP, HSI, ?). If you have a set of steps that can be used to reproduce the issue. If so, I will be happy to run through it and see that the issue may be. > Kerberos: Kerberos-related tasks occur after INSTALL stage of services being > added > -- > > Key: AMBARI-22654 > URL: https://issues.apache.org/jira/browse/AMBARI-22654 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.4.3, 2.5.2 >Reporter: Maciej Sitarz >Priority: Critical > > Kerberos-related configs (keytab,principal creation) are not applied before > INSTALL command is built on add service. > This occurs when new services and components are added to an existing cluster > where Kerberos is enabled. Due to the order Kerberos-related configurations > (keytabs, principals) will not be available for new services INSTALL step > functions. > This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22654) Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added
Maciej Sitarz created AMBARI-22654: -- Summary: Kerberos: Kerberos-related tasks occur after INSTALL stage of services being added Key: AMBARI-22654 URL: https://issues.apache.org/jira/browse/AMBARI-22654 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.4.3, 2.5.2 Reporter: Maciej Sitarz Priority: Critical Kerberos-related configs (keytab,principal creation) are not applied before INSTALL command is built on add service. This occurs when new services and components are added to an existing cluster where Kerberos is enabled. Due to the order Kerberos-related configurations (keytabs, principals) will not be available for new services INSTALL step functions. This seems to be identical problem to the one mentioned in #AMBARI-17772 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22642) LDAPS sync Connection Refused
[ https://issues.apache.org/jira/browse/AMBARI-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290996#comment-16290996 ] David F. Quiroga commented on AMBARI-22642: --- New tests shouldn't be needed. Only updating JVM opts here. > LDAPS sync Connection Refused > -- > > Key: AMBARI-22642 > URL: https://issues.apache.org/jira/browse/AMBARI-22642 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 > Environment: java version "1.8.0_121" > Java(TM) SE Runtime Environment (build 1.8.0_121-tdc1-b13) > Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) > AD Domain Controllers > LDAP v.3 > 2012 R2 OS >Reporter: David F. Quiroga >Priority: Minor > Labels: easyfix, patch > Attachments: ambari-22642.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Ambari server configured to use "secure" ldap authentication. > authentication.ldap.primaryUrl=:636 > authentication.ldap.useSSL=true > We call the ldap_sync_events REST endpoint frequently to synchronize > existing groups and a specific list groups. We had no issues with this until > mid-October at which point we began to see: > {code} > "status" : "ERROR", > "status_detail" : "Caught exception running LDAP sync. simple bind > failed: **:636; nested exception is > javax.naming.CommunicationException: simple bind failed: **:636 [Root > exception is java.net.SocketException: Connection reset]", > {code} > Troubleshooting: > * We saw random success and failure when attempting to sync a single group. > * With useSSL=false and an updated port ldap sync was consistently successful. > Cause: > * By default, ldap connection only uses pooled connections when connecting to > a directory server over LDAP. Enabling SSL causes it to disable the pooling, > resulting in poorer performance and failures due to connection resets. > * Around mid-October we increased the number of groups defined on the system > (50+), this pushed us outside the "safe zone". > Fix: > Enable the SSL connections pooling by adding the below argument to startup > options. > -Dcom.sun.jndi.ldap.connect.pool.protocol='plain ssl' > Reference: > [https://confluence.atlassian.com/jirakb/connecting-jira-to-active-directory-over-ldaps-fails-with-connection-reset-763004137.htm] > [https://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html] > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22642) LDAPS sync Connection Refused
[ https://issues.apache.org/jira/browse/AMBARI-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David F. Quiroga updated AMBARI-22642: -- Attachment: ambari-22642.patch Regenerated the patch using git diff > LDAPS sync Connection Refused > -- > > Key: AMBARI-22642 > URL: https://issues.apache.org/jira/browse/AMBARI-22642 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 > Environment: java version "1.8.0_121" > Java(TM) SE Runtime Environment (build 1.8.0_121-tdc1-b13) > Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) > AD Domain Controllers > LDAP v.3 > 2012 R2 OS >Reporter: David F. Quiroga >Priority: Minor > Labels: easyfix, patch > Attachments: ambari-22642.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Ambari server configured to use "secure" ldap authentication. > authentication.ldap.primaryUrl=:636 > authentication.ldap.useSSL=true > We call the ldap_sync_events REST endpoint frequently to synchronize > existing groups and a specific list groups. We had no issues with this until > mid-October at which point we began to see: > {code} > "status" : "ERROR", > "status_detail" : "Caught exception running LDAP sync. simple bind > failed: **:636; nested exception is > javax.naming.CommunicationException: simple bind failed: **:636 [Root > exception is java.net.SocketException: Connection reset]", > {code} > Troubleshooting: > * We saw random success and failure when attempting to sync a single group. > * With useSSL=false and an updated port ldap sync was consistently successful. > Cause: > * By default, ldap connection only uses pooled connections when connecting to > a directory server over LDAP. Enabling SSL causes it to disable the pooling, > resulting in poorer performance and failures due to connection resets. > * Around mid-October we increased the number of groups defined on the system > (50+), this pushed us outside the "safe zone". > Fix: > Enable the SSL connections pooling by adding the below argument to startup > options. > -Dcom.sun.jndi.ldap.connect.pool.protocol='plain ssl' > Reference: > [https://confluence.atlassian.com/jirakb/connecting-jira-to-active-directory-over-ldaps-fails-with-connection-reset-763004137.htm] > [https://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html] > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22642) LDAPS sync Connection Refused
[ https://issues.apache.org/jira/browse/AMBARI-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David F. Quiroga updated AMBARI-22642: -- Attachment: (was: ambari-22642.patch) > LDAPS sync Connection Refused > -- > > Key: AMBARI-22642 > URL: https://issues.apache.org/jira/browse/AMBARI-22642 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 > Environment: java version "1.8.0_121" > Java(TM) SE Runtime Environment (build 1.8.0_121-tdc1-b13) > Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) > AD Domain Controllers > LDAP v.3 > 2012 R2 OS >Reporter: David F. Quiroga >Priority: Minor > Labels: easyfix, patch > Original Estimate: 24h > Remaining Estimate: 24h > > Ambari server configured to use "secure" ldap authentication. > authentication.ldap.primaryUrl=:636 > authentication.ldap.useSSL=true > We call the ldap_sync_events REST endpoint frequently to synchronize > existing groups and a specific list groups. We had no issues with this until > mid-October at which point we began to see: > {code} > "status" : "ERROR", > "status_detail" : "Caught exception running LDAP sync. simple bind > failed: **:636; nested exception is > javax.naming.CommunicationException: simple bind failed: **:636 [Root > exception is java.net.SocketException: Connection reset]", > {code} > Troubleshooting: > * We saw random success and failure when attempting to sync a single group. > * With useSSL=false and an updated port ldap sync was consistently successful. > Cause: > * By default, ldap connection only uses pooled connections when connecting to > a directory server over LDAP. Enabling SSL causes it to disable the pooling, > resulting in poorer performance and failures due to connection resets. > * Around mid-October we increased the number of groups defined on the system > (50+), this pushed us outside the "safe zone". > Fix: > Enable the SSL connections pooling by adding the below argument to startup > options. > -Dcom.sun.jndi.ldap.connect.pool.protocol='plain ssl' > Reference: > [https://confluence.atlassian.com/jirakb/connecting-jira-to-active-directory-over-ldaps-fails-with-connection-reset-763004137.htm] > [https://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html] > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22614) Fix unit tests in feature branch to make them workable
[ https://issues.apache.org/jira/browse/AMBARI-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaly Brodetskyi updated AMBARI-22614: --- Attachment: (was: AMBARI-22614_part2.patch) > Fix unit tests in feature branch to make them workable > -- > > Key: AMBARI-22614 > URL: https://issues.apache.org/jira/browse/AMBARI-22614 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Vitaly Brodetskyi >Assignee: Vitaly Brodetskyi >Priority: Critical > Fix For: 3.0.0 > > Attachments: AMBARI-22614.branch-feature-AMBARI-14714.002.patch, > AMBARI-22614.patch, AMBARI-22614_part2.patch > > > Fix unit tests in feature branch to make them workable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22614) Fix unit tests in feature branch to make them workable
[ https://issues.apache.org/jira/browse/AMBARI-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaly Brodetskyi updated AMBARI-22614: --- Attachment: AMBARI-22614_part2.patch > Fix unit tests in feature branch to make them workable > -- > > Key: AMBARI-22614 > URL: https://issues.apache.org/jira/browse/AMBARI-22614 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Vitaly Brodetskyi >Assignee: Vitaly Brodetskyi >Priority: Critical > Fix For: 3.0.0 > > Attachments: AMBARI-22614.branch-feature-AMBARI-14714.002.patch, > AMBARI-22614.patch, AMBARI-22614_part2.patch > > > Fix unit tests in feature branch to make them workable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22623) Reporting host status was broken by merge
[ https://issues.apache.org/jira/browse/AMBARI-22623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Onischuk updated AMBARI-22623: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch-3.0-perf > Reporting host status was broken by merge > - > > Key: AMBARI-22623 > URL: https://issues.apache.org/jira/browse/AMBARI-22623 > Project: Ambari > Issue Type: Bug >Reporter: Andrew Onischuk >Assignee: Andrew Onischuk > Fix For: 3.0.0 > > Attachments: AMBARI-22623.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22635) Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed
[ https://issues.apache.org/jira/browse/AMBARI-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290824#comment-16290824 ] Mugdha Varadkar commented on AMBARI-22635: -- +1 for attached [^AMBARI-22635-trunk.1-addendum.patch] > Ambari should create a dummy core-site.xml for Ranger plugins when namenode > is not installed > > > Key: AMBARI-22635 > URL: https://issues.apache.org/jira/browse/AMBARI-22635 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0, 2.6.1 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22635-branch-2.6.1.patch, > AMBARI-22635-branch-2.6.2.patch, AMBARI-22635-branch-2.6.patch, > AMBARI-22635-trunk.1-addendum.patch, AMBARI-22635-trunk.1.patch, > AMBARI-22635-trunk.patch > > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22635) Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed
[ https://issues.apache.org/jira/browse/AMBARI-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Suvagia updated AMBARI-22635: Attachment: AMBARI-22635-branch-2.6.2.patch AMBARI-22635-trunk.1-addendum.patch Addendum fix for trunk branch, and updating revised patch for branch-2.6 > Ambari should create a dummy core-site.xml for Ranger plugins when namenode > is not installed > > > Key: AMBARI-22635 > URL: https://issues.apache.org/jira/browse/AMBARI-22635 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.0, 2.6.1 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia > Fix For: trunk, 2.6.2 > > Attachments: AMBARI-22635-branch-2.6.1.patch, > AMBARI-22635-branch-2.6.2.patch, AMBARI-22635-branch-2.6.patch, > AMBARI-22635-trunk.1-addendum.patch, AMBARI-22635-trunk.1.patch, > AMBARI-22635-trunk.patch > > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290796#comment-16290796 ] Hudson commented on AMBARI-22651: - SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8522 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8522/]) AMBARI-22651 Unable to add/change role for user. (atkach) (atkach: [http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=dc31e516d95a6636a27d5fbadb9b5529983587c2]) * (edit) ambari-admin/src/main/resources/ui/admin-web/app/views/userManagement/modals/userCreate.html * (edit) ambari-admin/src/main/resources/ui/admin-web/app/views/userManagement/groupEdit.html * (edit) ambari-admin/src/main/resources/ui/admin-web/app/views/userManagement/modals/groupCreate.html * (edit) ambari-admin/src/main/resources/ui/admin-web/app/views/userManagement/userEdit.html > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22653) Infra Manager: s3 upload support for archiving Infra Solr
Krisztian Kasa created AMBARI-22653: --- Summary: Infra Manager: s3 upload support for archiving Infra Solr Key: AMBARI-22653 URL: https://issues.apache.org/jira/browse/AMBARI-22653 Project: Ambari Issue Type: Improvement Components: ambari-infra Affects Versions: 3.0.0 Reporter: Krisztian Kasa Assignee: Krisztian Kasa Fix For: 3.0.0 Upload exported document from infa solr to a specified s3 server. s3 configuration should be defined in a property file and a property should be added to ambari-infra-manager.properties file for each jobs referencing the s3 property file. s3 upload should be optional, If s3 configuration reference presents upload the files into the specified s3 server and delete them from temporary folder, if not leave the files in the filesystem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22614) Fix unit tests in feature branch to make them workable
[ https://issues.apache.org/jira/browse/AMBARI-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaly Brodetskyi updated AMBARI-22614: --- Attachment: AMBARI-22614_part2.patch > Fix unit tests in feature branch to make them workable > -- > > Key: AMBARI-22614 > URL: https://issues.apache.org/jira/browse/AMBARI-22614 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Vitaly Brodetskyi >Assignee: Vitaly Brodetskyi >Priority: Critical > Fix For: 3.0.0 > > Attachments: AMBARI-22614.branch-feature-AMBARI-14714.002.patch, > AMBARI-22614.patch, AMBARI-22614_part2.patch > > > Fix unit tests in feature branch to make them workable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22577) Migrate user data for upgrade to improved user account management
[ https://issues.apache.org/jira/browse/AMBARI-22577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290765#comment-16290765 ] Hadoop QA commented on AMBARI-22577: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12901950/AMBARI-22577_branch-feature-AMBARI-20859_02.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12843//console This message is automatically generated. > Migrate user data for upgrade to improved user account management > - > > Key: AMBARI-22577 > URL: https://issues.apache.org/jira/browse/AMBARI-22577 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Robert Levas >Assignee: Robert Levas >Priority: Critical > Fix For: 3.0.0 > > Attachments: AMBARI-22577_branch-feature-AMBARI-20859_01.patch, > AMBARI-22577_branch-feature-AMBARI-20859_02.patch, > user_management_db_schema_upgrade.png > > > Migrate data from the {{users}} table (pre-Ambari 3.0.0) to the updated > {{users}} table and {{user_authentication}} tables. > See [^user_management_db_schema_upgrade.png] > !user_management_db_schema_upgrade.png|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290764#comment-16290764 ] Andrii Tkach commented on AMBARI-22651: --- committed to trunk > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Tkach updated AMBARI-22651: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290740#comment-16290740 ] Aleksandr Kovalenko commented on AMBARI-22651: -- +1 for the patch > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290722#comment-16290722 ] Hadoop QA commented on AMBARI-22651: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12902046/AMBARI-22651.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 core tests{color}. The patch passed unit tests in ambari-admin. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12841//console This message is automatically generated. > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22652) Ambari Server Fails to Install Packages with "No package found" Errors
[ https://issues.apache.org/jira/browse/AMBARI-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satheesh Nair updated AMBARI-22652: --- Description: The Ambari Server v2.6 was not able to install packages due to the following error : 2017-12-14 13:49:36,048 - No package found for hadoop_${stack_version}-yarn(hadoop_(\d|_)+-yarn$) It seems the problem is due to this piece of code : https://github.com/apache/ambari/blob/release-2.6.0/ambari-common/src/main/python/resource_management/libraries/script/script.py Offending Line : package_version = None Is there a reason why we are setting the Package Version to None Explictly which eventually leads to "None" Package Version and package installation failure ? Commenting this Line in "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py" seens to have resolved the problem and cluster installation was successfull. === Following are the version details being used for installation : OS : CentOS7 (Installation Using Local Repository Method - Repo Hosted with Apache Server) Ambari Server Version : 2.6.0.0-267 HDP Version - 2.6.2.0-205 HDP UTIL Version : HDP-UTILS-1.1.0.21 Ambari.repo File : [Updates-Ambari-2.6.0.0] name=Ambari-2.6.0.0-Updates baseurl=http://Master_Lab/ambari/centos7/2.6.0.0-267/ gpgcheck=1 gpgkey=http://Master_Lab/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 HDP & HDP UTIL REPO : [HDP-2.6-repo-101] name=HDP-2.6-repo-101 baseurl=http://master_lab.balwant.com/hdp/HDP/centos7/ path=/ enabled=1 gpgcheck=0 [HDP-UTILS-1.1.0.21-repo-101] name=HDP-UTILS-1.1.0.21-repo-101 baseurl=http://master_lab.balwant.com/hdp path=/ enabled=1 === was: The Ambari Server v2.6 was not able to install packages due to the following error : 2017-12-14 13:49:36,048 - No package found for hadoop_${stack_version}-yarn(hadoop_(\d|_)+-yarn$) It seems the problem is due to this piece of code : https://github.com/apache/ambari/blob/release-2.6.0/ambari-common/src/main/python/resource_management/libraries/script/script.py Offending Line : package_version = None Is there a reason why we are setting the Package Version to None Explictly which eventually leads. Commenting this Line in "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py" seens to have resolved the problem and cluster installation was successfull. === Following are the version details being used for installation : OS : CentOS7 (Installation Using Local Repository Method - Repo Hosted with Apache Server) Ambari Server Version : 2.6.0.0-267 HDP Version - 2.6.2.0-205 HDP UTIL Version : HDP-UTILS-1.1.0.21 Ambari.repo File : [Updates-Ambari-2.6.0.0] name=Ambari-2.6.0.0-Updates baseurl=http://Master_Lab/ambari/centos7/2.6.0.0-267/ gpgcheck=1 gpgkey=http://Master_Lab/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 HDP & HDP UTIL REPO : [HDP-2.6-repo-101] name=HDP-2.6-repo-101 baseurl=http://master_lab.balwant.com/hdp/HDP/centos7/ path=/ enabled=1 gpgcheck=0 [HDP-UTILS-1.1.0.21-repo-101] name=HDP-UTILS-1.1.0.21-repo-101 baseurl=http://master_lab.balwant.com/hdp path=/ enabled=1 === > Ambari Server Fails to Install Packages with "No package found" Errors > -- > > Key: AMBARI-22652 > URL: https://issues.apache.org/jira/browse/AMBARI-22652 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: Satheesh Nair > > The Ambari Server v2.6 was not able to install packages due to the following > error : > 2017-12-14 13:49:36,048 - No package found for > hadoop_${stack_version}-yarn(hadoop_(\d|_)+-yarn$) > It seems the problem is due to this piece of code : > https://github.com/apache/ambari/blob/release-2.6.0/ambari-common/src/main/python/resource_management/libraries/script/script.py > Offending Line : package_version = None > Is there a reason why we are setting the Package Version to None Explictly > which eventually leads to "None" Package Version and package installation > failure ? > Commenting this Line in > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py" > seens to have resolved the problem and cluster installation was successfull. > === > Following are the version details being used for installation : > OS : CentOS7 (Installation Using Local Repository Method - Repo Hosted with > Apache Server) > Ambari Server Version : 2.6.0.0-267 > HDP Version - 2.6.2.0-205 > HDP UTIL Version : HDP-UTILS-1.1.0.21 > Ambari.repo File : > [Updates-Ambari-2.6.0.0] >
[jira] [Created] (AMBARI-22652) Ambari Server Fails to Install Packages with "No package found" Errors
Satheesh Nair created AMBARI-22652: -- Summary: Ambari Server Fails to Install Packages with "No package found" Errors Key: AMBARI-22652 URL: https://issues.apache.org/jira/browse/AMBARI-22652 Project: Ambari Issue Type: Bug Affects Versions: 2.6.0 Reporter: Satheesh Nair The Ambari Server v2.6 was not able to install packages due to the following error : 2017-12-14 13:49:36,048 - No package found for hadoop_${stack_version}-yarn(hadoop_(\d|_)+-yarn$) It seems the problem is due to this piece of code : https://github.com/apache/ambari/blob/release-2.6.0/ambari-common/src/main/python/resource_management/libraries/script/script.py Offending Line : package_version = None Is there a reason why we are setting the Package Version to None Explictly which eventually leads. Commenting this Line in "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py" seens to have resolved the problem and cluster installation was successfull. === Following are the version details being used for installation : OS : CentOS7 (Installation Using Local Repository Method - Repo Hosted with Apache Server) Ambari Server Version : 2.6.0.0-267 HDP Version - 2.6.2.0-205 HDP UTIL Version : HDP-UTILS-1.1.0.21 Ambari.repo File : [Updates-Ambari-2.6.0.0] name=Ambari-2.6.0.0-Updates baseurl=http://Master_Lab/ambari/centos7/2.6.0.0-267/ gpgcheck=1 gpgkey=http://Master_Lab/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 HDP & HDP UTIL REPO : [HDP-2.6-repo-101] name=HDP-2.6-repo-101 baseurl=http://master_lab.balwant.com/hdp/HDP/centos7/ path=/ enabled=1 gpgcheck=0 [HDP-UTILS-1.1.0.21-repo-101] name=HDP-UTILS-1.1.0.21-repo-101 baseurl=http://master_lab.balwant.com/hdp path=/ enabled=1 === -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290710#comment-16290710 ] Andrii Tkach commented on AMBARI-22651: --- PhantomJS 1.9.8 (Linux): Executed 66 of 66 SUCCESS (1.253 secs / 1.25 secs) > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Tkach updated AMBARI-22651: -- Status: Patch Available (was: Open) > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22651) Unable to add/change role for user
[ https://issues.apache.org/jira/browse/AMBARI-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Tkach updated AMBARI-22651: -- Attachment: AMBARI-22651.patch > Unable to add/change role for user > -- > > Key: AMBARI-22651 > URL: https://issues.apache.org/jira/browse/AMBARI-22651 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: 3.0.0 >Reporter: Andrii Tkach >Assignee: Andrii Tkach > Fix For: 3.0.0 > > Attachments: AMBARI-22651.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22651) Unable to add/change role for user
Andrii Tkach created AMBARI-22651: - Summary: Unable to add/change role for user Key: AMBARI-22651 URL: https://issues.apache.org/jira/browse/AMBARI-22651 Project: Ambari Issue Type: Bug Components: ambari-web Affects Versions: 3.0.0 Reporter: Andrii Tkach Assignee: Andrii Tkach Fix For: 3.0.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22650) Add ability to define packages at stack level
Dmytro Sen created AMBARI-22650: --- Summary: Add ability to define packages at stack level Key: AMBARI-22650 URL: https://issues.apache.org/jira/browse/AMBARI-22650 Project: Ambari Issue Type: Task Components: ambari-server Affects Versions: 3.0.0 Reporter: Dmytro Sen Assignee: Dmytro Sen Priority: Blocker Fix For: 3.0.0 Today, we define the list of packages to be installed at individual service level metainfo.xml. In the new mpack model Ambari will no longer execute (i.e. "yum install hadoop", "yum install hadoop-client"), but instead Ambari will install individual mpack meta-rpms (i.e. "yum install hdpcore", "yum install edw") whenever a component from that mpack is installed. So we should add ability to define list of packages at the stack level (i.e. stack/metainfo.xml). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290572#comment-16290572 ] Swapan Shridhar edited comment on AMBARI-22649 at 12/14/17 9:00 AM: *Testing:* Tested on live cluster *=* *clusterSettings:* *=* *A.* get_cluster_setting_entries(): ** - 1. Retrieve *single* setting : 'recovery_enabled' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled']) *o/p*: {'recovery_enabled': True} - 2. Retrieve *two* settings : 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 3. Retrieve settings where passed in empty set -> Expected nothing returned -- In get_cluster_setting_entries(). Passed-in setting(s) : set([]) *o/p*: None - 4. Retrieve *three* settings : 'smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False, 'smokeuser': 'ambari-qa'} - 5. Retrieve *three* settings where *middle setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'abc', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 6. Retrieve *three* settings where *1st setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['abc', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 7. Retrieve *three* settings where *last setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce', 'abc']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 8. Retrieve passed in setting which is *non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1']) *o/p* : None - 9. Retrieve *two* passed in settings and both are non-existent -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1', 'non-existent2']) *o/p* : None - 10. Retrieve settings where set passed in is None. -> *returns all settings.* -- In get_cluster_setting_entries(). Passed-in setting(s) : None *o/p*: {'security_enabled': 'false', 'namenode_rolling_restart_timeout': '4200', 'enable_external_ranger': 'false', 'override_uid': 'true', 'kerberos_domain': 'EXAMPLE.COM', 'one_dir_per_partition': 'false', 'agent_mounts_ignore_list': '', 'repo_ubuntu_template': '{{package_type}} {{base_url}} {{components}}', 'ignore_groupsusers_create': 'false', 'alerts_repeat_tolerance': '1', 'hide_yarn_memory_widget': 'false', 'fetch_nonlocal_groups': 'true', 'manage_dirs_on_root': 'true', 'recovery_lifetime_max_count': '1024', 'recovery_type': 'AUTO_START', 'ignore_bad_mounts': 'false', 'recovery_window_in_minutes': '60', 'sysprep_skip_copy_tarballs_hdfs': 'false', 'user_group': 'hadoop', 'namenode_rolling_restart_safemode_exit_timeout': '3600', 'recovery_retry_interval': '5', 'sysprep_skip_copy_oozie_share_lib_to_hdfs': 'false', 'sysprep_skip_setup_jce': 'false', 'manage_hive_fsroot': 'true', 'service_check_type': 'full', 'recovery_enabled': 'true', 'recovery_max_count': '6', 'sysprep_skip_create_users_and_groups': 'false', 'smokeuser_keytab': '/etc/security/keytabs/smokeuser.headless.keytab', 'managed_hdfs_resource_property_names': 'false', 'smokeuser': 'ambari-qa', 'sysprep_skip_copy_fast_jar_hdfs': 'false'} *B.* get_cluster_setting_value(): ** - 1. Retrieve cluster_setting : 'hide_yarn_memory_widget' -- In get_cluster_setting_value(). Passed-in setting : hide_yarn_memory_widget *o/p* : false - 2. Retrieve cluster_setting : 'recovery_max_count' -- In get_cluster_setting_value(). Passed-in setting : recovery_max_count *o/p* : 6 - 3. Retrieve cluster_setting where passed in setting is 'non_existing' --> Empty string -- In get_cluster_setting_value(). Passed-in setting : non_existing *o/p* : None - 4. Retrieve cluster_setting setting passed in is "None" -- In get_cluster_setting_value(). Passed-in setting : None *o/p* : None *C*. is_security_enabled(): ** - 1. Retrieve security state of cluster by calling 'is_security_enabled()' which in turn calls *get_cluster_setting_value()* -- In get_cluster_setting_value().
[jira] [Commented] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290572#comment-16290572 ] Swapan Shridhar commented on AMBARI-22649: -- *Testing:* Tested on live cluster *=* *clusterSettings:* *=* *A.* get_cluster_setting_entries(): ** - 1. Retrieve *single* setting : 'recovery_enabled' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled']) *o/p*: {'recovery_enabled': True} - 2. Retrieve *two* settings : 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 3. Retrieve settings where passed in empty set -> Expected nothing returned -- In get_cluster_setting_entries(). Passed-in setting(s) : set([]) *o/p*: None - 4. Retrieve *three* settings : 'smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce' -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False, 'smokeuser': 'ambari-qa'} - 5. Retrieve *three* settings where *middle setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'abc', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 6. Retrieve *three* settings where *1st setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['abc', 'recovery_enabled', 'sysprep_skip_setup_jce']) *o/p* : {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 7. Retrieve *three* settings where *last setting is non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['recovery_enabled', 'sysprep_skip_setup_jce', 'abc']) *o/p*: {'recovery_enabled': True, 'sysprep_skip_setup_jce': False} - 8. Retrieve passed in setting which is *non-existent* -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1']) *o/p* : None - 9. Retrieve *two* passed in settings and both are non-existent -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['non-existent1', 'non-existent2']) *o/p* : None - 10. Retrieve settings where set passed in is None. -> *returns all settings.* -- In get_cluster_setting_entries(). Passed-in setting(s) : None *o/p*: {'security_enabled': 'false', 'namenode_rolling_restart_timeout': '4200', 'enable_external_ranger': 'false', 'override_uid': 'true', 'kerberos_domain': 'EXAMPLE.COM', 'one_dir_per_partition': 'false', 'agent_mounts_ignore_list': '', 'repo_ubuntu_template': '{{package_type}} {{base_url}} {{components}}', 'ignore_groupsusers_create': 'false', 'alerts_repeat_tolerance': '1', 'hide_yarn_memory_widget': 'false', 'fetch_nonlocal_groups': 'true', 'manage_dirs_on_root': 'true', 'recovery_lifetime_max_count': '1024', 'recovery_type': 'AUTO_START', 'ignore_bad_mounts': 'false', 'recovery_window_in_minutes': '60', 'sysprep_skip_copy_tarballs_hdfs': 'false', 'user_group': 'hadoop', 'namenode_rolling_restart_safemode_exit_timeout': '3600', 'recovery_retry_interval': '5', 'sysprep_skip_copy_oozie_share_lib_to_hdfs': 'false', 'sysprep_skip_setup_jce': 'false', 'manage_hive_fsroot': 'true', 'service_check_type': 'full', 'recovery_enabled': 'true', 'recovery_max_count': '6', 'sysprep_skip_create_users_and_groups': 'false', 'smokeuser_keytab': '/etc/security/keytabs/smokeuser.headless.keytab', 'managed_hdfs_resource_property_names': 'false', 'smokeuser': 'ambari-qa', 'sysprep_skip_copy_fast_jar_hdfs': 'false'} *B.* get_cluster_setting_value(): ** - 1. Retrieve cluster_setting : 'hide_yarn_memory_widget' -- In get_cluster_setting_value(). Passed-in setting : hide_yarn_memory_widget *o/p* : false - 2. Retrieve cluster_setting : 'recovery_max_count' -- In get_cluster_setting_value(). Passed-in setting : recovery_max_count *o/p* : 6 - 3. Retrieve cluster_setting where passed in setting is 'non_existing' --> Empty string -- In get_cluster_setting_value(). Passed-in setting : non_existing *o/p* : None - 4. Retrieve cluster_setting setting passed in is "None" -- In get_cluster_setting_value(). Passed-in setting : None *o/p* : None *C*. is_security_enabled(): ** - 1. Retrieve security state of cluster by calling 'is_security_enabled()' which in turn calls *get_cluster_setting_value()* -- In get_cluster_setting_value(). Passed-in setting : security_enabled
[jira] [Updated] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapan Shridhar updated AMBARI-22649: - Description: Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced "cluster settings" in Ambari. *=* *Library for querying _clusterSettings_ and _stackSettings_ for its contents in command\*.json.* *=* One should be able to query for a given *clusterSettings* or *stackSettings*: - by passing in the setting name(one or more) in order to get it back as key-value map, or - just get the value back for a passed-in setting. *Functions for clusterSettings:* *-* - get_cluster_setting_entries(setting_names) : -- Retrieves the passed-in cluster setting entr(y/ies) and their values as a map. If 'setting_names' is passed-in as None : all the settings names and their corresponding values will be returned as map. If 'setting_names' is passed-in as empty set : None will be returned. - get_cluster_setting_value(setting_name) : -- Retrieves the passed-in cluster setting entry's value. - is_security_enabled() : -- Retrieves the cluster's security status. *Functions for stackSettings:* *-* Stack settings as of now has 5 settings : stack_name, stack_root, stack_features, stack_tools, stack_packages. stack_name, stack_root have string as values, whereas stack_features, stack_tools, stack_packages have values as JSON. Further there already exists python functions in files : *stack_features.py*, *stack_tools.py* and *stack_select.py*. - get_stack_setting_entries(setting_names) : -- Retrieves the passed-in stack setting entr(y/ies) and their values as a map. If 'setting_names' is passed-in as None, all the settings names and their corresponding values will be returned as map. If 'setting_names' is passed-in as empty set : None will be returned. - get_stack_setting_value(setting_name): -- Retrieves the passed-in stack setting entry's value. - get_stack_name(): -- Retrieves the stack name. - get_stack_root(): -- Retrieves the stack root. *Modifications in _stack_features.py, stack_tools.py and stack_select.py_ files:* *--* - Given that these already exist and as of now they read the relevant stack setting from *configurations/cluster_env*. - Thus, code has been added to try reading from /stackSettings first by calling the new fn.() get_stack_setting_value(). if setting not found, go for the fall back *configurations/cluster_env* (which would be removed soon, when we remove cluster_env). > Library for querying cluster_settings and stack_settings in command*.json. > -- > > Key: AMBARI-22649 > URL: https://issues.apache.org/jira/browse/AMBARI-22649 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Reporter: Swapan Shridhar >Assignee: Swapan Shridhar > Fix For: 3.0.0 > > Attachments: AMBARI-22649.patch > > > Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced > "cluster settings" in Ambari. > *=* > *Library for querying _clusterSettings_ and _stackSettings_ for its contents > in command\*.json.* > *=* > One should be able to query for a given *clusterSettings* or *stackSettings*: > - by passing in the setting name(one or more) in order to get it back as > key-value map, or > - just get the value back for a passed-in setting. > *Functions for clusterSettings:* > *-* > - get_cluster_setting_entries(setting_names) : > -- Retrieves the passed-in cluster setting entr(y/ies) and their values > as a map. >If 'setting_names' is passed-in as None : all the settings names and > their corresponding values will be returned as map. >If 'setting_names' is passed-in as empty set : None will be returned. > - get_cluster_setting_value(setting_name) : > -- Retrieves the passed-in cluster setting entry's value. > - is_security_enabled() : > -- Retrieves the cluster's security status. > *Functions for stackSettings:* > *-* > Stack settings as of now has 5 settings : stack_name, stack_root, > stack_features, stack_tools, stack_packages. stack_name, stack_root have > string as values, whereas
[jira] [Updated] (AMBARI-22649) Library for querying cluster_settings and stack_settings in command*.json.
[ https://issues.apache.org/jira/browse/AMBARI-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapan Shridhar updated AMBARI-22649: - Attachment: AMBARI-22649.patch > Library for querying cluster_settings and stack_settings in command*.json. > -- > > Key: AMBARI-22649 > URL: https://issues.apache.org/jira/browse/AMBARI-22649 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Reporter: Swapan Shridhar >Assignee: Swapan Shridhar > Fix For: 3.0.0 > > Attachments: AMBARI-22649.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)