[jira] [Resolved] (AMBARI-24203) Improve Kafka service check to check for under-replicated partitions
[ https://issues.apache.org/jira/browse/AMBARI-24203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis resolved AMBARI-24203. --- Resolution: Fixed PR has been merged > Improve Kafka service check to check for under-replicated partitions > > > Key: AMBARI-24203 > URL: https://issues.apache.org/jira/browse/AMBARI-24203 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.7.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > During Ambari Rolling Upgrade, we invoke Kafka Service check to ensure that > Kafka is healthy. The current implementation only does basic topic > creation/deletion. We need to extend the implementation to check for number > of under-replicated partitions. > We have this support in kafka-topics.sh script which is already used in the > kafka service check script. We need to extend above script to call below > command > {noformat} > sh kafka-topics.sh --describe --zookeeper localhost:2181 > --under-replicated-partitions > {noformat} > If the command output is empty, then there are no under replicated partotions. > If the output contains "Topic:" string, then there are some under replicated > partitions, we can fail the service check. > {noformat} > sh kafka-topics.sh --describe --zookeeper localhost:2181 > --under-replicated-partitions > Topic: TEST Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1 > Topic: TEST Partition: 1 Leader: 1 Replicas: 2,1 Isr: 1{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AMBARI-24203) Improve Kafka service check to check for under-replicated partitions
Yolanda M. Davis created AMBARI-24203: - Summary: Improve Kafka service check to check for under-replicated partitions Key: AMBARI-24203 URL: https://issues.apache.org/jira/browse/AMBARI-24203 Project: Ambari Issue Type: Bug Components: stacks Affects Versions: 2.7.0 Reporter: Yolanda M. Davis Assignee: Yolanda M. Davis Fix For: 2.7.0 During Ambari Rolling Upgrade, we invoke Kafka Service check to ensure that Kafka is healthy. The current implementation only does basic topic creation/deletion. We need to extend the implementation to check for number of under-replicated partitions. We have this support in kafka-topics.sh script which is already used in the kafka service check script. We need to extend above script to call below command {noformat} sh kafka-topics.sh --describe --zookeeper localhost:2181 --under-replicated-partitions {noformat} If the command output is empty, then there are no under replicated partotions. If the output contains "Topic:" string, then there are some under replicated partitions, we can fail the service check. {noformat} sh kafka-topics.sh --describe --zookeeper localhost:2181 --under-replicated-partitions Topic: TEST Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1 Topic: TEST Partition: 1 Leader: 1 Replicas: 2,1 Isr: 1{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-22836) Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry
[ https://issues.apache.org/jira/browse/AMBARI-22836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22836: -- Attachment: (was: AMBARI-22836.patch) > Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry > - > > Key: AMBARI-22836 > URL: https://issues.apache.org/jira/browse/AMBARI-22836 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.1 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 2.6.2 > > > The following stack feature changes are required: > 1) Remove Kafka Stack Feature kafka_extended_sasl_support (doesn't apply to > current version) > 2) Remove registry entries: registry_allowed_resources_support, > registry_rewriteuri_filter_support > 3) Remove sam entries: sam_storage_core_in_registry > 4) Add NiFi encrypted authorizers > {noformat} > { > "name": "nifi_encrypted_authorizers_config", > "description": "Support encrypted authorizers.xml configuration for version > 3.1 onwards", > "min_version": "2.6.5.0” > }{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-22836) Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry
[ https://issues.apache.org/jira/browse/AMBARI-22836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22836: -- Attachment: AMBARI-22836.patch > Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry > - > > Key: AMBARI-22836 > URL: https://issues.apache.org/jira/browse/AMBARI-22836 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.1 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 2.6.2 > > Attachments: AMBARI-22836.patch > > > The following stack feature changes are required: > 1) Remove Kafka Stack Feature kafka_extended_sasl_support (doesn't apply to > current version) > 2) Remove registry entries: registry_allowed_resources_support, > registry_rewriteuri_filter_support > 3) Remove sam entries: sam_storage_core_in_registry > 4) Add NiFi encrypted authorizers > {noformat} > { > "name": "nifi_encrypted_authorizers_config", > "description": "Support encrypted authorizers.xml configuration for version > 3.1 onwards", > "min_version": "2.6.5.0” > }{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AMBARI-22483) Add SAM StackFeatures to HDP StackFeatures - DB Storage Support
[ https://issues.apache.org/jira/browse/AMBARI-22483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis resolved AMBARI-22483. --- Resolution: Won't Do This is no longer required (needs are replaced by AMBARI-22836) > Add SAM StackFeatures to HDP StackFeatures - DB Storage Support > --- > > Key: AMBARI-22483 > URL: https://issues.apache.org/jira/browse/AMBARI-22483 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 3.0.0 > > > { > "name": "sam_db_file_storage", > "description": "DB based file storage in SAM", > "min_version": "2.6.3.0" > } -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch was merged 11/30/2017 > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 2.6.1 > > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Resolution: Fixed Fix Version/s: 2.6.1 Status: Resolved (was: Patch Available) This patch was merged on 11/30/2017 > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 2.6.1 > > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AMBARI-22836) Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry
Yolanda M. Davis created AMBARI-22836: - Summary: Stack Feature Updates - SAM, KAFKA, NiFi, and Schema Registry Key: AMBARI-22836 URL: https://issues.apache.org/jira/browse/AMBARI-22836 Project: Ambari Issue Type: Bug Components: stacks Affects Versions: 2.6.1 Reporter: Yolanda M. Davis Assignee: Yolanda M. Davis Fix For: 2.6.2 The following stack feature changes are required: 1) Remove Kafka Stack Feature kafka_extended_sasl_support (doesn't apply to current version) 2) Remove registry entries: registry_allowed_resources_support, registry_rewriteuri_filter_support 3) Remove sam entries: sam_storage_core_in_registry 4) Add NiFi encrypted authorizers {noformat} { "name": "nifi_encrypted_authorizers_config", "description": "Support encrypted authorizers.xml configuration for version 3.1 onwards", "min_version": "2.6.5.0” }{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22537: -- Resolution: Fixed Status: Resolved (was: Patch Available) This has been merged. > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537-update.patch, AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274419#comment-16274419 ] Yolanda M. Davis edited comment on AMBARI-22485 at 12/1/17 1:52 PM: [~adoroszlai] thank you I will submit a fix for this. was (Author: yolandamdavis): [~adoroszlai] thank you I will change that and update the patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274419#comment-16274419 ] Yolanda M. Davis commented on AMBARI-22485: --- [~adoroszlai] thank you I will change that and update the patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273876#comment-16273876 ] Yolanda M. Davis commented on AMBARI-22537: --- The review received ship it (+1). cc: [~mradhakrishnan] . Patch was updated on the review (I will update this Jira with the latest version). > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537-update.patch, AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22537: -- Attachment: AMBARI-22537-update.patch Updated patch based on review > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537-update.patch, AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273123#comment-16273123 ] Yolanda M. Davis edited comment on AMBARI-22537 at 11/30/17 6:30 PM: - Pushing forward this patch on behalf of [~arunmahadevan]. I was able to test his patch in both rolling upgrade and express upgrade scenarios and it did allow the appropriate removal of the jmextric references during a patch uprrade scenario. Will submit patch for review. was (Author: yolandamdavis): Pushing forward this patch on behalf of Arun. I was able to test his patch in both rolling upgrade and express upgrade scenarios and it did allow the appropriate removal of the jmextric references during a patch uprrade scenario. Will submit patch for review. > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273123#comment-16273123 ] Yolanda M. Davis commented on AMBARI-22537: --- Pushing forward this patch on behalf of Arun. I was able to test his patch in both rolling upgrade and express upgrade scenarios and it did allow the appropriate removal of the jmextric references during a patch uprrade scenario. Will submit patch for review. > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AMBARI-22537) Storm jmxetric config not getting removed during patch upgrade
[ https://issues.apache.org/jira/browse/AMBARI-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis reassigned AMBARI-22537: - Assignee: Yolanda M. Davis (was: Arun Mahadevan) > Storm jmxetric config not getting removed during patch upgrade > -- > > Key: AMBARI-22537 > URL: https://issues.apache.org/jira/browse/AMBARI-22537 > Project: Ambari > Issue Type: Bug >Reporter: Arun Mahadevan >Assignee: Yolanda M. Davis > Attachments: AMBARI-22537.patch > > > The tasks to remove storm jmxetric settings from storm configs are not > getting executed during patch upgrade. > Add supports-patch flag to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272307#comment-16272307 ] Yolanda M. Davis commented on AMBARI-22505: --- This Jira passed review. cc: [~mradhakrishnan] > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272302#comment-16272302 ] Yolanda M. Davis commented on AMBARI-22485: --- This Jira passed review. cc: [~mradhakrishnan] > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Patch Available (was: Open) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Open (was: Patch Available) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267071#comment-16267071 ] Yolanda M. Davis commented on AMBARI-22505: --- Hadoop QA is attempting to apply branch patch to trunk in both cases. The trunk patch should be used for testing (removed branch patch to prevent from happening again) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Patch Available (was: In Progress) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: In Progress (was: Patch Available) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Patch Available (was: In Progress) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: In Progress (was: Patch Available) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Attachment: (was: AMBARI-22505_branch-2.6.patch) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Patch Available (was: In Progress) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch, AMBARI-22505_branch-2.6.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Attachment: AMBARI-22505_branch-2.6.patch > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch, AMBARI-22505_branch-2.6.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Attachment: (was: AMBARI-22505_branch2.6.patch) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Open (was: Patch Available) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch, AMBARI-22505_branch-2.6.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Status: Patch Available (was: In Progress) > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch, AMBARI-22505_branch2.6.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
[ https://issues.apache.org/jira/browse/AMBARI-22505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22505: -- Attachment: AMBARI-22505_branch2.6.patch AMBARI-22505.patch > Kafka service check fails when using a non-root user in kerberized environment > -- > > Key: AMBARI-22505 > URL: https://issues.apache.org/jira/browse/AMBARI-22505 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22505.patch, AMBARI-22505_branch2.6.patch > > > When Ambari agents are configured to run with a non-root user, if kerberos is > enabled, Kafka service check experiences errors in the logs which indicate > that the check is attempting to create a topic that already exists. However > the true failure, which isn't in the logs but captured in an error variable, > demonstrates the culprit (see below): > [2017-11-22 05:12:57,029] WARN SASL configuration failed: > javax.security.auth.login.LoginException: No password provided Will continue > connection to Zookeeper server without SASL authentication, if Zookeeper > server allows it. (org.apache.zookeeper.ClientCnxn) > Exception in thread "main" > org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure > at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) > at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) > at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) > at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) > at kafka.admin.TopicCommand.main(TopicCommand.scala) > This occurs because a prior check to see if the topic exists is not executed > as the kafka user. A subsequent call to create the topic does execute as that > user. The first check should also execute as the kakfa user. Also Ambari > should immediately raise a failure if an error occurs during the topic check > and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22505) Kafka service check fails when using a non-root user in kerberized environment
Yolanda M. Davis created AMBARI-22505: - Summary: Kafka service check fails when using a non-root user in kerberized environment Key: AMBARI-22505 URL: https://issues.apache.org/jira/browse/AMBARI-22505 Project: Ambari Issue Type: Bug Components: stacks Affects Versions: 2.6.0 Reporter: Yolanda M. Davis Assignee: Yolanda M. Davis When Ambari agents are configured to run with a non-root user, if kerberos is enabled, Kafka service check experiences errors in the logs which indicate that the check is attempting to create a topic that already exists. However the true failure, which isn't in the logs but captured in an error variable, demonstrates the culprit (see below): [2017-11-22 05:12:57,029] WARN SASL configuration failed: javax.security.auth.login.LoginException: No password provided Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn) Exception in thread "main" org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947) at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924) at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231) at org.I0Itec.zkclient.ZkClient.(ZkClient.java:157) at org.I0Itec.zkclient.ZkClient.(ZkClient.java:131) at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115) at kafka.utils.ZkUtils$.apply(ZkUtils.scala:97) at kafka.admin.TopicCommand$.main(TopicCommand.scala:56) at kafka.admin.TopicCommand.main(TopicCommand.scala) This occurs because a prior check to see if the topic exists is not executed as the kafka user. A subsequent call to create the topic does execute as that user. The first check should also execute as the kakfa user. Also Ambari should immediately raise a failure if an error occurs during the topic check and show that error in the logs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263260#comment-16263260 ] Yolanda M. Davis commented on AMBARI-22485: --- Appears due to checkstyle violations in trunk, unrelated to patch. > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Status: Patch Available (was: In Progress) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263211#comment-16263211 ] Yolanda M. Davis commented on AMBARI-22485: --- Saw the issue (appeared to be whitespace differences). Updated to also ensure ident was included in patch. > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: AMBARI-22485.patch AMBARI-22485_branch-2.6.patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: (was: AMBARI-22485.patch) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Status: Open (was: Patch Available) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: (was: AMBARI-22485_branch-2.6.patch) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262578#comment-16262578 ] Yolanda M. Davis commented on AMBARI-22485: --- Just tested branch against current branch branch-2.6 and trunk and did not encounter issue reported by Hadoop QA. [~jluniya] do you mind taking a look here? > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Status: Patch Available (was: Open) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261916#comment-16261916 ] Yolanda M. Davis commented on AMBARI-22485: --- Kafka supports *PLAINTEXT, SSL, SASL_PLAINTEXT(PLAINTEXTSASL), SASL_SSL* protocols to communicate with Kafka brokers. Brokers can be configured with all these protocols at the same using_ "listeners" _config property. listeners=PLAINTEXT://host.name:port1,SASL_PLAINTEXT://host.name:port2,SSL://host.name:port3,SASL_SSL://host.name:port4 *PLAINTEXT* This is the default protocol. No special configuration is required. This support is available in Ambari. *SSL* This can be configured in Ambari. Required configs are documented in https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/ch_wire-kafka.html *SASL_PLAINTEXT(PLAINTEXTSASL)* Under SASL protocol, Kafka supports below given SASL mechanisms SASL/GSSAPI (Kerberos) - [This is default mechanism) SASL/PLAIN SASL/SCRAM-SHA-256 SASL/SCRAM-SHA-512 Above mechanisms are configured using kafka_server_jaas.conf and below configs listeners sasl.enabled.mechanisms security.inter.broker.protocol sasl.mechanism.inter.broker.protocol Currently, on the kerberozied cluster, Ambari automates kafka_server_jaas.conf, configs for SASL/GSSAPI (Kerberos) mechanism and also pass below java system property to java runtime command. "-Djava.security.auth.login.config=/usr/hdf/current/kafka-broker/config/kafka_jaas.conf" For other mechanisms, we should support including kafka_server_jaas.conf in non-kerberozied environments. Enable below props to Custom Broker Section: sasl.enabled.mechanisms security.inter.broker.protocol sasl.mechanism.inter.broker.protocol *SASL_SSL* Above SASL mechanisms can be combined with SSL encryption. *THINGS TO DO:* 1. Write docs for configuring SASL/PLAIN, SASL/SCRAM-SHA-256, SASL/SCRAM-SHA-512 mechanisms, jaas conf files and configs 2. Enable below props to Custom Broker Section: sasl.enabled.mechanisms=GSSAPI security.inter.broker.protocol=PLAINTEXT sasl.mechanism.inter.broker.protocol=GSSAPI 3. Allow Ambari to pass kafka_server_jaas.conf on non-kerberozied cluster if listeners contains SASL_PLAINTEXT, SASL_SSL. We also need to pass kafka_client_jaas.conf files to kafka CLI scripts. 4. Resolve if any issues exist while enabling multiple listeners > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Description: Currently AMBARI support's SASL and SSL as the security options for Kafka. Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP system into Kafka. Also another important option is SASL_SSL. We need to expose necessary configs in Ambari to enable these mechanisms for users. Ambari should allow users to not only configure Kafka for non-kerberos based SASL mechanisms, but also ensure that jaas configuration files are written when these options are provided (as opposed to only writing those files when kerberos has been enabled).. was: Currently AMBARI support's SASL and SSL as the security options for Kafka. Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP system into Kafka. Also another important option is SASL_SSL. We need to expose necessary configs in Ambari to enable these mechanisms for users. > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. > Ambari should allow users to not only configure Kafka for non-kerberos based > SASL mechanisms, but also ensure that jaas configuration files are written > when these options are provided (as opposed to only writing those files when > kerberos has been enabled).. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: AMBARI-22485_branch-2.6.patch AMBARI-22485.patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: (was: AMBARI-22485-branch-2.6.patch) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Attachments: AMBARI-22485.patch, AMBARI-22485_branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AMBARI-22483) Add SAM StackFeatures to HDP StackFeatures - DB Storage Support
[ https://issues.apache.org/jira/browse/AMBARI-22483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis reassigned AMBARI-22483: - Assignee: Yolanda M. Davis > Add SAM StackFeatures to HDP StackFeatures - DB Storage Support > --- > > Key: AMBARI-22483 > URL: https://issues.apache.org/jira/browse/AMBARI-22483 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Fix For: 2.6.1 > > > { > "name": "sam_db_file_storage", > "description": "DB based file storage in SAM", > "min_version": "2.6.3.0" > } -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: (was: AMBARI-22485.patch) > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis > Attachments: AMBARI-22485-branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: AMBARI-22485-branch-2.6.patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis > Attachments: AMBARI-22485-branch-2.6.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
[ https://issues.apache.org/jira/browse/AMBARI-22485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22485: -- Attachment: AMBARI-22485.patch > Allow Ambari to support non-kerberos SASL mechanisms for Kafka > -- > > Key: AMBARI-22485 > URL: https://issues.apache.org/jira/browse/AMBARI-22485 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis > Attachments: AMBARI-22485.patch > > > Currently AMBARI support's SASL and SSL as the security options for Kafka. > Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other > mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP > system into Kafka. > Also another important option is SASL_SSL. > We need to expose necessary configs in Ambari to enable these mechanisms for > users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22485) Allow Ambari to support non-kerberos SASL mechanisms for Kafka
Yolanda M. Davis created AMBARI-22485: - Summary: Allow Ambari to support non-kerberos SASL mechanisms for Kafka Key: AMBARI-22485 URL: https://issues.apache.org/jira/browse/AMBARI-22485 Project: Ambari Issue Type: Bug Components: stacks Affects Versions: 2.6.0 Reporter: Yolanda M. Davis Currently AMBARI support's SASL and SSL as the security options for Kafka. Within SASL Ambari only supports GSSAPI(kerberos) only. Kafka supports other mechanisms such as plain, md5 etc.. This allows users to plug in their LDAP system into Kafka. Also another important option is SASL_SSL. We need to expose necessary configs in Ambari to enable these mechanisms for users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22483) Add SAM StackFeatures to HDP StackFeatures - DB Storage Support
Yolanda M. Davis created AMBARI-22483: - Summary: Add SAM StackFeatures to HDP StackFeatures - DB Storage Support Key: AMBARI-22483 URL: https://issues.apache.org/jira/browse/AMBARI-22483 Project: Ambari Issue Type: Bug Components: stacks Affects Versions: 2.6.0 Reporter: Yolanda M. Davis Fix For: 2.6.1 { "name": "sam_db_file_storage", "description": "DB based file storage in SAM", "min_version": "2.6.3.0" } -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22321) Add Registry StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248772#comment-16248772 ] Yolanda M. Davis commented on AMBARI-22321: --- [~jluniya] Since these features are aligned with the Stack Version I think they should be set to 2.6.3.0 instead of 2.6.0, Is that accurate? If so I have a suggested update below: {code:java} { "name": "registry_remove_rootpath", "description": "Registry remove root path setting", "min_version": "2.6.3.0" }, { "name": "registry_allowed_resources_support", "description": "Registry allowed resources", "min_version": "2.6.3.0" }, { "name": "registry_rewriteuri_filter_support", "description": "Registry RewriteUri servlet filter", "min_version": "2.6.3.0" } {code} > Add Registry StackFeatures to HDP StackFeatures > --- > > Key: AMBARI-22321 > URL: https://issues.apache.org/jira/browse/AMBARI-22321 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Jayush Luniya >Assignee: Jayush Luniya >Priority: Critical > Fix For: 2.6.1 > > > {code} > { > "name": "registry_remove_rootpath", > "description": "Registry remove root path setting", > "min_version": "2.6.0.0" > }, > { > "name": "registry_allowed_resources_support", > "description": "Registry allowed resources", > "min_version": "2.6.0.0" > }, > { > "name": "registry_rewriteuri_filter_support", > "description": "Registry RewriteUri servlet filter", > "min_version": "2.6.0.0" > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch was committed > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > Attachments: AMBARI-22338.patch > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226818#comment-16226818 ] Yolanda M. Davis commented on AMBARI-22338: --- cc: [~jluniya] > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > Attachments: AMBARI-22338.patch > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Status: Patch Available (was: Open) > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > Attachments: AMBARI-22338.patch > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Attachment: AMBARI-22338.patch > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > Attachments: AMBARI-22338.patch > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Attachment: (was: AMBARI-22338.patch) > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Attachment: AMBARI-22338.patch > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > Attachments: AMBARI-22338.patch > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Description: { "name": "sam_storage_core_in_registry", "description": "Storage core module moved to registry", "min_version": "2.6.0.0" } was: { "name": "sam_storage_core_in_registry", "description": "Storage core module moved to registry", "min_version": "3.1.0.0" } > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22338) Add SAM StackFeatures to HDP StackFeatures
[ https://issues.apache.org/jira/browse/AMBARI-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated AMBARI-22338: -- Description: {noformat} { "name": "sam_storage_core_in_registry", "description": "Storage core module moved to registry", "min_version": "2.6.0.0" } {noformat} was: { "name": "sam_storage_core_in_registry", "description": "Storage core module moved to registry", "min_version": "2.6.0.0" } > Add SAM StackFeatures to HDP StackFeatures > -- > > Key: AMBARI-22338 > URL: https://issues.apache.org/jira/browse/AMBARI-22338 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.6.0 >Reporter: Yolanda M. Davis >Priority: Critical > Fix For: 2.6.1 > > > {noformat} > { > "name": "sam_storage_core_in_registry", > "description": "Storage core module moved to registry", > "min_version": "2.6.0.0" > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)