[
https://issues.apache.org/jira/browse/AMBARI-26617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088376#comment-18088376
]
Basapuram Kumar commented on AMBARI-26617:
------------------------------------------
I am working on this, the PR will be out soon for this. Thanks.
> Add KAFKA_CLIENT component to deploy Kafka client configuration on non-broker
> hosts
> -----------------------------------------------------------------------------------
>
> Key: AMBARI-26617
> URL: https://issues.apache.org/jira/browse/AMBARI-26617
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
> Affects Versions: 3.0.0, 2.7.9
> Reporter: Basapuram Kumar
> Priority: Major
>
> h3. Summary
> Ambari’s Kafka service currently defines only the {{KAFKA_BROKER}} component.
> Client configuration files ({{{}kafka-env.sh{}}}, {{{}log4j.properties{}}},
> {{{}tools-log4j.properties{}}}, and Kerberos JAAS files) are generated only
> when the broker is installed and configured on a host.
> This makes it difficult to use Kafka client tools (for example
> {{{}kafka-topics.sh{}}}, {{{}kafka-console-producer.sh{}}},
> {{{}kafka-console-consumer.sh{}}}) on hosts that do not run a Kafka broker,
> especially in Kerberos-enabled clusters where JAAS and environment settings
> are required.
> This feature adds a new {{KAFKA_CLIENT}} CLIENT component to the Kafka
> service so Ambari can install the Kafka client package and push client
> configuration to any host in the cluster.
> h3. Problem
> * Kafka client configs are tied to broker install/configure lifecycle.
> * Client-only hosts do not receive required files under
> {{{}/etc/kafka/conf{}}}.
> * In secure clusters, client tools fail because {{kafka-env.sh}} and JAAS
> configs are missing.
> * Operators must manually copy or maintain client configs outside Ambari.
> h3. Proposed Solution
> Add a {{KAFKA_CLIENT}} component to the Kafka service stack definition with:
> # New component: {{KAFKA_CLIENT}} (category: {{{}CLIENT{}}}, cardinality:
> {{{}1+{}}})
> # Install/configure script: {{kafka_client.py}} following the existing
> Ambari CLIENT pattern (similar to {{{}HIVE_CLIENT{}}}, {{{}HDFS_CLIENT{}}})
> # Generated configs on client hosts:
> ** {{kafka-env.sh}}
> ** {{log4j.properties}}
> ** {{tools-log4j.properties}}
> ** {{kafka_client_jaas.conf}} (when Kerberos/SASL is enabled)
> # Kerberos support:
> ** Add {{KAFKA_CLIENT}} identity in {{kerberos.json}} referencing broker
> principal/keytab
> ** Update {{kafka_client_jaas_conf.xml}} to use keytab-based login
> ({{{}useKeyTab{}}}, {{{}principal{}}}) instead of ticket cache
> ** On client-only hosts, also create {{kafka_jaas.conf}} when absent (since
> {{kafka-env.sh}} references it and broker configure does not run there)
> # Stack packaging:
> ** Register {{KAFKA_CLIENT}} in {{stack_packages.json}} with
> {{kafka-broker}} stack-select package mapping
> # Config export support:
> ** Add client-related params ({{{}kafka_broker_hosts{}}},
> {{{}kafka_broker_port{}}}) for exported client config files
> h3. Scope
> * Target stack: {{BIGTOP/3.2.0}} (base Kafka service definition)
> h3. Benefits
> * Enables Ambari-managed Kafka client deployment on non-broker nodes
> * Improves Kerberos cluster usability for Kafka CLI/tools
> * Aligns Kafka with existing Ambari CLIENT component patterns
> * Reduces manual config management and operational errors
> h3. Acceptance Criteria
> * {{KAFKA_CLIENT}} appears as assignable component in Ambari UI for Kafka
> service
> * Install/Configure on a host without {{KAFKA_BROKER}} succeeds
> * Client config files are created under {{/etc/kafka/conf}}
> * In Kerberos-enabled cluster, client tools can authenticate using
> generated JAAS/env files
> * Stack upgrade pre-restart for client component selects correct package
> version
> * No regression for existing {{KAFKA_BROKER}} install/configure/start/stop
> behavior
> h3. Test Plan
> * Setup a 4-node cluster with 3-kafka brokers and 1-kafka client node.
> * {{Verify KAFKA_CLIENT}} Installations on client node.
> * Verify configuration files "/etc/kafka/conf/" at client node.
> * perform kafka CLI operations from client like kafka console topics list,
> producer, consumer related operations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
