goutamadwant commented on code in PR #19202:
URL: https://github.com/apache/pinot/pull/19202#discussion_r3763801177
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-3.0/src/main/java/org/apache/pinot/plugin/stream/kafka30/KafkaPartitionLevelConnectionHandler.java:
##########
@@ -98,28 +99,13 @@ private Properties buildProperties(StreamConfig
streamConfig) {
return consumerProp;
}
- /// Filter properties to only include the specified Kafka configurations.
- /// This prevents "was supplied but isn't a known config" warnings from
Kafka clients.
- ///
- /// @param props The properties to filter
- /// @param validConfigNames The set of valid configuration names for the
target Kafka client
- /// @return A new Properties object containing only the valid configurations
- private Properties filterKafkaProperties(Properties props, Set<String>
validConfigNames) {
- Properties filteredProps = new Properties();
- for (String key : props.stringPropertyNames()) {
- if (validConfigNames.contains(key)) {
- filteredProps.put(key, props.get(key));
- }
- }
- return filteredProps;
- }
-
private Consumer<Bytes, Bytes> createConsumer(Properties consumerProp,
RetryPolicy retryPolicy) {
AtomicReference<Consumer<Bytes, Bytes>> consumer = new AtomicReference<>();
try {
retryPolicy.attempt(() -> {
try {
- consumer.set(new KafkaConsumer<>(filterKafkaProperties(consumerProp,
CONSUMER_CONFIG_NAMES)));
+ consumer.set(new
KafkaConsumer<>(KafkaConfigUtils.filterAndValidateKafkaProperties(consumerProp,
Review Comment:
Fixed this. Filtering and validation now run once before entering the retry
logic for Kafka 3.x and 4.x consumer creation, direct AdminClient creation, and
shared AdminClient initialization. Retries now cover client construction only,
so deterministic ConfigException failures surface immediately with their
original message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
