[jira] [Commented] (FLINK-4004) Do not pass custom flink kafka connector properties to Kafka to avoid warnings
[
https://issues.apache.org/jira/browse/FLINK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334321#comment-17334321
]
Flink Jira Bot commented on FLINK-4004:
---
This issue was marked "stale-assigned" and has not received an update in 7
days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> Do not pass custom flink kafka connector properties to Kafka to avoid warnings
> --
>
> Key: FLINK-4004
> URL: https://issues.apache.org/jira/browse/FLINK-4004
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Robert Metzger
>Assignee: Luffy Tsai
>Priority: Major
> Labels: stale-assigned
>
> The FlinkKafkaConsumer has some custom properties, which we pass to the
> KafkaConsumer as well (such as {{flink.poll-timeout}}). This causes Kafka to
> log warnings about unused properties.
> We should not pass Flink-internal properties to Kafka, to avoid those
> warnings.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-4004) Do not pass custom flink kafka connector properties to Kafka to avoid warnings
[
https://issues.apache.org/jira/browse/FLINK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323716#comment-17323716
]
Flink Jira Bot commented on FLINK-4004:
---
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Do not pass custom flink kafka connector properties to Kafka to avoid warnings
> --
>
> Key: FLINK-4004
> URL: https://issues.apache.org/jira/browse/FLINK-4004
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Robert Metzger
>Assignee: Luffy Tsai
>Priority: Major
> Labels: stale-assigned
>
> The FlinkKafkaConsumer has some custom properties, which we pass to the
> KafkaConsumer as well (such as {{flink.poll-timeout}}). This causes Kafka to
> log warnings about unused properties.
> We should not pass Flink-internal properties to Kafka, to avoid those
> warnings.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-4004) Do not pass custom flink kafka connector properties to Kafka to avoid warnings
[
https://issues.apache.org/jira/browse/FLINK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178409#comment-16178409
]
Luffy Tsai commented on FLINK-4004:
---
Hi,
Before I started working, I thought the solution of the issue was defined a
filter function {{filterUnusedProperties}} and then just changed the line
{{this.kafkaProperties = checkNotNull(kafkaProperties);}} in the
[KafkaConsumerThread.java#L132|https://github.com/apache/flink/blob/4afca4b3a13b61c2754bc839c77ba4d4eb1d2da2/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.java#L132],
and finally the warning log would disappear.
However, I have had some problems. The following methods are not good enough.
h6. 1. Get config from Kafka ConsumerConfig.
Unfortunately, the public method {{configNames}} which was added from v0.10.1.
It couldn't be used in the flink-connector-kafka-0.8 and
flink-connector-kafka-0.9.
[ConsumerConfig#L417|https://github.com/apache/kafka/blob/2b663790733527488d7f33ebc47f383ff18b5a83/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L417]
Maybe we could get all public fields by the following code, but it's not a
appropriate design pattern.
{{Field[] fields = ConsumerConfig.class.getFields();}}
h6. 2. Maintain consumer config in the Flink.
It sounds workable but dirty.
Could you give me some suggestions?
Thank you!
> Do not pass custom flink kafka connector properties to Kafka to avoid warnings
> --
>
> Key: FLINK-4004
> URL: https://issues.apache.org/jira/browse/FLINK-4004
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
>Reporter: Robert Metzger
>Assignee: Luffy Tsai
>
> The FlinkKafkaConsumer has some custom properties, which we pass to the
> KafkaConsumer as well (such as {{flink.poll-timeout}}). This causes Kafka to
> log warnings about unused properties.
> We should not pass Flink-internal properties to Kafka, to avoid those
> warnings.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
