[jira] [Commented] (FLINK-20617) Kafka Consumer Deserializer Exception on application mode
[
https://issues.apache.org/jira/browse/FLINK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250851#comment-17250851
]
Georger commented on FLINK-20617:
-
[~fly_in_gis] I has try , it works ,but it looks so weird ...
> Kafka Consumer Deserializer Exception on application mode
> -
>
> Key: FLINK-20617
> URL: https://issues.apache.org/jira/browse/FLINK-20617
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.11.2
> Environment: application mode
> flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
>Reporter: Georger
>Priority: Critical
> Attachments: taskmanager.out
>
>
> Kafka source may has some issues on application mode
>
> when i run it with application mode on flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:789)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:643)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:623)
> at
> org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
> at
> org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
> at
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:688)
> ... 15 more
> The pom is:
>
> org.apache.flink
> flink-connector-kafka_2.11
> ${flink.version}
>
>
> org.slf4j
> slf4j-api
>
>
> org.apache.kafka
> kafka-clients
>
>
>
>
> org.apache.kafka
> kafka-clients
> 1.0.1
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20617) Kafka Consumer Deserializer Exception on application mode
[
https://issues.apache.org/jira/browse/FLINK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Georger updated FLINK-20617:
Attachment: taskmanager.out
> Kafka Consumer Deserializer Exception on application mode
> -
>
> Key: FLINK-20617
> URL: https://issues.apache.org/jira/browse/FLINK-20617
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.11.2
> Environment: application mode
> flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
>Reporter: Georger
>Priority: Critical
> Attachments: taskmanager.out
>
>
> Kafka source may has some issues on application mode
>
> when i run it with application mode on flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:789)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:643)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:623)
> at
> org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
> at
> org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
> at
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:688)
> ... 15 more
> The pom is:
>
> org.apache.flink
> flink-connector-kafka_2.11
> ${flink.version}
>
>
> org.slf4j
> slf4j-api
>
>
> org.apache.kafka
> kafka-clients
>
>
>
>
> org.apache.kafka
> kafka-clients
> 1.0.1
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-20617) Kafka Consumer Deserializer Exception on application mode
[
https://issues.apache.org/jira/browse/FLINK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250762#comment-17250762
]
Georger commented on FLINK-20617:
-
it seems the
org.apache.kafka.common.serialization.Deserializer has loaded twice, here is
the detail log:
[Loaded org.apache.kafka.common.serialization.Deserializer from
file:/mnt/data1/yarn/nm/usercache/hdfs/appcache/application_1606119213964_1657/blobStore-a0f843a0-737f-491c-9d8e-a7ec76568e07/job_24c0a330e49ff08ff3355fc9efeaa7bf/blob_p-bf8f8652268b5550585fcc184b2925c5e613a255-932296ba280a1ae7a544c24b249e68cd]
[Loaded org.apache.kafka.common.serialization.Deserializer from
file:/mnt/data0/yarn/nm/usercache/hdfs/appcache/application_1606119213964_1657/filecache/14/flink-examples-1.0-SNAPSHOT201217_104230_459_r57.jar]
> Kafka Consumer Deserializer Exception on application mode
> -
>
> Key: FLINK-20617
> URL: https://issues.apache.org/jira/browse/FLINK-20617
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.11.2
> Environment: application mode
> flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
>Reporter: Georger
>Priority: Critical
>
> Kafka source may has some issues on application mode
>
> when i run it with application mode on flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:789)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:643)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:623)
> at
> org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
> at
> org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
> at
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:688)
> ... 15 more
> The pom is:
>
> org.apache.flink
> flink-connector-kafka_2.11
> ${flink.version}
>
>
> org.slf4j
> slf4j-api
>
>
> org.apache.kafka
> kafka-clients
>
>
>
>
> org.apache.kafka
> kafka-clients
> 1.0.1
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-20617) Kafka Consumer Deserializer Exception on application mode
[
https://issues.apache.org/jira/browse/FLINK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250761#comment-17250761
]
Georger commented on FLINK-20617:
-
it seems not the problem , i have only use flink-connector-kafka-0.11_2.11 or
flink-connector-kafka_2.11, the same result it only with application mode
[~aljoscha]
> Kafka Consumer Deserializer Exception on application mode
> -
>
> Key: FLINK-20617
> URL: https://issues.apache.org/jira/browse/FLINK-20617
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.11.2
> Environment: application mode
> flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
>Reporter: Georger
>Priority: Critical
>
> Kafka source may has some issues on application mode
>
> when i run it with application mode on flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:789)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:643)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:623)
> at
> org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
> at
> org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
> at
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:688)
> ... 15 more
> The pom is:
>
> org.apache.flink
> flink-connector-kafka_2.11
> ${flink.version}
>
>
> org.slf4j
> slf4j-api
>
>
> org.apache.kafka
> kafka-clients
>
>
>
>
> org.apache.kafka
> kafka-clients
> 1.0.1
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-20617) Kafka Consumer Deserializer Exception on application mode
Georger created FLINK-20617:
---
Summary: Kafka Consumer Deserializer Exception on application mode
Key: FLINK-20617
URL: https://issues.apache.org/jira/browse/FLINK-20617
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.11.2
Environment: application mode
flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
Reporter: Georger
Kafka source may has some issues on application mode
when i run it with application mode on flink 1.11.2 it can't startup
the detail Excetion is:
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at
org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:789)
at
org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:643)
at
org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:623)
at
org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
at
org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
at
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
at
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.KafkaException:
org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance
of org.apache.kafka.common.serialization.Deserializer
at
org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
at
org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:688)
... 15 more
The pom is:
org.apache.flink
flink-connector-kafka_2.11
${flink.version}
org.slf4j
slf4j-api
org.apache.kafka
kafka-clients
org.apache.kafka
kafka-clients
1.0.1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20585) Document: The release note has some minor bug
[ https://issues.apache.org/jira/browse/FLINK-20585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Georger updated FLINK-20585: Summary: Document: The release note has some minor bug (was: the release note has some minor bug) > Document: The release note has some minor bug > - > > Key: FLINK-20585 > URL: https://issues.apache.org/jira/browse/FLINK-20585 > Project: Flink > Issue Type: Bug > Components: Documentation >Affects Versions: 1.12.0 >Reporter: Georger >Priority: Minor > > in this page > [https://flink.apache.org/news/2020/12/10/release-1.12.0.html#release-notes] > Please review the [release > notes|https://ci.apache.org/projects/flink/flink-docs-release-1.12/release-notes/flink-1.12.html] > carefully for a detailed list of changes and new features if you plan to > upgrade your setup to Flink 1.11. This version is API-compatible with > previous 1.x releases for APIs annotated with the @Public annotation > the setup to Flink should be Flink 1.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20585) the release note has some minor bug
Georger created FLINK-20585: --- Summary: the release note has some minor bug Key: FLINK-20585 URL: https://issues.apache.org/jira/browse/FLINK-20585 Project: Flink Issue Type: Bug Components: Documentation Affects Versions: 1.12.0 Reporter: Georger in this page [https://flink.apache.org/news/2020/12/10/release-1.12.0.html#release-notes] Please review the [release notes|https://ci.apache.org/projects/flink/flink-docs-release-1.12/release-notes/flink-1.12.html] carefully for a detailed list of changes and new features if you plan to upgrade your setup to Flink 1.11. This version is API-compatible with previous 1.x releases for APIs annotated with the @Public annotation the setup to Flink should be Flink 1.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
