jiangchunyang created FLINK-32550:
-
Summary: Single-parallel task consumes multi-partition data
exception in pulsar
Key: FLINK-32550
URL: https://issues.apache.org/jira/browse/FLINK-32550
Project: Flink
Issue Type: Bug
Components: Connectors / Pulsar
Affects Versions: 1.16.0
Environment: flink 1.16.0
pulsar 2.11.1
Reporter: jiangchunyang
Send data to pulsar every second through the program. When there is a
partitioned-topic with 3 partitions in pulsar, use the Exclusive consumption
mode for consumption (verified, it is not a data problem, because there is no
abnormal data in normal tasks), start A 3-parallel task consumes this topic,
and everything is normal; but when I use a single-parallel task to consume this
topic, an exception occurs when some data is deserialized, and the avro format
data is used, so an error is reported (Length is negative: -52); When I create
a partitioned-topic with a single partition, I use a single parallelism task to
consume, and there is no such problem, all the data is normal, and there is no
task error.
I checked the logs and found that the consumer and topic-partition are
allocated as expected.
Summarize:
3 topic-partitions and 1 consume work with exception (3 partitions are
allocated to a single consumer)
3 topic-partitions and 3 consumers work normally
1 topic-partition 1 consumer works normally
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
