[
https://issues.apache.org/jira/browse/FLINK-28609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570716#comment-17570716
]
Jacek Wislicki commented on FLINK-28609:
----------------------------------------
Thank you, but unfortunately we are constrained to used only official releases
that can be pulled from public Maven repos. For development purposes, we will
be using the old (deprecated) connector version from StreamNative/Pulsar.
Do you know a time plan from Flink 1.16 to be released? I see that there is a
lot of work done but still some issues are open and being discussed.
> Flink-Pulsar connector fails on larger schemas
> ----------------------------------------------
>
> Key: FLINK-28609
> URL: https://issues.apache.org/jira/browse/FLINK-28609
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
> Affects Versions: 1.14.3, 1.14.4, 1.14.5, 1.15.1
> Reporter: Jacek Wislicki
> Priority: Major
> Labels: pull-request-available
> Attachments:
> [FLINK-28609][Connector_Pulsar]_PulsarSchema_didn't_get_properly_serialized_.patch,
> exception.txt
>
>
> When a model results in a larger schema (this seems to be related to its byte
> array representation), the number of expected bytes to read is different than
> the number of actually read bytes: [^exception.txt]. The "read" is such a
> case is always 1018 while the expected "byteLen" gives a greater value. For
> smaller schemata, the numbers are equal (less than 1018) and no issue occurs.
> The problem reproduction is on
> [GitHub|https://github.com/JacekWislicki/vp-test2]. There are 2 simple jobs
> (SimpleJob1 and SimpleJob2) using basic models for the Pulsar source
> definition (PulsarMessage1 and PulsarMessage2, respectively). Each of the
> corresponding schemata is properly serialised and deserialised, unless an
> effective byte array length becomes excessive (marked with "the problem
> begins" in model classes). The fail condition can be achieved by a number of
> fields (PulsarMessage1) or just longer field names (PulsarMessage2). The
> problem occurs on either Avro or a JSON schema set in the Pulsar source
> definition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
