Hi SiNan, In the flink world, we don't always rely on the schema information provided by Pulsar or other connector systems. Flink application has its own (de)serialization schema logic, which treats the messages only in a binary format like a byte array.
In flink-connector-pulsar, we only use the schema when the users want to do some evolution check. Otherwise, we will only send messages in BYTES schema. On Tue, Mar 28, 2023 at 10:06 AM SiNan Liu <liusinan1...@gmail.com> wrote: > > Hi yufan. > Can you describe a bit the usage scenario of byte schema in > flink-connector-pulsa? > > > Thanks, > sinan > > Yufan Sheng <syh...@gmail.com> 于 2023年3月28日周二 上午9:53写道： > > > As the flink-connector-pulsar developer, I don't want to disable the > > BYTES schema upload. In my opinion, using BYTES schema means the users > > want to bypass the schema check and handle the schema validation by > > themselves. > > > > On Tue, Mar 28, 2023 at 8:58 AM SiNan Liu <liusinan1...@gmail.com> wrote: > > > > > > Hi, everyone. > > > When a user uploads bytes schema. We can warn the user and skip uploading > > > bytes schema. > > > Also check to see if the topic has a schema other than bytes. > > > 1. If yes, warn the user that it is not necessary to upload bytes schema. > > > You can subscribe to a topic using bytes schema. > > > 2. If there is no schema, warn the user that the topic does not have a > > > schema. The default is bytes schema, and there is no need to upload it. > > > Rather than simply throwing an exception rejecting the upload bytes > > schema. > > > > > > > > > Thanks, > > > sinan > > > > > > > > > Christophe Bornet <bornet.ch...@gmail.com> 于 2023年3月28日周二 上午1:15写道： > > > > > > > This change broke the Flink SQL Pulsar connector: > > > > https://github.com/streamnative/flink/issues/270 > > > > So I propose to revert it. > > > > > > > > Le ven. 9 déc. 2022 à 11:57, labuladong <labulad...@foxmail.com> a > > écrit : > > > > > > > > > > Hi pulsar community, > > > > > > > > > > > > > > > I'd like to discuss the behavior of schema uploading, for more > > context > > > > see https://github.com/apache/pulsar/issues/18825 > > > > > > > > > > > > > > > I think that forbidding users to upload `BYTES` schema is a > > recommended > > > > way to solve this issue. But this may change the existing behavior, so > > do > > > > you have any suggestion about this issue? > > > > > > > > > > > > > > > Thanks, > > > > > Donglai > > > > > >
