[jira] [Updated] (FLINK-25686) Support Pulsar Schema evolution in Pulsar Source Connector
[ https://issues.apache.org/jira/browse/FLINK-25686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-25686: --- Labels: Pulsar pull-request-available stale-assigned (was: Pulsar pull-request-available) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issue is assigned but has not received an update in 30 days, so it has been labeled "stale-assigned". If you are still working on the issue, please remove the label and add a comment updating the community on your progress. If this issue is waiting on feedback, please consider this a reminder to the committer/reviewer. Flink is a very active project, and so we appreciate your patience. If you are no longer working on the issue, please unassign yourself so someone else may work on it. > Support Pulsar Schema evolution in Pulsar Source Connector > -- > > Key: FLINK-25686 > URL: https://issues.apache.org/jira/browse/FLINK-25686 > Project: Flink > Issue Type: New Feature > Components: Connectors / Pulsar >Reporter: Yufei Zhang >Assignee: Yufei Zhang >Priority: Minor > Labels: Pulsar, pull-request-available, stale-assigned > > Currently the source connector reads all messages as Message, and > implements a deserialization abstraction to deserialize the byte array > message. This deserialization abstraction supports both flink schema and > pulsar schema. However because of this extra layer of abstraction, even when > using pulsar schema, the schema info is not provided to the consumer > (client), so it is not using any pulsar’s schema validation mechanism. (If > pulsar client has a schema, it will send the schema to broker on connect and > broker will validate if it is a valid schema according to a preset > compatibility rule :[ > https://pulsar.apache.org/docs/en/schema-evolution-compatibility/|https://pulsar.apache.org/docs/en/schema-evolution-compatibility/]). > we need to support this schema evolution -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-25686) Support Pulsar Schema evolution in Pulsar Source Connector
[ https://issues.apache.org/jira/browse/FLINK-25686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-25686: --- Issue Type: New Feature (was: Technical Debt) > Support Pulsar Schema evolution in Pulsar Source Connector > -- > > Key: FLINK-25686 > URL: https://issues.apache.org/jira/browse/FLINK-25686 > Project: Flink > Issue Type: New Feature > Components: Connectors / Pulsar >Reporter: Yufei Zhang >Assignee: Yufei Zhang >Priority: Minor > Labels: Pulsar, pull-request-available > > Currently the source connector reads all messages as Message, and > implements a deserialization abstraction to deserialize the byte array > message. This deserialization abstraction supports both flink schema and > pulsar schema. However because of this extra layer of abstraction, even when > using pulsar schema, the schema info is not provided to the consumer > (client), so it is not using any pulsar’s schema validation mechanism. (If > pulsar client has a schema, it will send the schema to broker on connect and > broker will validate if it is a valid schema according to a preset > compatibility rule :[ > https://pulsar.apache.org/docs/en/schema-evolution-compatibility/|https://pulsar.apache.org/docs/en/schema-evolution-compatibility/]). > we need to support this schema evolution -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-25686) Support Pulsar Schema evolution in Pulsar Source Connector
[ https://issues.apache.org/jira/browse/FLINK-25686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Zhang updated FLINK-25686: Summary: Support Pulsar Schema evolution in Pulsar Source Connector (was: Support Pulsar Schema evolution in Pulsar Connector) > Support Pulsar Schema evolution in Pulsar Source Connector > -- > > Key: FLINK-25686 > URL: https://issues.apache.org/jira/browse/FLINK-25686 > Project: Flink > Issue Type: Technical Debt > Components: Connectors / Pulsar >Reporter: Yufei Zhang >Assignee: Yufei Zhang >Priority: Minor > Labels: Pulsar, pull-request-available > > Currently the source connector reads all messages as Message, and > implements a deserialization abstraction to deserialize the byte array > message. This deserialization abstraction supports both flink schema and > pulsar schema. However because of this extra layer of abstraction, even when > using pulsar schema, the schema info is not provided to the consumer > (client), so it is not using any pulsar’s schema validation mechanism. (If > pulsar client has a schema, it will send the schema to broker on connect and > broker will validate if it is a valid schema according to a preset > compatibility rule :[ > https://pulsar.apache.org/docs/en/schema-evolution-compatibility/|https://pulsar.apache.org/docs/en/schema-evolution-compatibility/]). > we need to support this schema evolution -- This message was sent by Atlassian Jira (v8.20.1#820001)
