CloverDew commented on code in PR #9867:
URL: https://github.com/apache/seatunnel/pull/9867#discussion_r2375711413
##########
seatunnel-core/seatunnel-flink-starter/seatunnel-flink-starter-common/src/main/java/org/apache/seatunnel/core/starter/flink/execution/SinkExecuteProcessor.java:
##########
@@ -139,20 +152,20 @@ public List<DataStreamTableInfo>
execute(List<DataStreamTableInfo> upstreamDataS
((TableSinkFactory) (factory.orElse(null))));
sink.setJobContext(jobContext);
handleSaveMode(sink);
+
+ // check if sink supports schema evolution
+ if (sink instanceof SupportSchemaEvolutionSink) {
+ parallelism = 1;
+ sinkParallelism = true;
Review Comment:
Because I have customized a coordinator to receive requests, if the sink end
is multi-parallel, I cannot guarantee that all partitions will switch
partitions at the same time, and there is such a coordination risk. The second
point is that the implementation is relatively complicated. At the initial
stage of function implementation, this situation may not be considered for the
time being.
##########
seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SupportSchemaEvolutionSinkWriter.java:
##########
@@ -30,4 +32,48 @@ public interface SupportSchemaEvolutionSinkWriter {
* @throws IOException
*/
void applySchemaChange(SchemaChangeEvent event) throws IOException;
+
+ /**
+ * handle FlushEvent propagated from upstream
+ *
+ * @param event
+ * @throws IOException
+ */
+ default void handleFlushEvent(FlushEvent event) throws IOException {
Review Comment:
I've looked into it and found that Zeta uses a checkpoint barrier to
automatically trigger writer.prepareCommit() to flush data, eliminating the
need to manually send a FlushEvent. This is a standard sink interface that
naturally supports multiple degrees of parallelism and requires no additional
coordination logic. I'm not sure if I understand this correctly.
Our implementation is indeed quite heavyweight by comparison, and initially
only supports a single degree of parallelism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
