zhangshenghang commented on issue #10203: URL: https://github.com/apache/seatunnel/issues/10203#issuecomment-3707023668
> Thanks [@yzeng1618](https://github.com/yzeng1618) for the Flink CDC reference. Flink CDC's restore-time discovery requires a job restart, which doesn't meet this issue's "without restarting the job" requirement. I also considered the Kafka/Pulsar pattern (background thread directly modifying state), but this could cause race conditions during checkpoints or split assignments. I'm proposing asynchronous discovery with a pending queue to ensure thread safety: discovery thread only queues discovered tables(no state changes), and the main thread processes the queue at key points(assignSplits, snapshotState, etc) to ensure thread safety. What do you think about this direction? I don't quite understand what you mean. Is there any problem with discovering new tables during checkpoint? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
