dongwooooooo commented on issue #10203: URL: https://github.com/apache/seatunnel/issues/10203#issuecomment-3705617840
Thanks @yzeng1618 for the Flink CDC reference. Flink CDC's restore-time discovery requires a job restart, which doesn't meet this issue's "without restarting the job" requirement. I also considered the Kafka/Pulsar pattern (background thread directly modifying state), but this could cause race conditions during checkpoints or split assignments. I'm proposing asynchronous discovery with a pending queue to ensure thread safety: discovery thread only queues discovered tables(no state changes), and the main thread processes the queue at key points(assignSplits, snapshotState, etc) to ensure thread safety. What do you think about this direction? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
