tsreaper commented on issue #3420: URL: https://github.com/apache/paimon/issues/3420#issuecomment-2240948060
The main difficulty is to decide where you should use each class and call each method. For example, consider a distributed system with one master node and several workers node. `TableScan` should only be used in master, while `TableRead` and `TableWrite` should only be used in workers. Also you need to design how to distribute `Split`s generated from `TableScan` to the workers. You also need to be careful on `TableCommit` because it can only run with 1 parallelism (otherwise the consistency guarantee is broken). All in all, these things are exactly what you need to concern when designing a distributed system. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
