SEZ9 commented on issue #11741: URL: https://github.com/apache/seatunnel/issues/11741#issuecomment-5923826037
Hi @SiddhantShedge45, thanks for offering to pick this up. To recap the scope of this issue: the `connector-jdbc` `ChunkSplitter` currently picks only the first supported primary-key column as the split key, and `DynamicChunkSplitter` walks boundaries on that single column. For composite primary keys where the first column has low cardinality and sampling is disabled, that degrades into very few / very skewed chunks. The goal is multi-column (tuple-ordered) split-key selection and boundary walking inside the JDBC splitter, while keeping the existing single-column numeric / string / date paths unchanged. One important note before you start coding: there is already an open PR referenced earlier in this thread that targets exactly this `ChunkSplitter` / `DynamicChunkSplitter` path. A second implementation would overlap and conflict with it, so please don't open a parallel fix for now. Where help would be genuinely valuable right now is reviewing that PR against the cases we care about most: - repeated first-key values (the low-cardinality case that motivated this issue); - NULL key components in composite keys; - tuple-boundary inclusivity (no dropped or duplicated rows at chunk edges); - the single-column fallback staying byte-for-byte unchanged for existing users. If you find a concrete gap in any of those, please report it directly on that PR with a reproducible example or a test case. If that PR's implementation path ends up being closed or abandoned, we'll reassess assignment of this issue, and your interest here will be noted. Could you confirm whether you're happy to proceed on the review side for now? <!-- streview-comment:1443 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
