SEZ9 commented on issue #13330: URL: https://github.com/apache/seatunnel/issues/13330#issuecomment-6994106490
Thanks for the thorough report. The four cases, the config, and the measured read counts on MongoDB 8.0.4 make the problem clear, and the silent loss or re-read of documents with no error or warning is the most concerning part. Since you've indicated you're willing to submit a PR, that would be very welcome. A few suggestions: - The four cases look independent (mixed BSON types in the split key, missing/null/NaN bounds, `match.query` replaced when it names the split key, and a dotted split key such as `a.b`). Separate fixes, or at least separate commits, may be easier to review. - Please turn the rows of your table into regression tests for `SamplingSplitStrategy` / `MongoSplitUtils.createMongoSplit`, asserting that the splits together read every document exactly once and that `match.query` is preserved. - Before you start, could you briefly outline the approach you have in mind for each case, particularly how you plan to handle bounds of different BSON types and null/missing split-key values? That would help us agree on the direction early. <!-- streview-comment:2441 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
