SEZ9 commented on issue #13330:
URL: https://github.com/apache/seatunnel/issues/13330#issuecomment-7023083365
@SEPURI-SAI-KRISHNA thanks for the per-case breakdown. It lines up with the
four cases raised here: mixed BSON types, missing/null/NaN, the split key in
`match.query`, and dotted paths.
On your question about splitting: keeping 1 and 2 together is fine, since
they share the bracket and catch-all code. For 3 and 4, separate commits in the
same PR are enough, and separate PRs are not needed. That keeps each fix
reviewable on its own while the "every document read exactly once" checks still
run against the combined behaviour.
A few things I'd like confirmed, either here or in the PR description:
- **Catch-all vs. ranges:** Is the extra split strictly disjoint from the
per-bracket ranges, so that a document can never match both a bracket range and
the `$or` catch-all? Does `{key: null}` there also cover documents where the
key is missing entirely?
- **`match.query` on the split key:** Does `SamplingSplitStrategyTest`
include a case where the user's `match.query` itself has a condition on the
split key? That is the situation where merging went wrong, so I'd like the
`$and` combination asserted for it explicitly.
- **Exactly-once for 1, 2 and 4:** Does the `MongodbIT` data include
mixed-type, missing/null/NaN and dotted-key documents? If it doesn't, the
exact-rows check would not exercise those paths end to end.
The code-level review will happen on the PR itself.
<!-- streview-comment:2457 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]