SEPURI-SAI-KRISHNA commented on issue #13330:
URL: https://github.com/apache/seatunnel/issues/13330#issuecomment-7032243068
@SEZ9 thanks. Checked against #13332:
- Catch-all vs. ranges: disjoint for every value that is not an array. A
range only matches its own type bracket, and the catch-all only matches null or
missing, NaN (no range of numbers matches it) and the types no range covers.
`{key: null}` also matches a missing key: the IT data has documents without
`c_key`, and each `c_id` is written exactly once. Arrays are the exception the
docs tip names.
- `match.query` on the split key: `testEverySplitKeepsTheMatchQuery` splits
by `_id` with `match.query` `{_id: {$gte: 1500}}` and asserts that every split
is the `$and` of the match query and its bound.
- IT data: NaN and dotted keys were missing. I added NaN values of `c_key`
and a job split by `c_nested.k`. All jobs pass on Zeta, Flink and Spark.
Without the dotted fix that job still reads each document once, but in 1 split
instead of 18.
The PR now has three commits (1+2, 3, 4). Each new unit test fails on the
commit before it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]