morningman opened a new pull request, #68801:
URL: https://github.com/apache/doris/pull/68801
### What problem does this PR solve?
Issue Number: None
Related PR: #64757, #67704, #67153, #68250
Original authors: @yujun777 (#68250: the regression suite and the MTMV
assertion are ported from it), @Gabriel39 (#67153, #67704). The commit carries
Co-authored-by for both.
Problem Summary:
**In short.** These four branch-4.x fixes reached master through master's
own PRs, so master's production code already behaves the same. Some of the
tests the branch-4.x PRs added did not come along, mostly because the code they
test has moved on master. This PR adds that coverage in master's structure:
four unit-test cases and one regression suite. It does not change production
code.
**Background**
| branch-4.x PR | Master PR | Where the code lives on master |
|---|---|---|
| #64757: SQL Server / Oracle booleans pushed down as 1/0 | #62183, #64756 |
`fe-connector-jdbc` (`JdbcQueryBuilder.formatBooleanLiteral`) |
| #67704: fractional seconds in Iceberg `FOR TIME AS OF` | #67705 |
`fe-connector-iceberg` (`IcebergTimeUtils`,
`IcebergConnectorMetadata.resolveTimeTravel`) |
| #67153: fuzzy mode keeps FileScannerV2 on | #67574 |
`SessionVariable.initFuzzyModeVariables` |
| #68250: a base partition without a snapshot does not force a full MV
rebuild | #62606 | `MTMV.hasRefreshSnapshot`, `MTMVTask` refresh planning |
**The problem, and what it cost**
| Behavior | branch-4.x tests | Master before this PR |
|---|---|---|
| SQL Server `bit_value IN (TRUE, FALSE)` → `[bit_value] IN (1, 0)` |
`JdbcScanNodeTest` | not tested; the IN-list path has no unit test (only `=
FALSE` is covered) |
| SQL Server `= TRUE` → `= 1` | `JdbcScanNodeTest` | not tested |
| `FOR TIME AS OF '... .SSS'` picks the snapshot committed at that
millisecond | `IcebergUtilsTest` | only the parser is tested; nothing checks
that the fraction reaches snapshot selection |
| `enable_file_scanner_v2` defaults to true | `SessionVariablesTest` | only
the fuzzy-mode override is tested |
| An emptied partition snapshot map reads as "no baseline" | `MTMVTest` |
only the non-empty case is tested |
| A new base partition makes REFRESH AUTO refresh only that MV partition |
`test_base_table_add_partition_mtmv` | no suite; the existing ones only check
that the partition exists, refresh it by name, or go through IVM |
A regression in any of these would reach master's CI unnoticed.
**How this PR fixes it**
- `JdbcQueryBuilderTest`: `testSqlserverBooleanTrueLiteral` and
`testSqlserverBooleanInList`, written against `JdbcQueryBuilder` with SPI
expressions (`ConnectorComparison`, `ConnectorIn`). These replace the deleted
`JdbcScanNodeTest`.
-
`IcebergConnectorMetadataMvccTest.resolveTimestampDatetimeStringKeepsFractionalSeconds`.
The 4.x test mocked two snapshots in the same second, but this module has no
Mockito and its fixture is a real `InMemoryCatalog`. So the new test uses the
fixture's real S2 commit time instead:
- S2's exact millisecond, written with 3 or 6 fractional digits, must
resolve to S2;
- one millisecond earlier must not.
A parser that rejects the fraction throws, and one that truncates it lands
before S2. The comment of the existing whole-second test still described a
truncating parser and is corrected.
- `SessionVariablesTest`: assert the constructor default before the
fuzzy-mode case.
- `MTMVTest`: after the incomplete-map case, an emptied map must make
`hasRefreshSnapshot()` false.
-
`regression-test/suites/mtmv_p0/test_base_table_add_partition_mtmv.groovy`:
ported from #68250, with the table names hard-coded and the tables dropped
before use rather than after. The `.out` was generated on master.
**Results**
| Test | Result |
|---|---|
| `JdbcQueryBuilderTest` | 34 run, 0 failures |
| `IcebergConnectorMetadataMvccTest` | 32 run, 0 failures |
| `MTMVTest` | 49 run, 0 failures |
| `SessionVariablesTest` | 19 run, 0 failures |
| `mtmv_p0/test_base_table_add_partition_mtmv` | passes; the generated
`.out` is byte-identical to branch-4.x's (`["p_3"]`, `PARTIAL`, then
`NOT_REFRESH`) |
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
FE UT were run with `run-fe-ut.sh` (counts above). The regression suite
was run with `run-regression-test.sh` on a local single-FE / single-BE cluster
built from this branch, once with `-genOut` and once against the generated
`.out`. Checkstyle reports 0 violations in fe-core, fe-connector-jdbc and
fe-connector-iceberg.
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
