englefly opened a new pull request, #68214: URL: https://github.com/apache/doris/pull/68214
Cherry-pick of #62205, #62315, #62631, #62304, #63229, #63736, #64486 to branch-4.2 (the meta path series, applied in merge order). Adaptations for branch-4.2: * `AccessPathInfo` already lives in `org.apache.doris.analysis` on this branch; the new `ACCESS_STRING_OFFSET` constant was ported there (aliasing the existing `ACCESS_OFFSET`). * master-only types renamed to this branch's thrift types (`ColumnAccessPath* -> TColumnAccessPath`, `ColumnAccessPathType -> TAccessPathType`); `NestedColumnPruning` keeps branch-4.2's `comparePathSegments`/variant branch alongside upstream's new logic. * `LazyMaterializeTopN` merges upstream's restructure while preserving branch-4.2's Lance lazy-materialization gate; `LogicalJoin.getRightConditionSlot()` (a master-only helper) had to be added for the ported `PushDownTopNThroughJoin`. * `StringEmptyToLengthRuleTest` uses `ElementAt` (branch-4.2 folded `StructElement` into `ElementAt`); the regression suite expectation for the same reason. REQUIRED EXTRA COMMIT: the FE now emits offset-only / null-only nested access paths, which branch-4.2's BE could not read — `null_column_pruning.groovy` aborted the BE with `std::vector<...ColumnIterator>::operator[]: Assertion '__n < this->size()' failed`. This PR therefore also cherry-picks the BE-side prerequisite #61888 "[Exec](be) Support offset prue column and null column in BE" (adapted to branch-4.2's `IColumn::mutate`/Defer COW shape and `ColumnNullable::get_null_map_column_ptr()` signature). Please review that extra commit carefully. Testing on branch-4.2 (FE + BE rebuilt from this branch): * FE unit tests: PruneNestedColumnTest (57), PullUpProjectExprUnderTopNTest (28), OperativeColumnDeriveTest (4), StringEmptyToLengthRuleTest (11), MaterializeProbeVisitorTest (8) — 108 tests, 0 failures. * Regression: nereids_rules_p0/column_pruning (7 suites incl. the new string_length/null/nested_container_offset/topn_expr_pullup/topn_lazy_nested suites) 0 failures; shape_check clickbench/tpcds_sf100/tpcds_sf1000/tpcds_sf10t_orc/tpcds_sf1000_constraints suites touched by the series regenerated with -forceGenOut and green; nereids_rules_p0 push_filter_through / null_un_safe_equals / lazy_materialize_topn green. * Known deviations: #62446 (struct field slot type for OFFSET-only access) is not part of the requested series and was not included; the BE-side `MapAccessAllWithOffsetDoesNotPropagateOffsetToKey` unit test from #61888 was kept byte-identical to upstream although it may not hold (test binary not built with MAKE_TEST=ON). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
