This is an automated email from the ASF dual-hosted git repository.
924060929 pushed a change to branch fe_local_shuffle
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 6737acd386e [opt](local shuffle) support bucket shuffle for set
operation
add c23c9914ad4 [fix](fe) set cloud version_cache_ttl to 0 temporarily if
retry a query with -230 (#63721)
add 9d7d3a2c4f4 [fix](be) Avoid local runtime filter merge deadlock
(#64866)
add 03cb15cb8ba [fix](agg) Fix incorrect aggregate merge with duplicate
aliases (#65025)
add 7718451c8e4 [enhancement](limitreach) FE receives many limit reach
error messages and make very confused (#62842)
add 98b4a21b421 [improvement](regression) Align regression framework
arrow/netty with fe-core and target JDK17 (#64677)
add 32a2651f66b [fix](iceberg) Fix NPE in COUNT(*) pushdown when snapshot
summary omits total-* counters (#64648)
add aab2f22f071 [fix](binlog compaction): cumu policy may be null (#65032)
add c48cde7877b [fix](connection) Prevent timeout checker from stopping
after an exception (#65040)
add 4d99c7e4d32 [feature](workload) Support remote scan bytes breaker in
workload policy (#64649)
add b731c4a4681 [chore](github)Remove 'Build Broker' task from .asf.yaml
(#65069)
add ecda456d5fd [fix](set operation) Use regular child output for set
operation rules (#64908)
add 023b766f5ce [fix](be) Validate task executor scan handles (#65054)
add 94a791e6644 [fix](auth) add auth check for manager node and query
qerror REST APIs (#65042)
add 7b2c4b91f11 [fix](group commit) fix can not get a block queue (#63722)
add 7936ec5c712 [fix](case) Stabilize max compute schema metadata case
checks. (#64959)
add 9d93a33fa1a [fix](regression) Stabilize rowsets timestamp filters
(#65050)
add 9502e5757cd [fix](regression) stabilize multi leading scalar predicate
(#65045)
add c8dc6f6c0a9 [fix](regression) update audit log behavior output (#64986)
add fd142ddeec3 [fix](regression) stabilize eliminate_inner aggregate
phase (#64957)
add 9b26234bf1c [fix](column) avoid mutable nullable crc32c hashing
(#64944)
add bec8d9f0ea1 [fix](regression) stabilize colocate column order plan
test (#64940)
add ed823b6d8eb [test](docker) improve the prefix path of cloud mode
docker env (#65001)
add 6ee32534acd [fix](eager-agg) Fix NPE when pushing MAX below both sides
of join (#65005)
add 57672d180d2 [fix](be) Continue sorted merge when sender queue is ready
(#65004)
add b5c0bcd3434 [fix](simplify agg) SimplifyAggGroupBy should verify
injectivity (#64335)
new 935959445b6 [opt](local shuffle) support bucket shuffle for set
operation
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6737acd386e)
\
N -- N -- N refs/heads/fe_local_shuffle (935959445b6)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 1 -
be/src/common/config.cpp | 3 +
be/src/common/config.h | 2 +
be/src/common/status.h | 2 +-
be/src/core/column/column.h | 10 +
be/src/core/column/column_decimal.cpp | 10 +
be/src/core/column/column_decimal.h | 3 +
be/src/core/column/column_nullable.cpp | 20 +-
be/src/core/column/column_vector.cpp | 23 +-
be/src/core/column/column_vector.h | 4 +
be/src/exec/exchange/vdata_stream_recvr.cpp | 12 +-
be/src/exec/exchange/vdata_stream_recvr.h | 2 +
.../operator/group_commit_block_sink_operator.cpp | 37 +--
be/src/exec/pipeline/pipeline_fragment_context.cpp | 30 +-
be/src/exec/pipeline/pipeline_fragment_context.h | 7 +-
be/src/exec/runtime_filter/runtime_filter_merger.h | 27 +-
be/src/exec/runtime_filter/runtime_filter_mgr.cpp | 142 +++++----
be/src/exec/runtime_filter/runtime_filter_mgr.h | 35 ++-
.../runtime_filter/runtime_filter_producer.cpp | 42 +--
be/src/exec/scan/scanner_scheduler.h | 22 +-
.../time_sharing/time_sharing_task_executor.cpp | 28 +-
be/src/exec/sort/sort_cursor.h | 22 +-
be/src/exec/sort/vsorted_run_merger.cpp | 34 ++-
be/src/exec/sort/vsorted_run_merger.h | 8 +-
be/src/format/table/iceberg_reader_mixin.h | 8 +-
be/src/load/group_commit/group_commit_mgr.cpp | 329 +++++++++++++++++----
be/src/load/group_commit/group_commit_mgr.h | 41 ++-
be/src/load/group_commit/wal/wal_table.cpp | 11 +-
be/src/runtime/runtime_state.cpp | 2 +-
.../workload_management/workload_condition.cpp | 15 +-
.../workload_management/workload_condition.h | 33 ++-
.../workload_management/workload_sched_policy.cpp | 6 +
be/src/storage/tablet/tablet.cpp | 3 +-
be/src/storage/tablet/tablet_manager.cpp | 7 +-
be/test/core/column/column_nullable_test.cpp | 222 ++++++++++++++
be/test/core/value/sort_merger_test.cpp | 201 ++++++++++++-
.../time_sharing_task_executor_test.cpp | 57 ++++
.../runtime_filter/runtime_filter_merger_test.cpp | 26 +-
.../runtime_filter/runtime_filter_mgr_test.cpp | 33 ++-
be/test/runtime/workload_sched_policy_test.cpp | 34 ++-
docker/runtime/doris-compose/cluster.py | 3 +
.../doris/datasource/iceberg/IcebergUtils.java | 35 ++-
.../datasource/iceberg/source/IcebergScanNode.java | 19 +-
.../doris/httpv2/rest/manager/NodeAction.java | 6 +
.../httpv2/rest/manager/QueryProfileAction.java | 10 +-
.../nereids/rules/rewrite/MergeAggregate.java | 12 +-
.../rewrite/MergeOneRowRelationIntoUnion.java | 21 +-
.../rewrite/PushDownLimitDistinctThroughUnion.java | 7 +-
.../rewrite/PushDownTopNDistinctThroughUnion.java | 6 +-
.../nereids/rules/rewrite/SimplifyAggGroupBy.java | 57 +++-
.../rewrite/eageraggregation/EagerAggRewriter.java | 3 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 36 ---
.../java/org/apache/doris/qe/ConnectPoolMgr.java | 7 +-
.../java/org/apache/doris/qe/ConnectScheduler.java | 10 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 36 ++-
.../workloadschedpolicy/WorkloadCondition.java | 3 +
...loadConditionBeScanBytesFromRemoteStorage.java} | 19 +-
.../workloadschedpolicy/WorkloadMetricType.java | 3 +-
.../WorkloadSchedPolicyMgr.java | 7 +-
.../iceberg/source/IcebergCountPushDownTest.java | 86 ++++++
.../nereids/rules/rewrite/MergeAggregateTest.java | 28 +-
.../rewrite/SetOperationOutputMappingTest.java | 157 ++++++++++
.../rules/rewrite/SimplifyAggGroupByTest.java | 240 +++++++++++++++
.../org/apache/doris/qe/ConnectSchedulerTest.java | 36 +++
.../java/org/apache/doris/qe/StmtExecutorTest.java | 48 +++
.../WorkloadSchedPolicyMgrTest.java | 46 +++
gensrc/thrift/BackendService.thrift | 4 +-
.../data/audit/test_audit_log_behavior.out | 2 +-
.../maxcompute/test_max_compute_schema.out | 51 +++-
.../merge_aggregate/merge_aggregate.out | 5 +
.../query_p0/eager_agg/bilateral_eager_agg.out | 4 +
.../data/query_p0/hint/multi_leading.out | 2 +-
.../runtime_filter/test_pushdown_setop.out | 22 +-
regression-test/framework/pom.xml | 11 +-
regression-test/pipeline/cloud_p0/run.sh | 5 +-
regression-test/pipeline/common/doris-utils.sh | 8 +-
regression-test/pipeline/vault_p0/run.sh | 5 +-
.../auth_p0/test_http_node_action_auth.groovy | 113 +++++++
.../cloud_p0/query_retry/test_retry_e-230.groovy | 55 ++++
.../test_colocate_join_of_column_order.groovy | 2 +-
.../test_workload_policy_remote_scan_bytes.groovy | 143 +++++++++
.../maxcompute/test_max_compute_schema.groovy | 35 ++-
.../group_commit/test_group_commit_error.groovy | 46 ++-
...ommit_stream_load_high_concurrency_async.groovy | 138 +++++++++
.../merge_aggregate/merge_aggregate.groovy | 30 ++
.../merge_one_row_relation_into_union.groovy} | 21 +-
.../nereids_rules_p0/pkfk/eliminate_inner.groovy | 4 +-
.../push_down_limit_distinct_through_union.groovy} | 50 ++--
.../push_down_top_n_distinct_through_union.groovy | 21 +-
.../query_p0/eager_agg/bilateral_eager_agg.groovy | 59 +++-
.../suites/query_p0/hint/multi_leading.groovy | 2 +-
.../query_p0/system/test_query_sys_rowsets.groovy | 12 +-
.../system/test_query_sys_scan_rowsets.groovy | 7 +-
.../test_nereids_show_create_storage_vault.groovy | 4 +-
run-regression-test.sh | 30 +-
95 files changed, 2889 insertions(+), 497 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/{WorkloadConditionBeScanRows.java
=> WorkloadConditionBeScanBytesFromRemoteStorage.java} (64%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/source/IcebergCountPushDownTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/SetOperationOutputMappingTest.java
create mode 100644
regression-test/suites/auth_p0/test_http_node_action_auth.groovy
create mode 100644
regression-test/suites/external_table_p0/hive/test_workload_policy_remote_scan_bytes.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_group_commit_stream_load_high_concurrency_async.groovy
copy regression-test/suites/{nereids_syntax_p0/empty_relation.groovy =>
nereids_rules_p0/merge_one_row_relation/merge_one_row_relation_into_union.groovy}
(73%)
copy
regression-test/suites/nereids_rules_p0/{filter_push_down/filter_push_through_aggregate.groovy
=> push_down_limit_distinct/push_down_limit_distinct_through_union.groovy}
(54%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
