This is an automated email from the ASF dual-hosted git repository.
mrhhsg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 512b9cbde76 [fix](runtime-filter) Skip unnecessary local-only merge
metadata (#70305)
512b9cbde76 is described below
commit 512b9cbde76d19b9f4735c177a728929350501d6
Author: Jerry Hu <[email protected]>
AuthorDate: Sat Nov 21 16:12:34 2026 +0800
[fix](runtime-filter) Skip unnecessary local-only merge metadata (#70305)
### Background
A non-empty global runtime-filter merge controller makes the BE retain
an additional QueryContext reference. Its destruction triggers the final
fragment-profile report. Local-only filters are registered from plan
nodes and do not send global merge RPCs.
### What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Bugfix
Problem Summary: A successful query using only local runtime filters can
retain its final profiles until timeout because FE still populates
unused global merge metadata. This change omits that metadata for
eligible non-recursive queries. Remote-filter completion and reclamation
are intentionally outside this revision's scope.
### Details
- **Change map:** One FE production file (+34/-3 lines), one FE UT file,
and two SQL suites with their generated expected outputs. There are no
aggregate BE or Thrift changes.
- **Frequency:** Local-only runtime filters can establish the extra
merge owner even though they have no global merge work to perform.
- **Root cause:** FE populated merge descriptors for local filters,
making the controller non-empty and activating delayed QueryContext
ownership.
- **Why existing tests missed it:** They did not check the absence of
unnecessary merge metadata on the wire or timely final-profile
collection for an explicitly local-only query.
- **Fix:** Keep the eligibility decision in FE. Omit merge maps only
when the full query has no remote-target RF, no recursive CTE node/scan,
and no distinct workers sharing a BRPC endpoint. Keep plan-node RF
descriptors, the merge address, and TopN metadata unchanged. The
existing BE empty-controller path then avoids creating the extra delayed
owner.
- **Other paths and branches:** Remote-only, mixed local/remote,
recursive, incomplete, and duplicate-endpoint plans keep the original
metadata and lifetime. No new Thrift field, callback state, background
sweep, or BE map behavior is introduced. No release branch is changed.
### Testing
- On commit `44a950a9e60b4a5221fafda8d1e47f762c24330d`, the following
focused FE UT command passed, including compilation of the required
FE/reactor modules:
```bash
./run-fe-ut.sh --run
org.apache.doris.qe.runtime.ThriftPlansBuilderTest,org.apache.doris.qe.runtime.RuntimeFiltersThriftBuilderTest
```
Result: 12 tests passed (`ThriftPlansBuilderTest`: 8,
`RuntimeFiltersThriftBuilderTest`: 4), with 0 failures, 0 errors, and 0
skipped tests.
- Binary Thrift round-trip coverage checks local-only/no-RF omission;
remote/mixed, recursive, incomplete, and duplicate-endpoint fallback;
and preservation of the merge address and TopN metadata.
- Full and focused FE Checkstyle passed with zero violations. `git diff
--check` passed. The restored C++ files passed
`build-support/check-format.sh` and
`build-support/check-build-hygiene.sh`.
- Both SQL suites passed Groovy syntax parsing only. SQL/Profile
regression execution, controlled delayed duplicate-start, multi-BE, and
mixed-version execution were not run for this revision.
- No standalone full FE/BE build or BE UT was run for this FE-only
revision. The earlier BE early-release implementation and its validation
claims are not part of this revision.
### Notes for Reviewers
This revision replaces the previous BE early-release approach with a
narrow FE-only fix. In particular, it does not attempt to release
completed remote RF contexts. The local-only profile suite forces LOCAL
mode and a broadcast join; the remote query checks result correctness
without requiring early profile completion. The existing generated
expected-output files were not hand-edited.
### Release note
Eligible non-recursive queries using only local runtime filters can
report final profiles without waiting for query timeout.
### Check List (For Author)
- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: focused FE UT and static
checks passed; SQL/Profile, multi-BE, and mixed-version execution remain
unverified.
### AI Disclosure
- Tool: OpenAI Codex
- Generated: Code, tests, and this description.
- Human review: No human source review of this revision has been
performed.
---
.../doris/qe/runtime/ThriftPlansBuilder.java | 37 +++++-
.../doris/qe/runtime/ThriftPlansBuilderTest.java | 138 +++++++++++++++++++++
.../query_profile/rf_merge_context_profile.out | 8 ++
.../data/recursive_cte/rf_merge_context_reuse.out | 6 +
.../query_profile/rf_merge_context_profile.groovy | 57 +++++++++
.../recursive_cte/rf_merge_context_reuse.groovy | 42 +++++++
6 files changed, 285 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/qe/runtime/ThriftPlansBuilder.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/runtime/ThriftPlansBuilder.java
index e36b0978827..acdf1ff6d99 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/qe/runtime/ThriftPlansBuilder.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/qe/runtime/ThriftPlansBuilder.java
@@ -110,6 +110,7 @@ public class ThriftPlansBuilder {
List<PipelineDistributedPlan> distributedPlans =
coordinatorContext.distributedPlans;
Set<Integer> fragmentToNotifyClose =
setParamsForRecursiveCteNode(distributedPlans,
coordinatorContext.runtimeFilters);
+ boolean omitRfMergeParams = canOmitRfMergeParams(distributedPlans,
coordinatorContext.runtimeFilters);
// Determine whether this query is assigned to a single backend and
propagate it to
// TQueryOptions so that BE can apply more appropriate optimization
strategies (e.g.
@@ -157,7 +158,7 @@ public class ThriftPlansBuilder {
for (Entry<DistributedPlanWorker, TPipelineFragmentParams> kv :
workerToCurrentFragment.entrySet()) {
TPipelineFragmentParamsList fragments =
fragmentsGroupByWorker.computeIfAbsent(
kv.getKey(), w -> beToThrift(kv.getKey(),
runtimeFiltersThriftBuilder,
- topNFilterThriftSupplier));
+ topNFilterThriftSupplier, omitRfMergeParams));
fragments.addToParamsList(kv.getValue());
}
}
@@ -406,17 +407,47 @@ public class ThriftPlansBuilder {
return destination;
}
+ static boolean canOmitRfMergeParams(
+ List<PipelineDistributedPlan> distributedPlans,
List<RuntimeFilter> runtimeFilters) {
+ // Local RFs are registered from plan nodes, not the global merge
metadata. Omitting this
+ // metadata avoids retaining a QueryContext solely for local RFs that
never send merge RPCs.
+ if (distributedPlans.isEmpty() ||
runtimeFilters.stream().anyMatch(RuntimeFilter::hasRemoteTargets)) {
+ return false;
+ }
+ Map<String, DistributedPlanWorker> endpointOwners = Maps.newHashMap();
+ for (PipelineDistributedPlan plan : distributedPlans) {
+ for (AssignedJob job : plan.getInstanceJobs()) {
+ DistributedPlanWorker worker = job.getAssignedWorker();
+ DistributedPlanWorker owner =
endpointOwners.putIfAbsent(worker.brpcAddress(), worker);
+ // Different catalog workers can share a BE but receive
separate query-wide start RPCs.
+ // The first start may finish the query before the second
start reaches that BE.
+ if (owner != null && !owner.equals(worker)) {
+ return false;
+ }
+ }
+ PlanNode root = plan.getFragmentJob().getFragment().getPlanRoot();
+ // Even an empty recursive reset RPC looks up the merge BE's
QueryContext. Preserve
+ // the original metadata and its ownership for recursive queries
on any BE.
+ if (root == null || !root.collectInCurrentFragment(
+ node -> node instanceof RecursiveCteNode || node
instanceof RecursiveCteScanNode).isEmpty()) {
+ return false;
+ }
+ }
+ return true;
+ }
+
private static TPipelineFragmentParamsList beToThrift(
DistributedPlanWorker worker,
RuntimeFiltersThriftBuilder runtimeFiltersThriftBuilder,
- Supplier<List<TTopnFilterDesc>> topNFilterThriftSupplier) {
+ Supplier<List<TTopnFilterDesc>> topNFilterThriftSupplier,
+ boolean omitRfMergeParams) {
TPipelineFragmentParamsList beParam = new
TPipelineFragmentParamsList();
TRuntimeFilterInfo runtimeFilterInfo = new TRuntimeFilterInfo();
runtimeFilterInfo.setTopnFilterDescs(topNFilterThriftSupplier.get());
TRuntimeFilterParams runtimeFilterParams = new TRuntimeFilterParams();
runtimeFilterParams.setRuntimeFilterMergeAddr(runtimeFiltersThriftBuilder.mergeAddress);
- if
(worker.host().equals(runtimeFiltersThriftBuilder.mergeAddress.getHostname())
+ if (!omitRfMergeParams &&
worker.host().equals(runtimeFiltersThriftBuilder.mergeAddress.getHostname())
&& worker.brpcPort() ==
runtimeFiltersThriftBuilder.mergeAddress.getPort()) {
// only set following information for merge BE node
runtimeFiltersThriftBuilder.populateRuntimeFilterParams(runtimeFilterParams);
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/qe/runtime/ThriftPlansBuilderTest.java
b/fe/fe-core/src/test/java/org/apache/doris/qe/runtime/ThriftPlansBuilderTest.java
index c7706d9e19f..4f93e891f59 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/qe/runtime/ThriftPlansBuilderTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/qe/runtime/ThriftPlansBuilderTest.java
@@ -19,30 +19,47 @@ package org.apache.doris.qe.runtime;
import org.apache.doris.analysis.TupleDescriptor;
import org.apache.doris.analysis.TupleId;
+import org.apache.doris.common.IdGenerator;
import org.apache.doris.common.jmockit.Deencapsulation;
import org.apache.doris.datasource.scan.FileQueryScanNode;
+import org.apache.doris.nereids.trees.plans.distribute.PipelineDistributedPlan;
+import org.apache.doris.nereids.trees.plans.distribute.worker.BackendWorker;
import
org.apache.doris.nereids.trees.plans.distribute.worker.DistributedPlanWorker;
import org.apache.doris.nereids.trees.plans.distribute.worker.job.AssignedJob;
import
org.apache.doris.nereids.trees.plans.distribute.worker.job.DefaultScanSource;
import
org.apache.doris.nereids.trees.plans.distribute.worker.job.LocalShuffleAssignedJob;
import org.apache.doris.nereids.trees.plans.distribute.worker.job.ScanRanges;
import
org.apache.doris.nereids.trees.plans.distribute.worker.job.UnassignedJob;
+import org.apache.doris.planner.PlanFragmentId;
+import org.apache.doris.planner.PlanNode;
import org.apache.doris.planner.PlanNodeId;
+import org.apache.doris.planner.RecursiveCteNode;
import org.apache.doris.planner.RecursiveCteScanNode;
+import org.apache.doris.planner.RuntimeFilter;
+import org.apache.doris.planner.RuntimeFilterId;
import org.apache.doris.planner.ScanNode;
import org.apache.doris.planner.SortNode;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
import org.apache.doris.thrift.TExternalScanRange;
import org.apache.doris.thrift.TFileRangeDesc;
import org.apache.doris.thrift.TFileScanRange;
import org.apache.doris.thrift.TFileScanRangeParams;
import org.apache.doris.thrift.TPipelineFragmentParams;
+import org.apache.doris.thrift.TPipelineFragmentParamsList;
import org.apache.doris.thrift.TPipelineInstanceParams;
import org.apache.doris.thrift.TQueryOptions;
import org.apache.doris.thrift.TRecCTETarget;
+import org.apache.doris.thrift.TRuntimeFilterDesc;
+import org.apache.doris.thrift.TRuntimeFilterParams;
+import org.apache.doris.thrift.TRuntimeFilterType;
import org.apache.doris.thrift.TScanRange;
import org.apache.doris.thrift.TScanRangeParams;
+import org.apache.doris.thrift.TTopnFilterDesc;
import org.apache.doris.thrift.TUniqueId;
+import org.apache.thrift.TDeserializer;
+import org.apache.thrift.TSerializer;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
@@ -52,8 +69,129 @@ import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
+import java.util.Set;
+import java.util.function.Predicate;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
public class ThriftPlansBuilderTest {
+ private final IdGenerator<RuntimeFilterId> runtimeFilterIds =
RuntimeFilterId.createGenerator();
+
+ @Test
+ public void testLocalOnlyRfsOmitMergeParamsOnWire() throws Exception {
+ BackendWorker worker = newBackendWorker(0, 1, "127.0.0.1");
+ PipelineDistributedPlan plan =
mockPlanWithRoot(Mockito.mock(PlanNode.class), worker);
+ assertRfMergeParams(List.of(plan), List.of(newRuntimeFilter(false)),
true);
+ assertRfMergeParams(List.of(plan), Collections.emptyList(), true);
+ }
+
+ @Test
+ public void testRemoteAndMixedRfsKeepMergeParamsOnWire() throws Exception {
+ BackendWorker worker = newBackendWorker(0, 1, "127.0.0.1");
+ PipelineDistributedPlan plan =
mockPlanWithRoot(Mockito.mock(PlanNode.class), worker);
+ RuntimeFilter remote = newRuntimeFilter(true);
+ assertRfMergeParams(List.of(plan), List.of(remote), false);
+ assertRfMergeParams(List.of(plan), List.of(newRuntimeFilter(false),
remote), false);
+ }
+
+ @Test
+ public void testRecursiveAndIncompletePlansKeepMergeParamsOnWire() throws
Exception {
+ BackendWorker worker = newBackendWorker(0, 1, "127.0.0.1");
+ List<RuntimeFilter> filters = List.of(newRuntimeFilter(false));
+
Assertions.assertFalse(ThriftPlansBuilder.canOmitRfMergeParams(Collections.emptyList(),
filters));
+ PipelineDistributedPlan mergePlan =
mockPlanWithRoot(Mockito.mock(PlanNode.class), worker);
+ // A recursive node on another fragment can still send an empty
global-reset RPC.
+ for (PlanNode root : List.of(Mockito.mock(RecursiveCteNode.class),
Mockito.mock(RecursiveCteScanNode.class))) {
+ PipelineDistributedPlan recursivePlan = mockPlanWithRoot(root,
worker);
+ assertRfMergeParams(List.of(mergePlan, recursivePlan), filters,
false);
+ }
+
Mockito.when(mergePlan.getFragmentJob().getFragment().getPlanRoot()).thenReturn(null);
+ assertRfMergeParams(List.of(mergePlan), filters, false);
+ }
+
+ private PipelineDistributedPlan mockPlanWithRoot(PlanNode root,
BackendWorker worker) {
+ PipelineDistributedPlan plan =
Mockito.mock(PipelineDistributedPlan.class, Mockito.RETURNS_DEEP_STUBS);
+
Mockito.when(plan.getFragmentJob().getFragment().getPlanRoot()).thenReturn(root);
+ AssignedJob job = newAssignedJob(worker, new TUniqueId());
+ Mockito.when(plan.getInstanceJobs()).thenReturn(List.of(job));
+
Mockito.when(root.collectInCurrentFragment(Mockito.any())).thenAnswer(invocation
-> {
+ Predicate<PlanNode> predicate = invocation.getArgument(0);
+ return predicate.test(root) ? List.of(root) :
Collections.emptyList();
+ });
+ return plan;
+ }
+
+ @Test
+ public void testDistinctWorkersAtSameEndpointKeepMergeParamsOnWire()
throws Exception {
+ BackendWorker internalWorker = newBackendWorker(0, 1, "127.0.0.1");
+ BackendWorker catalogWorker = newBackendWorker(1, 1, "127.0.0.1");
+ BackendWorker remoteWorker = newBackendWorker(0, 2, "127.0.0.2");
+ AssignedJob catalogJob = newAssignedJob(catalogWorker, new
TUniqueId(2, 2));
+ AssignedJob sameWorkerJob = newAssignedJob(internalWorker, new
TUniqueId(2, 2));
+ AssignedJob remoteJob = newAssignedJob(remoteWorker, new TUniqueId(2,
2));
+
+ PipelineDistributedPlan firstPlan =
mockPlanWithRoot(Mockito.mock(PlanNode.class), internalWorker);
+ PipelineDistributedPlan secondPlan =
mockPlanWithRoot(Mockito.mock(PlanNode.class), catalogWorker);
+
Mockito.when(secondPlan.getInstanceJobs()).thenReturn(List.of(catalogJob));
+ Assertions.assertEquals(internalWorker.brpcAddress(),
catalogWorker.brpcAddress());
+ Assertions.assertNotEquals(internalWorker, catalogWorker);
+ List<RuntimeFilter> filters = List.of(newRuntimeFilter(false));
+ assertRfMergeParams(List.of(firstPlan, secondPlan), filters, false);
+
+
Mockito.when(secondPlan.getInstanceJobs()).thenReturn(List.of(sameWorkerJob));
+ assertRfMergeParams(List.of(firstPlan, secondPlan), filters, true);
+
+
Mockito.when(secondPlan.getInstanceJobs()).thenReturn(List.of(remoteJob));
+ assertRfMergeParams(List.of(firstPlan, secondPlan), filters, true);
+ }
+
+ private BackendWorker newBackendWorker(long catalogId, long backendId,
String host) {
+ Backend backend = new Backend(backendId, host, 9050);
+ backend.setBrpcPort(8060);
+ return new BackendWorker(catalogId, backend);
+ }
+
+ private RuntimeFilter newRuntimeFilter(boolean remote) {
+ RuntimeFilterId id = runtimeFilterIds.getNextId();
+ TRuntimeFilterDesc desc = new TRuntimeFilterDesc()
+ .setFilterId(id.asInt()).setSrcExpr(new
TExpr(Collections.emptyList())).setExprOrder(0)
+
.setPlanIdToTargetExpr(Collections.emptyMap()).setIsBroadcastJoin(false)
+
.setHasLocalTargets(!remote).setHasRemoteTargets(remote).setType(TRuntimeFilterType.BLOOM);
+ RuntimeFilter filter = Mockito.mock(RuntimeFilter.class);
+ Mockito.when(filter.getFilterId()).thenReturn(id);
+ Mockito.when(filter.hasRemoteTargets()).thenReturn(remote);
+ Mockito.when(filter.toThrift()).thenReturn(desc);
+ return filter;
+ }
+
+ private void assertRfMergeParams(List<PipelineDistributedPlan> plans,
List<RuntimeFilter> filters,
+ boolean omitted) throws Exception {
+ Set<RuntimeFilterId> ids =
filters.stream().map(RuntimeFilter::getFilterId).collect(Collectors.toSet());
+ for (int i = 0; i < plans.size(); i++) {
+
Mockito.when(plans.get(i).getFragmentJob().getFragment().getFragmentId()).thenReturn(new
PlanFragmentId(i));
+
Mockito.when(plans.get(i).getFragmentJob().getFragment().getBuilderRuntimeFilterIds()).thenReturn(ids);
+
Mockito.when(plans.get(i).getFragmentJob().getFragment().getTargetRuntimeFilterIds()).thenReturn(ids);
+ }
+ RuntimeFiltersThriftBuilder builder =
RuntimeFiltersThriftBuilder.compute(filters, plans);
+ DistributedPlanWorker worker = plans.get(plans.size() -
1).getInstanceJobs().get(0).getAssignedWorker();
+ boolean omitRfMergeParams =
ThriftPlansBuilder.canOmitRfMergeParams(plans, filters);
+ Assertions.assertEquals(omitted, omitRfMergeParams);
+ TTopnFilterDesc topn = new TTopnFilterDesc(9, true, false,
Collections.emptyMap());
+ Supplier<List<TTopnFilterDesc>> topnFilters = () -> List.of(topn);
+ TPipelineFragmentParamsList params = Deencapsulation.invoke(
+ ThriftPlansBuilder.class, "beToThrift", worker, builder,
topnFilters, omitRfMergeParams);
+ TPipelineFragmentParamsList decoded = new
TPipelineFragmentParamsList();
+ new TDeserializer().deserialize(decoded, new
TSerializer().serialize(params));
+
+ TRuntimeFilterParams expected = new
TRuntimeFilterParams().setRuntimeFilterMergeAddr(builder.mergeAddress);
+ if (!omitted) {
+ // Compare all merge metadata with the original builder, including
mixed local/remote RFs.
+ builder.populateRuntimeFilterParams(expected);
+ }
+ Assertions.assertEquals(expected,
decoded.getRuntimeFilterInfo().getRuntimeFilterParams());
+ Assertions.assertEquals(List.of(topn),
decoded.getRuntimeFilterInfo().getTopnFilterDescs());
+ }
+
@Test
public void testSetRuntimePredicateForNonOlapScanNode() {
ScanNode scanNode = Mockito.mock(ScanNode.class);
diff --git a/regression-test/data/query_profile/rf_merge_context_profile.out
b/regression-test/data/query_profile/rf_merge_context_profile.out
new file mode 100644
index 00000000000..36ab0fcde89
--- /dev/null
+++ b/regression-test/data/query_profile/rf_merge_context_profile.out
@@ -0,0 +1,8 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !join --
+2 20
+4 40
+
+-- !remote --
+2 20
+4 40
diff --git a/regression-test/data/recursive_cte/rf_merge_context_reuse.out
b/regression-test/data/recursive_cte/rf_merge_context_reuse.out
new file mode 100644
index 00000000000..1d2d81f8316
--- /dev/null
+++ b/regression-test/data/recursive_cte/rf_merge_context_reuse.out
@@ -0,0 +1,6 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !recursive --
+1
+2
+3
+4
diff --git
a/regression-test/suites/query_profile/rf_merge_context_profile.groovy
b/regression-test/suites/query_profile/rf_merge_context_profile.groovy
new file mode 100644
index 00000000000..f3811e17169
--- /dev/null
+++ b/regression-test/suites/query_profile/rf_merge_context_profile.groovy
@@ -0,0 +1,57 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import org.apache.doris.regression.action.ProfileAction
+
+suite('rf_merge_context_profile', 'nonConcurrent') {
+ sql 'drop table if exists rf_merge_profile_probe'
+ sql 'drop table if exists rf_merge_profile_build'
+ sql '''create table rf_merge_profile_probe (k int, v int) duplicate key(k)
+ distributed by hash(k) buckets 4 properties ("replication_num" =
"1")'''
+ sql '''create table rf_merge_profile_build (k int) duplicate key(k)
+ distributed by hash(k) buckets 4 properties ("replication_num" =
"1")'''
+ sql 'insert into rf_merge_profile_probe values (1, 10), (2, 20), (3, 30),
(4, 40)'
+ sql 'insert into rf_merge_profile_build values (2), (4)'
+
+ sql 'set enable_profile=true'
+ sql "set runtime_filter_mode='LOCAL'"
+ String localPlan = sql('''explain select p.k, p.v from
rf_merge_profile_probe p
+ join [broadcast] rf_merge_profile_build b on p.k
= b.k order by p.k''').toString()
+ assertTrue(localPlan.contains('runtime filters: RF'))
+ order_qt_join '''select p.k, p.v from rf_merge_profile_probe p
+ join [broadcast] rf_merge_profile_build b on p.k = b.k
order by p.k'''
+
+ // The profile check is tied to this query text, not to an unrelated
latest profile.
+ // Ten seconds is the FE's normal completion window; the helper's
30-second default
+ // would hide a delayed final report.
+ String token = UUID.randomUUID().toString()
+ sql """select '${token}' as profile_marker, p.k, p.v from
rf_merge_profile_probe p
+ join [broadcast] rf_merge_profile_build b on p.k = b.k
+ where p.k > 0 order by p.k"""
+ new ProfileAction(context).getProfileBySql(token, ['Fragment'], 10000, 200)
+
+ // The shuffle hint puts the probe scan across an Exchange from the join,
so
+ // these RFs exercise remote merge/publish even on a one-BE test cluster.
Only query
+ // correctness is checked: remote RF contexts keep their original delayed
lifetime.
+ sql "set runtime_filter_mode='GLOBAL'"
+ String remotePlan = sql('''explain select p.k, p.v from
rf_merge_profile_probe p
+ join [shuffle] rf_merge_profile_build b on p.k
= b.k order by p.k''').toString()
+ assertTrue(remotePlan.contains('INNER JOIN('), remotePlan)
+ assertTrue(remotePlan.contains('VEXCHANGE'), remotePlan)
+ assertTrue(remotePlan.contains('runtime filters: RF'), remotePlan)
+ order_qt_remote '''select p.k, p.v from rf_merge_profile_probe p
+ join [shuffle] rf_merge_profile_build b on p.k = b.k
order by p.k'''
+}
diff --git a/regression-test/suites/recursive_cte/rf_merge_context_reuse.groovy
b/regression-test/suites/recursive_cte/rf_merge_context_reuse.groovy
new file mode 100644
index 00000000000..05fec4b26a4
--- /dev/null
+++ b/regression-test/suites/recursive_cte/rf_merge_context_reuse.groovy
@@ -0,0 +1,42 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+suite('rf_merge_context_reuse') {
+ sql 'drop table if exists rf_merge_reuse_edges'
+ sql '''create table rf_merge_reuse_edges (src int, dst int) duplicate
key(src)
+ distributed by hash(src) buckets 4 properties ("replication_num" =
"1")'''
+ sql 'insert into rf_merge_reuse_edges values (1, 2), (2, 3), (3, 4)'
+
+ String plan = sql('''explain with recursive numbers(n) as (
+ select cast(1 as int)
+ union all
+ select cast(e.dst as int) from
rf_merge_reuse_edges e
+ inner join [shuffle] numbers r on e.src = r.n
where r.n < 4
+ ) select n from numbers order by n''').toString()
+ // The recursive join must publish RFs across an Exchange on every round;
+ // a constants-only recursive query never exercises the merge-context
reset.
+ assertTrue(plan.contains('INNER JOIN('), plan)
+ assertTrue(plan.contains('runtime filters: RF'), plan)
+ assertTrue(plan.contains('-> src'), plan)
+ assertTrue(plan.contains('VEXCHANGE'), plan)
+
+ order_qt_recursive '''with recursive numbers(n) as (
+ select cast(1 as int)
+ union all
+ select cast(e.dst as int) from
rf_merge_reuse_edges e
+ inner join [shuffle] numbers r on e.src = r.n
where r.n < 4
+ ) select n from numbers order by n'''
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]