[
https://issues.apache.org/jira/browse/CALCITE-4578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiatao Tao updated CALCITE-4578:
--------------------------------
Description:
Example like that: insert xxx VALUES (1, 'one'), (2, 'two'), (3,
'three')...(10000, 'ten thousand');
It generates a Union node with 10000 inputs, and the performance is poor,
here's the profile:
!image-2021-04-11-17-42-38-595.png|width=1086,height=240!
We create a program(HepProgram) to do the union merge, in this case, we only
have one union, so the rule is not applied, but it will still build the
buildFinalPlan:
{code:java}
public RelNode findBestExp() {
assert root != null;
executeProgram(mainProgram);
// Get rid of everything except what's in the final plan.
collectGarbage();
return buildFinalPlan(root);
}
{code}
And the union have large inputs, so the performance is poor(in
SetOp#replaceInput, we have to recomputeDigest ), we can skip recomputeDigest
if the inputs is not changed:
{code:java}
@Override public void replaceInput(int ordinalInParent, RelNode p) {
final List<RelNode> newInputs = new ArrayList<>(inputs);
newInputs.set(ordinalInParent, p);
if (!newInputs.equals(inputs)) {
inputs = ImmutableList.copyOf(newInputs);
recomputeDigest();
}
}
{code}
was:
Example like that: insert xxx VALUES (1, 'one'), (2, 'two'), (3,
'three')...(10000, 'ten thousand');
It generates a Union node with 10000 inputs, and the performance is poor,
here's the profile:
!image-2021-04-11-17-42-38-595.png|width=1086,height=240!
We create a program(HepProgram) to do the union merge, in this case, we only
have one union, so the rule is not applied, but it will still build the
buildFinalPlan:
{code:java}
public RelNode findBestExp() {
assert root != null;
executeProgram(mainProgram);
// Get rid of everything except what's in the final plan.
collectGarbage();
return buildFinalPlan(root);
}
{code}
And the union have large inputs, so the performance is poor(in ), we can skip
> skip SetOp#replaceInput when inputs are not changed
> ---------------------------------------------------
>
> Key: CALCITE-4578
> URL: https://issues.apache.org/jira/browse/CALCITE-4578
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Jiatao Tao
> Assignee: Jiatao Tao
> Priority: Major
> Attachments: image-2021-04-11-17-42-38-595.png
>
>
> Example like that: insert xxx VALUES (1, 'one'), (2, 'two'), (3,
> 'three')...(10000, 'ten thousand');
> It generates a Union node with 10000 inputs, and the performance is poor,
> here's the profile:
> !image-2021-04-11-17-42-38-595.png|width=1086,height=240!
> We create a program(HepProgram) to do the union merge, in this case, we only
> have one union, so the rule is not applied, but it will still build the
> buildFinalPlan:
> {code:java}
> public RelNode findBestExp() {
> assert root != null;
> executeProgram(mainProgram);
> // Get rid of everything except what's in the final plan.
> collectGarbage();
> return buildFinalPlan(root);
> }
> {code}
> And the union have large inputs, so the performance is poor(in
> SetOp#replaceInput, we have to recomputeDigest ), we can skip recomputeDigest
> if the inputs is not changed:
> {code:java}
> @Override public void replaceInput(int ordinalInParent, RelNode p) {
> final List<RelNode> newInputs = new ArrayList<>(inputs);
> newInputs.set(ordinalInParent, p);
> if (!newInputs.equals(inputs)) {
> inputs = ImmutableList.copyOf(newInputs);
> recomputeDigest();
> }
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
