[
https://issues.apache.org/jira/browse/CALCITE-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Frank Chen updated CALCITE-7809:
--------------------------------
Description:
RexShuttle and SqlShuttle currently copy operand collections before knowing
whether traversal will replace an operand. Read-only traversals therefore
allocate and discard unnecessary copies.
Make call-operand traversal copy-on-write: reuse unchanged immutable Rex lists
and SQL calls, and allocate only after an operand changes or when
SqlShuttle.alwaysCopy requires a new call.
Scope is limited to RexShuttle.visitList and SqlShuttle call operands. Plain IN
literals inside SqlNodeList and other eager-copy visitor paths are unchanged.
Verification:
- Focused tests cover unchanged inputs, first/middle/last replacements, mutable
and partial-view Rex lists, changed SQL operands, and alwaysCopy.
- With 100,000 VARCHAR ARRAY operands, the unchanged Calcite traversal drops
from 400,016 B/op to 0.029 B/op. The changed-operand path remains effectively
unchanged.
- The end-to-end VARCHAR = SOME(ARRAY[...]) benchmark reduces allocation by
6.27 MB/op (1.28%).
- The Druid benchmark reduces allocation by 2.72% at 100,000 string literals
and 1.55% at 1,000,000 literals.
Timing confidence intervals overlap, so no latency improvement is claimed.
Pull request: https://github.com/apache/calcite/pull/5283
was:
RexShuttle and SqlShuttle visit every child and return either the original
child or a replacement. They currently copy operand collections before knowing
whether any child changes, then discard those copies when the traversal is
read-only.
Make these operand collections copy-on-write. Allocate when the first changed
child is encountered, or during result construction when SqlShuttle.alwaysCopy
requires a new call, while preserving RexShuttle’s immutable-result contract.
The isolated Druid benchmark reduced allocation by 2.72% at 100,000 string
literals and 1.55% at 1,000,000 string literals.
Pull request: [PR #5283|https://github.com/apache/calcite/pull/5283]
> Avoid copying unchanged SQL and Rex operands
> --------------------------------------------
>
> Key: CALCITE-7809
> URL: https://issues.apache.org/jira/browse/CALCITE-7809
> Project: Calcite
> Issue Type: Sub-task
> Reporter: Frank Chen
> Priority: Major
> Labels: pull-request-available
>
> RexShuttle and SqlShuttle currently copy operand collections before knowing
> whether traversal will replace an operand. Read-only traversals therefore
> allocate and discard unnecessary copies.
> Make call-operand traversal copy-on-write: reuse unchanged immutable Rex
> lists and SQL calls, and allocate only after an operand changes or when
> SqlShuttle.alwaysCopy requires a new call.
> Scope is limited to RexShuttle.visitList and SqlShuttle call operands. Plain
> IN literals inside SqlNodeList and other eager-copy visitor paths are
> unchanged.
> Verification:
> - Focused tests cover unchanged inputs, first/middle/last replacements,
> mutable and partial-view Rex lists, changed SQL operands, and alwaysCopy.
> - With 100,000 VARCHAR ARRAY operands, the unchanged Calcite traversal drops
> from 400,016 B/op to 0.029 B/op. The changed-operand path remains effectively
> unchanged.
> - The end-to-end VARCHAR = SOME(ARRAY[...]) benchmark reduces allocation by
> 6.27 MB/op (1.28%).
> - The Druid benchmark reduces allocation by 2.72% at 100,000 string literals
> and 1.55% at 1,000,000 literals.
> Timing confidence intervals overlap, so no latency improvement is claimed.
> Pull request: https://github.com/apache/calcite/pull/5283
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
