Vladimir Sitnikov created CALCITE-7879:
------------------------------------------
Summary: SqlShuttle copies every SqlNodeList it visits, even when
no element changes
Key: CALCITE-7879
URL: https://issues.apache.org/jira/browse/CALCITE-7879
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Vladimir Sitnikov
Visiting a {{SqlNodeList}} with a {{SqlShuttle}} that changes no element
allocates a new {{ArrayList}} as large as the list, discards it, and returns
the original list. For a list of 100000 literals that is about 400 KB per
visit. A {{SqlNodeList}} holds, for example, the value list of {{x IN ('1',
..., 'N')}}.
h2. Reproduction
Main at 6ff13eaccd, JDK 21.0.9. The probe visits a list of 100000
{{SqlLiteral}} with a plain {{new SqlShuttle()}}, warms up for 2000 visits,
then averages the allocation of 1000 visits:
{code:java}
com.sun.management.ThreadMXBean mx =
(com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
List<SqlNode> nodes = new ArrayList<>();
for (int i = 0; i < 100000; i++) {
nodes.add(SqlLiteral.createCharString(Integer.toString(i),
SqlParserPos.ZERO));
}
SqlNodeList list = new SqlNodeList(nodes, SqlParserPos.ZERO);
SqlShuttle shuttle = new SqlShuttle();
SqlNode r = null;
for (int i = 0; i < 2000; i++) {
r = list.accept(shuttle);
}
int n = 1000;
long before = mx.getCurrentThreadAllocatedBytes();
for (int i = 0; i < n; i++) {
r = list.accept(shuttle);
}
long after = mx.getCurrentThreadAllocatedBytes();
System.out.println("PROBE size=" + list.size() + " sameInstance=" + (r == list)
+ " bytesPerVisit=" + (after - before) / n
+ " java=" + System.getProperty("java.version"));
{code}
Output:
{noformat}
PROBE size=100000 sameInstance=true bytesPerVisit=400072 java=21.0.9
{noformat}
h2. Expected behavior
A visit that changes no element returns the same {{SqlNodeList}} instance, as
it does today, and allocates nothing proportional to the list size;
{{bytesPerVisit}} in the probe stays near 0. A visit that changes an element
still returns a new list with the other elements kept as the same instances.
CALCITE-7809 (apache/calcite#5283) gives {{SqlShuttle}} call operands this
behavior: a call with no changed operand is returned without copying its
operands. That pull request leaves {{visit(SqlNodeList)}} out of scope, so
lists keep the old eager copy.
h2. Analysis (a reading of the code; the report stands without it)
{{SqlShuttle.visit(SqlNodeList)}} creates {{new ArrayList<>(nodeList.size())}}
and adds every visited element to it before it knows whether any element
changed. Allocating the list at the first changed element, and copying the
elements before it, would remove the copy for read-only traversals.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
