[
https://issues.apache.org/jira/browse/IGNITE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-16315:
---------------------------------------
Labels: calcite3-required (was: calcite2-required calcite3-required)
> Calcite engine. Query start request contains a lot of data
> ----------------------------------------------------------
>
> Key: IGNITE-16315
> URL: https://issues.apache.org/jira/browse/IGNITE-16315
> Project: Ignite
> Issue Type: Improvement
> Reporter: Aleksey Plekhanov
> Assignee: Aleksey Plekhanov
> Priority: Major
> Labels: calcite3-required
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> For simple queries SQL engine most of the time spend in writing/reading query
> start requests, which contains a lot of data. Nested instances of
> {{ColocationGroup}} class contain assignments for each partition
> ({{{}List<List<UUID>>{}}}). Transferred size can be reduced if we compact
> assignments somehow. The target colocation group from fragment description
> contains redundant synthetic partitions, this also can be optimized.
> Messages workflow is not optimal too. First, we send {{QueryStartRequest}} to
> the remote nodes, remotes reply with the QueryStartResponse messages. After
> that remotes send batches with data to the target nodes and receive acks for
> each batch (acks required to limit inbox workload). When query execution is
> finished, the node initiator sends {{QueryCloseMessage}} to the remote nodes,
> remotes close queries, and sends back {{ErrorMessage}} to the initiator with
> the {{ExecutionCancelledException}} error (which is ignored on the initiator
> node).
> Also, some other optimizations are possible. Proposed changes:
> * Implement compaction of assignments of {{ColocationGroup}}
> * Reduce target colocation group partitions count
> * Fix caching of query plans (store original SQL as key, not parsed SQL, to
> avoid redundant parsing)
> * Change messages workflow (don't send ack messages for the last batch since
> it is redundant, self-close remote queries, and don't send close query
> messages to remote nodes, if we know for sure that it's already self-closed,
> don't send query start response if we already have sent batch for the same
> fragment before)
> * Reduce count of {{RexBuilder}} creation on the execution phase (RexBuilder
> is stateless and can be used one static instance)
> * Reduce count of Calcite types creation on the execution phase
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
