[ https://issues.apache.org/jira/browse/CALCITE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lai Zhou updated CALCITE-3071: ------------------------------ Description: In real business, when sql queries become complex, the overhead of sql plan will increase quickly , and many of the sql queries are duplicates. We already have some caching issue about improving the performance, such as the issue https://issues.apache.org/jira/browse/CALCITE-2703, which reduce code generation and class loading overhead when executing queries in the EnumerableConvention, but I think it's not enough. I propose to cache the whole sql plan to reduce the latency ,for the same sql , ignoring the cost optimizing based on statistics here, we can cache the generated code for it. I use the FrameworkConfig API to execute sql queries, in this way I can easily do this job . but it's not easy to make a whole sql execution plan(that says code-gen) cache in the sql processing flow based on JDBC Connection, because there're many intermediate state in this processing flow. Let's discuss this feature and the probable solutions. was: In real business, when sql queries become complex, the overhead of sql plan will increase quickly , and many of the sql queries are duplicates. We already do something on cacheing to improve the performance, such as the issue https://issues.apache.org/jira/browse/CALCITE-2703, which reduce code generation and class loading overhead when executing queries in the EnumerableConvention, but I think it's not enough. I propose to cache the whole sql plan to reduce the latency ,for the same sql , ignoring the cost optimizing based on statistics here, we can cache the generated code for it. I use the FrameworkConfig API to execute sql queries, in this way I can easily do this job . but it's not easy to make a whole sql execution plan(that says code-gen) cache in the sql processing flow based on JDBC Connection, because there're many intermediate state in this processing flow. Let's discuss this feature and the probable solutions. > Cache the whole sql plan to reduce the latency and improve the performance > -------------------------------------------------------------------------- > > Key: CALCITE-3071 > URL: https://issues.apache.org/jira/browse/CALCITE-3071 > Project: Calcite > Issue Type: Improvement > Components: core > Affects Versions: 1.19.0 > Reporter: Lai Zhou > Priority: Major > > In real business, when sql queries become complex, the overhead of sql plan > will increase quickly , and many of the sql queries are duplicates. > We already have some caching issue about improving the performance, such as > the issue > https://issues.apache.org/jira/browse/CALCITE-2703, > which reduce code generation and class loading overhead when executing > queries in the EnumerableConvention, but I think it's not enough. > I propose to cache the whole sql plan to reduce the latency ,for the same sql > , ignoring the cost optimizing based on statistics here, we can cache the > generated code for it. > I use the FrameworkConfig API to execute sql queries, in this way I can > easily do this job . > but it's not easy to make a whole sql execution plan(that says code-gen) > cache in the sql processing flow based on JDBC Connection, because there're > many intermediate state in this processing flow. > > Let's discuss this feature and the probable solutions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)