[
https://issues.apache.org/jira/browse/CALCITE-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705738#comment-16705738
]
Stamatis Zampetakis commented on CALCITE-2703:
----------------------------------------------
{quote}
In other words, what if we add a caching layer in-between PreparedStatement and
EnumerableInterpretable#toBindable?
{quote}
[~vladimirsitnikov] That was kind of my original idea but the problem with this
is that whoever does not use Calcite as a server (e.g., me) thus does not pass
from the PreparedStatement. This mean, that if he uses the EnumerableConvention
he will potentially need to add another cache at the application layer.
[~julianhyde] I opened this JIRA case because I believe a cache will be
beneficial for the majority of users. If you believe it is not the case, I will
close this issue and we will add the cache outside of Calcite. Other than that
using Calcite's interpreter is not an alternative since it also passes from
Janino thus it suffers from the same compilation/ class loading overhead.
> Reduce code generation and class loading overhead when executing queries in
> the EnumerableConvention
> ----------------------------------------------------------------------------------------------------
>
> Key: CALCITE-2703
> URL: https://issues.apache.org/jira/browse/CALCITE-2703
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.17.0
> Reporter: Stamatis Zampetakis
> Assignee: Julian Hyde
> Priority: Major
> Labels: performance
> Fix For: 1.18.0
>
>
> The queries using Calcite's EnumerableConvention always end-up generating new
> java classes at runtime (using Janino) that are then instantiated using
> reflection. This combination of class generation and class loading introduces
> a big overhead in query response time.
> A quick profiling on our Company's internal test suite consisting in 4000
> tests with roughly 430000 SQL queries passing through Calcite we observed
> that a big amount of time is spend on code generation and class loading
> making the EnumerableInterpretable#toBindable method a performance
> bottleneck.
> Among the 430000 SQL queries there are many duplicates which are going to
> lead to the generation of exactly the same code Java. Introducing, a small
> cache at the level of EnumerableInterpretable class could avoid generating
> and loading the same code over and over again.
> A simple implementation based on Guava improved the overall execution time of
> the afforementioned test suite by more than 50%.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
