Re: Query Compilation happening more often then expected

2019-09-18 Thread Julian Hyde
See my comments in https://issues.apache.org/jira/browse/CALCITE-963 . > On Sep 18, 2019, at 1:58 PM, Scott Reynolds wrote: > > Hi all, > > I have submitted my pull request in github and wrote up a design document > as a comment on

Re: Query Compilation happening more often then expected

2019-09-18 Thread Scott Reynolds
Hi all, I have submitted my pull request in github and wrote up a design document as a comment on CALCITE-963 My implementation deviates

Re: Query Compilation happening more often then expected

2019-09-17 Thread Vladimir Sitnikov
Stamatis>Out of curiosity does anybody know if popular DBMS (Postgres) support "hoisting"? PostgreSQL does support it, and here's a reproducible case when that feature makes the query 300 times slower: https://gist.github.com/vlsi/df08cbef370b2e86a5c1 Vladimir

Re: Query Compilation happening more often then expected

2019-09-17 Thread Julian Hyde
I’m just saying the approach isn’t perfect. And it isn’t implemented. If you are able to change your application to use bind variables, that is superior in every way. You mention “a quick recheck of cost function”. This is a very nice use case for multi-objective parametric query

Re: Query Compilation happening more often then expected

2019-09-17 Thread Julian Feinauer
Hi, this is a good point Julian. So an implementation should consider a re-planning (possibly triggered by a quick recheck of cost function with the given Literal values). But this should not be a general issue with the approach, or? JulianF Am 16.09.19, 23:36 schrieb "Julian Hyde" : I

Re: Query Compilation happening more often then expected

2019-09-16 Thread Julian Hyde
I found evidence that MSSQL[1] and Sybase ASE[2] do it. I agree, it's not a free lunch. For instance, if a column has a non-uniform distribution, some values might be much more selective than others, and it would be much better to know which value you are dealing with at planning time, rather

Re: Query Compilation happening more often then expected

2019-09-16 Thread Stamatis Zampetakis
Out of curiosity does anybody know if popular DBMS (Postgres, Oracle, SQL Server, etc.) support "hoisting"? Performing it all the time does not seem a very good idea (constant reduction, histograms, and other optimization techniques would be impossible) while leaving its configuration to the

Re: Query Compilation happening more often then expected

2019-09-14 Thread Julian Hyde
The idea of converting literals into bind variables is called “hoisting”. We had the idea a while ago but have not implemented it. https://issues.apache.org/jira/browse/CALCITE-963 Until that feature is implemented, you will need to create bind variables explicitly, and bind them before

Query Compilation happening more often then expected

2019-09-13 Thread Scott Reynolds
Hi, Spent a bunch of time researching and staring at code today to understand the code compilation path within Calcite. I started down this path because we noticed whenever we changed the `startDate` or `endDate` for the query it went through compilation process again. We expected it to use the