date:20191002

Re: [DISCUSS] Draft board report for October 2019

2019-10-02 Thread Stamatis Zampetakis

Looks great, nothing to add! Thanks Francis.

Best,
Stamatis

On Thu, Oct 3, 2019 at 2:56 AM Michael Mior  wrote:

> Thanks Francis! Looks good to me.
> --
> Michael Mior
> mm...@apache.org
>
> Le mer. 2 oct. 2019 à 19:21, Francis Chuang  a
> écrit :
> >
> > Attached below is a draft of this month's board report. Please let me
> > know if you have any additions or corrections. Note that the format of
> > the report has changed slightly compared to the previous ones and the
> > ASF has revamped the reporting tool.
> >
> > ## Description:
> > Apache Calcite is a highly customizable framework for parsing and
> > planning queries on data in a wide variety of formats. It allows
> > database-like access, and in particular a SQL interface and advanced
> > query optimization, for data not residing in a traditional database.
> >
> > ## Issues:
> > There are no issues requiring board attention.
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-22 (4 years ago)
> > There are currently 45 committers and 20 PMC members in this project.
> > The Committer-to-PMC ratio is 9:4.
> >
> > Community changes, past quarter:
> > - No new PMC members. Last addition was Stamatis Zampetakis on
> 2019-04-13.
> > - Julian Feinauer was added as committer on 2019-09-10
> > - Mohamed Mohsen was added as committer on 2019-09-18
> >
> > ## Project Activity:
> > Development and mailing list activity is steady for both Calcite and its
> > Avatica sub-project.
> >
> > Calcite 1.21.0 was released in the middle of September, including more
> > than 100 resolved issues and maintaining a release cadence of roughly
> > one release per quarter.
> >
> > We are also seeing new faces on the mailing list and opening pull
> > requests on Github. In terms of pull requests, our committers and
> > contributors have made a lot of progress to provide feedback to open
> > pull requests and filed issues in a timely manner. This is evidenced by
> > the open pull requests on Github receiving comments within a couple of
> > days after being opened.
> >
> > Members of the project also participated in ApacheCon NA last month,
> > presenting 5 talks about Calcite.
> >
> > Finally, the Apache Ignite project has decided to adopt Calcite as its
> > SQL execution engine, replacing H2. This is an exciting development and
> > is a testament to the sound foundation and community the Calcite project
> > has developed.
> >
> > ## Community Health:
> > Activity levels on mailing lists, git and JIRA are normal for both
> > Calcite and Avatica with a slight decrease in code contributors and
> > commits (7% and 1% respectively).
> >
> > The rates of pull requests being closed and merged on Github has
> > increased by 16%, as we work to clear our backlog and we are also seeing
> > a 7% increase in opened pull requests.
> >
> > Since the last report, we have added 2 new committers, Julian Feinauer
> > and Mohamed Mohsen.
> >
> > We expect further growth in these numbers as Apache Ignite works to
> > integrate Calcite into their project, resulting in cross-pollination
> > between the 2 projects.
>

Re: [DISCUSS] Draft board report for October 2019

2019-10-02 Thread Michael Mior

Thanks Francis! Looks good to me.
--
Michael Mior
mm...@apache.org

Le mer. 2 oct. 2019 à 19:21, Francis Chuang  a écrit :
>
> Attached below is a draft of this month's board report. Please let me
> know if you have any additions or corrections. Note that the format of
> the report has changed slightly compared to the previous ones and the
> ASF has revamped the reporting tool.
>
> ## Description:
> Apache Calcite is a highly customizable framework for parsing and
> planning queries on data in a wide variety of formats. It allows
> database-like access, and in particular a SQL interface and advanced
> query optimization, for data not residing in a traditional database.
>
> ## Issues:
> There are no issues requiring board attention.
>
> ## Membership Data:
> Apache Calcite was founded 2015-10-22 (4 years ago)
> There are currently 45 committers and 20 PMC members in this project.
> The Committer-to-PMC ratio is 9:4.
>
> Community changes, past quarter:
> - No new PMC members. Last addition was Stamatis Zampetakis on 2019-04-13.
> - Julian Feinauer was added as committer on 2019-09-10
> - Mohamed Mohsen was added as committer on 2019-09-18
>
> ## Project Activity:
> Development and mailing list activity is steady for both Calcite and its
> Avatica sub-project.
>
> Calcite 1.21.0 was released in the middle of September, including more
> than 100 resolved issues and maintaining a release cadence of roughly
> one release per quarter.
>
> We are also seeing new faces on the mailing list and opening pull
> requests on Github. In terms of pull requests, our committers and
> contributors have made a lot of progress to provide feedback to open
> pull requests and filed issues in a timely manner. This is evidenced by
> the open pull requests on Github receiving comments within a couple of
> days after being opened.
>
> Members of the project also participated in ApacheCon NA last month,
> presenting 5 talks about Calcite.
>
> Finally, the Apache Ignite project has decided to adopt Calcite as its
> SQL execution engine, replacing H2. This is an exciting development and
> is a testament to the sound foundation and community the Calcite project
> has developed.
>
> ## Community Health:
> Activity levels on mailing lists, git and JIRA are normal for both
> Calcite and Avatica with a slight decrease in code contributors and
> commits (7% and 1% respectively).
>
> The rates of pull requests being closed and merged on Github has
> increased by 16%, as we work to clear our backlog and we are also seeing
> a 7% increase in opened pull requests.
>
> Since the last report, we have added 2 new committers, Julian Feinauer
> and Mohamed Mohsen.
>
> We expect further growth in these numbers as Apache Ignite works to
> integrate Calcite into their project, resulting in cross-pollination
> between the 2 projects.

[DISCUSS] Draft board report for October 2019

2019-10-02 Thread Francis Chuang

Attached below is a draft of this month's board report. Please let me 
know if you have any additions or corrections. Note that the format of 
the report has changed slightly compared to the previous ones and the 
ASF has revamped the reporting tool.


## Description:
Apache Calcite is a highly customizable framework for parsing and 
planning queries on data in a wide variety of formats. It allows 
database-like access, and in particular a SQL interface and advanced 
query optimization, for data not residing in a traditional database.


## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Calcite was founded 2015-10-22 (4 years ago)
There are currently 45 committers and 20 PMC members in this project.
The Committer-to-PMC ratio is 9:4.

Community changes, past quarter:
- No new PMC members. Last addition was Stamatis Zampetakis on 2019-04-13.
- Julian Feinauer was added as committer on 2019-09-10
- Mohamed Mohsen was added as committer on 2019-09-18

## Project Activity:
Development and mailing list activity is steady for both Calcite and its
Avatica sub-project.

Calcite 1.21.0 was released in the middle of September, including more 
than 100 resolved issues and maintaining a release cadence of roughly 
one release per quarter.


We are also seeing new faces on the mailing list and opening pull 
requests on Github. In terms of pull requests, our committers and 
contributors have made a lot of progress to provide feedback to open 
pull requests and filed issues in a timely manner. This is evidenced by 
the open pull requests on Github receiving comments within a couple of 
days after being opened.


Members of the project also participated in ApacheCon NA last month,
presenting 5 talks about Calcite.

Finally, the Apache Ignite project has decided to adopt Calcite as its 
SQL execution engine, replacing H2. This is an exciting development and 
is a testament to the sound foundation and community the Calcite project 
has developed.


## Community Health:
Activity levels on mailing lists, git and JIRA are normal for both 
Calcite and Avatica with a slight decrease in code contributors and 
commits (7% and 1% respectively).


The rates of pull requests being closed and merged on Github has 
increased by 16%, as we work to clear our backlog and we are also seeing 
a 7% increase in opened pull requests.


Since the last report, we have added 2 new committers, Julian Feinauer 
and Mohamed Mohsen.


We expect further growth in these numbers as Apache Ignite works to
integrate Calcite into their project, resulting in cross-pollination 
between the 2 projects.

Re: Ignite community is building Calcite-based prototype

2019-10-02 Thread Denis Magda

Julian,

Thanks a lot for the references and guidance! I do believe that from now on
our community guys will become frequent visitors of yours ;)

-
Denis


On Wed, Oct 2, 2019 at 12:40 PM Julian Hyde  wrote:

> Denis,
>
> I’ve been a follower and admirer of Ignite for several years, so I am
> delighted that you are considering Calcite.
>
> Ask questions on the dev list, log JIRA cases if you find them, and we’ll
> do our best to help.
>
> I’d like to bring to your attention RelBuilder. Some people want to go
> from SQL text to executable plan, but others want to drop in at the
> relational algebra level, and RelBuilder is a convenient interface for the
> latter.
>
> (The other) Julian
>
>
> > On Oct 1, 2019, at 3:43 PM, Denis Magda  wrote:
> >
> > Hi Julian,
> >
> > Nice to e-meet you and thanks for being ready to help! Hopefully, the
> > Ignite community will be able to contribute valuable changes back to
> > Calcite as part of this activity - "pay good for good" :)
> >
> > You are right that distributed computing, massive-parallel processing,
> and
> > calculations/querying at scale is what Ignite is targeted for. However,
> > while Drill is designed for analytics and IoTDB is for time-series,
> Ignite
> > is primarily used for OLTP with an increasing number of real-time
> analytics
> > use cases (no adhoc).
> >
> > Let's stay in touch!
> >
> > -
> > Denis
> >
> >
> > On Tue, Oct 1, 2019 at 6:42 AM Julian Feinauer <
> j.feina...@pragmaticminds.de>
> > wrote:
> >
> >> Hi Igor,
> >>
> >> I agree that it should be rather similar to what Drill did as
> distributed
> >> computing also is a big concern for Ignite, I guess, right?
> >>
> >> Julian
> >>
> >> Am 01.10.19, 15:06 schrieb "Seliverstov Igor" :
> >>
> >>Guys,
> >>
> >>The better link:
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-37%3A+New+query+execution+engine
> >> <
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-37:+New+query+execution+engine
> >>>
> >>
> >>Almost everything you may see by the link is the same as Drill guys
> >> already did, the difference is in details but the idea is the same.
> >>
> >>Of course we’ll face many issues while development and I'll
> appreciate
> >> if some of you assist us.
> >>
> >>Regards,
> >>Igor
> >>
> >>> 1 окт. 2019 г., в 12:32, Julian Feinauer <
> >> j.feina...@pragmaticminds.de> написал(а):
> >>>
> >>> Hi Denis,
> >>>
> >>> Nice to hear from you and the ignite team... that sounds like an
> >> excellent idea. I liked the idea of Ignite since I heard about it (I
> think
> >> when it became TLP back then). So I would be happy to help you if you
> have
> >> specific questions... I‘m currently working on a related topic, namely
> >> integrate calcite as SQL Layer into Apache IoTDB .
> >>>
> >>> Best
> >>> Julian
> >>>
> >>> Holen Sie sich Outlook für iOS
> >>> 
> >>> Von: Denis Magda 
> >>> Gesendet: Tuesday, October 1, 2019 2:37:20 AM
> >>> An: dev@calcite.apache.org ; dev <
> >> d...@ignite.apache.org>
> >>> Betreff: Ignite community is building Calcite-based prototype
> >>>
> >>> Hey ASF-mates,
> >>>
> >>> Just wanted to send a note for Ignite dev community who has started
> >>> prototyping
> >>> <
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/New-SQL-execution-engine-td43724.html
> >>>
> >>> with a new Ignite SQL engine and Calcite was selected as the most
> >> favorable
> >>> option.
> >>>
> >>> We will truly appreciate if you help us with questions that might
> >> hit your
> >>> dev list. Ignite folks have already studied Calcite well enough and
> >> carried
> >>> on with the integration, but there might be tricky parts that would
> >> require
> >>> your expertise.
> >>>
> >>> Btw, if anybody is interested in Ignite (memory-centric database and
> >>> compute platform) or would like to learn more details about the
> >> prototype
> >>> or join its development, please check these links or send us a note:
> >>>
> >>>  - https://ignite.apache.org
> >>>  -
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-33%3A+New+SQL+executor+engine+infrastructure
> >>>
> >>>
> >>> -
> >>> Denis,
> >>> Ignite PMC Chair
> >>
> >>
> >>
> >>
>
>

Re: Ignite community is building Calcite-based prototype

2019-10-02 Thread Julian Hyde

Denis,

I’ve been a follower and admirer of Ignite for several years, so I am delighted 
that you are considering Calcite.

Ask questions on the dev list, log JIRA cases if you find them, and we’ll do 
our best to help.

I’d like to bring to your attention RelBuilder. Some people want to go from SQL 
text to executable plan, but others want to drop in at the relational algebra 
level, and RelBuilder is a convenient interface for the latter.

(The other) Julian


> On Oct 1, 2019, at 3:43 PM, Denis Magda  wrote:
> 
> Hi Julian,
> 
> Nice to e-meet you and thanks for being ready to help! Hopefully, the
> Ignite community will be able to contribute valuable changes back to
> Calcite as part of this activity - "pay good for good" :)
> 
> You are right that distributed computing, massive-parallel processing, and
> calculations/querying at scale is what Ignite is targeted for. However,
> while Drill is designed for analytics and IoTDB is for time-series, Ignite
> is primarily used for OLTP with an increasing number of real-time analytics
> use cases (no adhoc).
> 
> Let's stay in touch!
> 
> -
> Denis
> 
> 
> On Tue, Oct 1, 2019 at 6:42 AM Julian Feinauer 
> wrote:
> 
>> Hi Igor,
>> 
>> I agree that it should be rather similar to what Drill did as distributed
>> computing also is a big concern for Ignite, I guess, right?
>> 
>> Julian
>> 
>> Am 01.10.19, 15:06 schrieb "Seliverstov Igor" :
>> 
>>Guys,
>> 
>>The better link:
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-37%3A+New+query+execution+engine
>> <
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-37:+New+query+execution+engine
>>> 
>> 
>>Almost everything you may see by the link is the same as Drill guys
>> already did, the difference is in details but the idea is the same.
>> 
>>Of course we’ll face many issues while development and I'll appreciate
>> if some of you assist us.
>> 
>>Regards,
>>Igor
>> 
>>> 1 окт. 2019 г., в 12:32, Julian Feinauer <
>> j.feina...@pragmaticminds.de> написал(а):
>>> 
>>> Hi Denis,
>>> 
>>> Nice to hear from you and the ignite team... that sounds like an
>> excellent idea. I liked the idea of Ignite since I heard about it (I think
>> when it became TLP back then). So I would be happy to help you if you have
>> specific questions... I‘m currently working on a related topic, namely
>> integrate calcite as SQL Layer into Apache IoTDB .
>>> 
>>> Best
>>> Julian
>>> 
>>> Holen Sie sich Outlook für iOS
>>> 
>>> Von: Denis Magda 
>>> Gesendet: Tuesday, October 1, 2019 2:37:20 AM
>>> An: dev@calcite.apache.org ; dev <
>> d...@ignite.apache.org>
>>> Betreff: Ignite community is building Calcite-based prototype
>>> 
>>> Hey ASF-mates,
>>> 
>>> Just wanted to send a note for Ignite dev community who has started
>>> prototyping
>>> <
>> http://apache-ignite-developers.2346864.n4.nabble.com/New-SQL-execution-engine-td43724.html
>>> 
>>> with a new Ignite SQL engine and Calcite was selected as the most
>> favorable
>>> option.
>>> 
>>> We will truly appreciate if you help us with questions that might
>> hit your
>>> dev list. Ignite folks have already studied Calcite well enough and
>> carried
>>> on with the integration, but there might be tricky parts that would
>> require
>>> your expertise.
>>> 
>>> Btw, if anybody is interested in Ignite (memory-centric database and
>>> compute platform) or would like to learn more details about the
>> prototype
>>> or join its development, please check these links or send us a note:
>>> 
>>>  - https://ignite.apache.org
>>>  -
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-33%3A+New+SQL+executor+engine+infrastructure
>>> 
>>> 
>>> -
>>> Denis,
>>> Ignite PMC Chair
>> 
>> 
>> 
>>

Re: Volcano Optimizer JDBC Table Scan -> Join

2019-10-02 Thread Julian Hyde

No need to apologize! We really appreciate you following up and saying you 
found a solution. Your solution may help someone else.

Julian


> On Oct 2, 2019, at 9:34 AM, Madhav Suresh  wrote:
> 
> I apologize for the immediate follow up, however, I found out my issue was
> not adding the EnumerableRules.ENUMERABLE_JOIN_RULE.
> 
> On Wed, Oct 2, 2019 at 10:50 AM Madhav Suresh 
> wrote:
> 
>> Hi All,
>> 
>> I'm trying to run the volcano optimizer on a simple join query with the
>> TPC-H schema. I'm a little unclear on how to solve this, I thought I needed
>> to add a converter rules from JDBC->Enumerable, but it seems like those
>> rules are included at runtime. I then tried adding a None->Bindable rule,
>> but that also didn't fix the issue. I get the error below, code included
>> below error:
>> 
>> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
>>> enough rules to produce a node with desired properties:
>>> convention=ENUMERABLE.
>>> Missing conversions are LogicalJoin[convention: NONE -> JDBC.name],
>>> LogicalJoin[convention: NONE -> ENUMERABLE]
>>> There are 2 empty subsets:
>>> Empty subset 0: rel#51:Subset#4.ENUMERABLE, the relevant part of the
>>> original plan is as follows
>>> 14:LogicalJoin(condition=[true], joinType=[inner])
>>>  11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
>>> joinType=[inner])
>>>0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name],
>>> table=[[customer]])
>>>1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>>>  4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>>> 
>>> Empty subset 1: rel#49:Subset#4.JDBC.name, the relevant part of the
>>> original plan is as follows
>>> 14:LogicalJoin(condition=[true], joinType=[inner])
>>>  11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
>>> joinType=[inner])
>>>0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name],
>>> table=[[customer]])
>>>1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>>>  4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>>> 
>> 
>> 
>> The test case I'm running is here (
>> https://gist.github.com/madhavsuresh/e95631776eceb47ab8eeb608268531ce):
>> 
>>>  @Test
>>>  public void testSimpleJoin() throws SqlParseException,
>>> ValidationException {
>>>String sql =
>>>"select\n"
>>>+ "  l.l_orderkey\n"
>>>+ "from\n"
>>>+ "  customer c,\n"
>>>+ "  orders o,\n"
>>>+ "  lineitem l\n"
>>>+ "\n"
>>>+ "where\n"
>>>+ "  c.c_mktsegment = 'HOUSEHOLD'\n"
>>>+ "  and c.c_custkey = o.o_custkey\n";
>>>optimizer = new VolcanoPlanner();
>>>optimizer.addRelTraitDef(ConventionTraitDef.INSTANCE);
>>>optimizer.addRule(new OptToyRules.OptToyTestFilter());
>>>// add rules
>>>optimizer.addRule(FilterJoinRule.FilterIntoJoinRule.FILTER_ON_JOIN);
>>>optimizer.addRule(ReduceExpressionsRule.PROJECT_INSTANCE);
>>>optimizer.addRule(PruneEmptyRules.PROJECT_INSTANCE);
>>> 
>>>// add ConverterRule
>>>optimizer.addRule(EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE);
>>>optimizer.addRule(EnumerableRules.ENUMERABLE_SORT_RULE);
>>>optimizer.addRule(EnumerableRules.ENUMERABLE_VALUES_RULE);
>>>optimizer.addRule(EnumerableRules.ENUMERABLE_PROJECT_RULE);
>>>optimizer.addRule(EnumerableRules.ENUMERABLE_FILTER_RULE);
>>>optimizer.addRule(NoneToBindableConverterRule.INSTANCE);
>>>SqlNode node = planner.parse(sql);
>>>node = planner.validate(node);
>>>SqlToRelConverter converter = createSqlToRelConverter();
>>>RelRoot n = converter.convertQuery(node, true, true);
>>>RelNode relNode = n.rel;
>>> 
>>>// TODO(madhavsuresh): only works with needsValidation set to true.
>>>RelTraitSet desiredTraits =
>>> 
>>> relNode.getCluster().traitSet().replace(EnumerableConvention.INSTANCE);
>>>relNode = optimizer.changeTraits(relNode, desiredTraits);
>>>optimizer.setRoot(relNode);
>>>optimizer.findBestExp();
>>> 
>>> 
>> 
>> Thanks for the help!
>> 
>> Madhav
>>

Re: Volcano Optimizer JDBC Table Scan -> Join

2019-10-02 Thread Madhav Suresh

I apologize for the immediate follow up, however, I found out my issue was
not adding the EnumerableRules.ENUMERABLE_JOIN_RULE.

On Wed, Oct 2, 2019 at 10:50 AM Madhav Suresh 
wrote:

> Hi All,
>
> I'm trying to run the volcano optimizer on a simple join query with the
> TPC-H schema. I'm a little unclear on how to solve this, I thought I needed
> to add a converter rules from JDBC->Enumerable, but it seems like those
> rules are included at runtime. I then tried adding a None->Bindable rule,
> but that also didn't fix the issue. I get the error below, code included
> below error:
>
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
>> enough rules to produce a node with desired properties:
>> convention=ENUMERABLE.
>> Missing conversions are LogicalJoin[convention: NONE -> JDBC.name],
>> LogicalJoin[convention: NONE -> ENUMERABLE]
>> There are 2 empty subsets:
>> Empty subset 0: rel#51:Subset#4.ENUMERABLE, the relevant part of the
>> original plan is as follows
>> 14:LogicalJoin(condition=[true], joinType=[inner])
>>   11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
>> joinType=[inner])
>> 0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name],
>> table=[[customer]])
>> 1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>>   4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>>
>> Empty subset 1: rel#49:Subset#4.JDBC.name, the relevant part of the
>> original plan is as follows
>> 14:LogicalJoin(condition=[true], joinType=[inner])
>>   11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
>> joinType=[inner])
>> 0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name],
>> table=[[customer]])
>> 1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>>   4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>>
>
>
> The test case I'm running is here (
> https://gist.github.com/madhavsuresh/e95631776eceb47ab8eeb608268531ce):
>
>>   @Test
>>   public void testSimpleJoin() throws SqlParseException,
>> ValidationException {
>> String sql =
>> "select\n"
>> + "  l.l_orderkey\n"
>> + "from\n"
>> + "  customer c,\n"
>> + "  orders o,\n"
>> + "  lineitem l\n"
>> + "\n"
>> + "where\n"
>> + "  c.c_mktsegment = 'HOUSEHOLD'\n"
>> + "  and c.c_custkey = o.o_custkey\n";
>> optimizer = new VolcanoPlanner();
>> optimizer.addRelTraitDef(ConventionTraitDef.INSTANCE);
>> optimizer.addRule(new OptToyRules.OptToyTestFilter());
>> // add rules
>> optimizer.addRule(FilterJoinRule.FilterIntoJoinRule.FILTER_ON_JOIN);
>> optimizer.addRule(ReduceExpressionsRule.PROJECT_INSTANCE);
>> optimizer.addRule(PruneEmptyRules.PROJECT_INSTANCE);
>>
>> // add ConverterRule
>> optimizer.addRule(EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE);
>> optimizer.addRule(EnumerableRules.ENUMERABLE_SORT_RULE);
>> optimizer.addRule(EnumerableRules.ENUMERABLE_VALUES_RULE);
>> optimizer.addRule(EnumerableRules.ENUMERABLE_PROJECT_RULE);
>> optimizer.addRule(EnumerableRules.ENUMERABLE_FILTER_RULE);
>> optimizer.addRule(NoneToBindableConverterRule.INSTANCE);
>> SqlNode node = planner.parse(sql);
>> node = planner.validate(node);
>> SqlToRelConverter converter = createSqlToRelConverter();
>> RelRoot n = converter.convertQuery(node, true, true);
>> RelNode relNode = n.rel;
>>
>> // TODO(madhavsuresh): only works with needsValidation set to true.
>> RelTraitSet desiredTraits =
>>
>> relNode.getCluster().traitSet().replace(EnumerableConvention.INSTANCE);
>> relNode = optimizer.changeTraits(relNode, desiredTraits);
>> optimizer.setRoot(relNode);
>> optimizer.findBestExp();
>>
>>
>
> Thanks for the help!
>
> Madhav
>

Volcano Optimizer JDBC Table Scan -> Join

2019-10-02 Thread Madhav Suresh

Hi All,

I'm trying to run the volcano optimizer on a simple join query with the
TPC-H schema. I'm a little unclear on how to solve this, I thought I needed
to add a converter rules from JDBC->Enumerable, but it seems like those
rules are included at runtime. I then tried adding a None->Bindable rule,
but that also didn't fix the issue. I get the error below, code included
below error:

org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE.
> Missing conversions are LogicalJoin[convention: NONE -> JDBC.name],
> LogicalJoin[convention: NONE -> ENUMERABLE]
> There are 2 empty subsets:
> Empty subset 0: rel#51:Subset#4.ENUMERABLE, the relevant part of the
> original plan is as follows
> 14:LogicalJoin(condition=[true], joinType=[inner])
>   11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
> joinType=[inner])
> 0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name], table=[[customer]])
> 1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>   4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>
> Empty subset 1: rel#49:Subset#4.JDBC.name, the relevant part of the
> original plan is as follows
> 14:LogicalJoin(condition=[true], joinType=[inner])
>   11:LogicalJoin(subset=[rel#12:Subset#2.NONE], condition=[true],
> joinType=[inner])
> 0:JdbcTableScan(subset=[rel#9:Subset#0.JDBC.name], table=[[customer]])
> 1:JdbcTableScan(subset=[rel#10:Subset#1.JDBC.name], table=[[orders]])
>   4:JdbcTableScan(subset=[rel#13:Subset#3.JDBC.name], table=[[lineitem]])
>


The test case I'm running is here (
https://gist.github.com/madhavsuresh/e95631776eceb47ab8eeb608268531ce):

>   @Test
>   public void testSimpleJoin() throws SqlParseException,
> ValidationException {
> String sql =
> "select\n"
> + "  l.l_orderkey\n"
> + "from\n"
> + "  customer c,\n"
> + "  orders o,\n"
> + "  lineitem l\n"
> + "\n"
> + "where\n"
> + "  c.c_mktsegment = 'HOUSEHOLD'\n"
> + "  and c.c_custkey = o.o_custkey\n";
> optimizer = new VolcanoPlanner();
> optimizer.addRelTraitDef(ConventionTraitDef.INSTANCE);
> optimizer.addRule(new OptToyRules.OptToyTestFilter());
> // add rules
> optimizer.addRule(FilterJoinRule.FilterIntoJoinRule.FILTER_ON_JOIN);
> optimizer.addRule(ReduceExpressionsRule.PROJECT_INSTANCE);
> optimizer.addRule(PruneEmptyRules.PROJECT_INSTANCE);
>
> // add ConverterRule
> optimizer.addRule(EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE);
> optimizer.addRule(EnumerableRules.ENUMERABLE_SORT_RULE);
> optimizer.addRule(EnumerableRules.ENUMERABLE_VALUES_RULE);
> optimizer.addRule(EnumerableRules.ENUMERABLE_PROJECT_RULE);
> optimizer.addRule(EnumerableRules.ENUMERABLE_FILTER_RULE);
> optimizer.addRule(NoneToBindableConverterRule.INSTANCE);
> SqlNode node = planner.parse(sql);
> node = planner.validate(node);
> SqlToRelConverter converter = createSqlToRelConverter();
> RelRoot n = converter.convertQuery(node, true, true);
> RelNode relNode = n.rel;
>
> // TODO(madhavsuresh): only works with needsValidation set to true.
> RelTraitSet desiredTraits =
>
> relNode.getCluster().traitSet().replace(EnumerableConvention.INSTANCE);
> relNode = optimizer.changeTraits(relNode, desiredTraits);
> optimizer.setRoot(relNode);
> optimizer.findBestExp();
>
>

Thanks for the help!

Madhav

Re: [DISCUSS] Draft board report for October 2019

Re: [DISCUSS] Draft board report for October 2019

[DISCUSS] Draft board report for October 2019

Re: Ignite community is building Calcite-based prototype

Re: Ignite community is building Calcite-based prototype

Re: Volcano Optimizer JDBC Table Scan -> Join

Re: Volcano Optimizer JDBC Table Scan -> Join

Volcano Optimizer JDBC Table Scan -> Join

8 matches

Site Navigation

Mail list logo

Footer information