Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Андрей Цвелодуб
Hello everyone, First of all, thanks for this great effort of improving the core parts of the framework we all are using, I believe this is long overdue and hope this will have benefits both for the maintainers and users of the library. I don't have anything to say about the general idea at the

[jira] [Created] (CALCITE-3942) Move type-coercion configurations into SqlValidator.Config

2020-04-21 Thread Danny Chen (Jira)
Danny Chen created CALCITE-3942: --- Summary: Move type-coercion configurations into SqlValidator.Config Key: CALCITE-3942 URL: https://issues.apache.org/jira/browse/CALCITE-3942 Project: Calcite

[jira] [Created] (CALCITE-3943) Remove the JSON functions keyword from parser

2020-04-21 Thread Danny Chen (Jira)
Danny Chen created CALCITE-3943: --- Summary: Remove the JSON functions keyword from parser Key: CALCITE-3943 URL: https://issues.apache.org/jira/browse/CALCITE-3943 Project: Calcite Issue Type:

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Chunwei Lei
Haisheng and Xiening, thanks for sharing these wonderful ideas. I believe this will be a huge improvement and definitely benefits all users. >From my experience of upgrading calcite version, there are always some changes in the new version which may lead to unexpected behavior due to a lack of

[jira] [Created] (CALCITE-3944) Move dumpSets and dumpGraphviz out of VolcanoPlanner

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3944: -- Summary: Move dumpSets and dumpGraphviz out of VolcanoPlanner Key: CALCITE-3944 URL: https://issues.apache.org/jira/browse/CALCITE-3944 Project: Calcite

[jira] [Created] (CALCITE-3945) Update Streaming documentation

2020-04-21 Thread Rui Wang (Jira)
Rui Wang created CALCITE-3945: - Summary: Update Streaming documentation Key: CALCITE-3945 URL: https://issues.apache.org/jira/browse/CALCITE-3945 Project: Calcite Issue Type: Sub-task

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Андрей Цвелодуб
Hello Haisheng, > To keep backward compatibility, all the un-marked rules will be treated as logical rules, except rules that uses AbstractConverter as rule operand, these rules still need to applied top-down, or random order. Obviously, from what is written here, I could guess that this would

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-21 Thread Stamatis Zampetakis
I've seen it also quite a few times and most often the reason was cycles in the subsets that were triggering some rules (e.g., ProjectMergeRule [1]) to fire an infinite number of times [2, 3, 4]. Best, Stamatis [1]

[jira] [Created] (CALCITE-3947) AbstractRelOptPlanner.classes should be LinkedHashSet so that rule match order is deterministic across runs

2020-04-21 Thread Botong Huang (Jira)
Botong Huang created CALCITE-3947: - Summary: AbstractRelOptPlanner.classes should be LinkedHashSet so that rule match order is deterministic across runs Key: CALCITE-3947 URL:

Re: Understanding annotations of SqlGroupingFunction

2020-04-21 Thread Vineet G
I expect that the user behavior for the GROUPING in both hive and calcite is same. It’s just the documentation which is a bit confusing. e.g. comment line on grouping : if both deptno and gender are being grouped should really mean that the row which represents the grand total i.e without

Calcite-Master - Build # 1712 - Still Failing

2020-04-21 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #1712) Status: Still Failing Check console output at https://builds.apache.org/job/Calcite-Master/1712/ to view the results.

[jira] [Created] (CALCITE-3948) Improve operand's RelSubset matching handling in VolcanoRuleCall

2020-04-21 Thread Botong Huang (Jira)
Botong Huang created CALCITE-3948: - Summary: Improve operand's RelSubset matching handling in VolcanoRuleCall Key: CALCITE-3948 URL: https://issues.apache.org/jira/browse/CALCITE-3948 Project:

Re: Stored Proc to Relational Expression

2020-04-21 Thread Stamatis Zampetakis
Hi Ravi, As Julian already mentioned, we are quite far from what you would like to achieve. Nevertheless, I would like to mention that you can model certain kind of loops and recursion using the RepeatUnion [1] and Spool [2] relational expressions. Note that these APIs are experimental.

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-21 Thread Haisheng Yuan
I think ProjectMergeRule related cyclic triggering has been walked around by CALCITE-3774 [1]. Haisheng [1] https://issues.apache.org/jira/browse/CALCITE-3774 On 2020/04/21 21:20:35, Stamatis Zampetakis wrote: > I've seen it also quite a few times and most often the reason was cycles in >

Calcite-Master - Build # 1711 - Failure

2020-04-21 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #1711) Status: Failure Check console output at https://builds.apache.org/job/Calcite-Master/1711/ to view the results.

Re: Building a Calcite Adapter

2020-04-21 Thread Stamatis Zampetakis
Hi Jon, Thanks for your kind words. I'm sure people working on the project are very happy to receive some positive feedback for their work from time to time :) I had a quick look on your project and definitely looks interesting. If your engine (Crux) uses better join algorithms than the ones

[jira] [Created] (CALCITE-3949) RelDistributions.of() and RelCollations.of() should canonize trait instance

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3949: -- Summary: RelDistributions.of() and RelCollations.of() should canonize trait instance Key: CALCITE-3949 URL: https://issues.apache.org/jira/browse/CALCITE-3949

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
Hi Andrii, > Obviously, from what is written here, I could guess that this would require > me to change my physical planning rules, even if only by implementing a > marker interface. You don't need to change your physical rules, it will be treated as equal as logical rules and be applied

Re: Understanding annotations of SqlGroupingFunction

2020-04-21 Thread Julian Hyde
Suppose we have one row that represents the total for department 10, and another that represents the grand total of all departments. Which row would we say that department is “grouped” (in Calcite’s parlance) or “aggregated” in (Hive’s parlance)? I find the terms confusing. It’s possible that

[jira] [Created] (CALCITE-3946) Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements

2020-04-21 Thread dasch (Jira)
dasch created CALCITE-3946: -- Summary: Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements Key: CALCITE-3946 URL: https://issues.apache.org/jira/browse/CALCITE-3946 Project:

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
Hi Andrii, > I guess changing the planner would lead to changes in tons of rules and even > more tests. Obviously you didn't read through my email. You are not required to do any changes to your rule if you don't want to, but if you do, just need to mark the rule to tell planner whether it is