[jira] [Created] (CALCITE-1536) Initialize cluster before planner

2016-12-12 Thread Julian Hyde (JIRA) 
Julian Hyde created CALCITE-1536:


 Summary: Initialize cluster before planner
 Key: CALCITE-1536
 URL: https://issues.apache.org/jira/browse/CALCITE-1536
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde


We should initialize the cluster ({{RelOptCluster}}) before planner 
({{RelOptPlanner}}, or a sub-class such as {{VolcanoPlanner}} or 
{{HepPlanner}}). Currently the planner contains important information such as 
executor ({{RelOptPlanner.Executor}}), the set of active traits (epitomized by 
the {{RelOptPlanner.emptyTraitSet}} method) and the metadata providers, and the 
cluster contains a link to a planner, so the planner has to be created first.

This makes it difficult to use a succession of planners for query planning. 
Fixing this issue is a first step towards CALCITE-1525.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Ask Advices on Calcite Streaming SQL

2016-12-12 Thread 邓小勇(静行) 
Hi there,

We're working on a streaming engine used internally at Alibaba, and would like 
to support some sort of streaming SQL on top of it.

The current plan is to follow Calcite Streaming SQL, though we might need to 
extend the current implementation in certain ways (e.g., allowing creating 
streams connecting to external sources/sink, multi-insert, the support for 
multiple statements in a script, different kinds of window functions, etc).

Since we'd like to have it ready for production use within the next 3-6 months, 
I'm wondering if you have any suggestions or comments on how we should proceed. 
Thank you very much!

Best,
Xiaoyong
Alibaba Cloud

Group by alias name ?

2016-12-12 Thread Fang, Ningli 
Dear Calcite developers,

When we execute this query:
select "apiName" as "API_NAME", SUM("respSize") AS FUNC_SUM_RESP_SIZE from 
"api_metrics" group by "API_NAME";

we receive the exception:
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Column 
'API_NAME' not found in any table

Stack trace:
Error: Error while executing SQL "select "apiName" as "API_NAME", 
SUM("respSize") AS FUNC_SUM_RESP_SIZE from "api_metrics" group by "API_NAME"": 
From line 1, co
 line 1, column 108: Column 'API_NAME' not found in any table (state=,code=0)
java.sql.SQLException: Error while executing SQL "select "apiName" as 
"API_NAME", SUM("respSize") AS FUNC_SUM_RESP_SIZE from "api_metrics" group by 
"API_NAME"":
 1, column 99 to line 1, column 108: Column 'API_NAME' not found in any table
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at 
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:209)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:813)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, 
column 99 to line 1, column 108: Column 'API_NAME' not found in any table
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown 
Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at 
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:405)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:799)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:784)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4102)
at 
org.apache.calcite.sql.validate.EmptyScope.findQualifyingTableName(EmptyScope.java:104)
at 
org.apache.calcite.sql.validate.DelegatingScope.findQualifyingTableName(DelegatingScope.java:149)
at 
org.apache.calcite.sql.validate.ListScope.findQualifyingTableName(ListScope.java:122)
at 
org.apache.calcite.sql.validate.DelegatingScope.fullyQualify(DelegatingScope.java:198)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:4601)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:4583)
at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:324)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.expand(SqlValidatorImpl.java:4276)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause(SqlValidatorImpl.java:3444)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3081)
at 
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:898)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:879)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:208)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:854)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:565)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:549)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:195)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:735)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:598)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:568)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:215)
at 
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:594)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:616)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
... 7 more
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Co

Re: Group by alias name ?

2016-12-12 Thread Julian Hyde 
Your query is not standard SQL; some databases support it, but Calcite does 
not. It is covered by https://issues.apache.org/jira/browse/CALCITE-1306 
.

> On Dec 12, 2016, at 2:01 PM, Fang, Ningli  wrote:
> 
> Dear Calcite developers,
>  
> When we execute this query:
> select "apiName" as "API_NAME", SUM("respSize") AS FUNC_SUM_RESP_SIZE from 
> "api_metrics" group by "API_NAME";
> 
> we receive the exception:
> Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Column 
> 'API_NAME' not found in any table
> 
> Stack trace:
> Error: Error while executing SQL "select "apiName" as "API_NAME", 
> SUM("respSize") AS FUNC_SUM_RESP_SIZE from "api_metrics" group by 
> "API_NAME"": From line 1, co
>  line 1, column 108: Column 'API_NAME' not found in any table (state=,code=0)
> java.sql.SQLException: Error while executing SQL "select "apiName" as 
> "API_NAME", SUM("respSize") AS FUNC_SUM_RESP_SIZE from "api_metrics" group by 
> "API_NAME"":
>  1, column 99 to line 1, column 108: Column 'API_NAME' not found in any table
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at 
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:209)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:813)
> at sqlline.SqlLine.begin(SqlLine.java:686)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, 
> column 99 to line 1, column 108: Column 'API_NAME' not found in any table
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown 
> Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown 
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at 
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:405)
> at 
> org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:799)
> at 
> org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:784)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4102)
> at 
> org.apache.calcite.sql.validate.EmptyScope.findQualifyingTableName(EmptyScope.java:104)
> at 
> org.apache.calcite.sql.validate.DelegatingScope.findQualifyingTableName(DelegatingScope.java:149)
> at 
> org.apache.calcite.sql.validate.ListScope.findQualifyingTableName(ListScope.java:122)
> at 
> org.apache.calcite.sql.validate.DelegatingScope.fullyQualify(DelegatingScope.java:198)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:4601)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:4583)
> at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:324)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.expand(SqlValidatorImpl.java:4276)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause(SqlValidatorImpl.java:3444)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3081)
> at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:898)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:879)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:208)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:854)
> at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:565)
> at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:549)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:195)
> at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:735)
> at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:598)
> at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:568)
> at 
> org.apache.calcite.jdbc.CalciteC

[jira] [Created] (CALCITE-1537) Un-necessary project expression in multi-subquery plan

2016-12-12 Thread Vineet Garg (JIRA) 
Vineet Garg created CALCITE-1537:


 Summary: Un-necessary project expression in multi-subquery plan
 Key: CALCITE-1537
 URL: https://issues.apache.org/jira/browse/CALCITE-1537
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Vineet Garg
Assignee: Julian Hyde


Query
{code} select sal from emp where empno IN (select deptno from dept where 
emp.job = dept.name)  AND empno IN (select empno from emp e where emp.ename = 
e.ename) {code}

Plan just before calling *SubqueryRemoveRule*
{code}
LogicalProject(SAL=[$5])
  LogicalFilter(condition=[AND(IN($0, {
LogicalProject(DEPTNO=[$0])
  LogicalFilter(condition=[=($cor0.JOB, $1)])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
}), IN($0, {
LogicalProject(EMPNO=[$0])
  LogicalFilter(condition=[=($cor0.ENAME, $1)])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
}))], variablesSet=[[$cor0]])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}

Plan just after *SubqueryRemoveRule*
{code}
LogicalProject(SAL=[$5])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], DEPTNO0=[$9])
  LogicalJoin(condition=[=($0, $10)], joinType=[inner])
LogicalFilter(condition=[=($0, $9)])
  LogicalCorrelate(correlation=[$cor0], joinType=[INNER], 
requiredColumns=[{2}])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0}])
  LogicalProject(DEPTNO=[$0])
LogicalFilter(condition=[=($cor0.JOB, $1)])
  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
LogicalAggregate(group=[{0}])
  LogicalProject(EMPNO=[$0])
LogicalFilter(condition=[=($cor0.ENAME, $1)])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}

Plan just after *decorrelation*
{code}
LogicalProject(SAL=[$5], ENAME0=[$9])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], ENAME0=[$10])
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], DEPTNO0=[$9], ENAME0=[$12])
  LogicalJoin(condition=[=($0, $11)], joinType=[inner])
LogicalJoin(condition=[AND(=($2, $10), =($0, $9))], joinType=[inner])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0, 1}])
LogicalProject(DEPTNO=[$0], JOB=[$1])
  LogicalProject(DEPTNO=[$0], JOB=[$2])
LogicalJoin(condition=[=($2, $1)], joinType=[inner])
  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
  LogicalAggregate(group=[{0}])
LogicalProject(JOB=[$2])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}])
  LogicalProject(EMPNO=[$0], ENAME0=[$1])
LogicalProject(EMPNO=[$0], ENAME0=[$9])
  LogicalJoin(condition=[=($9, $1)], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0}])
  LogicalProject(ENAME=[$1])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}

As you can notice the top *LogicalProject* has unnecessary expression *ENAME0* 
after decorrelation




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ask Advices on Calcite Streaming SQL

2016-12-12 Thread Albert 
Hi,
the question seems too broad and not well framed. but in general if you
want to understand more about streaming within `calcite`.

- Firstly:  The document 
- Secondly: Julian also led several discussions in the dev thread, you
might be able to found them within mail archive.
- Last but not least: code is always there for you to reference. I suggest
you start from the CSV adapter, you might have a clearer view regarding
what to proceed after you look into that.

anyways, discussion is always welcomed here.



On Mon, Dec 12, 2016 at 5:38 PM, 邓小勇(静行) 
wrote:

> Hi there,
>
> We're working on a streaming engine used internally at
> Alibaba, and would like to support some sort of
> streaming SQL on top of it.
>
> The current plan is to follow Calcite Streaming SQL, though
> we might need to extend the current implementation in
> certain ways (e.g., allowing creating streams connecting
> to external sources/sink, multi-insert, the support for
> multiple statements in a script, different kinds of window functions,
> etc).
>
> Since we'd like to have it ready for production use
> within the next 3-6 months, I'm wondering if you have any
> suggestions or comments on how we should proceed. Thank you very much!
>
> Best,
> Xiaoyong
> Alibaba Cloud




-- 
~~~
no mistakes
~~