date:20200402

Re: Calcite geode adapter query

2020-04-02 Thread aashish choudhary

+Christian

With best regards,
Ashish

On Thu, Apr 2, 2020, 5:00 PM Danny Chan  wrote:

> I believe few people in the community are familiar with geode, maybe you
> can ask the author of this module [1]
>
> [1] https://github.com/tzolov
>
> Best,
> Danny Chan
> 在 2020年4月2日 +0800 PM6:41，aashish choudhary  >，写道：
> > Can anyone help me with this?
> >
> > With best regards,
> > Ashish
> >
> > On Wed, Apr 1, 2020, 7:27 PM aashish choudhary <
> aashish.choudha...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am trying to setup querying geode using calcite geode adapter. So
> far i
> > > am successfully able to connect to geode cluster which is SSL enabled
> after
> > > making few changes in the adapter and query data for regions which are
> > > having flat hierarchy.
> > >
> > > However I am not able to run any query for region having Array list as
> > > values and String as key. I am using Dbeaver to query following this
> link.
> > > https://youtu.be/I_iaMgqdV3Q
> > >
> > > Can someone help me with this?
> > >
> > > With geode you can query those complex region in geode Pulse data
> browser.
> > > You can do .entrySet or .entries to run queries like this.
> > >
> > > Also in Dbeaver in under Tables->Columns I can view fields as well for
> > > regions having flat hierarchy but it doesn't show anything for regions
> > > storing arrays.
> > >
> > >
> > > With best regards,
> > > Ashish
> > >
>

Re: Draft board report for April 2020

2020-04-02 Thread Julian Hyde

“A talk  was given ...”. You’re too modest, Stamatis. And as my copy editor 
said, passive voice should rarely be used. 

“Stamatis gave a talk ...” is better; fewer words, more information, more 
interesting to read. 

Julian

> On Apr 1, 2020, at 3:08 PM, Stamatis Zampetakis  wrote:
> 
> Attached below is a draft of this month's board report. I plan to submit it
> on April 6. Please let me know if you have any additions or corrections.
> 
> ## Description:
> Apache Calcite is a highly customizable framework for parsing and planning
> queries on data in a wide variety of formats. It allows database-like
> access,
> and in particular a SQL interface and advanced query optimization, for data
> not residing in a traditional database.
> 
> Avatica is a sub-project within Calcite and provides a framework for
> building
> local and remote JDBC and ODBC database drivers. Avatica has an independent
> release schedule and its own repository.
> 
> ## Issues:
> There are no issues requiring board attention.
> 
> ## Membership Data:
> Apache Calcite was founded 2015-10-22 (4 years ago)
> There are currently 46 committers and 22 PMC members in this project.
> The Committer-to-PMC ratio is roughly 2:1.
> 
> Community changes, past quarter:
> - No new PMC members. Last addition was Haisheng Yuan on 2019-11-11.
> - Feng Zhu was added as committer on 2020-02-29
> 
> ## Project Activity:
> Avatica 1.16.0 was released in the middle of December, including numerous
> bug fixes and security improvements while the build system has been
> migrated from Maven to gradle.
> 
> Calcite 1.22.0 was released at the beginning of March, including more than
> 250 resolved issues including long-awaited features such as support of SQL
> hints, important bug fixes in the core of the query planner, and many
> changes in build and test infrastructure. This release was the biggest in
> the history of the project and one of the most complicated to pull out due
> to the big number of contributions.
> 
> An introductory talk about query planning and Calcite was given at the
> university of Cergy-Pontoise, Paris, France, in March. The audience was not
> aware of the existence of the project which shows that we should organize
> such events more often, at least in
> Europe, if we want to expand our community. [Do we have something else to
> add here?]
> 
> Finally, we were informed that the SuperSQL project by Tencent uses Calcite
> in order to provide a unified entrance to various data platforms such as
> RDBMS, ElasticSearch, Hive, Flink, Spark, Presto, ClickHouse, etc.
> 
> ## Community Health:
> 
> Activity levels on mailing lists (-8%), git (-24%) and JIRA (opened -20%,
> closed
> -30%) have decreased in the first quarter of 2020. Most of the big changes
> that started during the last quarter of 2019 (and initiated a lot of
> activity) have
> been finalized and incorporated in the release of Calcite 1.22.0, making
> things
> calmer in the community. The big decrease of closed issues in JIRA may be
> explained by the frequency of releases that have slightly dropped. To the
> above it is worth adding the period of holidays in December and January
> that in general slows things down.
> 
> Following the decreased activity in the lists, git, and JIRA it is normal
> to see
> the activity rates on pull requests drop (-20% opened, -31% closed) more or
> less
> for the same reasons. Although our backlog keeps increasing (~174 open
> pullvrequests), the community is very reactive commenting and reviewing
> almost every contribution.

Re: Asking help for parsing MySql Alter Table statements

2020-04-02 Thread Julian Hyde

Calcite server supports a minimal set of DDL commands so that you can create a 
Calcite server with state. However I don’t see us adding a lot of other 
commands, or supporting other engines’ syntax (which would probably reference 
concepts that do not exist in Calcite). 

So the Babel module is probably a better place for this.  

Julian

> On Apr 2, 2020, at 4:27 AM, Danny Chan  wrote:
> 
> Dear , you are right that Calcite does not support ALTER TABLE yet.
> 
> ALTER TABLE is meaningful and necessary when the engine has a 
> Catalog/MetadataStore that can persist the table definitions. Calcite does 
> not support external metadata-store read/write, we only support static schema 
> read(from json).
> 
> Here is the full syntax for DDL so far [1]
> 
> [1] 
> https://github.com/apache/calcite/blob/master/server/src/main/codegen/includes/parserImpls.ftl
> 
> Best,
> Danny Chan
> 在 2020年3月27日 +0800 AM8:23，Antonio Gelameris 
> ，写道：
>> Hello to everyone!
>> I was trying to understand whether it's possible to parse ALTER TABLE
>> statements or not (like the ones supported in MYSQL).
>> After some hours googling, I have the feeling that according to
>> documentation (https://calcite.apache.org/docs/reference.html#ddl-extensions)
>> it's not possible (they're not mentioned), neither using
>> SqlDdlParserImpl.FACTORY in calcite-server, since the internal
>> representation required to describe a state change like this is too
>> different from the current one, am I right?
>> Thanks for your time and help!
>> ---
>> Antonio
>> 
>> --
>> 
>> 
>> 
>> 
>> Listed on SIX Swiss Exchange
>> 
>> 
>> Our websites: lmgroup.lastminute.com |
>>  lastminute.com  |
>> weg.de  | jetcost.com  |
>> volagratis.com  | rumbo.es  |
>> bravofly.com  | hotelscan 
>> | crocierissime 
>> 
>> 
>> This e-mail may contain
>> confidential and/or privileged information. If you are not the intended
>> recipient (or have received this e-mail in error) please notify the sender
>> immediately and destroy this e-mail. Any unauthorized copying, disclosure
>> or distribution of the material in this e-mail is strictly forbidden and
>> could be a crime. The contents of this email are not personal and therefore
>> the answers to this email may be known, within the limits allowed by
>> company policies, by the organization of the sender.

Re: TIMESTAMPDIFF and datetime subtraction can not work well with JDBC adapter

2020-04-02 Thread Julian Hyde

I think you have identified the issues correctly.

I prefer SqlDatetimeSubtractionOperator to SqlTimestampDiffFunction because it 
is standard and more powerful. But some work is needed to make SQL generation 
run smoothly. Especially with data types, as you point out. 

I would approach this by writing tests in RelToSqlConverterTest with the 
desired SQL for each database and change RelToSqlConverter to make them pass. 
Pragmatic, not very elegant. 

Julian

> On Apr 2, 2020, at 6:45 AM, TANG Wen-hui  
> wrote:
> 
> Dear All,
> 
> As for issue https://issues.apache.org/jira/browse/CALCITE-3312(I'm working 
> on this issue), I have noticed several problems. So I am writing this mail 
> for some advices.
> 
> 1.Calcite support the syntax like that "(datetime - datetime) 
> IntervalQualifier";
> A simple test in JdbcTest:
>  @Test public void testTimestampMinus() {
>CalciteAssert.that()
>.with(CalciteAssert.Config.JDBC_SCOTT)
>.query("select (date'2003-12-30' - date'2001-11-30') year from 
> JDBC_SCOTT.emp")
>.returns("");
>  }
> However, the result returns "+63" which is wrong. This is because that 
> RelToSqlConverter simply convert it to "SELECT (DATE '2003-12-30' - DATE 
> '2001-11-30') FROM "SCOTT"."EMP"" (see 
> SqlDatetimeSubtractionOperator#unparse).
> As I konw, most database systems cannot deal with this sitution properly, and 
> the standard format of datetime value expression is like that
> "
>  ::=
> 
> |   
> |   
> |   
>  ::=
> 
>  ::=
>  [  ]
>  ::=
> 
> | 
>  ::=
> AT 
>  ::=
> LOCAL
> | TIME ZONE 
> "[1]
> 
> 2.Calcite will translate SqlTimestampDiffFunction to 
> SqlDatetimeSubtractionOperator during the conversion from SqlNode to RexNode 
> for SqlTimestampDiffFunction(see TimestampDiffConvertlet#convertCall). But 
> the different return types between SqlTimestampDiffFunction whose return type 
> is SqlTypeName.BIGINT or SqlTypeName.INTEGER and 
> SqlDatetimeSubtractionOperator whose return type is IntervalSqlType introduce 
> the internal operator REINTERPRET and /INT(see makeCastIntervalToExact). The 
> internal operator should not be converted to sql. I did a test that I 
> overrided the "unparse" method for REINTERPRET and / and still got the wrong 
> result like above.
> 
> I have a few thoughts about this kind of situation, but I am not sure it goes 
> right way.
> 1.We can forbidden the conversion from Project to JdbcProject for the Project 
> which contains RexCall whose operator is SqlDatetimeSubtractionOperator and 
> operands both are datetime.
> 2.Or we can translate SqlDatetimeSubtractionOperator to 
> SqlTimestampDiffFunction or other similar functions during unparsing(for 
> example, MySQL supports TIMESTAMPDIFF rather than datetime subtraction), but 
> the internal operator and the situation that databases which support this 
> feature take different approaches to achieve this feature make thing 
> confusing.
> 
> Best wishes
> Wenhui Tang
> 
> [1] ISO/IEC CD 9075-2
> 
> 
> 
> 
> winifred.wenhui.t...@gmail.com

TIMESTAMPDIFF and datetime subtraction can not work well with JDBC adapter

2020-04-02 Thread TANG Wen-hui

Dear All,

As for issue https://issues.apache.org/jira/browse/CALCITE-3312(I'm working on 
this issue), I have noticed several problems. So I am writing this mail for 
some advices.

1.Calcite support the syntax like that "(datetime - datetime) 
IntervalQualifier";
 A simple test in JdbcTest:
  @Test public void testTimestampMinus() {
CalciteAssert.that()
.with(CalciteAssert.Config.JDBC_SCOTT)
.query("select (date'2003-12-30' - date'2001-11-30') year from 
JDBC_SCOTT.emp")
.returns("");
  }
However, the result returns "+63" which is wrong. This is because that 
RelToSqlConverter simply convert it to "SELECT (DATE '2003-12-30' - DATE 
'2001-11-30') FROM "SCOTT"."EMP"" (see SqlDatetimeSubtractionOperator#unparse).
As I konw, most database systems cannot deal with this sitution properly, and 
the standard format of datetime value expression is like that
"
 ::=

|   
|   
|   
 ::=

 ::=
 [  ]
 ::=

| 
 ::=
AT 
 ::=
LOCAL
| TIME ZONE 
"[1]

2.Calcite will translate SqlTimestampDiffFunction to 
SqlDatetimeSubtractionOperator during the conversion from SqlNode to RexNode 
for SqlTimestampDiffFunction(see TimestampDiffConvertlet#convertCall). But the 
different return types between SqlTimestampDiffFunction whose return type is 
SqlTypeName.BIGINT or SqlTypeName.INTEGER and SqlDatetimeSubtractionOperator 
whose return type is IntervalSqlType introduce the internal operator 
REINTERPRET and /INT(see makeCastIntervalToExact). The internal operator should 
not be converted to sql. I did a test that I overrided the "unparse" method for 
REINTERPRET and / and still got the wrong result like above.

I have a few thoughts about this kind of situation, but I am not sure it goes 
right way.
1.We can forbidden the conversion from Project to JdbcProject for the Project 
which contains RexCall whose operator is SqlDatetimeSubtractionOperator and 
operands both are datetime.
2.Or we can translate SqlDatetimeSubtractionOperator to 
SqlTimestampDiffFunction or other similar functions during unparsing(for 
example, MySQL supports TIMESTAMPDIFF rather than datetime subtraction), but 
the internal operator and the situation that databases which support this 
feature take different approaches to achieve this feature make thing confusing.

Best wishes
Wenhui Tang

[1] ISO/IEC CD 9075-2




winifred.wenhui.t...@gmail.com

Re: Calcite geode adapter query

2020-04-02 Thread Danny Chan

I believe few people in the community are familiar with geode, maybe you can 
ask the author of this module [1]

[1] https://github.com/tzolov

Best,
Danny Chan
在 2020年4月2日 +0800 PM6:41，aashish choudhary ，写道：
> Can anyone help me with this?
>
> With best regards,
> Ashish
>
> On Wed, Apr 1, 2020, 7:27 PM aashish choudhary 
> wrote:
>
> > Hi,
> >
> > I am trying to setup querying geode using calcite geode adapter. So far i
> > am successfully able to connect to geode cluster which is SSL enabled after
> > making few changes in the adapter and query data for regions which are
> > having flat hierarchy.
> >
> > However I am not able to run any query for region having Array list as
> > values and String as key. I am using Dbeaver to query following this link.
> > https://youtu.be/I_iaMgqdV3Q
> >
> > Can someone help me with this?
> >
> > With geode you can query those complex region in geode Pulse data browser.
> > You can do .entrySet or .entries to run queries like this.
> >
> > Also in Dbeaver in under Tables->Columns I can view fields as well for
> > regions having flat hierarchy but it doesn't show anything for regions
> > storing arrays.
> >
> >
> > With best regards,
> > Ashish
> >

Re: Asking help for parsing MySql Alter Table statements

2020-04-02 Thread Danny Chan

Dear , you are right that Calcite does not support ALTER TABLE yet.

ALTER TABLE is meaningful and necessary when the engine has a 
Catalog/MetadataStore that can persist the table definitions. Calcite does not 
support external metadata-store read/write, we only support static schema 
read(from json).

Here is the full syntax for DDL so far [1]

[1] 
https://github.com/apache/calcite/blob/master/server/src/main/codegen/includes/parserImpls.ftl

Best,
Danny Chan
在 2020年3月27日 +0800 AM8:23，Antonio Gelameris 
，写道：
> Hello to everyone!
> I was trying to understand whether it's possible to parse ALTER TABLE
> statements or not (like the ones supported in MYSQL).
> After some hours googling, I have the feeling that according to
> documentation (https://calcite.apache.org/docs/reference.html#ddl-extensions)
> it's not possible (they're not mentioned), neither using
> SqlDdlParserImpl.FACTORY in calcite-server, since the internal
> representation required to describe a state change like this is too
> different from the current one, am I right?
> Thanks for your time and help!
> ---
> Antonio
>
> --
>
>
>
>
> Listed on SIX Swiss Exchange
>
>
> Our websites: lmgroup.lastminute.com |
>  lastminute.com  |
> weg.de  | jetcost.com  |
> volagratis.com  | rumbo.es  |
> bravofly.com  | hotelscan 
> | crocierissime 
>
>
> This e-mail may contain
> confidential and/or privileged information. If you are not the intended
> recipient (or have received this e-mail in error) please notify the sender
> immediately and destroy this e-mail. Any unauthorized copying, disclosure
> or distribution of the material in this e-mail is strictly forbidden and
> could be a crime. The contents of this email are not personal and therefore
> the answers to this email may be known, within the limits allowed by
> company policies, by the organization of the sender.

Re: Draft board report for April 2020

2020-04-02 Thread Danny Chan

Thanks for the summary Stamatis ~

Sigh to see the activity levels decrease, that may because some of the most 
committers/PMC has less activity recently, such as me, personally I’m really 
busy these days for my Flink work. But I have always try to check the daily 
mailing list and give some help if I have time (on the weekend).

I think things would be better and better, Calcite is a great project and we 
deserve to have more active committers in the future ~

Best,
Danny Chan
在 2020年4月2日 +0800 AM6:08，Stamatis Zampetakis ，写道：
> Attached below is a draft of this month's board report. I plan to submit it
> on April 6. Please let me know if you have any additions or corrections.
>
> ## Description:
> Apache Calcite is a highly customizable framework for parsing and planning
> queries on data in a wide variety of formats. It allows database-like
> access,
> and in particular a SQL interface and advanced query optimization, for data
> not residing in a traditional database.
>
> Avatica is a sub-project within Calcite and provides a framework for
> building
> local and remote JDBC and ODBC database drivers. Avatica has an independent
> release schedule and its own repository.
>
> ## Issues:
> There are no issues requiring board attention.
>
> ## Membership Data:
> Apache Calcite was founded 2015-10-22 (4 years ago)
> There are currently 46 committers and 22 PMC members in this project.
> The Committer-to-PMC ratio is roughly 2:1.
>
> Community changes, past quarter:
> - No new PMC members. Last addition was Haisheng Yuan on 2019-11-11.
> - Feng Zhu was added as committer on 2020-02-29
>
> ## Project Activity:
> Avatica 1.16.0 was released in the middle of December, including numerous
> bug fixes and security improvements while the build system has been
> migrated from Maven to gradle.
>
> Calcite 1.22.0 was released at the beginning of March, including more than
> 250 resolved issues including long-awaited features such as support of SQL
> hints, important bug fixes in the core of the query planner, and many
> changes in build and test infrastructure. This release was the biggest in
> the history of the project and one of the most complicated to pull out due
> to the big number of contributions.
>
> An introductory talk about query planning and Calcite was given at the
> university of Cergy-Pontoise, Paris, France, in March. The audience was not
> aware of the existence of the project which shows that we should organize
> such events more often, at least in
> Europe, if we want to expand our community. [Do we have something else to
> add here?]
>
> Finally, we were informed that the SuperSQL project by Tencent uses Calcite
> in order to provide a unified entrance to various data platforms such as
> RDBMS, ElasticSearch, Hive, Flink, Spark, Presto, ClickHouse, etc.
>
> ## Community Health:
>
> Activity levels on mailing lists (-8%), git (-24%) and JIRA (opened -20%,
> closed
> -30%) have decreased in the first quarter of 2020. Most of the big changes
> that started during the last quarter of 2019 (and initiated a lot of
> activity) have
> been finalized and incorporated in the release of Calcite 1.22.0, making
> things
> calmer in the community. The big decrease of closed issues in JIRA may be
> explained by the frequency of releases that have slightly dropped. To the
> above it is worth adding the period of holidays in December and January
> that in general slows things down.
>
> Following the decreased activity in the lists, git, and JIRA it is normal
> to see
> the activity rates on pull requests drop (-20% opened, -31% closed) more or
> less
> for the same reasons. Although our backlog keeps increasing (~174 open
> pullvrequests), the community is very reactive commenting and reviewing
> almost every contribution.

Re: Calcite geode adapter query

2020-04-02 Thread aashish choudhary

Can anyone help me with this?

With best regards,
Ashish

On Wed, Apr 1, 2020, 7:27 PM aashish choudhary 
wrote:

> Hi,
>
> I am trying to setup querying geode using calcite geode adapter. So far i
> am successfully able to connect to geode cluster which is SSL enabled after
> making few changes in the adapter and query data for regions which are
> having flat hierarchy.
>
> However I am not able to run any query for region having Array list as
> values and String as key. I am using Dbeaver to query following this link.
> https://youtu.be/I_iaMgqdV3Q
>
> Can someone help me with this?
>
> With geode you can query those complex region in geode Pulse data browser.
> You can do .entrySet or .entries to run queries like this.
>
> Also in Dbeaver in under Tables->Columns I can view fields as well for
> regions having flat hierarchy but it doesn't show anything for regions
> storing arrays.
>
>
> With best regards,
> Ashish
>

Calcite-Master - Build # 1669 - Failure

2020-04-02 Thread Apache Jenkins Server

The Apache Jenkins build system has built Calcite-Master (build #1669)

Status: Failure

Check console output at https://builds.apache.org/job/Calcite-Master/1669/ to 
view the results.

[jira] [Created] (CALCITE-3893) Sql with GROUP_ID may generate wrong plan

2020-04-02 Thread Shuo Cheng (Jira)

Shuo Cheng created CALCITE-3893:
---

 Summary: Sql with GROUP_ID may generate wrong plan
 Key: CALCITE-3893
 URL: https://issues.apache.org/jira/browse/CALCITE-3893
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.22.0
Reporter: Shuo Cheng
 Fix For: 1.23.0


Consider the following SQL:
{code:java}
select 
  deptno, group_id() as g, count(*) as c
from emp 
group by grouping sets (deptno, (), ())
{code}
the plan after SqlToRel is:
{code:java}
LogicalUnion(all=[true])
  LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
  LogicalProject(DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
  LogicalProject(DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}
I'm afraid there's some semantic problems here. As `groups` of the aggregate is 
empty, then what is `$0` meaning in the project above the aggregate. Maybe that 
we want is:
{code:java}

LogicalUnion(all=[true])
  LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
  LogicalProject(DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
  LogicalProject(DEPTNO=[null])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
I noticed this is introduced by CALCITE-1824, cc [~donnyzone], 
[~vladimirsitnikov].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Calcite geode adapter query

Re: Draft board report for April 2020

Re: Asking help for parsing MySql Alter Table statements

Re: TIMESTAMPDIFF and datetime subtraction can not work well with JDBC adapter

TIMESTAMPDIFF and datetime subtraction can not work well with JDBC adapter

Re: Calcite geode adapter query

Re: Asking help for parsing MySql Alter Table statements

Re: Draft board report for April 2020

Re: Calcite geode adapter query

Calcite-Master - Build # 1669 - Failure

[jira] [Created] (CALCITE-3893) Sql with GROUP_ID may generate wrong plan

11 matches

Site Navigation

Mail list logo

Footer information