[jira] [Created] (CALCITE-3794) Return directly if there is no pulled up predicate

[Chunwei Lei (Jira)]

Chunwei Lei created CALCITE-3794:


 Summary: Return directly if there is no pulled up predicate
 Key: CALCITE-3794
 URL: https://issues.apache.org/jira/browse/CALCITE-3794
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.21.0
Reporter: Chunwei Lei
Assignee: Chunwei Lei
 Fix For: 1.22.0


If there is no pulled up predicates, it can not simplify anything in 
{{simplifyUsingPredicates}}[1]. So it'd better return directly to reduce 
overhead.

[1][https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L1572-L1586]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-3795) Promote the Project digest to only print "0" instead of "0..0" when there is only one trivial field

[Danny Chen (Jira)]

Danny Chen created CALCITE-3795:
---

 Summary: Promote the Project digest to only print "0" instead of 
"0..0" when there is only one trivial field
 Key: CALCITE-3795
 URL: https://issues.apache.org/jira/browse/CALCITE-3795
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.22.0
Reporter: Danny Chen
Assignee: Danny Chen
 Fix For: 1.22.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Not able use PostgresDialect.

[Danny Chan]

If you want to parse the sql in PostgreSQL dialect, there is no way to do
that now. You need a new SqlConformance actually.

Shivraj Singh 于2020年2月13日 周四下午3:50写道：

> Hi, I want to parse the query of PostgreSQL via Calcite and while I am
> parsing the query using the calcite parser it gives me exception for system
> keyword.
>
> This is the program:
> This is the query :
> " select system from my_table;"
>
> val  parser:SqlParser = SqlParser.create(query)
> parser.parseQuery()
>
> Exception I got.
> org.apache.calcite.sql.parser.SqlParseException: Encountered "*system*" at
> line 1, column 8.
> Was expecting one of:
> "ABS" ...
> "ALL" ...
>
> And I also got exception for this scenario to :
> query :
> INSERT INTO my_table(id,system_state)
>  VALUES (?, ?::system_state_type)
>
> exception I got:
> org.apache.calcite.sql.parser.SqlParseException: Encountered "*:*" at line.
> Was expecting one of:
> 
> "EXCEPT" ...
>
> Please help me
> 1. How to parse these queries?
> 2. Can we parse query using  PostgresqlSqlDialect, if yes then how.
>
> Thanks.
>
> *Shivraj Singh*
> Software Consultant
> Knoldus Inc. 
> +91-8800782123
> Canada - USA - India - Singapore
>  
>  
>

Re: Not able use PostgresDialect.

[Julian Hyde]

Note that Calcite's default parser only parses SQL in Calcite's
dialect (with a little leeway such as choice of quoting characters).

To parse other dialects, use the Babel parser. It can handle (or could
handle) SQL extensions that are not in standard SQL and which we don't
want to bring into Calcite SQL. PostgreSQL's "::" cast operator is an
example of that.

On Thu, Feb 13, 2020 at 4:00 AM Danny Chan  wrote:
>
> If you want to parse the sql in PostgreSQL dialect, there is no way to do
> that now. You need a new SqlConformance actually.
>
> Shivraj Singh 于2020年2月13日 周四下午3:50写道：
>
> > Hi, I want to parse the query of PostgreSQL via Calcite and while I am
> > parsing the query using the calcite parser it gives me exception for system
> > keyword.
> >
> > This is the program:
> > This is the query :
> > " select system from my_table;"
> >
> > val  parser:SqlParser = SqlParser.create(query)
> > parser.parseQuery()
> >
> > Exception I got.
> > org.apache.calcite.sql.parser.SqlParseException: Encountered "*system*" at
> > line 1, column 8.
> > Was expecting one of:
> > "ABS" ...
> > "ALL" ...
> >
> > And I also got exception for this scenario to :
> > query :
> > INSERT INTO my_table(id,system_state)
> >  VALUES (?, ?::system_state_type)
> >
> > exception I got:
> > org.apache.calcite.sql.parser.SqlParseException: Encountered "*:*" at line.
> > Was expecting one of:
> > 
> > "EXCEPT" ...
> >
> > Please help me
> > 1. How to parse these queries?
> > 2. Can we parse query using  PostgresqlSqlDialect, if yes then how.
> >
> > Thanks.
> >
> > *Shivraj Singh*
> > Software Consultant
> > Knoldus Inc. 
> > +91-8800782123
> > Canada - USA - India - Singapore
> >  
> >  
> >

RelNode rewrite

[Christian Beikov]

Hello,

I'm trying to generate SQL for a trigger that requires a little rewrite 
of a RelNode structure and I wanted to know if anyone could help me 
figure out how to do a few things


 * I need to swap left and right of a LogicalJoin. Doing that messes up
   predicates and projections though. Is there a utility I could use or
   how would I do that?
 * I need to introduce an "alias" like "NEW" into a RelNode for which I
   want to create a RexNode "NEW.key1" for. I want to add a WHERE
   condition based on the NEW alias. Not sure how I would do that
   though. On SqlNode level only maybe?
 * Can I somehow determine what columns of a table are unique or part
   of the primary key? I want to create a predicate like (NEW.key1,
   NEW.key2) = (existing.key1, existing.key2)

Thanks in advance!

Regards,

Christian

Re: RelNode rewrite

[Jess Balint]

Hey Christian,

On Thu, Feb 13, 2020 at 6:30 PM Christian Beikov 
wrote:

> Hello,
>
> I'm trying to generate SQL for a trigger that requires a little rewrite
> of a RelNode structure and I wanted to know if anyone could help me
> figure out how to do a few things
>
>   * I need to swap left and right of a LogicalJoin. Doing that messes up
> predicates and projections though. Is there a utility I could use or
> how would I do that?
>

You can use JoinCommuteRule, or check it's code for your needs.


>   * I need to introduce an "alias" like "NEW" into a RelNode for which I
> want to create a RexNode "NEW.key1" for. I want to add a WHERE
> condition based on the NEW alias. Not sure how I would do that
> though. On SqlNode level only maybe?
>

I don't understand what you're after here. The WHERE condition will be a
filter node. Can you just add it with attaching something to the rel node?


>   * Can I somehow determine what columns of a table are unique or part
> of the primary key? I want to create a predicate like (NEW.key1,
> NEW.key2) = (existing.key1, existing.key2)
>

You can use RelMetadataQuery.

Best,

Jess


>
> Thanks in advance!
>
> Regards,
>
> Christian
>
>

Calcite-Master - Build # 1605 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #1605)

Status: Still Failing

Check console output at https://builds.apache.org/job/Calcite-Master/1605/ to 
view the results.

Build failed in Jenkins: Calcite-Snapshots #113

[Apache Jenkins Server]

See 


Changes:

[rubenql] [CALCITE-3783] PruneEmptyRules#JOIN_RIGHT_INSTANCE wrong behavior for

[h.yuan] [CALCITE-3785] HepPlanner.belongToDag() doesn't have to use

[jhyde] Add RelBuilder.transform, which allows you to clone a RelBuilder with

[jhyde] [CALCITE-3763] RelBuilder.aggregate should prune unused fields from the

[jhyde] [CALCITE-3774] In RelBuilder and ProjectMergeRule, prevent merges when

[25229979+asereda-gs] [CALCITE-2442] Remove .toDelete cassandra temp folder on 
Windows after

[yuzhao.cyz] [CALCITE-3792] Remove the generic type declaration of method

[yuzhao.cyz] [CALCITE-3793] AssertionError throws for planner digest cache key 
when

[yuzhao.cyz] [CALCITE-3795] Promote the Project digest to only print 0 instead 
of


--
[...truncated 106.55 KB...]

> Configure project :example:csv
Evaluating project ':example:csv' using build file 
'

> Configure project :example:function
Evaluating project ':example:function' using build file 
'
All projects evaluated.
Selected primary task 'publish' from project :
Tasks to be executed: [task ':publish', task ':linq4j:compileJava', task 
':core:compileKotlin', task ':core:fmppMain', task ':core:javaCCMain', task 
':core:versionClass', task ':core:compileJava', task ':babel:compileKotlin', 
task ':babel:fmppMain', task ':babel:javaCCMain', task ':babel:compileJava', 
task ':babel:processResources', task ':babel:classes', task 
':babel:inspectClassesForKotlinIC', task ':babel:jar', task 
':babel:generateMetadataFileForBabelPublication', task 
':babel:generatePomFileForBabelPublication', task ':gitProps', task ':rat', 
task ':validateBeforeBuildingReleaseArtifacts', task 
':validateNexusCredentials', task ':babel:initializeNexusStagingRepository', 
task ':babel:javadoc', task ':babel:javadocJar', task ':babel:sourcesJar', task 
':babel:publishBabelPublicationToNexusRepository', task ':babel:publish', task 
':cassandra:compileJava', task ':cassandra:processResources', task 
':cassandra:classes', task ':cassandra:jar', task 
':cassandra:generateMetadataFileForCassandraPublication', task 
':cassandra:generatePomFileForCassandraPublication', task 
':cassandra:initializeNexusStagingRepository', task ':cassandra:javadoc', task 
':cassandra:javadocJar', task ':cassandra:sourcesJar', task 
':cassandra:publishCassandraPublicationToNexusRepository', task 
':cassandra:publish', task ':core:processResources', task ':core:classes', task 
':core:inspectClassesForKotlinIC', task ':core:jar', task 
':core:generateMetadataFileForCorePublication', task 
':core:generatePomFileForCorePublication', task 
':core:initializeNexusStagingRepository', task ':core:javadoc', task 
':core:javadocJar', task ':core:sourcesJar', task ':core:compileTestKotlin', 
task ':core:fmppTest', task ':core:javaCCTest', task ':core:compileTestJava', 
task ':core:processTestResources', task ':core:testClasses', task 
':core:testJar', task ':core:publishCorePublicationToNexusRepository', task 
':core:publish', task ':druid:compileJava', task ':druid:processResources', 
task ':druid:classes', task ':druid:jar', task 
':druid:generateMetadataFileForDruidPublication', task 
':druid:generatePomFileForDruidPublication', task 
':druid:initializeNexusStagingRepository', task ':druid:javadoc', task 
':druid:javadocJar', task ':druid:sourcesJar', task 
':druid:publishDruidPublicationToNexusRepository', task ':druid:publish', task 
':elasticsearch:compileJava', task ':elasticsearch:processResources', task 
':elasticsearch:classes', task ':elasticsearch:jar', task 
':elasticsearch:generateMetadataFileForElasticsearchPublication', task 
':elasticsearch:generatePomFileForElasticsearchPublication', task 
':elasticsearch:initializeNexusStagingRepository', task 
':elasticsearch:javadoc', task ':elasticsearch:javadocJar', task 
':elasticsearch:sourcesJar', task 
':elasticsearch:publishElasticsearchPublicationToNexusRepository', task 
':elasticsearch:publish', task ':example:csv:compileJava', task 
':file:compileJava', task ':file:processResources', task ':file:classes', task 
':file:jar', task ':file:generateMetadataFileForFilePublication', task 
':file:generatePomFileForFilePublication', task 
':file:initializeNexusStagingRepository', task ':file:javadoc', task 
':file:javadocJar', task ':file:sourcesJar', task 
':file:publishFilePublicationToNexusRepository', task ':file:publish', task 
':geode:compileJava', task ':geode:processResources', task ':geode:classes', 
task ':geode:jar', task ':geode:generateMetadataFileForGeodePublication', task 
':geode:generatePomFileForGeodePublication', task 
':geode:initializeNexusStagingRepository', task ':geode:javadoc', task 
':geode:javadocJar', task ':geode:sourcesJar', task 
':geo

Calcite-Master - Build # 1607 - Failure

[Apache Jenkins Server]

The Apache Jenkins build system has built Calcite-Master (build #1607)

Status: Failure

Check console output at https://builds.apache.org/job/Calcite-Master/1607/ to 
view the results.

[jira] [Created] (CALCITE-3796) Support Array operator <, <=, >, >=

[Rui Wang (Jira)]

Rui Wang created CALCITE-3796:
-

 Summary: Support Array operator <, <=, >, >=
 Key: CALCITE-3796
 URL: https://issues.apache.org/jira/browse/CALCITE-3796
 Project: Calcite
  Issue Type: Improvement
Reporter: Rui Wang
Assignee: Rui Wang


Seems like Calcite has an  =, <> support for array type. But it does not have 
<, <=, >, >= support.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ DISCUSS ] Revert change: CALCITE-3713 Remove column names from Project#digest

[Danny Chan]

The change in CALCITE-3713 make a radical change to the plan digest, but it 
only brings in little promotion(at least from Calcite tests and Flink cases)

During the upgrade of Flink Calcite version to 1.22.0-SNAPSHOT, the 2000+ plan 
changes to due to this patch, it is hard for me to figure out the reason of the 
change: because of this patch, or because of other bug-fix/promotion.

Even though I change the plan verification to EXPLAIN_ATTRIBUTE, there are 
still 500+ plan diffs.

This makes the Flink Calcite upgrade almost impossible to go on and I’m 
burdened out of it.

So, PLEASE, if possible, can we REVERT this patch, 2000+ plan changes for no 
promotion(useless) (at lease for Flink).

I did show my options when discussing if the patch should be merged, me and 
Julian all gave -1 for that but the code still merged, sigh !

[1] 
https://ponymail-vm.apache.org/_GUI_/thread.html/54bf3ed733eb7e725ce3ea397334aad8f1323ead13e450b1753b1521%40%3Cdev.calcite.apache.org%3E

Best,
Danny Chan

Re: Not able use PostgresDialect.

[Shivraj Singh]

I tried for BABEL conformance but it also doesn't work.
This is the way I used it:

val sqlParserConfig = SqlParser.configBuilder()
  .setParserFactory(SqlParserImpl.FACTORY)
  .setConformance(SqlConformanceEnum.BABEL)
  .setConfig(sqlParserA)
  .build()

SqlParser.create(new
SourceStringReader(sql),sqlParserConfig).parseQuery()

Am I doing something wrong?

*Shivraj Singh*
Software Consultant
Knoldus Inc. 
+91-8800782123
Canada - USA - India - Singapore
 
 


On Thu, Feb 13, 2020 at 11:42 PM Julian Hyde  wrote:

> Note that Calcite's default parser only parses SQL in Calcite's
> dialect (with a little leeway such as choice of quoting characters).
>
> To parse other dialects, use the Babel parser. It can handle (or could
> handle) SQL extensions that are not in standard SQL and which we don't
> want to bring into Calcite SQL. PostgreSQL's "::" cast operator is an
> example of that.
>
> On Thu, Feb 13, 2020 at 4:00 AM Danny Chan  wrote:
> >
> > If you want to parse the sql in PostgreSQL dialect, there is no way to do
> > that now. You need a new SqlConformance actually.
> >
> > Shivraj Singh 于2020年2月13日 周四下午3:50写道：
> >
> > > Hi, I want to parse the query of PostgreSQL via Calcite and while I am
> > > parsing the query using the calcite parser it gives me exception for
> system
> > > keyword.
> > >
> > > This is the program:
> > > This is the query :
> > > " select system from my_table;"
> > >
> > > val  parser:SqlParser = SqlParser.create(query)
> > > parser.parseQuery()
> > >
> > > Exception I got.
> > > org.apache.calcite.sql.parser.SqlParseException: Encountered
> "*system*" at
> > > line 1, column 8.
> > > Was expecting one of:
> > > "ABS" ...
> > > "ALL" ...
> > >
> > > And I also got exception for this scenario to :
> > > query :
> > > INSERT INTO my_table(id,system_state)
> > >  VALUES (?, ?::system_state_type)
> > >
> > > exception I got:
> > > org.apache.calcite.sql.parser.SqlParseException: Encountered "*:*" at
> line.
> > > Was expecting one of:
> > > 
> > > "EXCEPT" ...
> > >
> > > Please help me
> > > 1. How to parse these queries?
> > > 2. Can we parse query using  PostgresqlSqlDialect, if yes then how.
> > >
> > > Thanks.
> > >
> > > *Shivraj Singh*
> > > Software Consultant
> > > Knoldus Inc. 
> > > +91-8800782123
> > > Canada - USA - India - Singapore
> > >  <
> https://twitter.com/Knolspeak>
> > >   >
> > >
>

Re: [ DISCUSS ] Revert change: CALCITE-3713 Remove column names from Project#digest

[Vladimir Sitnikov]

The subject of the mail does not match the body.

Please double check and ensure you mention a single change.


PS if you use explain_digest attributes, it means you expect it might
change due to implementation details.

Vladimir

Re: Not able use PostgresDialect.

[Danny Chan]

You should extend the babel parser by yourself.

Shivraj Singh 于2020年2月14日 周五下午2:33写道：

> I tried for BABEL conformance but it also doesn't work.
> This is the way I used it:
>
> val sqlParserConfig = SqlParser.configBuilder()
>   .setParserFactory(SqlParserImpl.FACTORY)
>   .setConformance(SqlConformanceEnum.BABEL)
>   .setConfig(sqlParserA)
>   .build()
>
> SqlParser.create(new
> SourceStringReader(sql),sqlParserConfig).parseQuery()
>
> Am I doing something wrong?
>
> *Shivraj Singh*
> Software Consultant
> Knoldus Inc. 
> +91-8800782123
> Canada - USA - India - Singapore
>  
>  
>
>
> On Thu, Feb 13, 2020 at 11:42 PM Julian Hyde  wrote:
>
> > Note that Calcite's default parser only parses SQL in Calcite's
> > dialect (with a little leeway such as choice of quoting characters).
> >
> > To parse other dialects, use the Babel parser. It can handle (or could
> > handle) SQL extensions that are not in standard SQL and which we don't
> > want to bring into Calcite SQL. PostgreSQL's "::" cast operator is an
> > example of that.
> >
> > On Thu, Feb 13, 2020 at 4:00 AM Danny Chan  wrote:
> > >
> > > If you want to parse the sql in PostgreSQL dialect, there is no way to
> do
> > > that now. You need a new SqlConformance actually.
> > >
> > > Shivraj Singh 于2020年2月13日 周四下午3:50写道：
> > >
> > > > Hi, I want to parse the query of PostgreSQL via Calcite and while I
> am
> > > > parsing the query using the calcite parser it gives me exception for
> > system
> > > > keyword.
> > > >
> > > > This is the program:
> > > > This is the query :
> > > > " select system from my_table;"
> > > >
> > > > val  parser:SqlParser = SqlParser.create(query)
> > > > parser.parseQuery()
> > > >
> > > > Exception I got.
> > > > org.apache.calcite.sql.parser.SqlParseException: Encountered
> > "*system*" at
> > > > line 1, column 8.
> > > > Was expecting one of:
> > > > "ABS" ...
> > > > "ALL" ...
> > > >
> > > > And I also got exception for this scenario to :
> > > > query :
> > > > INSERT INTO my_table(id,system_state)
> > > >  VALUES (?, ?::system_state_type)
> > > >
> > > > exception I got:
> > > > org.apache.calcite.sql.parser.SqlParseException: Encountered "*:*" at
> > line.
> > > > Was expecting one of:
> > > > 
> > > > "EXCEPT" ...
> > > >
> > > > Please help me
> > > > 1. How to parse these queries?
> > > > 2. Can we parse query using  PostgresqlSqlDialect, if yes then how.
> > > >
> > > > Thanks.
> > > >
> > > > *Shivraj Singh*
> > > > Software Consultant
> > > > Knoldus Inc. 
> > > > +91-8800782123
> > > > Canada - USA - India - Singapore
> > > >  <
> > https://twitter.com/Knolspeak>
> > > >  <
> https://blog.knoldus.com/
> > >
> > > >
> >
>

Re: [ DISCUSS ] Revert change: CALCITE-3713 Remove column names from Project#digest

[Danny Chan]

> if you use explain_digest attributes, it means you expect it might
change due to implementation details.

That is what you just defined that, why a digest should be changed just because 
of the implementation.

Best,
Danny Chan
在 2020年2月14日 +0800 PM2:54，Vladimir Sitnikov ，写道：
> The subject of the mail does not match the body.
>
> Please double check and ensure you mention a single change.
>
>
> PS if you use explain_digest attributes, it means you expect it might
> change due to implementation details.
>
> Vladimir

Re: Not able use PostgresDialect.

[Shivraj Singh]

Thanks @Danny Chan ,

Please tell me one more thing:
Just curious what does the PostgresSqlDialect accomplish in Calcite, if not
to parse postgres sqls?
*Shivraj Singh*
Software Consultant
Knoldus Inc. 
+91-8800782123
Canada - USA - India - Singapore
 
 


On Fri, Feb 14, 2020 at 12:26 PM Danny Chan  wrote:

> You should extend the babel parser by yourself.
>
> Shivraj Singh 于2020年2月14日 周五下午2:33写道：
>
> > I tried for BABEL conformance but it also doesn't work.
> > This is the way I used it:
> >
> > val sqlParserConfig = SqlParser.configBuilder()
> >   .setParserFactory(SqlParserImpl.FACTORY)
> >   .setConformance(SqlConformanceEnum.BABEL)
> >   .setConfig(sqlParserA)
> >   .build()
> >
> > SqlParser.create(new
> > SourceStringReader(sql),sqlParserConfig).parseQuery()
> >
> > Am I doing something wrong?
> >
> > *Shivraj Singh*
> > Software Consultant
> > Knoldus Inc. 
> > +91-8800782123
> > Canada - USA - India - Singapore
> >   >
> >  
> >
> >
> > On Thu, Feb 13, 2020 at 11:42 PM Julian Hyde  wrote:
> >
> > > Note that Calcite's default parser only parses SQL in Calcite's
> > > dialect (with a little leeway such as choice of quoting characters).
> > >
> > > To parse other dialects, use the Babel parser. It can handle (or could
> > > handle) SQL extensions that are not in standard SQL and which we don't
> > > want to bring into Calcite SQL. PostgreSQL's "::" cast operator is an
> > > example of that.
> > >
> > > On Thu, Feb 13, 2020 at 4:00 AM Danny Chan 
> wrote:
> > > >
> > > > If you want to parse the sql in PostgreSQL dialect, there is no way
> to
> > do
> > > > that now. You need a new SqlConformance actually.
> > > >
> > > > Shivraj Singh 于2020年2月13日 周四下午3:50写道：
> > > >
> > > > > Hi, I want to parse the query of PostgreSQL via Calcite and while I
> > am
> > > > > parsing the query using the calcite parser it gives me exception
> for
> > > system
> > > > > keyword.
> > > > >
> > > > > This is the program:
> > > > > This is the query :
> > > > > " select system from my_table;"
> > > > >
> > > > > val  parser:SqlParser = SqlParser.create(query)
> > > > > parser.parseQuery()
> > > > >
> > > > > Exception I got.
> > > > > org.apache.calcite.sql.parser.SqlParseException: Encountered
> > > "*system*" at
> > > > > line 1, column 8.
> > > > > Was expecting one of:
> > > > > "ABS" ...
> > > > > "ALL" ...
> > > > >
> > > > > And I also got exception for this scenario to :
> > > > > query :
> > > > > INSERT INTO my_table(id,system_state)
> > > > >  VALUES (?, ?::system_state_type)
> > > > >
> > > > > exception I got:
> > > > > org.apache.calcite.sql.parser.SqlParseException: Encountered "*:*"
> at
> > > line.
> > > > > Was expecting one of:
> > > > > 
> > > > > "EXCEPT" ...
> > > > >
> > > > > Please help me
> > > > > 1. How to parse these queries?
> > > > > 2. Can we parse query using  PostgresqlSqlDialect, if yes then how.
> > > > >
> > > > > Thanks.
> > > > >
> > > > > *Shivraj Singh*
> > > > > Software Consultant
> > > > > Knoldus Inc. 
> > > > > +91-8800782123
> > > > > Canada - USA - India - Singapore
> > > > >  <
> > > https://twitter.com/Knolspeak>
> > > > >  <
> > https://blog.knoldus.com/
> > > >
> > > > >
> > >
> >
>