rewrite an optimized RelNode

[Yang Liu]

In Calcite, after optimization provided by the default VolcanoPlanner, we can 
get an optimized RelNode, but can we have a further optimization? For example I 
want to add an ElasticsearchSort or something like that to limit the dataset we 
will handle.

Someone suggests we can define a RelOptRule, but since VolcanoPlanner handles 
the optimization in dynamic programming way, not sure if the rule can be 
applied in the right order.

Any ideas are welcome~

Build failed in Jenkins: Calcite-Snapshots #123

[Apache Jenkins Server]

See 


Changes:

[h.yuan] [CALCITE-3821] RelOptUtil::containsMultisetOrWindowedAgg doesn't really


--
[...truncated 106.87 KB...]

> Configure project :example:csv
Evaluating project ':example:csv' using build file 
'

> Configure project :example:function
Evaluating project ':example:function' using build file 
'
All projects evaluated.
Selected primary task 'publish' from project :
Tasks to be executed: [task ':publish', task ':linq4j:compileJava', task 
':core:compileKotlin', task ':core:fmppMain', task ':core:javaCCMain', task 
':core:versionClass', task ':core:compileJava', task ':babel:compileKotlin', 
task ':babel:fmppMain', task ':babel:javaCCMain', task ':babel:compileJava', 
task ':babel:processResources', task ':babel:classes', task 
':babel:inspectClassesForKotlinIC', task ':babel:jar', task 
':babel:generateMetadataFileForBabelPublication', task 
':babel:generatePomFileForBabelPublication', task ':gitProps', task ':rat', 
task ':validateBeforeBuildingReleaseArtifacts', task 
':validateNexusCredentials', task ':babel:initializeNexusStagingRepository', 
task ':babel:javadoc', task ':babel:javadocJar', task ':babel:sourcesJar', task 
':babel:publishBabelPublicationToNexusRepository', task ':babel:publish', task 
':cassandra:compileJava', task ':cassandra:processResources', task 
':cassandra:classes', task ':cassandra:jar', task 
':cassandra:generateMetadataFileForCassandraPublication', task 
':cassandra:generatePomFileForCassandraPublication', task 
':cassandra:initializeNexusStagingRepository', task ':cassandra:javadoc', task 
':cassandra:javadocJar', task ':cassandra:sourcesJar', task 
':cassandra:publishCassandraPublicationToNexusRepository', task 
':cassandra:publish', task ':core:processResources', task ':core:classes', task 
':core:inspectClassesForKotlinIC', task ':core:jar', task 
':core:generateMetadataFileForCorePublication', task 
':core:generatePomFileForCorePublication', task 
':core:initializeNexusStagingRepository', task ':core:javadoc', task 
':core:javadocJar', task ':core:sourcesJar', task ':core:compileTestKotlin', 
task ':core:fmppTest', task ':core:javaCCTest', task ':core:compileTestJava', 
task ':core:processTestResources', task ':core:testClasses', task 
':core:testJar', task ':core:publishCorePublicationToNexusRepository', task 
':core:publish', task ':druid:compileJava', task ':druid:processResources', 
task ':druid:classes', task ':druid:jar', task 
':druid:generateMetadataFileForDruidPublication', task 
':druid:generatePomFileForDruidPublication', task 
':druid:initializeNexusStagingRepository', task ':druid:javadoc', task 
':druid:javadocJar', task ':druid:sourcesJar', task 
':druid:publishDruidPublicationToNexusRepository', task ':druid:publish', task 
':elasticsearch:compileJava', task ':elasticsearch:processResources', task 
':elasticsearch:classes', task ':elasticsearch:jar', task 
':elasticsearch:generateMetadataFileForElasticsearchPublication', task 
':elasticsearch:generatePomFileForElasticsearchPublication', task 
':elasticsearch:initializeNexusStagingRepository', task 
':elasticsearch:javadoc', task ':elasticsearch:javadocJar', task 
':elasticsearch:sourcesJar', task 
':elasticsearch:publishElasticsearchPublicationToNexusRepository', task 
':elasticsearch:publish', task ':example:csv:compileJava', task 
':file:compileJava', task ':file:processResources', task ':file:classes', task 
':file:jar', task ':file:generateMetadataFileForFilePublication', task 
':file:generatePomFileForFilePublication', task 
':file:initializeNexusStagingRepository', task ':file:javadoc', task 
':file:javadocJar', task ':file:sourcesJar', task 
':file:publishFilePublicationToNexusRepository', task ':file:publish', task 
':geode:compileJava', task ':geode:processResources', task ':geode:classes', 
task ':geode:jar', task ':geode:generateMetadataFileForGeodePublication', task 
':geode:generatePomFileForGeodePublication', task 
':geode:initializeNexusStagingRepository', task ':geode:javadoc', task 
':geode:javadocJar', task ':geode:sourcesJar', task 
':geode:publishGeodePublicationToNexusRepository', task ':geode:publish', task 
':kafka:compileJava', task ':kafka:processResources', task ':kafka:classes', 
task ':kafka:jar', task ':kafka:generateMetadataFileForKafkaPublication', task 
':kafka:generatePomFileForKafkaPublication', task 
':kafka:initializeNexusStagingRepository', task ':kafka:javadoc', task 
':kafka:javadocJar', task ':kafka:sourcesJar', task 
':kafka:publishKafkaPublicationToNexusRepository', task ':kafka:publish', task 
':linq4j:processResources', task ':linq4j:classes', task ':linq4j:jar', task 
':linq4j:generateMetadataFileForLinq4jPublication', task 

Re: Why can't RexLiteral.getValue2() get a double/float value?

[Julian Hyde]

As the java doc says, RexLiteral.getValue2 returns values in the form that the 
calculator builder needs them. That may not be the form that you need them. In 
which case, don’t use that method.

If you want a Double, have you tried literal.getValueAs(Double.class)?

(We don’t tend to use Float and Double much because when we’re compiling 
queries we can’t afford any loss of precision. If the user typed 1.1 we want 
exactly 1.1, whereas Decimal might be something like 1.0987.)

> On Mar 4, 2020, at 9:43 PM, Xiangwei Wei  wrote:
> 
> In RexLiteral.getValue2(), it treats Decimal just by `return
> getValueAs(Long.class);`.
> In this instance, we can't get a correct double/float value because it's
> considered as Decimal type here. Is this a problem or is there a reason
> here?
> 
> -- 
> Best,
> Xiangwei Wei

Re: SQL has order by, there will be one more column(the sort column) in result plan

[JiaTao Tao]

Seems `root.project()` works, thanks a lot!


Regards!

Aron Tao


JiaTao Tao  于2020年3月5日周四 上午11:05写道：

> Thanks, Julian
>
> I can see the fields in RelRoot, and I'm a little confused, I want to
> use RelVisitor to visit the RelNode, and here comes the RelRoot, any advice
> to deal with RelRoot when using RelVisitor?
>
>
> Regards!
>
> Aron Tao
>
>
> Julian Hyde  于2020年3月5日周四 上午2:03写道：
>
>> There is a class RelRoot that wraps the top-most RelNode, and projects
>> away columns that are only used for sorting. Read through
>> https://issues.apache.org/jira/browse/CALCITE-2999 <
>> https://issues.apache.org/jira/browse/CALCITE-2999>, where someone had
>> the same misunderstanding that you have.
>>
>>
>> > On Mar 4, 2020, at 3:43 AM, JiaTao Tao  wrote:
>> >
>> > SQL;
>> > ```
>> > SELECT u.ID, u.NAME
>> > FROM USERS u
>> > ORDER BY u.AGE
>> > ```
>> >
>> > PLAN:
>> > ```
>> > LogicalSort(sort0=[$2], dir0=[ASC])
>> >  LogicalProject(ID=[$0], NAME=[$1], AGE=[$2])
>> >EnumerableTableScan(table=[[RECR, USERS]])
>> > ```
>> > Plan's row type: RecordType(INTEGER ID, VARCHAR NAME, INTEGER AGE)
>> >
>> > But the SQL only project ID and NAME, there should be a LogicalProject
>> > (ID, NAME) on LogicalSort. Is this by design? And what about this extra
>> > column?
>> >
>> >
>> >
>> > Regards!
>> >
>> > Aron Tao
>>
>>

Why can't RexLiteral.getValue2() get a double/float value?

[Xiangwei Wei]

In RexLiteral.getValue2(), it treats Decimal just by `return
getValueAs(Long.class);`.
In this instance, we can't get a correct double/float value because it's
considered as Decimal type here. Is this a problem or is there a reason
here?

-- 
Best,
Xiangwei Wei

Build failed in Jenkins: Calcite-Snapshots #122

[Apache Jenkins Server]

See 


Changes:

[yuzhao.cyz] [CALCITE-3818] Upgrade Avatica version to 1.16.0

[yuzhao.cyz] [CALCITE-3822] Source distribution must not contain fonts under 
SIL OFL

[yuzhao.cyz] [CALCITE-3830] The ‘approximate’ field should be considered when

[yuzhao.cyz] [CALCITE-3826] UPDATE assigns wrong type to bind variables

[yuzhao.cyz] Following CALCITE-3769: Add BindableTableScanRule into the default

[yuzhao.cyz] Stop building zip archives when building using gradle

[yuzhao.cyz] [CALCITE-3808] Release Calcite 1.22.0

[yuzhao.cyz] Improve 1.22.0 release note and document

[yuzhao.cyz] Prepare for next development iteration

[yuzhao.cyz] [CALCITE-3817] VolcanoPlanner does not remove the entry in 
ruleNames

[yuzhao.cyz] [CALCITE-3734] MySQL JDBC rewrite is producing queries with CHAR 
with

[yuzhao.cyz] [CALCITE-3825] Split AbstractMaterializedViewRule into multiple 
classes

[yuzhao.cyz] [CALCITE-3824] JoinProjectTransposeRule should skip Projects 
containing

[yuzhao.cyz] [CALCITE-3825] Split AbstractMaterializedViewRule into multiple 
classes


--
[...truncated 107.18 KB...]

> Configure project :example:csv
Evaluating project ':example:csv' using build file 
'

> Configure project :example:function
Evaluating project ':example:function' using build file 
'
All projects evaluated.
Selected primary task 'publish' from project :
Tasks to be executed: [task ':publish', task ':linq4j:compileJava', task 
':core:compileKotlin', task ':core:fmppMain', task ':core:javaCCMain', task 
':core:versionClass', task ':core:compileJava', task ':babel:compileKotlin', 
task ':babel:fmppMain', task ':babel:javaCCMain', task ':babel:compileJava', 
task ':babel:processResources', task ':babel:classes', task 
':babel:inspectClassesForKotlinIC', task ':babel:jar', task 
':babel:generateMetadataFileForBabelPublication', task 
':babel:generatePomFileForBabelPublication', task ':gitProps', task ':rat', 
task ':validateBeforeBuildingReleaseArtifacts', task 
':validateNexusCredentials', task ':babel:initializeNexusStagingRepository', 
task ':babel:javadoc', task ':babel:javadocJar', task ':babel:sourcesJar', task 
':babel:publishBabelPublicationToNexusRepository', task ':babel:publish', task 
':cassandra:compileJava', task ':cassandra:processResources', task 
':cassandra:classes', task ':cassandra:jar', task 
':cassandra:generateMetadataFileForCassandraPublication', task 
':cassandra:generatePomFileForCassandraPublication', task 
':cassandra:initializeNexusStagingRepository', task ':cassandra:javadoc', task 
':cassandra:javadocJar', task ':cassandra:sourcesJar', task 
':cassandra:publishCassandraPublicationToNexusRepository', task 
':cassandra:publish', task ':core:processResources', task ':core:classes', task 
':core:inspectClassesForKotlinIC', task ':core:jar', task 
':core:generateMetadataFileForCorePublication', task 
':core:generatePomFileForCorePublication', task 
':core:initializeNexusStagingRepository', task ':core:javadoc', task 
':core:javadocJar', task ':core:sourcesJar', task ':core:compileTestKotlin', 
task ':core:fmppTest', task ':core:javaCCTest', task ':core:compileTestJava', 
task ':core:processTestResources', task ':core:testClasses', task 
':core:testJar', task ':core:publishCorePublicationToNexusRepository', task 
':core:publish', task ':druid:compileJava', task ':druid:processResources', 
task ':druid:classes', task ':druid:jar', task 
':druid:generateMetadataFileForDruidPublication', task 
':druid:generatePomFileForDruidPublication', task 
':druid:initializeNexusStagingRepository', task ':druid:javadoc', task 
':druid:javadocJar', task ':druid:sourcesJar', task 
':druid:publishDruidPublicationToNexusRepository', task ':druid:publish', task 
':elasticsearch:compileJava', task ':elasticsearch:processResources', task 
':elasticsearch:classes', task ':elasticsearch:jar', task 
':elasticsearch:generateMetadataFileForElasticsearchPublication', task 
':elasticsearch:generatePomFileForElasticsearchPublication', task 
':elasticsearch:initializeNexusStagingRepository', task 
':elasticsearch:javadoc', task ':elasticsearch:javadocJar', task 
':elasticsearch:sourcesJar', task 
':elasticsearch:publishElasticsearchPublicationToNexusRepository', task 
':elasticsearch:publish', task ':example:csv:compileJava', task 
':file:compileJava', task ':file:processResources', task ':file:classes', task 
':file:jar', task ':file:generateMetadataFileForFilePublication', task 
':file:generatePomFileForFilePublication', task 
':file:initializeNexusStagingRepository', task ':file:javadoc', task 
':file:javadocJar', task ':file:sourcesJar', task 
':file:publishFilePublicationToNexusRepository', task ':file:publish', task 
':geode:compileJava', task 

Re: SQL has order by, there will be one more column(the sort column) in result plan

[JiaTao Tao]

Thanks, Julian

I can see the fields in RelRoot, and I'm a little confused, I want to
use RelVisitor to visit the RelNode, and here comes the RelRoot, any advice
to deal with RelRoot when using RelVisitor?


Regards!

Aron Tao


Julian Hyde  于2020年3月5日周四 上午2:03写道：

> There is a class RelRoot that wraps the top-most RelNode, and projects
> away columns that are only used for sorting. Read through
> https://issues.apache.org/jira/browse/CALCITE-2999 <
> https://issues.apache.org/jira/browse/CALCITE-2999>, where someone had
> the same misunderstanding that you have.
>
>
> > On Mar 4, 2020, at 3:43 AM, JiaTao Tao  wrote:
> >
> > SQL;
> > ```
> > SELECT u.ID, u.NAME
> > FROM USERS u
> > ORDER BY u.AGE
> > ```
> >
> > PLAN:
> > ```
> > LogicalSort(sort0=[$2], dir0=[ASC])
> >  LogicalProject(ID=[$0], NAME=[$1], AGE=[$2])
> >EnumerableTableScan(table=[[RECR, USERS]])
> > ```
> > Plan's row type: RecordType(INTEGER ID, VARCHAR NAME, INTEGER AGE)
> >
> > But the SQL only project ID and NAME, there should be a LogicalProject
> > (ID, NAME) on LogicalSort. Is this by design? And what about this extra
> > column?
> >
> >
> >
> > Regards!
> >
> > Aron Tao
>
>

[RESULT] [VOTE] Release apache-calcite-1.22.0 (release candidate 3)

[Danny Chan]

Thanks to everyone who has tested the release candidate and given
their comments and votes.

The tally is as follows.

4 binding +1s:
Danny Chan
Vladimir Sitnikov
Francis Chuang
Julian Hyde

5 non-binding +1s:
Rui Wang
Feng Zhu
Enrico Olivelli
Ruben Q L
Chunwei Lei

1 binding 0:
Stamatis Zampetakis

1 non-binding 0:
Anton Haidai

No -1s.

Therefore I am delighted to announce that the proposal to release
Apache Calcite 1.22.0 has passed.

Thanks everyone. We’ll now roll the release out to the mirrors.

There was some feedback during voting. I shall open a separate
thread to discuss.

Best,
Danny Chan

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 3)

[Chunwei Lei]

+1 (non-binding)

macOS 10.14.6, jdk 1.8.0_151, maven 3.5.4
 * Went over release note OK
 * Run Calcite tests OK
* Checked signatures and checksums OK


Best,
Chunwei


On Wed, Mar 4, 2020 at 2:50 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Stamatis>It is worth mentioning that the build passed with the 3rd attempt.
> In the
> Stamatis>first attempts the build was stuck while performing style
> checking.
>
> Do you have threaddumps?
> Have you filed JIRA for the locking?
>
> Vladimir
>

Re: Quicksql

[Chunwei Lei]

Thank you for your introduction, Siyuan.

Quicksql is an interesting project and full of potential. I'd like to learn
more about it.


Best,
Chunwei


On Tue, Mar 3, 2020 at 3:26 AM Siyuan Liu  wrote:

> Hi, everyone:
>
> Glad to see a lot of old friends here. Quicksql is a project born in early
> 2019. It was designed to solve the problem of long and complex work flow in
> the big data field with many data sources, many compute engines, and many
> types of syntax. The core idea is `Connect All Data Sources with One Extra
> Parsing Cost`.
>
> Because it involves standard SQL parsing, we finally chose Calcite as the
> parsing engine that has the best SQL compatibility. Thanks to the excellent
> architecture and toolkits provided by Calcite, Quicksql has made some
> extensions on this basis and made more logical plans Rich definitions
> enable single data source and multi-source queries to be described. For
> single data sources, an end-to-end connection query is directly
> established, and for multiple data sources, logical plans are divided and
> pushed down, final interpreted as the code of the compute engine (such as
> Spark, Flink) with distributed computing capabilities for data merge.
>
> Based on this design, Quicksql makes extensive use of the ability of
> Calcite Adapter \ Dialect \ UDF to provide syntax adaptation compatibility
> for various data sources and compute engines, and also uses Avatica as a
> JDBC protocol. We are very grateful for the excellent artwork provided by
> the Calcite community.
>
> At the beginning of the project, Quicksql was confused about the
> application areas. After one year of polishing, Quicksql has successfully
> applied two areas:
> 1. Interactive Query Engine: Provides big data interactive query and BI
> analysis with standard SQL syntax, and response time is in seconds to
> minutes.
> 2. ETL Compute Engine: SQL-based ETL for multi-data source, which can use
> optimization capabilities of SQL for data cleaning \ transformation \ join,
> etc.
> In the future, we will also focus on dynamic engine selection, so that
> engines such as Hive, Spark, and Presto can run more suitable SQL.
>
> Looking forward to working with the Calcite community to do some
> interesting things and explore the unlimited possibilities of SQL
>
> Siyuan Liu
>
> On Mon, Mar 2, 2020 at 3:45 PM Francis Du  wrote:
>
> > Hi everyone:
> >
> > Allow me to introduce my good friend Siyuan Liu, who is the leader of
> > Quicksql project.
> >
> > I CC to him and ask him to introduce the project to us.Here is the
> > documentation link for
> >
> > Quicksql [1].
> >
> > [1].  https://quicksql.readthedocs.io/en/latest/
> >
> > Regards,
> > Francis
> >
> > Juan Pan  于2019年12月23日周一 上午11:44写道：
> >
> >> Thanks Gelbana,
> >>
> >>
> >> Very appreciated your explanation, which sheds me some light on
> exploring
> >> Calcite. :)
> >>
> >>
> >> Best wishes,
> >> Trista
> >>
> >>
> >>  Juan Pan (Trista)
> >>
> >> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> >> E-mail: panj...@apache.org
> >>
> >>
> >>
> >>
> >> On 12/22/2019 05:58，Muhammad Gelbana wrote：
> >> I am curious how to join the tables from different datasources.
> >> Based on Calcite's conventions concept, the Join operator and its input
> >> operators should all have the same convention. If they don't, the
> >> convention different from the Join operator's convention will have to
> >> register a converter rule. This rule should produce an operator that
> only
> >> converts from that convention to the Join operator's convention.
> >>
> >> This way the Join operator will be able to handle the data obtained from
> >> its input operators because it understands the data structure.
> >>
> >> Thanks,
> >> Gelbana
> >>
> >>
> >> On Wed, Dec 18, 2019 at 5:08 AM Juan Pan  wrote:
> >>
> >> Some updates.
> >>
> >>
> >> Recently i took a look at their doc and source code, and found this
> >> project uses SQL parsing and Relational algebra of Calcite to get query
> >> plan, and also translates to spark SQL for joining different
> datasources,
> >> or corresponding query for single datasource.
> >>
> >>
> >> Although it copies many classes from Calcite, the idea of QuickSQL seems
> >> some of interests, and code is succinct.
> >>
> >>
> >> Best,
> >> Trista
> >>
> >>
> >> Juan Pan (Trista)
> >>
> >> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> >> E-mail: panj...@apache.org
> >>
> >>
> >>
> >>
> >> On 12/13/2019 17:16，Juan Pan wrote：
> >> Yes, indeed.
> >>
> >>
> >> Juan Pan (Trista)
> >>
> >> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> >> E-mail: panj...@apache.org
> >>
> >>
> >>
> >>
> >> On 12/12/2019 18:00，Alessandro Solimando >
> >> wrote：
> >> Adapters must be needed by data sources not supporting SQL, I think this
> >> is
> >> what Juan Pan was asking for.
> >>
> >> On Thu, 12 Dec 2019 at 04:05, Haisheng Yuan  wrote:
> >>
> >> Nope, it doesn't use any adapters. It just submits partial SQL query to
> >> different engines.
> >>

Re: Increasing number of concurrent queries

[Julian Hyde]

Enumerable is flawed because it uses the Volcano execution model, which 
requires blocking threads.

Interpreter might be a better execution model. Take a look at 
org.apache.calcite.interpreter.Node. The ‘run()’ method returns void but we 
could make it return boolean, where ‘false’ means ‘I’m not yet done, call me in 
a while’. Then you could have many more queries running than you have threads.

Julian


> On Mar 4, 2020, at 9:48 AM, Scott Reynolds  wrote:
> 
> List,
> 
> Our team's current application runs a thread pool for handling queries.
> Each time a query comes it it is placed in the run queue and executed
> outside of the HTTP RPC thread. This allows us to handle N concurrent
> queries where N is the size of the thread pool. I would like to change this
> to a N:M system allowing our service to handle more concurrent queries then
> the thread pool size. Our TableScan uses Netty RPCs and Queues to fetch
> results from the remote database giving use the opportunity to park the
> current query while awaiting RPC responses.
> 
> Has anyone been able to achieve this? Is the best way to own Convention?
> 
> Our current service doesn't use JDBC and therefore have direct access to
> the compiled Enumerable. One solution I haven't been able to get working is
> using InterruptedException (or something similar -- Kafka created a
> WakeUpException), which would allow a sub-enumerable the ability to signal
> above that it needs to save it's state and yield.
> 
> Is this something that could be added to EnumerableDefaults? If not would
> it be a InterruptableEnumerable convention instead?

Re: SQL has order by, there will be one more column(the sort column) in result plan

[Julian Hyde]

There is a class RelRoot that wraps the top-most RelNode, and projects away 
columns that are only used for sorting. Read through 
https://issues.apache.org/jira/browse/CALCITE-2999 
, where someone had the 
same misunderstanding that you have.


> On Mar 4, 2020, at 3:43 AM, JiaTao Tao  wrote:
> 
> SQL;
> ```
> SELECT u.ID, u.NAME
> FROM USERS u
> ORDER BY u.AGE
> ```
> 
> PLAN:
> ```
> LogicalSort(sort0=[$2], dir0=[ASC])
>  LogicalProject(ID=[$0], NAME=[$1], AGE=[$2])
>EnumerableTableScan(table=[[RECR, USERS]])
> ```
> Plan's row type: RecordType(INTEGER ID, VARCHAR NAME, INTEGER AGE)
> 
> But the SQL only project ID and NAME, there should be a LogicalProject
> (ID, NAME) on LogicalSort. Is this by design? And what about this extra
> column?
> 
> 
> 
> Regards!
> 
> Aron Tao

[jira] [Created] (CALCITE-3841) Change downloads page to use downloads.apache.org

[Julian Hyde (Jira)]

Julian Hyde created CALCITE-3841:


 Summary: Change downloads page to use downloads.apache.org
 Key: CALCITE-3841
 URL: https://issues.apache.org/jira/browse/CALCITE-3841
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Hyde


Infra has 
[decided|https://lists.apache.org/thread.html/rcd2864e75e417597d342b8eb83080eb2d7a0cafea84fd4521a4d9cfd%40%3Cusers.infra.apache.org%3E]
 (login required for that email link) to deprecate 
[www.apache.org/dist|https://www.apache.org/dist] and move downloads to 
[https://downloads.apache.org|https://downloads.apache.org].

On [Calcite's downloads page|https://calcite.apache.org/downloads/], we need to 
change the 'digest' link from (for example) 
{{https://www.apache.org/dist/calcite/apache-calcite-1.21.0/apache-calcite-1.21.0-src.tar.gz.sha256}}
 to 
{{https://downloads.apache.org/calcite/apache-calcite-1.21.0/apache-calcite-1.21.0-src.tar.gz.sha256}},
 and similarly the 'gpg' link.

I believe that the 'tar' link can remain as 
{{https://www.apache.org/dyn/closer.lua?filename=calcite/apache-calcite-1.21.0/apache-calcite-1.21.0-src.tar.gz=download}}
 for the latest release and 
{{https://archive.apache.org/dist/calcite/apache-calcite-1.20.0/apache-calcite-1.20.0-src.tar.gz}}
 for older releases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Increasing number of concurrent queries

[Scott Reynolds]

List,

Our team's current application runs a thread pool for handling queries.
Each time a query comes it it is placed in the run queue and executed
outside of the HTTP RPC thread. This allows us to handle N concurrent
queries where N is the size of the thread pool. I would like to change this
to a N:M system allowing our service to handle more concurrent queries then
the thread pool size. Our TableScan uses Netty RPCs and Queues to fetch
results from the remote database giving use the opportunity to park the
current query while awaiting RPC responses.

Has anyone been able to achieve this? Is the best way to own Convention?

Our current service doesn't use JDBC and therefore have direct access to
the compiled Enumerable. One solution I haven't been able to get working is
using InterruptedException (or something similar -- Kafka created a
WakeUpException), which would allow a sub-enumerable the ability to signal
above that it needs to save it's state and yield.

Is this something that could be added to EnumerableDefaults? If not would
it be a InterruptableEnumerable convention instead?

Re: [ANNOUNCE] New committer: Feng Zhu

[Enrico Olivelli]

Welcome !

Enrico

Il giorno mer 4 mar 2020 alle ore 10:42 Matt Wang  ha scritto:
>
> Congrats Feng!
>
>
> ---
> Best,
> Matt Wang
>
>
> On 03/4/2020 11:30，Xin Wang wrote：
> Congrats Feng!
>
> - Xin
>
> Kevin Risden  于2020年3月4日周三 上午2:32写道：
>
> Congrats Feng!
>
> Sounds like an interesting project.
>
> Kevin Risden
>
>
> On Sun, Mar 1, 2020 at 5:34 AM Feng Zhu  wrote:
>
> Thanks Julian.
> Let me make a brief introduction about the project.
> SuperSQL relies on both Calcite and Avatica (with inner branch). It aims
> to
> take the role as a unified entrance and middleware to connect various
> data
> platforms in Tencent, including
> RDBMS、ES、Hive、Flink、Spark、Presto、ClickHouse
> and some inner-built systems. They may be integrated as datasource or
> execution engine.
>
> As we are still busy on migrating massive workloads and supporting
> in-house
> business with SuperSQL, current now we do not have enough resources on
> public release (e.g., presentation, talks or posts). Therefore, it is not
> suprising that you wasn't aware of it before.
>
> Is it better to add the paragraph when we have website or an open-sourced
> version (maybe several months later)?
>
> Tencent's logo can be found in bottom right-hand corner in (
> https://www.tencent.com/en-us/index.html).
> I think you can just pick up the english part (i.e. "Tencent").
>
> Best regards,
> Feng
>
> Julian Hyde  于2020年3月1日周日 上午7:33写道：
>
> Congratulations, thanks, and welcome!
>
> Feng, I wasn’t aware of Tencent’s SuperSQL project. It would be a great
> addition to the “powered by” page. If you would like to add it, please
> create a PR to add a paragraph to that page (no need for a JIRA case),
> and
> provide a link to a logo for Tencent that I can add to the picture.
>
> Julian
>
>
>
> On Feb 29, 2020, at 3:15 PM, Rui Wang  wrote:
>
> Congratulations Feng! Well deserved!
>
>
>
> -Rui
>
> On Sat, Feb 29, 2020 at 2:10 AM Feng Zhu 
> wrote:
>
> Thank you everyone for your warm welcome!
> Currently I am working at SuperSQL team of Tencent in Shenzhen,
> Guangdong,
> China.
> The project relies heavily on Calcite to analyze data residing in
> thousands
> of heterogeneous data sources.
>
> It is my greate honor to be part of the community. Calcite is an
> excellent
> and promising project!
> I will do my best to contribute to the project.
>
> Best regards,
> Feng
>
> Chunwei Lei  于2020年2月29日周六 下午4:46写道：
>
> Congratulations！
>
>
> Best,
> Chunwei
>
>
> On Sat, Feb 29, 2020 at 4:41 PM Danny Chan 
> wrote:
>
> Congratulations!
>
> Francis Chuang 于2020年2月29日 周六下午4:35写道：
>
> Congrats, well-deserved!
>
> On 29/02/2020 6:26 pm, Stamatis Zampetakis wrote:
> Apache Calcite's Project Management Committee (PMC) has invited
> Feng
> Zhu to become a committer, and we are pleased to announce that
> he
> has accepted.
>
> Feng is an active contributor and has contributed a lot of code
> to
> some
> fairly complex parts of Calcite. He has been very helpful in
> discussions
> and reviews, especially around code generation, and his work is
> always
> of high quality.
>
> Feng, welcome, thank you for your contributions, and we look
> forward
> your further interactions with the community! If you wish,
> please
> feel
> free to tell us more about yourself and what you are working on.
>
> Stamatis (on behalf of the Apache Calcite PMC)
>
>
>
>
>
>
>
>
>
>
>
> --
> Thanks,
> Xin

SQL has order by, there will be one more column(the sort column) in result plan

[JiaTao Tao]

SQL;
```
SELECT u.ID, u.NAME
FROM USERS u
ORDER BY u.AGE
```

PLAN:
```
LogicalSort(sort0=[$2], dir0=[ASC])
  LogicalProject(ID=[$0], NAME=[$1], AGE=[$2])
EnumerableTableScan(table=[[RECR, USERS]])
```
Plan's row type: RecordType(INTEGER ID, VARCHAR NAME, INTEGER AGE)

But the SQL only project ID and NAME, there should be a LogicalProject
(ID, NAME) on LogicalSort. Is this by design? And what about this extra
column?



Regards!

Aron Tao

[jira] [Created] (CALCITE-3840) Re-aliasing of VALUES that has column aliases produces wrong SQL in the JDBC adapter

[Christian Beikov (Jira)]

Christian Beikov created CALCITE-3840:
-

 Summary: Re-aliasing of VALUES that has column aliases produces 
wrong SQL in the JDBC adapter
 Key: CALCITE-3840
 URL: https://issues.apache.org/jira/browse/CALCITE-3840
 Project: Calcite
  Issue Type: Bug
  Components: jdbc-adapter
Affects Versions: 1.21.0
Reporter: Christian Beikov


Rendering a VALUES relnode to e.g. PostgreSQL will produce \{{FROM 
(VALUES((1))) AS t(col_alias)}} where "t" is a static alias. When e.g. joining 
with such a VALUES, the RelToSqlConverter tries to re-alias this with a unique 
alias, but fails because it produces \{{FROM (VALUES((1))) AS t(col_alias) AS 
newAlias}}.

The fix is to replace the static table alias instead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New committer: Feng Zhu

[Matt Wang]

Congrats Feng!


---
Best,
Matt Wang


On 03/4/2020 11:30，Xin Wang wrote：
Congrats Feng!

- Xin

Kevin Risden  于2020年3月4日周三 上午2:32写道：

Congrats Feng!

Sounds like an interesting project.

Kevin Risden


On Sun, Mar 1, 2020 at 5:34 AM Feng Zhu  wrote:

Thanks Julian.
Let me make a brief introduction about the project.
SuperSQL relies on both Calcite and Avatica (with inner branch). It aims
to
take the role as a unified entrance and middleware to connect various
data
platforms in Tencent, including
RDBMS、ES、Hive、Flink、Spark、Presto、ClickHouse
and some inner-built systems. They may be integrated as datasource or
execution engine.

As we are still busy on migrating massive workloads and supporting
in-house
business with SuperSQL, current now we do not have enough resources on
public release (e.g., presentation, talks or posts). Therefore, it is not
suprising that you wasn't aware of it before.

Is it better to add the paragraph when we have website or an open-sourced
version (maybe several months later)?

Tencent's logo can be found in bottom right-hand corner in (
https://www.tencent.com/en-us/index.html).
I think you can just pick up the english part (i.e. "Tencent").

Best regards,
Feng

Julian Hyde  于2020年3月1日周日 上午7:33写道：

Congratulations, thanks, and welcome!

Feng, I wasn’t aware of Tencent’s SuperSQL project. It would be a great
addition to the “powered by” page. If you would like to add it, please
create a PR to add a paragraph to that page (no need for a JIRA case),
and
provide a link to a logo for Tencent that I can add to the picture.

Julian



On Feb 29, 2020, at 3:15 PM, Rui Wang  wrote:

Congratulations Feng! Well deserved!



-Rui

On Sat, Feb 29, 2020 at 2:10 AM Feng Zhu 
wrote:

Thank you everyone for your warm welcome!
Currently I am working at SuperSQL team of Tencent in Shenzhen,
Guangdong,
China.
The project relies heavily on Calcite to analyze data residing in
thousands
of heterogeneous data sources.

It is my greate honor to be part of the community. Calcite is an
excellent
and promising project!
I will do my best to contribute to the project.

Best regards,
Feng

Chunwei Lei  于2020年2月29日周六 下午4:46写道：

Congratulations！


Best,
Chunwei


On Sat, Feb 29, 2020 at 4:41 PM Danny Chan 
wrote:

Congratulations!

Francis Chuang 于2020年2月29日 周六下午4:35写道：

Congrats, well-deserved!

On 29/02/2020 6:26 pm, Stamatis Zampetakis wrote:
Apache Calcite's Project Management Committee (PMC) has invited
Feng
Zhu to become a committer, and we are pleased to announce that
he
has accepted.

Feng is an active contributor and has contributed a lot of code
to
some
fairly complex parts of Calcite. He has been very helpful in
discussions
and reviews, especially around code generation, and his work is
always
of high quality.

Feng, welcome, thank you for your contributions, and we look
forward
your further interactions with the community! If you wish,
please
feel
free to tell us more about yourself and what you are working on.

Stamatis (on behalf of the Apache Calcite PMC)











--
Thanks,
Xin