[jira] [Created] (CALCITE-3964) Support in DESCRIPTOR operator
Rui Wang created CALCITE-3964:
-
Summary: Support in DESCRIPTOR operator
Key: CALCITE-3964
URL: https://issues.apache.org/jira/browse/CALCITE-3964
Project: Calcite
Issue Type: Sub-task
Reporter: Rui Wang
Assignee: Rui Wang
>From SQL standard 2016
{code:sql}
8.15
::=
::=
DESCRIPTOR
::=
[ { }... ]
::=
[ ]
A is the keyword DESCRIPTOR followed by a parenthesized
list of column names; each column name may optionally have a data type. If
every column name has a data type, then the descriptor describes a row type. In
the examples, CSVreader and Pivot use descriptor arguments that are just lists
of column names; ExecR is an example that uses a descriptor to pass a complete
row type.
{code}
DESCRIPTOR can include an optional data type, which will enable type checking
in SQL validator. It is useful for streaming windowing: the windowing is
required to be applied on TIMESTAMP type, and we can rely on descriptor to do
type validation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3963) Maintains logical properties at RelSet (equivalent group) instead of RelNode
Xiening Dai created CALCITE-3963: Summary: Maintains logical properties at RelSet (equivalent group) instead of RelNode Key: CALCITE-3963 URL: https://issues.apache.org/jira/browse/CALCITE-3963 Project: Calcite Issue Type: Bug Reporter: Xiening Dai Currently the logical properties (such as row count, distinct row count, etc) are maintained at RelNode level. This creates a number of meta data consistency problems, e.g. CALCITE-1048, CALCITE-2166. In theory, all RelNodes in a RelSet should share the same logical properties per definition of relational equivalence. So it makes more sense to keep logical properties at RelSet level, rather than the RelNode. And such properties shouldn't change when new sub set is created or subset's best is changed. Specifically I think below build in metadata should fall into the logical properties category - Selectivity UniqueKeys ColumnUniqueness RowCount MaxRowCount MinRowCount DistinctRowCount Size (averageRowSize, averageColumnSize) -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] New committer: Wang Yanlin
Congrats, Wang Yanlin! Best, Danny Chan 在 2020年4月29日 +0800 AM8:26，dev@calcite.apache.org，写道： > > Congrats, Wang Yanlin!
Re: How to give type in case of type mismatch while doing union
> On Apr 29, 2020, at 21:31, Danny Chan wrote: > > Or try SQL, it has the implicit type coercion ~ That’s true. SQL is a great way to build relational algebra expressions. Highly recommended. Julian
Re: How to give type in case of type mismatch while doing union
Or try SQL, it has the implicit type coercion ~ Best, Danny Chan 在 2020年4月28日 +0800 AM4:36，Julian Hyde ，写道： > Anjali, > > If you’re using RelBuilder to create the union, or creating the union > manually, it is your responsibility to make sure that the input RelNodes have > compatible types. > > RelDataTypeFactory.leastRestrictive(List) may be useful. > > Julian > > > > On Apr 27, 2020, at 1:32 PM, Rui Wang wrote: > > > > Did a quick test by running a SQL query that has UNION on two different > > types. Validator gave a correct error message (not NPE) to remind type > > mismatch. > > > > Agreed with Jin, could you provide more context/example how you reach the > > NPE? (It could be better if you can file a Jira with your context). > > > > > > -Rui > > > > On Mon, Apr 27, 2020 at 4:35 AM XING JIN wrote: > > > > > Hi, Anjali ~ > > > Are you doing the UNION by Sql ? If so, can you give the Sql content ? > > > Are you doing the UNION on RelNodes ?, If so, you need to do type CAST. > > > > > > Jin > > > > > > Anjali Shrishrimal 于2020年4月27日周一 > > > 下午4:25写道： > > > > > > > Hi, > > > > > > > > While doing union of 2 RelNodes with different types, I am getting NPE. > > > (I > > > > am using calcite 1.21.0) > > > > java.lang.NullPointerException: at index 0 > > > > at > > > > > > > com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:225) > > > > at > > > > > > > com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:215) > > > > at > > > > > > > com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:209) > > > > at > > > > com.google.common.collect.ImmutableList.construct(ImmutableList.java:346) > > > > at > > > > com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:258) > > > > at > > > > > > > org.apache.calcite.rel.type.RelDataTypeFactoryImpl.canonize(RelDataTypeFactoryImpl.java:373) > > > > at > > > > > > > org.apache.calcite.rel.type.RelDataTypeFactoryImpl.createStructType(RelDataTypeFactoryImpl.java:155) > > > > at > > > > > > > org.apache.calcite.rel.type.RelDataTypeFactoryImpl.createStructType(RelDataTypeFactoryImpl.java:146) > > > > at > > > > > > > org.apache.calcite.rel.type.RelDataTypeFactory$Builder.build(RelDataTypeFactory.java:569) > > > > at > > > > > > > org.apache.calcite.rel.type.RelDataTypeFactoryImpl.leastRestrictiveStructuredType(RelDataTypeFactoryImpl.java:257) > > > > at > > > > > > > org.apache.calcite.sql.type.SqlTypeFactoryImpl.leastRestrictiveSqlType(SqlTypeFactoryImpl.java:285) > > > > at > > > > > > > org.apache.calcite.sql.type.SqlTypeFactoryImpl.leastRestrictive(SqlTypeFactoryImpl.java:156) > > > > at > > > > org.apache.calcite.rel.core.SetOp.deriveRowType(SetOp.java:107) > > > > > > > > If the column types (family types) are different, currently the derived > > > > type is null. Is there any way to control that? > > > > Where can I define the type in case of mismatch ? > > > > > > > > > > > > Thank you, > > > > Anjali Shrishrimal > > > > > > > >
Re: Re: [ANNOUNCE] New committer: Jin Xing
Congrats, Jin Xing! Best, Danny Chan 在 2020年4月30日 +0800 PM12:15，dev@calcite.apache.org，写道： > > Congrats, Jin Xing!
Re: Re: [ANNOUNCE] New committer: Forward Xu
Congratulations, Forward! Best, Liya Fan On Wed, Apr 29, 2020 at 8:51 PM Wang Yanlin <1989yanlinw...@163.com> wrote: > Congrations! Forward!-- > > Best, > Wang Yanlin > > > > > > At 2020-04-29 10:52:25, "Feng Zhu" wrote: > > Congrations! Forward! > > > >best, > >Feng > > > >Chunwei Lei 于2020年4月29日周三 上午10:17写道： > > > >> Congrats, Forward! > >> > >> > >> > >> Best, > >> Chunwei > >> > >> > >> On Wed, Apr 29, 2020 at 6:46 AM Rui Wang wrote: > >> > >> > Congrats! > >> > > >> > > >> > -Rui > >> > > >> > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang < > francischu...@apache.org > >> > > >> > wrote: > >> > > >> > > Congrats, Forward! > >> > > > >> > > Francis > >> > > > >> > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote: > >> > > > Apache Calcite's Project Management Committee (PMC) has invited > >> Forward > >> > > Xu > >> > > > to > >> > > > become a committer, and we are pleased to announce that he has > >> > accepted. > >> > > > > >> > > > Forward has been helping the project for some time now. He added > many > >> > new > >> > > > SQL > >> > > > functions to the project and is one of our JSON experts. On top of > >> > that, > >> > > and > >> > > > other fixes, he is the one who added the Redis adapter to the > >> project. > >> > > > > >> > > > Forward, welcome, thank you for your contributions, and we look > >> forward > >> > > to > >> > > > your > >> > > > further interactions with the community! If you wish, please feel > >> free > >> > to > >> > > > tell > >> > > > us more about yourself and what you are working on. > >> > > > > >> > > > Stamatis (on behalf of the Apache Calcite PMC) > >> > > > > >> > > > >> > > >> >
Re: Re: [ANNOUNCE] New committer: Jin Xing
Congratulations, Jin! Best, Liya Fan On Wed, Apr 29, 2020 at 8:51 PM Wang Yanlin <1989yanlinw...@163.com> wrote: > Congrats, Jin Xing! > > > -- > > Best, > Wang Yanlin > > > > > > 在 2020-04-29 13:58:53，"Zoltan Haindrich" 写道： > >Congrats! > > > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: > >> Congratulations! > >> > >> Enrico > >> > >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: > >> > >>> Congrations! > >>> > >>> best, > >>> Feng > >>> > >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道： > >>> > Congrats, Jin! > > > Best, > Chunwei > > > On Wed, Apr 29, 2020 at 10:07 AM Forward Xu > wrote: > > > Congrats > > > > > > best, > > > > Forward > > > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道： > > > >> Congrats, Jin Xing! > >> > >> > >> ---Original--- > >> From: "Stamatis Zampetakis" >> Date: Wed, Apr 29, 2020 05:47 AM > >> To: "dev" >> Subject: [ANNOUNCE] New committer: Jin Xing > >> > >> > >> Apache Calcite's Project Management Committee (PMC) has invited Jin > Xing > > to > >> become a committer, and we are pleased to announce that he has > accepted. > >> > >> Jin has contributed a lot of code in the project and many > >> recent improvements in > >> materialized view matching have his signature on them. Apart from > >>> code > >> contributions, Jin provides valuable help to the community by doing > > reviews > >> and > >> answering questions in the devlist. > >> > >> Jin, welcome, thank you for your contributions, and we look forward > >>> to > > your > >> further interactions with the community! If you wish, please feel > >>> free > to > >> tell > >> us more about yourself and what you are working on. > >> > >> Stamatis (on behalf of the Apache Calcite PMC) > > > > >>> > >> >
Re: Re: [ANNOUNCE] New committer: Wang Yanlin
Congratulations, Yanlin! Best, Liya Fan On Wed, Apr 29, 2020 at 8:50 PM Wang Yanlin <1989yanlinw...@163.com> wrote: > Hi, guys, thanks for your warm welcome. > > > > I'm working in Ant Finical, Alibaba Group. Currently my team is working > on building a system to process big data in form of sql. > We use calcite to parse sql, optimize Relnode and rewrite SqlNode to > execute on different engines, like Spark，MaxCompute, HBase and so on. > Calcite is really a great community, and it's really an honor for me to > become calcite committer, hops to make more contribution to calcite. > > > Thanks again. > > -- > > Best, > Wang Yanlin > > > > > > 在 2020-04-29 13:58:35，"Zoltan Haindrich" 写道： > >Congratulations! > > > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: > >> Congrats! > >> > >> Enrico > >> > >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: > >> > >>> Congrations! Yanlin! > >>> > >>> best, > >>> Feng > >>> > >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道： > >>> > Congrats, Yanlin! > > > Best, > Chunwei > > > On Wed, Apr 29, 2020 at 10:07 AM Forward Xu > wrote: > > > Congrats > > > > > > Best, > > > > Forward > > > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道： > > > >> Congrats, Wang Yanlin! > >> > >> > >> > >> > >> ---Original--- > >> From: "Stamatis Zampetakis" >> Date: Wed, Apr 29, 2020 05:51 AM > >> To: "dev" >> Subject: [ANNOUNCE] New committer: Wang Yanlin > >> > >> > >> Apache Calcite's Project Management Committee (PMC) has invited Wang > > Yanlin > >> to > >> become a committer, and we are pleased to announce that he has > accepted. > >> > >> Wang has pushed numerous fixes and improvements to the project, > >>> landing > > in > >> total > >> the impressive number of 30 commits to the master. Among other > >>> things, > he > >> contributed some important features in the Interpreter. > >> > >> Wang, welcome, thank you for your contributions, and we look forward > your > >> further interactions with the community! If you wish, please feel > >>> free > to > >> tell > >> us more about yourself and what you are working on. > >> > >> Stamatis (on behalf of the Apache Calcite PMC) > > > > >>> > >> >
Re:Re: [ANNOUNCE] New committer: Wang Yanlin
Hi, guys, thanks for your warm welcome. I'm working in Ant Finical, Alibaba Group. Currently my team is working on building a system to process big data in form of sql. We use calcite to parse sql, optimize Relnode and rewrite SqlNode to execute on different engines, like Spark，MaxCompute, HBase and so on. Calcite is really a great community, and it's really an honor for me to become calcite committer, hops to make more contribution to calcite. Thanks again. -- Best, Wang Yanlin 在 2020-04-29 13:58:35，"Zoltan Haindrich" 写道： >Congratulations! > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: >> Congrats! >> >> Enrico >> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: >> >>> Congrations! Yanlin! >>> >>> best, >>> Feng >>> >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道： >>> Congrats, Yanlin! Best, Chunwei On Wed, Apr 29, 2020 at 10:07 AM Forward Xu wrote: > Congrats > > > Best, > > Forward > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道： > >> Congrats, Wang Yanlin! >> >> >> >> >> ---Original--- >> From: "Stamatis Zampetakis"> Date: Wed, Apr 29, 2020 05:51 AM >> To: "dev"> Subject: [ANNOUNCE] New committer: Wang Yanlin >> >> >> Apache Calcite's Project Management Committee (PMC) has invited Wang > Yanlin >> to >> become a committer, and we are pleased to announce that he has accepted. >> >> Wang has pushed numerous fixes and improvements to the project, >>> landing > in >> total >> the impressive number of 30 commits to the master. Among other >>> things, he >> contributed some important features in the Interpreter. >> >> Wang, welcome, thank you for your contributions, and we look forward your >> further interactions with the community! If you wish, please feel >>> free to >> tell >> us more about yourself and what you are working on. >> >> Stamatis (on behalf of the Apache Calcite PMC) > >>> >>
Re:Re: [ANNOUNCE] New committer: Jin Xing
Congrats, Jin Xing! -- Best, Wang Yanlin 在 2020-04-29 13:58:53，"Zoltan Haindrich" 写道： >Congrats! > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: >> Congratulations! >> >> Enrico >> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: >> >>> Congrations! >>> >>> best, >>> Feng >>> >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道： >>> Congrats, Jin! Best, Chunwei On Wed, Apr 29, 2020 at 10:07 AM Forward Xu wrote: > Congrats > > > best, > > Forward > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道： > >> Congrats, Jin Xing! >> >> >> ---Original--- >> From: "Stamatis Zampetakis"> Date: Wed, Apr 29, 2020 05:47 AM >> To: "dev"> Subject: [ANNOUNCE] New committer: Jin Xing >> >> >> Apache Calcite's Project Management Committee (PMC) has invited Jin Xing > to >> become a committer, and we are pleased to announce that he has accepted. >> >> Jin has contributed a lot of code in the project and many >> recent improvements in >> materialized view matching have his signature on them. Apart from >>> code >> contributions, Jin provides valuable help to the community by doing > reviews >> and >> answering questions in the devlist. >> >> Jin, welcome, thank you for your contributions, and we look forward >>> to > your >> further interactions with the community! If you wish, please feel >>> free to >> tell >> us more about yourself and what you are working on. >> >> Stamatis (on behalf of the Apache Calcite PMC) > >>> >>
Re:Re: [ANNOUNCE] New committer: Forward Xu
Congrations! Forward!-- Best, Wang Yanlin At 2020-04-29 10:52:25, "Feng Zhu" wrote: > Congrations! Forward! > >best, >Feng > >Chunwei Lei 于2020年4月29日周三 上午10:17写道： > >> Congrats, Forward! >> >> >> >> Best, >> Chunwei >> >> >> On Wed, Apr 29, 2020 at 6:46 AM Rui Wang wrote: >> >> > Congrats! >> > >> > >> > -Rui >> > >> > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang > > >> > wrote: >> > >> > > Congrats, Forward! >> > > >> > > Francis >> > > >> > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote: >> > > > Apache Calcite's Project Management Committee (PMC) has invited >> Forward >> > > Xu >> > > > to >> > > > become a committer, and we are pleased to announce that he has >> > accepted. >> > > > >> > > > Forward has been helping the project for some time now. He added many >> > new >> > > > SQL >> > > > functions to the project and is one of our JSON experts. On top of >> > that, >> > > and >> > > > other fixes, he is the one who added the Redis adapter to the >> project. >> > > > >> > > > Forward, welcome, thank you for your contributions, and we look >> forward >> > > to >> > > > your >> > > > further interactions with the community! If you wish, please feel >> free >> > to >> > > > tell >> > > > us more about yourself and what you are working on. >> > > > >> > > > Stamatis (on behalf of the Apache Calcite PMC) >> > > > >> > > >> > >>
Re: [ANNOUNCE] New committer: Jin Xing
Thanks a lot ~ Calcite is a great project and it's great honor for me to work with you guys. I really appreciate the help from community. I'm working in Alibaba. My team builds big data system to optimize batch and streaming jobs. We use Calcite to process Sql queries and accommodate to different physical engines. I'm very excited to become Calcite committer and looking forward to make more contributions. Best regards, Jin Zoltan Haindrich 于2020年4月29日周三 下午1:58写道： > Congrats! > > On 4/29/20 7:32 AM, Enrico Olivelli wrote: > > Congratulations! > > > > Enrico > > > > Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: > > > >> Congrations! > >> > >> best, > >> Feng > >> > >> Chunwei Lei 于2020年4月29日周三 上午10:16写道： > >> > >>> Congrats, Jin! > >>> > >>> > >>> Best, > >>> Chunwei > >>> > >>> > >>> On Wed, Apr 29, 2020 at 10:07 AM Forward Xu > >>> wrote: > >>> > Congrats > > > best, > > Forward > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道： > > > Congrats, Jin Xing! > > > > > > ---Original--- > > From: "Stamatis Zampetakis" > Date: Wed, Apr 29, 2020 05:47 AM > > To: "dev" > Subject: [ANNOUNCE] New committer: Jin Xing > > > > > > Apache Calcite's Project Management Committee (PMC) has invited Jin > >>> Xing > to > > become a committer, and we are pleased to announce that he has > >>> accepted. > > > > Jin has contributed a lot of code in the project and many > > recent improvements in > > materialized view matching have his signature on them. Apart from > >> code > > contributions, Jin provides valuable help to the community by doing > reviews > > and > > answering questions in the devlist. > > > > Jin, welcome, thank you for your contributions, and we look forward > >> to > your > > further interactions with the community! If you wish, please feel > >> free > >>> to > > tell > > us more about yourself and what you are working on. > > > > Stamatis (on behalf of the Apache Calcite PMC) > > >>> > >> > > >
Re: [ANNOUNCE] New committer: Forward Xu
Thank you everyone for your warm welcome! I'm working in the TBDS team of Tencent in Shenzhen. TBDS (Tencent Big Data Suite) is similar to Alibaba's EMR, TBDS is a big data ecosystem. I am responsible for Oceanus(flink streaming jobs) and Tdbank (Tencent real-time data collection system). I‘m very happy to become calcite committer and looking forward to make more contributions. Best, Forward Zoltan Haindrich 于2020年4月29日周三 下午1:58写道： > Congratulations! > > On 4/29/20 7:31 AM, Enrico Olivelli wrote: > > Congrats! > > > > Enrico > > > > Il Mer 29 Apr 2020, 04:52 Feng Zhu ha scritto: > > > >> Congrations! Forward! > >> > >> best, > >> Feng > >> > >> Chunwei Lei 于2020年4月29日周三 上午10:17写道： > >> > >>> Congrats, Forward! > >>> > >>> > >>> > >>> Best, > >>> Chunwei > >>> > >>> > >>> On Wed, Apr 29, 2020 at 6:46 AM Rui Wang wrote: > >>> > Congrats! > > > -Rui > > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang < > >> francischu...@apache.org > > wrote: > > > Congrats, Forward! > > > > Francis > > > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote: > >> Apache Calcite's Project Management Committee (PMC) has invited > >>> Forward > > Xu > >> to > >> become a committer, and we are pleased to announce that he has > accepted. > >> > >> Forward has been helping the project for some time now. He added > >> many > new > >> SQL > >> functions to the project and is one of our JSON experts. On top of > that, > > and > >> other fixes, he is the one who added the Redis adapter to the > >>> project. > >> > >> Forward, welcome, thank you for your contributions, and we look > >>> forward > > to > >> your > >> further interactions with the community! If you wish, please feel > >>> free > to > >> tell > >> us more about yourself and what you are working on. > >> > >> Stamatis (on behalf of the Apache Calcite PMC) > >> > > > > >>> > >> > > >
