Re: [ANNOUNCE] New Calcite PMC chair: Benchao Li
Congratulations, Benchao! Best, Dan Zou > 2023年12月21日 17:18，Stamatis Zampetakis 写道： > > Calcite community members, > > I am pleased to announce that we have a new PMC chair and VP as per > our tradition of rotating the chair once a year. I have resigned, and > Benchao Li was duly elected by the PMC and approved unanimously by the > Board. > > Please join me in congratulating Benchao! > > Best regards, > Stamatis
Re: [ANNOUNCE] New committer: Hongyu Guo
Congrats, Hongyu! Best, Dan Zou > 2023年11月2日 22:27，Benchao Li 写道： > > Apache Calcite's Project Management Committee (PMC) has invited Hongyu > Guo to become a committer, and we are pleased to announce that he has > accepted. > > Hongyu's contributions are around the functions, parser, DDL(CREATE > TABLE LIKE, GRANT, REVOKE), and simplication. Apart from the code > contribution, Hongyu is also very active in the mailing list and code > reviews. > > Hongyu, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please > feel free to tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Benchao Li (on behalf of the Apache Calcite PMC) > > [1] https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members > > -- > > Best, > Benchao Li
Re: [ANNOUNCE] New committer: Runkang He
Congrats, Runkang! Best, Dan Zou > 2023年11月3日 09:54，Jiajun Xie 写道： > > Congrats, Runkang! > > On Fri, 3 Nov 2023 at 04:46, Francis Chuang > wrote: > >> Congrats, Runkang! >> >> On 3/11/2023 3:54 am, Tanner Clary wrote: >>> Congratulations, Runkang! >>> >>> Best, >>> Tanner >>> >>> On Thu, Nov 2, 2023 at 9:11 AM Ran Tao wrote: >>> >>>> Congrats, Runkang! >>>> >>>> Best Regards, >>>> Ran Tao >>>> >>>> >>>> Benchao Li 于2023年11月2日周四 22:27写道： >>>> >>>>> Apache Calcite's Project Management Committee (PMC) has invited >>>>> Runkang He to become a committer, and we are pleased to announce that >>>>> they have accepted. >>>>> >>>>> Runkang has added many new functions to Hive and Spark libraries, and >>>>> various improvements around simplification and sub-query. Apart from >>>>> code contributions, Runkang also helps a lot with code reviews. >>>>> >>>>> Runkang, welcome, thank you for your contributions, and we look >>>>> forward to your further interactions with the community! If you wish, >>>>> please feel free to tell us more about yourself and what you are >>>>> working on. >>>>> >>>>> As your first commit, please add yourself to the contributors list [1] >>>>> and the community page will re-generate [2]. >>>>> >>>>> Benchao Li (on behalf of the Apache Calcite PMC) >>>>> >>>>> [1] >>>>> >> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >>>>> [2] https://calcite.apache.org/community/#project-members >>>>> >>>>> -- >>>>> >>>>> Best, >>>>> Benchao Li >>>>> >>>> >>> >>
Re: [ANNOUNCE] New committer: Ran Tao
Congrats Ran! Best, Dan Zou > 2023年10月27日 23:38，Tanner Clary 写道： > > Congrats Ran!! Have learned a lot from your pull requests! > > Best, > Tanner > > On Fri, Oct 27, 2023 at 8:32 AM Stamatis Zampetakis > wrote: > >> Apache Calcite's Project Management Committee (PMC) has invited Ran Tao to >> become a committer, and we are pleased to announce that he has accepted. >> >> Ran has implemented numerous improvements and fixes in the past few >> months. He has added new SQL functions to the Spark library, addressed >> various issues related to casts and type inference, enhanced >> documentation, and made various changes with the aim of improving code >> quality >> >> Ran, welcome, thank you for your contributions, and we look forward to your >> further interactions with the community! If you wish, please feel free to >> tell >> us more about yourself and what you are working on. >> >> As your first commit, please add yourself to the contributors list [1] >> and the community page will re-generate [2]. >> >> Stamatis (on behalf of the Apache Calcite PMC) >> >> [1] >> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >> [2] https://calcite.apache.org/community/#project-members >>
Re: [ANNOUNCE] New committer: Mihai Budiu
Congratulations, Mihai! Best, Dan Zou > 2023年10月26日 19:19，Sergey Nuyanzin 写道： > > Congratulations
Re: [ANNOUNCE] New committer: Dan Zou
Thank you everyone, it is a great honor for me and I will continue to contribute to the project. I have worked in ByteDance for 5 years, mainly engaged in Big data computing and OLAP related work, in which calcite played a very important role. Looking forward to continue cooperating with all you guys. Best, Dan Zou > 2023年7月6日 03:11，Julian Hyde 写道： > > Welcome, Dan! And thank you for your contributions so far. > >> On Jul 4, 2023, at 7:12 PM, Jiajun Xie wrote: >> >> Congratulations, Dan! >> >> On Tue, 4 Jul 2023 at 19:48, Runkang He wrote: >> >>> Congratulations, Dan! >>> >>> Best, >>> Runkang He >>> >>> Jacky Lau 于2023年7月4日周二 18:56写道： >>> >>>> Congratulations, Dan! >>>> >>>> Best Jacky! >>>> >>>> Benchao Li 于2023年7月4日周二 17:55写道： >>>> >>>>> Congratulations, Dan! >>>>> >>>>> Zhe Hu 于2023年7月4日周二 17:26写道： >>>>> >>>>>> Congrats! Dan. >>>>>> Replied Message >>>>>> | From | Francis Chuang | >>>>>> | Date | 07/04/2023 15:18 | >>>>>> | To | dev | >>>>>> | Subject | Re: [ANNOUNCE] New committer: Dan Zou | >>>>>> Congrats, Dan! >>>>>> >>>>>> On 4/07/2023 5:16 pm, Stamatis Zampetakis wrote: >>>>>>> Apache Calcite's Project Management Committee (PMC) has invited Dan >>>> Zou >>>>>> to >>>>>>> become a committer, and we are pleased to announce that they have >>>>>> accepted. >>>>>>> >>>>>>> Dan has been doing some great work for the project over the past >>> few >>>>>>> months. They implemented and enabled multiple new SQL functions for >>>>>>> BigQuery and MSSQL dialects, fixed some optimization rules, and >>>>>>> improved documentation and test code. >>>>>>> >>>>>>> Dan, welcome, thank you for your contributions, and we look forward >>>> to >>>>>> your >>>>>>> further interactions with the community! If you wish, please feel >>>> free >>>>>> to tell >>>>>>> us more about yourself and what you are working on. >>>>>>> >>>>>>> As your first commit, please add yourself to the contributors list >>>> [1] >>>>>>> and the community page will re-generate [2]. >>>>>>> >>>>>>> Stamatis (on behalf of the Apache Calcite PMC) >>>>>>> >>>>>>> [1] >>>>>> >>>> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >>>>>>> [2] https://calcite.apache.org/community/#project-members >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Best, >>>>> Benchao Li >>>>> >>>> >>>
Re: [ANNOUNCE] New committer: Jacky Lau
Congrats, Jacky! Best, Dan Zou > 2023年6月28日 20:01，Jiajun Xie 写道： > > Congratulations, Jacky! > > Looking forward to your greater contribution in the future. > > On Wed, 28 Jun 2023 at 19:40, xiong duan wrote: > >> Congratulations, Jacky! >> >> Francis Chuang 于2023年6月28日周三 18:57写道： >> >>> Congrats, Jacky! >>> >>> On 28/06/2023 8:46 pm, Stamatis Zampetakis wrote: >>>> Apache Calcite's Project Management Committee (PMC) has invited Jacky >>> Lau to >>>> become a committer, and we are pleased to announce that they have >>> accepted. >>>> >>>> Jacky has started contributing to the project very recently and in a >>>> very short time they landed many notable improvements around ARRAY and >>>> MAP functions with a particular focus on the Spark dialect. >>>> >>>> Jacky, welcome, thank you for your contributions, and we look forward >> to >>> your >>>> further interactions with the community! If you wish, please feel free >>> to tell >>>> us more about yourself and what you are working on. >>>> >>>> As your first commit, please add yourself to the contributors list [1] >>>> and the community page will re-generate [2]. >>>> >>>> Stamatis (on behalf of the Apache Calcite PMC) >>>> >>>> [1] >>> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >>>> [2] https://calcite.apache.org/community/#project-members >>> >>
Re: [ANNOUNCE] New committer: Zhe Hu
Congratulations, Zhe Hu! Best, Dan Zou > 2023年6月28日 20:24，Jiajun Xie 写道： > > Congratulations, Zhe Hu! > > Welcome! > > > On Wed, 28 Jun 2023 at 19:39, xiong duan wrote: > >> Congratulations, Zhe Hu! >> >> Francis Chuang 于2023年6月28日周三 19:05写道： >> >>> Congrats, Zhe Hu! >>> >>> On 28/06/2023 9:04 pm, Stamatis Zampetakis wrote: >>>> Apache Calcite's Project Management Committee (PMC) has invited Zhe Hu >> to >>>> become a committer, and we are pleased to announce that they have >>> accepted. >>>> >>>> Zhe Hu has been contributing to the project for a while now. They >>>> improved the stability of the ElasticSearch adapter, worked on >>>> supporting new java versions, and landed various enhancements around >>>> CONCAT and CONVERT functions. >>>> >>>> Zhe Hu, welcome, thank you for your contributions, and we look forward >>> to your >>>> further interactions with the community! If you wish, please feel free >>> to tell >>>> us more about yourself and what you are working on. >>>> >>>> As your first commit, please add yourself to the contributors list [1] >>>> and the community page will re-generate [2]. >>>> >>>> Stamatis (on behalf of the Apache Calcite PMC) >>>> >>>> [1] >>> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >>>> [2] https://calcite.apache.org/community/#project-members >>> >>
[jira] [Created] (CALCITE-5782) Add TO_HEX and FROM_HEX functions (enabled in BigQuery library)
Dan Zou created CALCITE-5782:
Summary: Add TO_HEX and FROM_HEX functions (enabled in BigQuery
library)
Key: CALCITE-5782
URL: https://issues.apache.org/jira/browse/CALCITE-5782
Project: Calcite
Issue Type: New Feature
Reporter: Dan Zou
Assignee: Dan Zou
Add TO_HEX and FROM_HEX functions (enabled in BigQuery library)
* TO_HEX: Converts a sequence of {{BYTES}} into a hexadecimal {{{}STRING{}}}.
* FROM_HEX: Converts a hexadecimal-encoded {{STRING}} into {{BYTES}} format
See more details in BigQuery doc
[TO_HEX|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#to_hex],
[FROM_HEX|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#from_hex]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5777) Simplify 'OperandTypes.STRING.or(OperandTypes.BINARY)' to 'OperandTypes.STRING'
Dan Zou created CALCITE-5777:
Summary: Simplify 'OperandTypes.STRING.or(OperandTypes.BINARY)' to
'OperandTypes.STRING'
Key: CALCITE-5777
URL: https://issues.apache.org/jira/browse/CALCITE-5777
Project: Calcite
Issue Type: Bug
Reporter: Dan Zou
We have used 'OperandTypes.STRING.or(OperandTypes.BINARY)' multiple times as
the SqlOperandTypeChecker in 'SqlLibraryOperators'. But this is not reasonable,
as the STRING family already contains BINARY family, which means we could
simplify 'OperandTypes.STRING.or(OperandTypes.BINARY)' to 'OperandTypes.STRING'.
{code:java}
// From 'SqlTypeName'
public static final List STRING_TYPES =
combine(CHAR_TYPES, BINARY_TYPES);
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [ANNOUNCE] New committer: Oliver Lee
Congratulations Oliver! Best, Dan Zou > 2023年6月13日 18:44，Stamatis Zampetakis 写道： > > Apache Calcite's Project Management Committee (PMC) has invited Oliver > Lee to become a committer, and we are pleased to announce that they > have accepted. > > Oliver started working with us around November 2022 and since then > they contributed multiple SQL functions to the Calcite repository > bringing lots of improvements to the BigQuery dialect. They improved > the extensibility of the SQL validator and pushed various fixes in > RelNode serialization to JSON. > > Oliver, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please > feel free to tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members
[jira] [Created] (CALCITE-5770) Implement SAFE_SUBTRACT function (enabled for BigQuery)
Dan Zou created CALCITE-5770: Summary: Implement SAFE_SUBTRACT function (enabled for BigQuery) Key: CALCITE-5770 URL: https://issues.apache.org/jira/browse/CALCITE-5770 Project: Calcite Issue Type: Sub-task Reporter: Dan Zou Add support for SAFE_SUBTRACT in BigQuery. SAFE_ADD: Equivalent to the addition operator '-', but returns NULL if overflow occurs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5729) Add BYTE_LENGTH function (enabled in BigQuery library)
Dan Zou created CALCITE-5729: Summary: Add BYTE_LENGTH function (enabled in BigQuery library) Key: CALCITE-5729 URL: https://issues.apache.org/jira/browse/CALCITE-5729 Project: Calcite Issue Type: New Feature Reporter: Dan Zou Assignee: Dan Zou BYTE_LENGTH(value): Returns the length of the STRING or BYTES value in BYTES, regardless of whether the type of the value is STRING or BYTES. For more details: [BYTE_LENGTH|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#byte_length] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New committer: Tanner Clary
Congratulations, Tanner! Best, Dan Zou > 2023年5月29日 03:26，Julian Hyde 写道： > > Welcome to the community, Tanner! You've earned it. > > On Sun, May 28, 2023 at 5:11 AM Jacky Lau wrote: >> >> Congratulations, Tanner! >> >> Best regards, >> Jacky >> >> Jiajun Xie 于2023年5月27日周六 11:28写道： >> >>> Congratulations and welcome, Tanner! >>> >>> On Sat, 27 May 2023 at 02:46, Sergey Nuyanzin wrote: >>> >>>> Congratulations, Tanner! >>>> >>>> On Fri, May 26, 2023 at 8:40 PM Bertil Chapuis >>> wrote: >>>> >>>>> Congratulations and welcome! >>>>> >>>>> Bertil >>>>> >>>>>> On 26 May 2023, at 18:58, Tanner Clary >>> .INVALID> >>>>> wrote: >>>>>> >>>>>> Hey everyone, >>>>>> >>>>>> Thank you all for the warm welcome! I have really enjoyed >>> contributing >>>> to >>>>>> the project so far and look forward to continuing to do so. >>>>>> >>>>>> About me: I graduated in May 2022 and since then I've been working on >>>>>> Looker @ Google. I'm eager to learn more about other areas of the >>>> project >>>>>> and hopefully collaborate with many of you along the way. >>>>>> >>>>>> Thanks again, >>>>>> Tanner Clary >>>>>> >>>>>> >>>>>> On Fri, May 26, 2023 at 5:12 AM Michael Mior >>> wrote: >>>>>> >>>>>>> Congratulations and welcome Tanner! >>>>>>> -- >>>>>>> Michael Mior >>>>>>> mm...@apache.org >>>>>>> >>>>>>> >>>>>>> On Fri, May 26, 2023 at 5:05 AM Stamatis Zampetakis < >>>> zabe...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Apache Calcite's Project Management Committee (PMC) has invited >>>> Tanner >>>>>>>> Clary to become a committer, and we are pleased to announce that >>> they >>>>>>>> have accepted. >>>>>>>> >>>>>>>> In less than 5 months, Tanner has introduced, enabled, and tested >>>> over >>>>>>>> 30 SQL functions in Calcite. They have been a driving force in >>>>>>>> improving the BigQuery dialect and by now an expert in library and >>>>>>>> parser changes. >>>>>>>> >>>>>>>> Tanner, welcome, thank you for your contributions, and we look >>>> forward >>>>>>>> to your further interactions with the community! If you wish, >>> please >>>>>>>> feel free to tell us more about yourself and what you are working >>> on. >>>>>>>> >>>>>>>> As your first commit, please add yourself to the contributors list >>>> [1] >>>>>>>> and the community page will re-generate [2]. >>>>>>>> >>>>>>>> Stamatis (on behalf of the Apache Calcite PMC) >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>> >>> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml >>>>>>>> >>>>>>>> [2] https://calcite.apache.org/community/#project-members >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> -- >>>> Best regards, >>>> Sergey >>>> >>>
[jira] [Created] (CALCITE-5728) Add ARRAY_TO_STRING function (enabled in BigQuery library)
Dan Zou created CALCITE-5728: Summary: Add ARRAY_TO_STRING function (enabled in BigQuery library) Key: CALCITE-5728 URL: https://issues.apache.org/jira/browse/CALCITE-5728 Project: Calcite Issue Type: New Feature Reporter: Dan Zou Assignee: Dan Zou ARRAY_TO_STRING: Returns a concatenation of the elements in array_expression as a STRING. For more details: [ARRAY_TO_STRING|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#array_to_string] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Share intermediate data between nodes during plan execution
Hi Vladislav, Could you please provide more details? Do you want to reuse part of RelNode trees? Best, Dan Zou > 2023年5月24日 06:44，Vladislav Matyushevski 写道： > > Vladislav
Re: subscribe calcite
Hi Jacky,
Welcome! You could subscribe to the lists by sending email to
{list}-subscr...@calcite.apache.org (where {list} is either “dev”, “issues”, or
“commits”).
Best,
Dan Zou
> 2023年5月23日 13:51，Jacky Lau 写道：
>
> Hi every one:
> I am Jacky Lau, a flink contributor, 7+ years of experience in big data
> development. I want to subscribe mails. glads to see you guys
> Best Jacky!
[jira] [Created] (CALCITE-5709) Add TO_BASE32 and FROM_BASE32 functions (enabled in BigQuery library)
Dan Zou created CALCITE-5709: Summary: Add TO_BASE32 and FROM_BASE32 functions (enabled in BigQuery library) Key: CALCITE-5709 URL: https://issues.apache.org/jira/browse/CALCITE-5709 Project: Calcite Issue Type: New Feature Reporter: Dan Zou - TO_BASE32(bytes_expr): Converts a sequence of BYTES into a base32-encoded STRING. - FROM_BASE32(string_expr): Converts the base32-encoded input string_expr into BYTES format. See more details in BigQuery doc: [TO_BASE32|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#to_base32] and [FROM_BASE32|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#from_base32] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Translate join predicates into filters
Hi Luca,
If I understand you correctly, what you are looking for is temporal table join.
Flink has implemented this based on Calcite, maybe is a good reference. You
could find more details in [1], [2], [3].
- [1] https://issues.apache.org/jira/browse/CALCITE-1912
- [2] https://lists.apache.org/thread/s8rx569p6tqbh8ybomodo5w3h2rbfvkr
- [3] https://issues.apache.org/jira/browse/FLINK-12269
Best,
Dan Zou
> 2023年5月17日 23:41，Luca Marchi 写道：
>
> Morning everyone,
> in our company we are running a POC using Apache Calcite, and we would like
> to collect some feedbacks from you for the scenario mentioned below.
>
> There is a service API that allows retrieving some `Book`s, and we would like
> to build a table adapter on top of this service; this API
> only accepts a set of IDs, and if no IDs are provided, no result is returned.
>
> ```
> interface BookService {
>/** Returns the books matching the given IDs.
> *
> * If not IDs is provided, no result is returned.
> */
>List findBooksByIds(Set ids);
> }
>
> record Book(String id, String title) {};
> ```
>
> A requirement of this table is that it has to support join, and we would like
> to support joining by ID in an efficient way.
>
> The goal is to define a rule that forces the query planner to always push
> down join predicates into a table scan.
>
> Given the following `book` table:
>
> ```java
> /** A table which represents books, queryable only by their ID. */
> final class BookTable extends AbstractTable implements FilterableTable {
> private final BookService service;
>
> BookTable(BookService service) {
>this.service = service;
> }
>
> @Override
> public RelDataType getRowType(RelDataTypeFactory typeFactory) {
>return new RelDataTypeFactory.Builder(typeFactory)
>.add("id", SqlTypeName.VARCHAR)
>.add("title", SqlTypeName.VARCHAR)
>.build();
> }
>
> @Override
> public Enumerable scan(DataContext root, List filters) {
>Set bookIds = getBooksId(filters);
>List result = service.findBooksByIds(bookIds)
>.stream()
>.map(b -> new Object[]{b.id, b.title})
>.toList();
>
>return Linq4j.asEnumerable(result);
> }
>
> private static Set getBooksId(List filters) {
>if (filters.size() != 1) {
> throw new IllegalArgumentException("Expected one filter to the ID,
> found: %d".formatted(filters.size()));
>}
>
>RexNode filter = filters.get(0);
>RexNode leftCondition = ((RexCall) filter).getOperands().get(0);
>RexNode rightCondition = ((RexCall) filter).getOperands().get(1);
>
>if (leftCondition instanceof RexInputRef left
>&& rightCondition instanceof RexLiteral right
>// The index of the ID column is 1.
>&& left.getIndex() == 1) {
> if (filter.isA(SqlKind.EQUALS)) {
>String bookId = right.getValue2().toString();
>return ImmutableSet.of(bookId);
> }
> if (filter.isA(SqlKind.SEARCH)) {
>@SuppressWarnings("unchecked")
>Sarg searchArguments = right.getValueAs(Sarg.class);
>return searchArguments.rangeSet.asRanges().stream()
>.map(Range::lowerEndpoint)
>.map(NlsString::getValue)
>.collect(toSet());
> }
>}
>throw new IllegalArgumentException("Unexpected operator, found:
> %s".formatted(filter.getKind()));
> }
> }
> ```
>
> The API of the `BookService` always expects a set of IDs, and in case of
> query like (assuming an entry in `book` matching the ID `'a'`):
> ```sql
> WITH config (id, val) AS (
>VALUES ('a', 3), ('b', 5)
> )
> SELECT b.* FROM books b
> INNER JOIN config ON b.id = config.id
> WHERE config.val > 4
> ```
>
> Calcite produces the following plan:
> ```
> EnumerableCalc(expr#0..2=[{inputs}], id=[$t1], title=[$t2])
> EnumerableMergeJoin(condition=[=($0, $1)], joinType=[inner])
>EnumerableSort(sort0=[$0], dir0=[ASC])
> EnumerableCalc(expr#0..1=[{inputs}], id=[$t0])
>EnumerableTableScan(table=[[books]])
>EnumerableSort(sort0=[$0], dir0=[ASC])
> EnumerableTableScan(table=[[books]])
> ```
>
> This means Calcite performs a full table scan of the `book` table, and since
> the `RexNode` filters in the `scan` method are empty, no result is returned
> (in this example we are using a value statement scoped views, but ideally the
> solution we are looking for should be valid for other table).
> Under some defined circustances, Postgres generates Nested-Loop query plan
> for join: it first selects the row of the table A matching
Re: RelDataTypeSystemImpl#deriveSumType question.
Hi stanilovsky, You can customize your behavior in RelDataTypeSystem. We have discussed this issue before, you could find more details in [1] and [2]. - [1] https://issues.apache.org/jira/browse/CALCITE-5604 - [2] https://issues.apache.org/jira/browse/CALCITE-1945 Best, Dan Zou > 2023年4月25日 19:14，stanilovsky evgeny 写道： > > Hello ! > Probably can anyone explain why RelDataTypeSystemImpl#deriveSumType returns > INTEGER for incoming INTEGER type ? > I run simple: SELECT i, SUM(i) FROM integers i1 GROUP BY i ORDER BY i; and i > expect that return type will be BIGINT. > Did i miss something here ? > Thanks !
Re: [DISCUSS] Disable JIRA worklog for GitHub PRs
+1, it would reduce a lot of noise. Best, Dan Zou > 2023年4月19日 19:37，Benchao Li 写道： > > +1, > > One thing that may affect current workflow is sometimes I only watch the > Jira, and will get notified from the PR notification. If we are going to > disable it, I need to watch the PR too to get the notification for cases > that I'm interested in both the Jira discussion and PR comments. But that > won't be a big problem for me. > > Michael Mior 于2023年4月19日周三 18:43写道： > >> +1 from me as well >> >> On Wed, Apr 19, 2023, 04:19 Stamatis Zampetakis wrote: >> >>> Hello, >>> >>> Everything that happens in a GitHub PR creates a worklog entry under >>> the respective JIRA ticket. >>> For every worklog entry we receive a notification from j...@apache.org >>> when we are watching an issue. The worklog entry and email >>> notification usually appear messy. >>> >>> Moreover, if we are watching the GitHub PR we are going to get a >>> notification from notificati...@github.com which has the same content >>> with the JIRA worklog entry and is much more readable. >>> >>> Finally, the PR notification is also going to >>> comm...@calcite.apache.org so those who are subscribed to that list >>> will get the same notification three times. >>> >>> Personally, I never read the JIRA worklog notifications and I largely >>> prefer those from notificati...@github.com. >>> >>> How do you feel about disabling the worklog entries in JIRA coming >>> from GitHub PRs? >>> >>> For archiving purposes, the notifications already go to commits@ so we >>> don't lose anything from disabling the worklog entries. On the >>> contrary, I find that this would reduce the noise and redundancy on >>> our inboxes. >>> >>> Concretely this is what I have in mind in terms of change: >>> https://github.com/apache/calcite/pull/3166 >>> >>> Best, >>> Stamatis >>> >> > > > -- > > Best, > Benchao Li
[jira] [Created] (CALCITE-5642) Add SHA256, SHA512 functions
Dan Zou created CALCITE-5642: Summary: Add SHA256, SHA512 functions Key: CALCITE-5642 URL: https://issues.apache.org/jira/browse/CALCITE-5642 Project: Calcite Issue Type: New Feature Reporter: Dan Zou Add SHA256, SHA512 functions and enable them for BigQuery. SHA256: Computes the hash of the input using the [SHA-256 algorithm|https://en.wikipedia.org/wiki/SHA-2]. This function returns 32 bytes. SHA512: Computes the hash of the input using the [SHA-512 algorithm|https://en.wikipedia.org/wiki/SHA-2]. This function returns 64 bytes. See more details in [BigQuery Doc|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#sha256] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Requesting PR review
Hi all, Could someone help reviewing my PR? Here is the PR list: - CALCITE-5599: Add SqlLibrary.ALL and change the library of TANH, COSH, SINH to it - https://github.com/apache/calcite/pull/3128 - CALCITE-4771: change the value of the CAST function to be nullable - https://github.com/apache/calcite/pull/3136 Any help would be appreciated. Best, Dan Zou
[jira] [Created] (CALCITE-5640) Implement SAFE_ADD function
Dan Zou created CALCITE-5640: Summary: Implement SAFE_ADD function Key: CALCITE-5640 URL: https://issues.apache.org/jira/browse/CALCITE-5640 Project: Calcite Issue Type: Sub-task Reporter: Dan Zou Assignee: Dan Zou Add support for SAFE_ADD in BigQuery. *SAFE_ADD*: Equivalent to the addition operator '+', but returns NULL if overflow occurs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Sharing the load of reviewing PRs
+1, thanks Julian for proposing this. From my observation, there are many pending PRs in Calcite and only a few active committers, this puts a lot of pressure on these committers. For example Julian have reviewed 34 PR in 2023 Q1, it is an unimaginable number. I am very supportive of achieving a mechanism to improve the review efficiency of PRs, and also I would like to make contribution in reviewing PRs. Best, Dan Zou > 2023年4月11日 01:56，Julian Hyde 写道： > > I don't enjoy reviewing and merging PRs. And every time I do, I feel > like a sucker, because there are over a few dozen committers who are > enjoying the project and not doing the work. (There is a small group > of committers who regularly review and merge PRs. I don't know how > they feel about the task, but I am immensely grateful.) > > I think I would review more PRs if I saw others doing the same. > > Can we figure out a fairer way to distribute the load? For release > managers (approximately the same amount of work, but compressed into a > few hours or days) we have successfully run a rota for several years. > Could we do something similar with PRs? > > I propose the following. For each calendar month, there is a PR > manager and 6 - 8 reviewers. The PR manager does not review PRs, but > assigns them to reviewers, and politely reminds reviews to keep the PR > moving. > > The PR manager's goals are: > * every non-draft PR is reviewed within 3 days of submission, > * every PR is merged within 3 days of being done; > * rotate duties so that no reviewer is asked to review more than 4 > PRs per month; > * email a report at the end of the month; > * work down the backlog of historic PRs if it's a slow month. > > The PR manager rotates every month. The reviewers can rotate if they > wish, but I suspect most will stay in the pool for several months, > because the reviewing load is not very heavy, and because they see > others doing the work. > > Other notes: > * Non-committers would be welcome to join the pool of reviews (and > that would be a good way to earn the committer bit) and a committer > could merge when the PR is approved. > * If committers join the pool, that's a good way to earn PMC membership. > * Committers who are not in the pool are welcome to review PRs and > assign PRs to themselves (but expect to be nagged by the PR manager if > you don't review in a timely manner). > > What do you think? Would you join this scheme if we introduced it? If > you agree please +1; also happy to see revisions to this suggestion or > other ideas to share the work. > > Julian
Re: Question about the deprecated CachingRelMetadataProvider
Hi Stephen, I can provide some information that I have observed, there are two ways to drive metadata in Calcite: code generated based handler and another way with java reflection. And we prefer the former for performance and safety reasons. And the 'CachingRelMetadataProvider' is used for java reflection based metadata caching, and this is the reason it has been deprecated. You could find more info in[1] - [1] https://issues.apache.org/jira/browse/CALCITE-4544 Best, Dan Zou > 2023年4月6日 20:01，Stephen Carlin 写道： > > JaninoRelMetadataProvider
Re: Draft: board report for 2023 Q1
+1, Thanks Stamatis Best, Dan Zou > 2023年4月7日 10:11，Benchao Li 写道： > > +1, it's very insightful, thanks Stamatis! > > Francis Chuang 于2023年4月7日周五 05:48写道： > >> +1 Thanks Stamatis! >> >> On 7/04/2023 4:44 am, Ruben Q L wrote: >>> Thanks Stamatis for preparing the report. >>> LGTM. >>> >>> >>> On Thu, Apr 6, 2023 at 1:17 PM Stamatis Zampetakis >>> wrote: >>> >>>> Hello, >>>> >>>> Below you can find a draft of this quarter's board report. I plan to >>>> submit it next Tuesday (April 11, 2023). >>>> Please let me know if you have any additions or corrections. >>>> >>>> Best regards, >>>> Stamatis >>>> - >>>> >>>> ## Description: >>>> Apache Calcite is a highly customizable framework for parsing and >> planning >>>> queries on data in a wide variety of formats. It allows database-like >>>> access, >>>> and in particular a SQL interface and advanced query optimization, for >> data >>>> not residing in a traditional database. >>>> >>>> Avatica is a sub-project within Calcite and provides a framework for >>>> building >>>> local and remote JDBC and ODBC database drivers. Avatica has an >> independent >>>> release schedule and its own repository. >>>> >>>> ## Issues: >>>> There are no issues requiring board attention. >>>> >>>> ## Membership Data: >>>> Apache Calcite was founded 2015-10-22 (7 years ago) >>>> There are currently 63 committers and 27 PMC members in this project. >>>> The Committer-to-PMC ratio is 7:3. >>>> >>>> Community changes, past quarter: >>>> - Benchao Li was added to the PMC on 2023-01-27 >>>> - Alex Plehanov was added as committer on 2023-01-06 >>>> - Jiajun Xie was added as committer on 2023-02-10 >>>> - Istvan Toth was added as committer on 2023-01-25 >>>> >>>> ## Project Activity: >>>> Apache Calcite 1.34.0 was released on 2023-03-14. It contains >> contributions >>>> from 18 contributors, and resolves 34 issues. It’s worth highlighting >> the >>>> introduction of QUALIFY clause (CALCITE-5268), which facilitates >> filtering >>>> the >>>> results of window functions. Among other improvements and fixes, it adds >>>> roughly 15 new functions in BigQuery library for handling dates, times, >> and >>>> timestamps. >>>> >>>> Apache Calcite 1.33.0 was released on 2023-02-06. It contains >> contributions >>>> from 33 contributors, and resolves 107 issues. It’s worth highlighting >> the >>>> support for custom time frames (CALCITE-5155), the new MEASURE type and >>>> AGGREGATE aggregate function (CALCITE-5105) as well as the many >>>> improvements >>>> to the BigQuery dialect (CALCITE-5180). >>>> >>>> Apache Calcite Avatica 1.23.0 was released on 2023-01-19. It fixes bugs >> in >>>> Statement.getUpdateCount(), ResultSet.getObject; and supports >>>> HTTP_BAD_REQUEST >>>> and configuring fetch size and SSL key-store type. Also, there are >> various >>>> improvements to DateTimeUtils and ByteString. >>>> >>>> On 2023-03-15, there was an online meetup of the Calcite community with >>>> approximately 50 participants joining the call. We had three very >>>> interesting >>>> presentations around adding measures in SQL, incremental view >> maintenance >>>> for >>>> streaming engines, and debugging planner issues, followed by open >>>> discussion. >>>> The videos from the meetup as well as the slides were published online >>>> shortly >>>> after the event. >>>> >>>> ## Community Health: >>>> The project remains super healthy and there is a general increase in >>>> traffic >>>> almost in every aspect of the project (dev@, JIRA, and GitHub). >>>> >>>> The most notable increase in traffic was for dev mailing list (46%). One >>>> factor that led to this increase is the addition of four new members in >> the >>>> project and the traditional welcoming emails that usually come along. >>>> Another >>>> one, would proba
Re: Calcite development video
Michael, thanks for sharing this, it is very interesting! Best, Dan Zou > 2023年4月7日 03:15，Michael Mior 写道： > > Hi all, > > I happened to come across the Gource visualization tool today. It takes a > log of repository history and generates a timelapse history of development. > I decided to do one for Calcite. It's fairly entertaining to watch. Thanks > to all contributors! > > https://www.youtube.com/watch?v=qSpokrc0Ofs > > -- > Michael Mior > mm...@apache.org
Re: CI requiring approval for external contributors
+1 for this, it is something out of my expectation when I created my first PR in calcite and informed that approval is required. Best, Dan Zou > 2023年3月28日 21:37，Benchao Li 写道： > > I'm +1 for changing it back. > > I was also thinking about starting a discussion about this before, but I > didn't know how to achieve it. Thanks Julian for sharing this with us. > > Julian Hyde 于2023年3月28日周二 20:57写道： > >> (Forwarding from the Druid list for discussion.) >> >> Julian >> >> Begin forwarded message: >> >>> From: Gian Merlino >>> Date: March 28, 2023 at 1:24:42 AM CDT >>> To: d...@druid.apache.org >>> Subject: CI requiring approval for external contributors >>> Reply-To: d...@druid.apache.org >>> >>> Recently, ASF GitHub repos had their defaults for GitHub Actions >> changed to >>> "always require approval for external contributors". In Slack, Karan >>> pointed out that Airflow has recently submitted a ticket to have that >>> changed back: https://issues.apache.org/jira/browse/INFRA-24200. IMO, we >>> should do the same. I don't think we have a problem with fake PRs, but we >>> can always improve our responsiveness to contributors from outside the >>> project! Every little bit helps, including running CI automatically. >>> >>> If others have opinions on this, let me know. I'd like to raise our own >>> ticket to change our default. >>> >>> Gian >> > > > -- > > Best, > Benchao Li
[jira] [Created] (CALCITE-5608) Implement ACOSH function
Dan Zou created CALCITE-5608: Summary: Implement ACOSH function Key: CALCITE-5608 URL: https://issues.apache.org/jira/browse/CALCITE-5608 Project: Calcite Issue Type: Sub-task Reporter: Dan Zou ACOSH(X): Computes the inverse hyperbolic cosine of X. See more details in [BigQuery DOC|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#acosh] h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5606) Add SqlLibrary.ALL and change the library of TANH, COSH, SINH to it
Dan Zou created CALCITE-5606: Summary: Add SqlLibrary.ALL and change the library of TANH, COSH, SINH to it Key: CALCITE-5606 URL: https://issues.apache.org/jira/browse/CALCITE-5606 Project: Calcite Issue Type: Sub-task Reporter: Dan Zou Assignee: Dan Zou Add a new SqlLibrary named 'ALL' with abbreviation '*', if an operator is marked with this SqlLibrary, it means it's a common one and should belong to all the library. TANH, COSH, SINH are three of these operators. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: register a custom udf
Hi Guillaume, are you using your own udf and some functions in
SqlStdOperatorTable at the same time? If yes, SqlOperatorTables#chain is the
right way to organize multiple SqlOperatorTable.
Best,
Dan Zou
> 2023年3月21日 02:59，Guillaume Masse 写道：
>
> I found the solution here:
> https://stackoverflow.com/questions/58268828/apache-calcite-registering-udfs-for-use-in-relbuilder
>
> The idea is to use
>
> `SqlOperatorTables.of`` to register a list of SqlFunction
>
> then use `SqlOperatorTables.chain` to join it with
> `SqlStdOperatorTable.instance()`
>
> On Mon, Mar 20, 2023 at 1:39 PM Guillaume Masse <
> masse.guilla...@narrative.io> wrote:
>
>> ah this is Scala, it's a bit shorter to write
>>
>> On Mon, Mar 20, 2023 at 1:12 PM Julian Hyde
>> wrote:
>>
>>> What language is this?
>>>
>>>> On Mar 20, 2023, at 9:01 AM, Guillaume Masse <
>>> masse.guilla...@narrative.io> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> How can we register a custom udf? I found some solutions on
>>> Stackoverflow
>>>> but they are out of date (
>>>>
>>> https://stackoverflow.com/questions/44147819/adding-a-user-defined-function-to-calcite
>>>> )
>>>>
>>>> So far what I have:
>>>>
>>>> // top level class
>>>> class CustomUdf {
>>>> def foo(in: Double): Double = in + 1
>>>> }
>>>>
>>>> // somewhere else
>>>> val typeFactory = new JavaTypeFactoryImpl()
>>>>
>>>> val root = CalciteSchema.createRootSchema(false)
>>>> root.plus().add("foo", ScalarFunctionImpl.create(classOf[CustomUdf],
>>> "foo"))
>>>>
>>>> val catalogReader = new CalciteCatalogReader(
>>>> root,
>>>> Collections.emptyList(),
>>>> typeFactory,
>>>> CalciteConnectionConfig.DEFAULT
>>>> )
>>>>
>>>> val operators =
>>>> SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
>>>> SqlLibrary.STANDARD,
>>>> SqlLibrary.SPARK
>>>> )
>>>>
>>>> val sqlValidator = SqlValidatorUtil.newValidator(
>>>> operators,
>>>> catalogReader,
>>>> typeFactory,
>>>> SqlValidator.Config.DEFAULT
>>>> )
>>>>
>>>> simple expression failed to
>>>> validate,org.apache.calcite.runtime.CalciteContextException: From line
>>> 1,
>>>> column 1 to line 1, column 22: No match found for function signature
>>>> toto())
>>>>
>>>> From what I saw on Stackoverflow, it looks like I need to also add the
>>>> function to the operator table.
>>>>
>>>> Thanks!
>>>> Guillaume Massé (马赛卫)
>>>
>>
Re: Requesting PR Reviews
I can provide some help to review these PRs. Best, Dan Zou > 2023年3月18日 02:32，Tanner Clary 写道： > > Hello Calcite Developers, > > > My name is Tanner and I have been working to enable several functions for > the BigQuery dialect over the last few months. I have four open pull > requests at the moment that could use a review if anyone has the time. I > would really appreciate it. Three of them have already been reviewed once > and are now awaiting re-review/merge (denoted with *). > > The JIRA Issues and their corresponding Pull Requests are: > > > - [CALCITE-5554 <https://issues.apache.org/jira/browse/CALCITE-5554>] - [PR > #3094 <https://github.com/apache/calcite/pull/3094>]* > - [CALCITE-5564 <https://issues.apache.org/jira/browse/CALCITE-5565>] - [PR > #3099 <https://github.com/apache/calcite/pull/3099>]* > - [CALCITE-5580 <https://issues.apache.org/jira/browse/CALCITE-5580>] - [PR > #3109 <https://github.com/apache/calcite/pull/3109>] > - [CALCITE-5585 <https://issues.apache.org/jira/browse/CALCITE-5585>] - [PR > #3114 <https://github.com/apache/calcite/pull/3114>]* > > > Thank you in advance to anyone who is able to assist! > > Best, > > Tanner Clary
Re: [DISCUSS] Towards Calcite 1.34.0
+1, Thank you for being the RM Best, Dan Zou > 2023年3月10日 07:18，Stamatis Zampetakis 写道： > > Draft release notes: https://github.com/apache/calcite/pull/3103 > > On Thu, Mar 9, 2023 at 11:06 PM Charles Givre wrote: > >> +1. Thank you very much for doing this! >> -- C >> >> >> >> >>> On Mar 9, 2023, at 5:01 PM, Francis Chuang >> wrote: >>> >>> +1 Thanks for being RM, Stamatis! >>> >>> On 10/03/2023 8:57 am, Stamatis Zampetakis wrote: >>>> Hello, >>>> It's been a bit more than a month since our last release [1] and there >> are >>>> currently ~30 new commits in master. >>>> Although we could wait a bit longer before cutting a new release, it >> would >>>> be nice to facilitate Drills efforts [2] in upgrading to the latest >> Calcite >>>> release by providing a version with the fix for the small breaking >> change >>>> reported under CALCITE-5522. >>>> If there are no objections, I will create an RC in the following days. >>>> If there are other must fix for 1.34.0 please let us know so that we can >>>> plan accordingly. >>>> Best, >>>> Stamatis >>>> [1] https://calcite.apache.org/docs/history.html#v1-33-0 >>>> [2] https://lists.apache.org/thread/yo4075khphgbw1dzzo6w4kpvyl7280x9 >> >>
Re: [REVIEW REQUEST] CALCITE-5160 and CALCITE-5403
I would like to participate in the review of these two PRs Best, Dan Zou > 2023年3月6日 23:03，Dmitry Sysolyatin 写道： > > Hello, > I would like to request review for the following tickets: > > CALCITE-5160: ANY/SOME, ALL operators should support collection expressions > [1] > CALCITE-5403: PostgreSQL dialect should support SET, RESET, BEGIN, SHOW, > ROLLBACK, COMMIT [2] > > [1] https://issues.apache.org/jira/browse/CALCITE-5160 > [2] https://issues.apache.org/jira/browse/CALCITE-5403 > > Thanks!
Re: [ANNOUNCE] New self-serve portal for projects, committers, and contributors
+1, this allows newcomers to quickly participate in the project. Best, Dan Zou > 2023年3月2日 10:10，Benchao Li 写道： > > +1, the new process is simpler and easier for users. > > Francis Chuang 于2023年3月2日周四 08:38写道： > >> The ASF has published a new process for contributors to request Jira >> accounts. We should update our documentation to direct contributors to >> this new form and use it as part of the onboarding process. >> >> What do you all think? >> >> Francis >> >> Forwarded Message >> Subject: [ANNOUNCE] New self-serve portal for projects, committers, and >> contributors >> Date: Wed, 1 Mar 2023 18:12:25 -0600 >> From: Daniel Gruno >> Reply-To: us...@infra.apache.org >> To: annou...@infra.apache.org >> >> Hi folks, >> During today's Infrastructure round-table session[1] we launched a new >> infrastructure self-serve portal that replaces our old self-serve site. >> >> The URL remains the same, https://selfserve.apache.org/ , and while the >> self-serve offerings are largely the same as well, I do want to call out >> the new "Request a Jira account" feature on there. >> >> Prior to today, projects were tasked with managing new Jira accounts >> themselves, including handling the initial request from users. The new >> self-serve workflow for Jira accounts is aimed at making this a faster, >> more streamlined experience for all projects, while also addressing data >> privacy concerns and time constraints of projects. Any project using >> Jira can now refer their users/contributors to >> https://selfserve.apache.org/jira-account.html for new Jira account >> requests, which will take care of all the needed paperwork for new >> account requests and loop in projects for the final review and approval >> of the accounts. >> >> We hope this new feature will be well received. If there are any >> questions, concerns, or other feedback regarding the new self-serve >> portal, please do reach out to us at: us...@infra.apache.org >> >> With regards, >> Daniel on behalf of ASF Infra. >> >> >> [1] https://infra.apache.org/roundtable.html >> > > > -- > > Best, > Benchao Li
Re: How to use TO_DATE function in Calcite SQL,fun option not effective
Hi Ein, I have checked the calcite doc(https://calcite.apache.org/docs/reference.html) and do not find the 'STR_TO_DATE'. And I also quickly checked the calcite code, and it seem 'STR_TO_DATE' only supported with MySQL Dialect, you could find the usage in RelToSqlConverterTest#testFloorWeek. Best, Dan Zou > 2023年2月25日 20:38，Ein 写道： > > STR_TO_DATE
