[jira] [Created] (CALCITE-4514) Prefer merge new relset into old relset when they are parent set of each other
Botong Huang created CALCITE-4514: - Summary: Prefer merge new relset into old relset when they are parent set of each other Key: CALCITE-4514 URL: https://issues.apache.org/jira/browse/CALCITE-4514 Project: Calcite Issue Type: Improvement Reporter: Botong Huang When merging two relsets, we have two preferences: 1. Merge parent relset into child relset 2. Merge newer relset into older relset Currently, when the two relsets are parent set of each other, we randomly pick a merge order without checking the second condition above. For performance reasons, we should, to avoid unnecessary churn. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Google Summer of Code 2021
There was some discussion about having students contribute to the project as part of GSoC in the past, but I don't think it has happened before. See: - https://lists.apache.org/thread.html/b4cb29fdfc5128386081401f75d99e888cd2af7cc79a8d0d63df2af7%401455998281%40%3Cdev.calcite.apache.org%3E - https://lists.apache.org/thread.html/c0ef0701658caaa0c51b04c33da99bdde621d83fc0fb5aa0e319347e%40%3Cdev.calcite.apache.org%3E In the past, Michael had some students working on parts of Calcite, but not in GSoC capacity: - https://lists.apache.org/thread.html/f0ea9594f151e7f6ad9b7f305fc2c87d99f64acac2cc76e179a1a1f7%40%3Cdev.calcite.apache.org%3E I would definitely love to see this happen. It would be a great opportunity for students to learn something new and interesting and a good opportunity for the project as well. Francis On 26/02/2021 6:44 am, Rui Wang wrote: Hi community, Does Calcite ever participate in Google Summer of Code (GSoC) [1]. What do you think of creating smaller projects and allowing students to pick them up? I used to host two GSoC students for Apache Beam and I found the collaboration was smooth and students could add valuable additions to the project. [1]: https://community.apache.org/gsoc.html -Rui
Google Summer of Code 2021
Hi community, Does Calcite ever participate in Google Summer of Code (GSoC) [1]. What do you think of creating smaller projects and allowing students to pick them up? I used to host two GSoC students for Apache Beam and I found the collaboration was smooth and students could add valuable additions to the project. [1]: https://community.apache.org/gsoc.html -Rui
[jira] [Created] (CALCITE-4513) RelBuilder.aggregate destroys input traits when pruning unused fields of project
Stamatis Zampetakis created CALCITE-4513:
Summary: RelBuilder.aggregate destroys input traits when pruning
unused fields of project
Key: CALCITE-4513
URL: https://issues.apache.org/jira/browse/CALCITE-4513
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.23.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
RelBuilder attempts to prune unused fields when the underlying input is a
project and the respective configuration
({{RelBuilder.Config#pruneInputOfAggregate}}) is enabled (CALCITE-3763). As a
result of this optimization a new project operator is introduced in the plan.
The new operator is created via {{Project#copy}} method but instead of
retaining or adapting the previous traitset a new one is created
(https://github.com/apache/calcite/blob/94502f51e6a12cd53cec88b39d21b4fec6070428/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L1922).
{code:java}
r = project.copy(cluster.traitSet(), project.getInput(), newProjects,
builder.build());
{code}
The main problem with that is that a {{HiveProject}} operator which used to be
in {{HiveConvention}} will be transformed to a {{HiveProject}} operator in
{{Convention.None}}. Obviously other traits (apart from {{Convention}})
potentially present in the operator are also lost after the transformation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Assistance with basic calcite program
Interestingly, the error is now different. Earlier, the error I used to get was: Exception in thread "main" org.apache.calcite.runtime.CalciteContextException: From line 9, column 9 to line 9, column 26: No match found for function signature SUBSTR(, , ) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:467) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:867) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:852) at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5003) at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1835) After your suggestion, I get the following error. It feels like it has made progress but now it does not have an implementation for the SUBSTR function. Is there a way for me to map SUBSTR to the underlying SUBSTRING operator? Exception in thread "main" java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlFunction: SUBSTR at org.apache.calcite.util.Util.needToImplement(Util.java:975) at org.apache.calcite.sql.SqlOperator.getOperandCountRange(SqlOperator.java:198) at org.apache.calcite.sql.SqlUtil.lambda$filterRoutinesByParameterCount$3(SqlUtil.java:584) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:637) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.common.collect.Iterators.addAll(Iterators.java:355) at com.google.common.collect.Lists.newArrayList(Lists.java:143) at org.apache.calcite.sql.SqlUtil.lookupSubjectRoutines(SqlUtil.java:515) at org.apache.calcite.sql.SqlUtil.lookupRoutine(SqlUtil.java:443) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:241) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5766) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5753) at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1753) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1738) at org.apache.calcite.sql.type.InferTypes.lambda$static$0(InferTypes.java:47) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1933) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1941) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateWhereOrOn(SqlValidatorImpl.java:4093) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateWhereClause(SqlValidatorImpl.java:4085) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3424) at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1064) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1038) at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:232) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1013) at
Re: Assistance with basic calcite program
On Wed, Feb 24, 2021 at 9:51 AM Ruben Q L wrote: > Honestly, I don't know the details. I just know that on our application we > use "withIdentifierExpansion(true)", to avoid potential issues (I guess > like the one you had). > Several internal Calcite classes do the same, maybe someone with more > experience on the SqlValidator could give more info. > > > Regarding the SUBSTR issue, I was expecting that changing the > CalciteConnectionConfig as: > CalciteConnectionConfig.DEFAULT > .set(CalciteConnectionProperty.CASE_SENSITIVE, "false") > .set(CalciteConnectionProperty.FUN, "oracle") > would do the trick; but it seems it does not. > > Instead, what I think would work in your test would be leaving the > CalciteConnectionConfig as you have it, and enlarge the > SqlStdOperatorTable: instead of just > SqlStdOperatorTable.instance() > use > SqlOperatorTables.chain(SqlStdOperatorTable.instance(), > > SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(SqlLibrary.ORACLE)) > (or whatever SqlLibrary suits your needs). Could you verify it? > Unfortunately, the suggestion did not seem to work :( I tried a) setting the FUN property, b) just using the chained operator table and c) both of those together. I am continuing to look at various Calcite unit tests etc to see if I can spot any ideas. Meanwhile, any other suggestions are welcome. Thanks!
Re: count(if(a=1, 1, null)) syntax support
Hi,you Zhuang I find clacite have SqlIfFunction,but it don't accomplish. you Zhuang 于2021年2月21日周日 上午2:39写道： > when does calcite support count(if(a=1, 1, null)) syntax ? spark, hive, > clickhouse, impala, presto support the syntax for a long time. >
Re: Performance degradation during query planning (vs Calcite 1.25)
Well done, thanks Lgor. Regards! Aron Tao Chunwei Lei 于2021年2月22日周一 上午11:15写道： > Thank you for your work, Lgor. > > Would love to review the PR. > > > Best, > Chunwei > > > On Fri, Feb 12, 2021 at 1:49 AM Igor Lozynskyi > wrote: > > > Hi Calcite community! > > > > Before the upcoming 1.27 release, in a downstream project, my team > started > > the migration procedure. We had to accommodate our code for SARGs > changes, > > but other than that, everything looks great. Out test suites are green, > and > > everything works like a charm. However, when compared with 1.25, Calcite > > optimizes queries slower, especially queries with many Rels, especially > > with many JOINs. > > > > Here, I described the essence of the issue: > > https://issues.apache.org/jira/browse/CALCITE-4494. > > > > Fortunately, the potential fix is trivial: > > https://github.com/apache/calcite/pull/2347. > > > > I am ready to assign the issue to myself. And it would be great if the PR > > is reviewed and accepted (if no objections) before the 1.27 release. > > > > Thanks! > > >
