Re: Request to be a contributor in jira

[Francis Chuang]

Hey Xiong,

I've added you as a contributor in JIRA and have assigned the issue to you.

Francis

On 16/04/2021 12:54 pm, 段雄 wrote:

Hi, My name is xiong duan,I'm working for Hikvision. I create a issue about
CALCITE-4585  . I want
try to be a contributor. can you assign this issue to me? Thank you very
much.

[jira] [Created] (CALCITE-4586) Make "RelBuilder.Config#simplify" configurable in "piglet.PigRelBuilder"

[Jiatao Tao (Jira)]

Jiatao Tao created CALCITE-4586:
---

 Summary: Make "RelBuilder.Config#simplify" configurable in 
"piglet.PigRelBuilder"
 Key: CALCITE-4586
 URL: https://issues.apache.org/jira/browse/CALCITE-4586
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiatao Tao
Assignee: Jiatao Tao






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Request to be a contributor in jira

[段雄]

Hi, My name is xiong duan,I'm working for Hikvision. I create a issue about
CALCITE-4585  . I want
try to be a contributor. can you assign this issue to me? Thank you very
much.

Re: "PigRelBuilder#create" maybe a wrong place to set Hook.REL_BUILDER_SIMPLIFY

[JiaTao Tao]

Sure, thanks, Julian! I'll take a look.

Regards!

Aron Tao


Julian Hyde  于2021年4月16日周五 上午1:43写道：

> Note that there are TWO PigRelBuilder.java files in the code base.
> (Apologies for that! Both names made sense at the time.)
>
> This particular line of code is in piglet. I suspect that it was added
> by me or Khai Tran in order to make some tests pass, and get a big PR
> ( https://github.com/apache/calcite/pull/1265 ) across the finishing
> line. If you want to find out what it does, remove the line and see
> what tests fail.
>
> I doubt that it is very harmful (at worst, PigRelBuilder misses some
> optimizations), but I suppose some users might like some more control.
> Feel free to enhance it.
>
> On Thu, Apr 15, 2021 at 6:25 AM JiaTao Tao  wrote:
> >
> > Hi, in PigRelBuilder#create, we set Hook.REL_BUILDER_SIMPLIFY to false:
> >
> >   public static PigRelBuilder create(FrameworkConfig config) {
> > final RelBuilder relBuilder = RelBuilder.create(config);
> > Hook.REL_BUILDER_SIMPLIFY.addThread(Hook.propertyJ(false));
> > return new PigRelBuilder(
> > transform(config.getContext(), c -> c.withBloat(-1)),
> > relBuilder.getCluster(),
> > relBuilder.getRelOptSchema());
> >   }
> >
> > I have two doubts:
> > 1. Why we set FALSE here cuz its default value is true;
> > 2. This maybe not the suitable place to set  "Hook.REL_BUILDER_SIMPLIFY",
> > IMO we should use this in user code, not in the frame code, cuz once we
> > done this in frame code, we can not change this config anymore.
> >
> > Regards!
> >
> > Aron Tao
>

[jira] [Created] (CALCITE-4585) when use prepareStatement parse and run RelNode or Queryable, Maybe Confusing log information be thrown

[duan xiong (Jira)]

duan xiong created CALCITE-4585:
---

 Summary: when use prepareStatement parse and run RelNode or 
Queryable, Maybe Confusing log information be thrown
 Key: CALCITE-4585
 URL: https://issues.apache.org/jira/browse/CALCITE-4585
 Project: Calcite
  Issue Type: New Feature
  Components: core
Affects Versions: 1.26.0
Reporter: duan xiong


This code capture the exception,But Only the output of SQL exception is 
processed.So need to add the procedure to handle extra exception. 
{code:java}
// 
try {
  final Meta.Signature signature =
  parseQuery(query, createPrepareContext(), -1);
  final CalcitePreparedStatement calcitePreparedStatement =
  (CalcitePreparedStatement) factory.newPreparedStatement(this, null,
  signature, resultSetType, resultSetConcurrency, resultSetHoldability);
  server.getStatement(calcitePreparedStatement.handle).setSignature(signature);
  return calcitePreparedStatement;
} catch (Exception e) {
  throw Helper.INSTANCE.createException(
  "Error while preparing statement [" + query.sql + "]", e);
}

{code}
When we run RelNode or  queryable show wrong,this exception can be:

"Error while preparing statement [null]"

 can't log the real exception



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Default disable the RexNode normalization(or operands reorder)

[Julian Hyde]

I have logged https://issues.apache.org/jira/browse/CALCITE-4559,
"Create 'interface RexRule', a modular rewrite for row-expressions".
Abstracting RexNode rewrites as objects would be a major step toward
achieving the goals in this thread.

Now is a great chance to give feedback on this design. The APIs for
registering rules and applying rules will be expensive to change
later.

On Sat, Mar 13, 2021 at 9:37 AM Vladimir Ozerov  wrote:
>
> Hi Julian,
>
> I agree that in your example normalization may have some different concerns
> comparing to simplification. However, both normalization and simplification
> sometimes address similar problems either. For example, the simplification
> may decrease the search space, but so does the normalization. E.g.
> normalized reordering of operands in a join condition may allow for the
> merge of equivalent nodes that otherwise would be considered
> non-equivalent. Do any of the currently implemented rules depend on some
> normalized representation?
>
> Also, as many rules (such as join reorder rules) generate filters, I would
> argue that moving the normalization to a separate phase might cause the
> unnecessary expansion of the search space.
>
> The idea I expressed above is inspired by CockroachDB (again :-)). In
> CockroachDB, expressions are part of the MEMO and treated similarly to
> relational operators, which allows for the unified rule infrastructure for
> both operators and expressions. Expressions are created using a
> context-aware builder, which knows the set of active normalization rules.
> Whenever a builder is to create a new expression (not necessarily
> the top-level), the normalization rules are invoked in a heuristic manner.
> The code generation is used to build the heuristic rule executor. Both
> normalization and simplification (in our terms) rules are invoked here. For
> example, see [1] (normalization) and [2] (simplification). Finally, the
> expression is registered in MEMO. As a result, every expression ever
> produced is always in a normalized/simplified form.
>
> I am not saying that we should follow this approach. But IMO (1) unified
> handling of simplification and normalization through rules and (2) a single
> entry point for all normalization (builder) are interesting design
> decisions, as they offer both flexibility and convenience.
>
> Regards,
> Vladimir.
>
> [1]
> https://github.com/cockroachdb/cockroach/blob/release-21.1/pkg/sql/opt/norm/rules/scalar.opt#L8
> [2]
> https://github.com/cockroachdb/cockroach/blob/release-21.1/pkg/sql/opt/norm/rules/bool.opt#L30
>
> пт, 12 мар. 2021 г. в 07:15, Julian Hyde :
>
> > Without simplifications, many trivial RelNodes would be produced. It is
> > beneficial to have those in RelBuilder; if they were in rules, the trivial
> > RelNodes (and equivalence sets) would still be present, increasing the size
> > of the search space.
> >
> > I want to draw a distinction between simplification and normalization. A
> > normalized form is relied upon throughout the system. Suppose for example,
> > that we always normalize ‘RexLiteral = RexInputRef’ to ‘RexInputRef =
> > RexLiteral’. If a rule encountered the latter case, it would not be a bug
> > if the rule failed with, say, a ClassCastException.
> >
> > So, I disagree with Vladimir that 'RexSimplify may also be considered a
> > “normalization”’. If simplification is turned off, each rule must be able
> > to deal with the unsimplified expressions.
> >
> > Also, the very idea of normalizations being optional, enabled by system
> > properties or other config, is rather disturbing, because the rules
> > probably don’t know that the normalization has been turned off.
> >
> > The only place for normalization, in my opinion, is explicitly, in a
> > particular planner phase. For example, pulling up all filters before
> > attempting to match materialized views.
> >
> > Julian
> >
> > > On Mar 11, 2021, at 10:37 AM, Vladimir Ozerov 
> > wrote:
> > >
> > > in our practice, we also had some problems with normalization. First, we
> > > observed problems with the unwanted (and sometimes
> > > incorrect) simplification of expressions with CASTs and literals which
> > came
> > > from RexSimplify. I couldn't find an easy way to disable that behavior.
> > > Note, that RexSimplify may also be considered a "normalization". Second,
> > we
> > > implemented a way to avoid Project when doing join reordering but had
> > some
> > > issues with operator signatures due to lack of automatic normalization
> > for
> > > expressions for permuted inputs. These two cases demonstrate two opposite
> > > views: sometimes you want a specific normalization to happen
> > automatically,
> > > but sometimes you want to disable it.
> > >
> > > Perhaps an alternative approach could be to unify all simplification and
> > > normalization logic and split it into configurable rules. Then, we may
> > add
> > > these rules as a separate rule set to the planner, which would be invoked
> > > heuristically 

Re: "PigRelBuilder#create" maybe a wrong place to set Hook.REL_BUILDER_SIMPLIFY

[Julian Hyde]

Note that there are TWO PigRelBuilder.java files in the code base.
(Apologies for that! Both names made sense at the time.)

This particular line of code is in piglet. I suspect that it was added
by me or Khai Tran in order to make some tests pass, and get a big PR
( https://github.com/apache/calcite/pull/1265 ) across the finishing
line. If you want to find out what it does, remove the line and see
what tests fail.

I doubt that it is very harmful (at worst, PigRelBuilder misses some
optimizations), but I suppose some users might like some more control.
Feel free to enhance it.

On Thu, Apr 15, 2021 at 6:25 AM JiaTao Tao  wrote:
>
> Hi, in PigRelBuilder#create, we set Hook.REL_BUILDER_SIMPLIFY to false:
>
>   public static PigRelBuilder create(FrameworkConfig config) {
> final RelBuilder relBuilder = RelBuilder.create(config);
> Hook.REL_BUILDER_SIMPLIFY.addThread(Hook.propertyJ(false));
> return new PigRelBuilder(
> transform(config.getContext(), c -> c.withBloat(-1)),
> relBuilder.getCluster(),
> relBuilder.getRelOptSchema());
>   }
>
> I have two doubts:
> 1. Why we set FALSE here cuz its default value is true;
> 2. This maybe not the suitable place to set  "Hook.REL_BUILDER_SIMPLIFY",
> IMO we should use this in user code, not in the frame code, cuz once we
> done this in frame code, we can not change this config anymore.
>
> Regards!
>
> Aron Tao

"PigRelBuilder#create" maybe a wrong place to set Hook.REL_BUILDER_SIMPLIFY

[JiaTao Tao]

Hi, in PigRelBuilder#create, we set Hook.REL_BUILDER_SIMPLIFY to false:

  public static PigRelBuilder create(FrameworkConfig config) {
final RelBuilder relBuilder = RelBuilder.create(config);
Hook.REL_BUILDER_SIMPLIFY.addThread(Hook.propertyJ(false));
return new PigRelBuilder(
transform(config.getContext(), c -> c.withBloat(-1)),
relBuilder.getCluster(),
relBuilder.getRelOptSchema());
  }

I have two doubts:
1. Why we set FALSE here cuz its default value is true;
2. This maybe not the suitable place to set  "Hook.REL_BUILDER_SIMPLIFY",
IMO we should use this in user code, not in the frame code, cuz once we
done this in frame code, we can not change this config anymore.

Regards!

Aron Tao

[jira] [Created] (CALCITE-4584) Using function in partition by list of over window cause converting exception

[Wang Yanlin (Jira)]

Wang Yanlin created CALCITE-4584:


 Summary: Using function in partition by list of over window cause 
converting exception
 Key: CALCITE-4584
 URL: https://issues.apache.org/jira/browse/CALCITE-4584
 Project: Calcite
  Issue Type: Bug
Reporter: Wang Yanlin


When try to using function expr in the partition by list of over window in sql, 
got
{noformat}
java.lang.UnsupportedOperationException: class 
org.apache.calcite.sql.SqlBasicCall: CHAR_LENGTH(`ENAME`)
{noformat}
The stack trace is as follow
{noformat}
java.lang.RuntimeException: while converting CHAR_LENGTH(`EMP`.`ENAME`)

at 
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerOpTypeMethod$3(ReflectiveConvertletTable.java:145)
at 
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5232)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4486)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5088)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertOver(SqlToRelConverter.java:2037)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.access$2000(SqlToRelConverter.java:224)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5081)
at 
org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$9(StandardConvertletTable.java:209)
at 
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5232)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4486)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5088)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:4327)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:707)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:664)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3538)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:589)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:638)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:799)
at 
org.apache.calcite.test.SqlToRelConverterTest.testFunctionExprInOver(SqlToRelConverterTest.java:4196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81)
at 
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at 
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
at 
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at 

[jira] [Created] (CALCITE-4583) Make simplification API more conservative in "RelBuilder#filter"

[Jiatao Tao (Jira)]

Jiatao Tao created CALCITE-4583:
---

 Summary: Make simplification API more conservative in 
"RelBuilder#filter"
 Key: CALCITE-4583
 URL: https://issues.apache.org/jira/browse/CALCITE-4583
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiatao Tao
Assignee: Jiatao Tao


Previous in CALCITE-2338, we make simplification configurable in most methods 
except RelBuilder#filter, this Jira aims to cover this method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4582) The distinct row count and population size should be 1 for deduced constant columns

[Liya Fan (Jira)]

Liya Fan created CALCITE-4582:
-

 Summary: The distinct row count and population size should be 1 
for deduced constant columns
 Key: CALCITE-4582
 URL: https://issues.apache.org/jira/browse/CALCITE-4582
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Liya Fan


As a follow-up issue for CALCITE-4511, we should support more scenarios for 
which the columns are deduced to have constant values, so they must have the 
distinct row count equal to 1.

Some examples of such cases:

{noformat}
select one
from ( select *, 1 as one from emp);

select deptno
from emp
where deptno = 20;
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4581) The digest of TableScan should consider table hints

[godfrey he (Jira)]

godfrey he created CALCITE-4581:
---

 Summary: The digest of TableScan should consider table hints
 Key: CALCITE-4581
 URL: https://issues.apache.org/jira/browse/CALCITE-4581
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: godfrey he


A table with different hints should be treated as different table, otherwise 
the TableScans with same table but different hints will be treated as one. See 
[FLINK-21627|https://issues.apache.org/jira/browse/FLINK-21627] for more info



--
This message was sent by Atlassian Jira
(v8.3.4#803005)