[jira] [Created] (CALCITE-3189) Multiple fixes for Oracle SQL dialect

[Jesus Camacho Rodriguez (JIRA)]

Jesus Camacho Rodriguez created CALCITE-3189:


 Summary: Multiple fixes for Oracle SQL dialect
 Key: CALCITE-3189
 URL: https://issues.apache.org/jira/browse/CALCITE-3189
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Among others, it includes i) SQL translation support for custom types (e.g. 
{{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type, 
iii) creating datetime literals correctly, and iv) method to infer whether a 
given data type is supported or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: How to create a rule in calcite

[Muhammad Gelbana]

Did you look into the Geode and Druide example projects ? They have some
rules that might help.
I remember very well that Apache Calcite does exactly what you say, its
join rule decides which join algorithm will be executed. Check that out.
Here [1] is Michael's answer on SoF that might shed some light on what you
need.

Straight into your question, you need to match exactly what you need, else,
you might end up modifying operators in the plan tree that you didn't want
to touch.
To do that, you need to identify the inputs to your join operator, their
convention and the join operator's convention. All this can be specified in
the rule's constructor super method call.
If you need more control, you can run extra checks in the *matches* method.
Or you can simply abort your rule if it's mistakenly matched by aborting
the execution on *onMatch* method by simply returning.

Forgive me if I'm answer is too general but I would say that your question
isn't very specific itself. Perhaps you can start by sharing your trials so
everyone can have a better idea about what's wrong with your rule.

[1]
https://stackoverflow.com/questions/56234480/whats-the-difference-between-calcites-converterrule-and-reloptrule

Thanks,
Gelbana


On Tue, Jul 9, 2019 at 5:22 PM Felipe Gutierrez <
felipe.o.gutier...@gmail.com> wrote:

> Hi,
>
> Is there any tutorial teaching how to create my own rule (using Java) in
> apache Calcite?
> I want to create a rule for join operators which I can decide which
> implementation of join I use.
>
> Or, maybe, is there any example that I can see how does it work in Calcite?
>
> thanks
> Felipe
> *--*
> *-- Felipe Gutierrez*
>
> *-- skype: felipe.o.gutierrez*
> *--* *https://felipeogutierrez.blogspot.com
> *
>

[jira] [Created] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)

[Stamatis Zampetakis (JIRA)]

Stamatis Zampetakis created CALCITE-3188:


 Summary: IndexOutOfBoundsException in ProjectFilterTransposeRule 
when executing SELECT COUNT(*)
 Key: CALCITE-3188
 URL: https://issues.apache.org/jira/browse/CALCITE-3188
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.21.0
Reporter: Stamatis Zampetakis
 Fix For: 1.21.0


The exception is raised by many tests in {{DruidAdapterIT}}. 

What they all have in common is a query of the following form:

{code:java}
SELECT COUNT(*) FROM ...
{code}

where the query projects only a single column. 

The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the 
following.


{noformat}
Caused by: java.lang.RuntimeException: Error while applying rule 
DruidProjectFilterTransposeRule, args 
[rel#226:LogicalProject.NONE.[](input=RelSubset#50), 
rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
 $0), 1), 3), 1)):INTEGER NOT NULL, 1)), 
rel#1:DruidQuery.BINDABLE.[](table=[foodmart, 
foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
at 
org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
at 
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
at 
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 28 more
Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than 
size (0)
at 
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
at 
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
at 
com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
at 
org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
... 41 more

{noformat}

It is regression that must be solved before the release.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-3187) Derive all decimal return type through type factory

[Praveen Kumar Desabandu (JIRA)]

Praveen Kumar Desabandu created CALCITE-3187:


 Summary: Derive all decimal return type through type factory
 Key: CALCITE-3187
 URL: https://issues.apache.org/jira/browse/CALCITE-3187
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Praveen Kumar Desabandu


Currently decimal product and quotient return types are derived through type 
factory, this allows clients to override the return type if they so desire.

But decimal sum is embedded in return types, also decimal mod does not have a 
return type inference defined.

This task is to derive all of the return types through type factory, so that 
clients can override if they wish to.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

How to create a rule in calcite

[Felipe Gutierrez]

Hi,

Is there any tutorial teaching how to create my own rule (using Java) in
apache Calcite?
I want to create a rule for join operators which I can decide which
implementation of join I use.

Or, maybe, is there any example that I can see how does it work in Calcite?

thanks
Felipe
*--*
*-- Felipe Gutierrez*

*-- skype: felipe.o.gutierrez*
*--* *https://felipeogutierrez.blogspot.com
*

[jira] [Created] (CALCITE-3186) IN expressions in UPDATE statements throwing Exceptions

[Pressenna (JIRA)]

Pressenna created CALCITE-3186:
--

 Summary: IN expressions in UPDATE statements throwing Exceptions
 Key: CALCITE-3186
 URL: https://issues.apache.org/jira/browse/CALCITE-3186
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Pressenna


The patch to get correlated sub-queries working in UPDATE statements had this 
side-effect.

 
{code:java}
CREATE TABLE t1 (id1 INT, val1 TEXT);
CREATE TABLE t2 (id2 INT, val2 TEXT);

UPDATE t1 SET val1 = 't2' WHERE id1 IN (1, 2, 3);

-- or

UPDATE t1 SET val1 = 't2' WHERE id1 IN (SELECT id2 FROM t2);{code}
 

These seem to raise exceptions now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-3185) Calcite Web: API doc search does not work (404)

[Ruben Quesada Lopez (JIRA)]

Ruben Quesada Lopez created CALCITE-3185:


 Summary: Calcite Web: API doc search does not work (404)
 Key: CALCITE-3185
 URL: https://issues.apache.org/jira/browse/CALCITE-3185
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Ruben Quesada Lopez


In https://calcite.apache.org/apidocs/ when we type something in the SEARCH 
input, e.g. "RelBuilder", and we click on the corresponding item, we get 
redirected to a wrong url (notice the 'undefined'  in the middle):
https://calcite.apache.org/apidocs/undefined/org/apache/calcite/tools/RelBuilder.html
which shows the message:
{code}
Not Found
The requested URL /apidocs/undefined/org/apache/calcite/tools/RelBuilder.html 
was not found on this server.
{code}

The proper url should be:
https://calcite.apache.org/apidocs/org/apache/calcite/tools/RelBuilder.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Modify Calcite Planner in Hive to remove GROUP BY

[Krzysztof Zarzycki]

Thanks for your explanation, it helps a lot!
I was in a big mistake thinking that the result should be the same.
Possibly then Tableau puts "group by " intentionally, to receive
zero rows.

But that means... It is an even bigger bug/miss that Apache Kylin does not
handle grouping by constant. And so I'm afraid I cannot do anything on
Calcite level (like rewrite), I need to work on Kylin. (Or someone has a
different idea?)
I will raise an issue on Kylin Jira then.

Krzysztof



czw., 27 cze 2019 o 04:03 Vineet Garg  napisał(a):

> Hi Julian,
>
> You are right it should produce zero rows not NULL. Thanks for the
> correction.
>
> Vineet
>
>
> On Wed, Jun 26, 2019 at 4:49 PM Julian Hyde  wrote:
>
> > > Select count(*) from empty_table group by  will produce NULL
> >
> > Really? I thought it should produce zero rows.
> >
> > Hsqldb:
> >
> > > select count(*) from "foodmart"."days" where false group by true;
> > +-+
> > |   C1|
> > +-+
> > +-+
> > No rows selected (0.001 seconds)
> >
> >
> > Julian
> >
> >
> > > On Jun 26, 2019, at 1:12 PM, Vineet Garg  wrote:
> > >
> > > Hello Krzysztof,
> > >
> > > The rewrite you mention in Hive was done in HIVE-19674
> > >  to be able to push
> > such
> > > group by to Druid. Currently there is no way to disable this rewrite.
> > >
> > > As for removing Group by , there are rules/rewrites which can
> > > reduce grouping keys by removing constants but removing whole group by
> is
> > > not safe since it can lead to semantically different query.
> > > e.g. Select count(*) from empty_table group by  will produce
> > NULL
> > > but Select count(*) from empty_table will produce 0.
> > >
> > > P.S. There was a bug in HIVE-19674' patch which was further fixed by
> > > HIVE-21539 .
> > >
> > > Regards,
> > > Vineet Garg
> > >
> > > On Wed, Jun 26, 2019 at 7:08 AM Haisheng Yuan 
> > > wrote:
> > >
> > >> Calcite has the rule that does the work. But you can't remove the
> group
> > by
> > >> clause if the constant is the only group key. The semantic is
> different
> > >> without group key. Try it on empty relation, you will see the
> > difference.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Thanks~
> > >> Haisheng
> > >> Yuan--
> > >> 发件人：Krzysztof Zarzycki
> > >> 日 期：2019年06月26日 21:52:41
> > >> 收件人：
> > >> 主 题：Modify Calcite Planner in Hive to remove GROUP BY 
> > >>
> > >> Hello,
> > >>
> > >> While the question I have might look like regards to Hive, I believe
> is
> > >> more about Calcite. I need to add a Calcite plan rule to Hive, that
> > removes
> > >> "Group by" clause when it groups by some constant value (GROUP BY TRUE
> > more
> > >> precisely). As far as I believe, the query semantically is the same.
> > >> Could anyone on this mailing list help me how to do it properly? While
> > I'm
> > >> an experienced java engineer, I have no clue how to achieve this.
> > >> I was trying to modify hive code to do this myself, but unfortunately
> I
> > got
> > >> only NullPointerExceptions.
> > >>
> > >>
> > >> More context below:
> > >> I want to use JdbcStorageHandler in Hive, that connects to Apache
> Kylin
> > and
> > >> forward queries there. Then I put Tableau on top of Hive.
> Unfortunately,
> > >> the queries produced by Tableau to Hive and then reproduced by Calcite
> > >> Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite
> as
> > >> well). I disabled some of the hive optimizations which fixed some of
> my
> > >> queries. But I'm stuck on one I cannot disable. Tableau generates a
> > query
> > >> with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE",
> > by
> > >> Hive/Calcite. But neither of those can be handled by Kylin. I got an
> > idea
> > >> that I will remove GROUP BY completely, because in my understanding
> it's
> > >> unecessary.
> > >>
> > >> I will be very grateful for your help,
> > >> Kind Regards,
> > >> Krzysztof
> > >>
> > >>
> >
> >
>