??????[ANNOUNCE] New Calcite PMC chair: Ruben Q L

[953396112]

Congratulations to Ruben!  
Thanks for serving as Chair, Haisheng!


Best regards,
Zhaohui Xu




--  --
??: 
   "dev"

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Francis Chuang]

Congrats Ruben!

On 20/01/2022 4:23 pm, Zhe Hu wrote:

Congratulations to Ruben!  Thanks for serving as Chair during the last year, 
Haisheng!


Best regards,
ZheHu






On 01/20/2022 13:09，Jacques Nadeau wrote：
Ditto, thanks Haisheng and congrats Ruben!

On Wed, Jan 19, 2022, 6:35 PM Julian Hyde  wrote:

Congratulations, Ruben, and good luck!

Haisheng, Thank you for serving as Chair.

Julian

On Wed, Jan 19, 2022 at 6:29 PM Haisheng Yuan  wrote:

Calcite community members,

I am pleased to announce that we have a new PMC chair and VP as per our
tradition of rotating the chair once a year. I have resigned, and Ruben
Q L
was duly elected by the PMC and approved unanimously by the Board.

Please join me in congratulating Ruben!

Best,
Haisheng Yuan

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Zhe Hu]

Congratulations to Ruben!  Thanks for serving as Chair during the last year, 
Haisheng!


Best regards,
ZheHu






On 01/20/2022 13:09，Jacques Nadeau wrote：
Ditto, thanks Haisheng and congrats Ruben!

On Wed, Jan 19, 2022, 6:35 PM Julian Hyde  wrote:

Congratulations, Ruben, and good luck!

Haisheng, Thank you for serving as Chair.

Julian

On Wed, Jan 19, 2022 at 6:29 PM Haisheng Yuan  wrote:

Calcite community members,

I am pleased to announce that we have a new PMC chair and VP as per our
tradition of rotating the chair once a year. I have resigned, and Ruben
Q L
was duly elected by the PMC and approved unanimously by the Board.

Please join me in congratulating Ruben!

Best,
Haisheng Yuan

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Jacques Nadeau]

Ditto, thanks Haisheng and congrats Ruben!

On Wed, Jan 19, 2022, 6:35 PM Julian Hyde  wrote:

> Congratulations, Ruben, and good luck!
>
> Haisheng, Thank you for serving as Chair.
>
> Julian
>
> On Wed, Jan 19, 2022 at 6:29 PM Haisheng Yuan  wrote:
> >
> > Calcite community members,
> >
> > I am pleased to announce that we have a new PMC chair and VP as per our
> > tradition of rotating the chair once a year. I have resigned, and Ruben
> Q L
> > was duly elected by the PMC and approved unanimously by the Board.
> >
> > Please join me in congratulating Ruben!
> >
> > Best,
> > Haisheng Yuan
>

Re: [DISCUSS] Refactoring tests

[Jacques Nadeau]

Unfortunately, the last minute attendance of the meetup today threw my
schedule off and I won't be able to look at this for at least a few days.
Don't feel obligated to hold up for me.

On Wed, Jan 19, 2022, 9:04 AM Jacques Nadeau  wrote:

> FYI, I'm trying to do a thorough review today (as much as possible with
> patch this size).
>
> On Wed, Jan 19, 2022, 4:36 AM Michael Mior  wrote:
>
>> Thanks for doing this Julian! I haven't been able to do a detailed review,
>> but from my skim of the PR, this looks like a nice improvement. I think
>> it's always been a bit difficult for new Calcite developers to write good
>> tests, especially for new modules. So anything that helps that experience
>> is very welcome.
>>
>> --
>> Michael Mior
>> mm...@apache.org
>>
>>
>> Le mer. 17 nov. 2021 à 01:15, Vladimir Sitnikov <
>> sitnikov.vladi...@gmail.com>
>> a écrit :
>>
>> > Jacques>This sounds like it will mean we will need to make calcite-core
>> > test artifacts available
>> >
>> > Test artifacts publication is yet another anti-pattern just like "base
>> test
>> > class".
>> > This change has been discussed:
>> > https://lists.apache.org/thread/fz96p94h016p11g777otqntjxg2oxgh1
>> >
>> > If you want to depend on a class from tests, consider moving it to
>> /testkit
>> > module:
>> >
>> >
>> https://github.com/apache/calcite/tree/0899e6c157632ba1c5369a942cfe2be15fb4ed9f/testkit
>> >
>> > Jacques>We should think about the rules around Kotlin
>> >
>> > What happens in calcite-core/tests stays in calcite-core/tests :)
>> >
>> > It is reasonable to assume that testkit module would have dependencies,
>> > and testkit would provide API that is usable from Java and other JVM
>> > languages.
>> >
>> > In that regard, Kotlin dependency in testkit is not much different from
>> > Quidem or commons-lang3.
>> > Consumers might use Quidem if it fits just like they could use Kotlin
>> if it
>> > fits.
>> >
>> > Vladimir
>> >
>>
>

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Gavin Ray]

Thank you for hosting this, and it was nice to listen to the discussion and
learn.

I was sad Julian did not make it, I hoped he might be there to share wisdom.

There were two things I wanted to drop as a note really quick:

1) If there's enough interest, maybe these sorts of meetups could happen in
a more regular interval? Something like every month or few months?

2) Eugen asked a question at the end when I had to leave, about how to
manage and proxy to multiple databases. (The second part was about
implementing security/access controls)

I can help with the first part of the question with some code, as I am
having to do the same thing.
Here is an example of dynamically adding a Postgres + SQL Server DB while
the application is running to a global schema, then querying across them:

https://gist.github.com/GavinRay97/9e5f32c6664be13a27dd3c676f544719

Hope this helps and is roughly what you meant =)

On Wed, Jan 19, 2022 at 2:28 PM Eugen Stan  wrote:

> Hi,
>
> I published my presentation here https://ieugen.github.io/calcite-clj/
> for anyone who is interested.
>
> Once the video is up I will add a link to it as well.
>
> It was nice meeting with you!
>
> Regards,
> --
> Eugen Stan
>
> +40770 941 271  / https://www.netdava.com

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Julian Hyde]

Congratulations, Ruben, and good luck!

Haisheng, Thank you for serving as Chair.

Julian

On Wed, Jan 19, 2022 at 6:29 PM Haisheng Yuan  wrote:
>
> Calcite community members,
>
> I am pleased to announce that we have a new PMC chair and VP as per our
> tradition of rotating the chair once a year. I have resigned, and Ruben Q L
> was duly elected by the PMC and approved unanimously by the Board.
>
> Please join me in congratulating Ruben!
>
> Best,
> Haisheng Yuan

[ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Haisheng Yuan]

Calcite community members,

I am pleased to announce that we have a new PMC chair and VP as per our
tradition of rotating the chair once a year. I have resigned, and Ruben Q L
was duly elected by the PMC and approved unanimously by the Board.

Please join me in congratulating Ruben!

Best,
Haisheng Yuan

Re: New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Julian Hyde]

Justin,

For planning table or index scans, I would recommend using a single TableScan 
with a Filter that uses a Sarg, rather than using multiple TableScans connected 
by a Union. So I think this rule will be useful.

But I do agree that this proposed rule is not a “no brainer”. It may not do 
what people want/expect in all cases, and therefore it probably should not be 
enabled it by default.

Julian





> On Jan 19, 2022, at 3:38 PM, Justin Swanhart  wrote:
> 
> Hi,
> 
> Note that this will negate the optimization that one usually is looking for
> when writing such queries:
> 
> Select * from TAB where a = 1
> UNION ALL
> Select * from TAB where b = 1
> 
> In a database with indexes (most databases) this will allow indexes to be
> used on both the a column and the b column.
> Databases with bitmap indexes or without indexes would benefit from the
> rule.
> 
> On Wed, Jan 19, 2022 at 4:32 PM Julian Hyde  > wrote:
> 
>> Can you log a Jira case for this?
>> 
>> I think you should make your rule work for N-way Union, not just 2-way
>> Union. And I think you should make it work whether or not a Project is
>> present.
>> 
>>> On Jan 19, 2022, at 1:25 PM, Julian Hyde  wrote:
>>> 
>>> It sounds useful.
>>> 
>>> What do you think the rule should be called? UnionFilterTransposeRule,
>> perhaps?
>>> 
>>> A challenge when writing the rule will be to ensure that all of the
>> inputs to the Union are the same. The Volcano framework is not very good at
>> that.
>>> 
>>> You should be careful of the case that the conditions overlap. For
>> example, the rewrite
>>> 
>>>  SELECT * FROM Emp WHERE deptno < 30
>>>  UNION ALL
>>>  SELECT * FROM Emp WHERE deptno IN (25, 35, 45)
>>> 
>>> to
>>> 
>>>  SELECT * FROM Emp WHERE deptno < 30 OR deptno IN (25, 35, 45)
>>> 
>>> Is not valid, because rows with deptno = 25 will appear twice in the
>> first query, once in the second. Maybe that problem does not occur when
>> applied to UNION than when applied to UNION ALL.
>>> 
>>> There would seem to be analogous rules for INTERSECT (combine the
>> conditions using AND) and EXCEPT (combine the conditions using AND NOT).
>> Perhaps one rule could cover all set operations (see
>> FilterSetOpTransposeRule).
>>> 
>>> Julian
>>> 
>>> 
>>> 
 On Jan 19, 2022, at 2:38 AM, Yanjing Wang > >> wrote:
 
 A simple example is converting SELECT a, b FROM t WHERE c = 1 UNION ALL
>> SELECT a, b FROM t WHERE c = 2 to SELECT a, b FROM t WHERE c in (1, 2)
 
 Yanjing Wang mailto:zhuangzixiao...@gmail.com> 
 > zhuangzixiao...@gmail.com >> 于2022年1月19日周三 
>> 18:35写道：
 Hi, community
 
 Here I recommend a new rule for converting UNION ALL sub plan to a
>> single input with an OR filter, the following is its conversion diagram.
 
 
 The conversion prerequisites are
 1. left filter range has no intersection with right filter range.
 2. Project and Input Sub Tree must be identical.
 
 The rule will be used when Input Sub Tree is a computing-intensive or
>> large IO operation.
 
 I don't know whether the community supports it or not, any suggestions
>> will be appreciated.

Re: New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Justin Swanhart]

Hi,

Note that this will negate the optimization that one usually is looking for
when writing such queries:

Select * from TAB where a = 1
UNION ALL
Select * from TAB where b = 1

In a database with indexes (most databases) this will allow indexes to be
used on both the a column and the b column.
Databases with bitmap indexes or without indexes would benefit from the
rule.

On Wed, Jan 19, 2022 at 4:32 PM Julian Hyde  wrote:

> Can you log a Jira case for this?
>
> I think you should make your rule work for N-way Union, not just 2-way
> Union. And I think you should make it work whether or not a Project is
> present.
>
> > On Jan 19, 2022, at 1:25 PM, Julian Hyde  wrote:
> >
> > It sounds useful.
> >
> > What do you think the rule should be called? UnionFilterTransposeRule,
> perhaps?
> >
> > A challenge when writing the rule will be to ensure that all of the
> inputs to the Union are the same. The Volcano framework is not very good at
> that.
> >
> > You should be careful of the case that the conditions overlap. For
> example, the rewrite
> >
> >   SELECT * FROM Emp WHERE deptno < 30
> >   UNION ALL
> >   SELECT * FROM Emp WHERE deptno IN (25, 35, 45)
> >
> > to
> >
> >   SELECT * FROM Emp WHERE deptno < 30 OR deptno IN (25, 35, 45)
> >
> > Is not valid, because rows with deptno = 25 will appear twice in the
> first query, once in the second. Maybe that problem does not occur when
> applied to UNION than when applied to UNION ALL.
> >
> > There would seem to be analogous rules for INTERSECT (combine the
> conditions using AND) and EXCEPT (combine the conditions using AND NOT).
> Perhaps one rule could cover all set operations (see
> FilterSetOpTransposeRule).
> >
> > Julian
> >
> >
> >
> >> On Jan 19, 2022, at 2:38 AM, Yanjing Wang  > wrote:
> >>
> >> A simple example is converting SELECT a, b FROM t WHERE c = 1 UNION ALL
> SELECT a, b FROM t WHERE c = 2 to SELECT a, b FROM t WHERE c in (1, 2)
> >>
> >> Yanjing Wang  zhuangzixiao...@gmail.com>> 于2022年1月19日周三 18:35写道：
> >> Hi, community
> >>
> >> Here I recommend a new rule for converting UNION ALL sub plan to a
> single input with an OR filter, the following is its conversion diagram.
> >>
> >>
> >> The conversion prerequisites are
> >> 1. left filter range has no intersection with right filter range.
> >> 2. Project and Input Sub Tree must be identical.
> >>
> >> The rule will be used when Input Sub Tree is a computing-intensive or
> large IO operation.
> >>
> >> I don't know whether the community supports it or not, any suggestions
> will be appreciated.
> >>
> >>
> >
>
>

Re: New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Julian Hyde]

Can you log a Jira case for this?

I think you should make your rule work for N-way Union, not just 2-way Union. 
And I think you should make it work whether or not a Project is present.

> On Jan 19, 2022, at 1:25 PM, Julian Hyde  wrote:
> 
> It sounds useful.
> 
> What do you think the rule should be called? UnionFilterTransposeRule, 
> perhaps?
> 
> A challenge when writing the rule will be to ensure that all of the inputs to 
> the Union are the same. The Volcano framework is not very good at that.
> 
> You should be careful of the case that the conditions overlap. For example, 
> the rewrite
> 
>   SELECT * FROM Emp WHERE deptno < 30
>   UNION ALL
>   SELECT * FROM Emp WHERE deptno IN (25, 35, 45)
> 
> to
> 
>   SELECT * FROM Emp WHERE deptno < 30 OR deptno IN (25, 35, 45) 
> 
> Is not valid, because rows with deptno = 25 will appear twice in the first 
> query, once in the second. Maybe that problem does not occur when applied to 
> UNION than when applied to UNION ALL.
> 
> There would seem to be analogous rules for INTERSECT (combine the conditions 
> using AND) and EXCEPT (combine the conditions using AND NOT). Perhaps one 
> rule could cover all set operations (see FilterSetOpTransposeRule).
> 
> Julian
> 
> 
> 
>> On Jan 19, 2022, at 2:38 AM, Yanjing Wang > > wrote:
>> 
>> A simple example is converting SELECT a, b FROM t WHERE c = 1 UNION ALL 
>> SELECT a, b FROM t WHERE c = 2 to SELECT a, b FROM t WHERE c in (1, 2)
>> 
>> Yanjing Wang mailto:zhuangzixiao...@gmail.com>> 
>> 于2022年1月19日周三 18:35写道：
>> Hi, community
>> 
>> Here I recommend a new rule for converting UNION ALL sub plan to a single 
>> input with an OR filter, the following is its conversion diagram.
>> 
>>  
>> The conversion prerequisites are 
>> 1. left filter range has no intersection with right filter range.
>> 2. Project and Input Sub Tree must be identical.
>> 
>> The rule will be used when Input Sub Tree is a computing-intensive or large 
>> IO operation.
>> 
>> I don't know whether the community supports it or not, any suggestions will 
>> be appreciated.
>> 
>> 
> 

Re: New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Julian Hyde]

It sounds useful.

What do you think the rule should be called? UnionFilterTransposeRule, perhaps?

A challenge when writing the rule will be to ensure that all of the inputs to 
the Union are the same. The Volcano framework is not very good at that.

You should be careful of the case that the conditions overlap. For example, the 
rewrite

  SELECT * FROM Emp WHERE deptno < 30
  UNION ALL
  SELECT * FROM Emp WHERE deptno IN (25, 35, 45)

to

  SELECT * FROM Emp WHERE deptno < 30 OR deptno IN (25, 35, 45) 

Is not valid, because rows with deptno = 25 will appear twice in the first 
query, once in the second. Maybe that problem does not occur when applied to 
UNION than when applied to UNION ALL.

There would seem to be analogous rules for INTERSECT (combine the conditions 
using AND) and EXCEPT (combine the conditions using AND NOT). Perhaps one rule 
could cover all set operations (see FilterSetOpTransposeRule).

Julian



> On Jan 19, 2022, at 2:38 AM, Yanjing Wang  wrote:
> 
> A simple example is converting SELECT a, b FROM t WHERE c = 1 UNION ALL 
> SELECT a, b FROM t WHERE c = 2 to SELECT a, b FROM t WHERE c in (1, 2)
> 
> Yanjing Wang mailto:zhuangzixiao...@gmail.com>> 
> 于2022年1月19日周三 18:35写道：
> Hi, community
> 
> Here I recommend a new rule for converting UNION ALL sub plan to a single 
> input with an OR filter, the following is its conversion diagram.
> 
>  
> The conversion prerequisites are 
> 1. left filter range has no intersection with right filter range.
> 2. Project and Input Sub Tree must be identical.
> 
> The rule will be used when Input Sub Tree is a computing-intensive or large 
> IO operation.
> 
> I don't know whether the community supports it or not, any suggestions will 
> be appreciated.
> 
> 

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Eugen Stan]

Hi,

I published my presentation here https://ieugen.github.io/calcite-clj/ 
for anyone who is interested.


Once the video is up I will add a link to it as well.

It was nice meeting with you!

Regards,
--
Eugen Stan

+40770 941 271  / https://www.netdava.combegin:vcard
fn:Eugen Stan
n:Stan;Eugen
email;internet:eugen.s...@netdava.com
tel;cell:+40720898747
x-mozilla-html:FALSE
url:https://www.netdava.com
version:2.1
end:vcard

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Jacques Nadeau]

In future, I suggest we just post the zoom link here. This one is still
going here:

https://cloudera.zoom.us/j/91701199263

On Wed, Jan 19, 2022 at 10:33 AM Julian Hyde  wrote:

> For what it’s worth, I had trouble logging into Meetup.com (from both an
> iPhone and a macOS laptop) and so missed the meetup.
>
> > On Jan 19, 2022, at 9:08 AM, Jacques Nadeau  wrote:
> >
> > Shoot, totally missed that this was using meetup.com. Figured the
> date/time
> > would be discussed here. In the future would be helpful to also mention
> > time/date on the mailing list.
> >
> > On Wed, Jan 19, 2022, 8:43 AM Stamatis Zampetakis 
> wrote:
> >
> >> Forgo the link to the event:
> >> https://www.meetup.com/Apache-Calcite/events/282836907/
> >>
> >> On Wed, Jan 19, 2022 at 5:42 PM Stamatis Zampetakis 
> >> wrote:
> >>
> >>> Small reminder: the meetup starts in ~15 minutes.
> >>>
> >>> Best,
> >>> Stamatis
> >>>
> >>> On Wed, Jan 12, 2022 at 11:30 PM Stamatis Zampetakis <
> zabe...@gmail.com>
> >>> wrote:
> >>>
>  Hi,
> 
>  I read the previous emails about the GraphQL project and it looks
>  very interesting.
>  @Gavin: Please send us a title, abstract, and duration of the talk.
>  Don't hesitate to take more time if you need to. I think we still
> have a
>  bit of margin.
> 
>  With this we can freeze the agenda for this meetup. If there is more
>  interest we can immediately start discussing the next one.
> 
>  Although I appear as an organizer in the Apache Calcite Meetup group,
>  this does not mean I am the only one deciding about the agenda or
> when a
>  meetup should happen.
>  If other committers/PMC members want to take a more active role in the
>  organization I would be more than happy to add them as co-organizers
> in
> >> the
>  group.
>  Apart from that feedback & suggestions from anyone are always welcome.
> 
>  Best,
>  Stamatis
> 
>  On Wed, Jan 12, 2022 at 8:36 PM Eugen Stan 
>  wrote:
> 
> > I would love to see GraphQL over Calcite for sure.
> >
> > On 12.01.2022 17:37, Gavin Ray wrote:
> >> If anyone is interested, I can spend 5-10 minutes briefly presenting
> > the
> >> work I've been doing on automatic GraphQL API generation from
> Calcite
> > data
> >> sources.
> >> GraphQL is still somewhat niche though, so I'll hold off on asking
> to
> > speak
> >> about it unless there are folks attending who are interested/using
> >> it.
> >
> >
> > --
> > Eugen Stan
> >
> > +40770 941 271  / https://www.netdava.com
> 
> 
> >>
>
>

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Julian Hyde]

For what it’s worth, I had trouble logging into Meetup.com (from both an iPhone 
and a macOS laptop) and so missed the meetup.

> On Jan 19, 2022, at 9:08 AM, Jacques Nadeau  wrote:
> 
> Shoot, totally missed that this was using meetup.com. Figured the date/time
> would be discussed here. In the future would be helpful to also mention
> time/date on the mailing list.
> 
> On Wed, Jan 19, 2022, 8:43 AM Stamatis Zampetakis  wrote:
> 
>> Forgo the link to the event:
>> https://www.meetup.com/Apache-Calcite/events/282836907/
>> 
>> On Wed, Jan 19, 2022 at 5:42 PM Stamatis Zampetakis 
>> wrote:
>> 
>>> Small reminder: the meetup starts in ~15 minutes.
>>> 
>>> Best,
>>> Stamatis
>>> 
>>> On Wed, Jan 12, 2022 at 11:30 PM Stamatis Zampetakis 
>>> wrote:
>>> 
 Hi,
 
 I read the previous emails about the GraphQL project and it looks
 very interesting.
 @Gavin: Please send us a title, abstract, and duration of the talk.
 Don't hesitate to take more time if you need to. I think we still have a
 bit of margin.
 
 With this we can freeze the agenda for this meetup. If there is more
 interest we can immediately start discussing the next one.
 
 Although I appear as an organizer in the Apache Calcite Meetup group,
 this does not mean I am the only one deciding about the agenda or when a
 meetup should happen.
 If other committers/PMC members want to take a more active role in the
 organization I would be more than happy to add them as co-organizers in
>> the
 group.
 Apart from that feedback & suggestions from anyone are always welcome.
 
 Best,
 Stamatis
 
 On Wed, Jan 12, 2022 at 8:36 PM Eugen Stan 
 wrote:
 
> I would love to see GraphQL over Calcite for sure.
> 
> On 12.01.2022 17:37, Gavin Ray wrote:
>> If anyone is interested, I can spend 5-10 minutes briefly presenting
> the
>> work I've been doing on automatic GraphQL API generation from Calcite
> data
>> sources.
>> GraphQL is still somewhat niche though, so I'll hold off on asking to
> speak
>> about it unless there are folks attending who are interested/using
>> it.
> 
> 
> --
> Eugen Stan
> 
> +40770 941 271  / https://www.netdava.com
 
 
>> 

Re: Projection for SELECT COUNT(*)

[Jacques Nadeau]

I think Viliam's complaint is that the fields aren't being trimmed from the
scan.

I don't remember the details but my question is whether this code is
running [1]

And whether it is behaving as expected. I assume you've enabled trim fields
[2] on SqlToRel?

[1]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java#L1252
[2]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6290

On Wed, Jan 19, 2022 at 7:32 AM Viliam Durina  wrote:

> The issue here is not with zero-field records. The issue is that when doing
> `SELECT COUNT(*)`, the sql-to-rel conversion doesn't produce a projection
> with just a dummy constant to have one field, but there's no projection at
> all and all fields are read from the TableScan.
>
> Viliam
>
> On Wed, 19 Jan 2022 at 03:08, Julian Hyde  wrote:
>
> > As Stamatis said, we don’t have a consistent policy on zero-length
> > records. But in that thread I logged
> > https://issues.apache.org/jira/browse/CALCITE-4597 <
> > https://issues.apache.org/jira/browse/CALCITE-4597> to clarify the
> > situation. It would be great if someone worked on it.
> >
> > I see Viliam’s point that it makes physical optimization easier if there
> > is an explicit Project telling you which columns (if any) need to be read
> > from the TableScan. AggregateExtractProjectRule [1] may make it easier to
> > accomplish this. But in the usual case, when this rule is not enabled, I
> > don’t think we should create a Project.
> >
> > Julian
> >
> > [1]
> >
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53
> > <
> >
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53
> >
> >
> >
> > > On Jan 15, 2022, at 2:07 PM, Stamatis Zampetakis 
> > wrote:
> > >
> > > Hi Viliam,
> > >
> > > I don't see a problem with the current plan. It seems correct and more
> > > intuitive than the one with the DUMMY projection.
> > >
> > > LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >LogicalTableScan(table=foo)
> > >
> > > The code you cited in SqlToRelConverter seems an attempt to handle
> empty
> > > records/tuples that we are not handling very well in general [1].
> > > Doesn't seem related to performance as the use-case you mentioned.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1] https://lists.apache.org/thread/dtsz159x4nk3l9b3topgykqpsml024tv
> > >
> > > On Fri, Jan 14, 2022 at 12:57 PM Viliam Durina 
> > wrote:
> > >
> > >> I noticed this two pieces of code:
> > >>
> > >> 1. in SqlToRelConverter:
> > >>
> > >>  if (preExprs.size() == 0) {
> > >>// Special case for COUNT(*), where we can end up with no inputs
> > >>// at all.  The rest of the system doesn't like 0-tuples, so we
> > >>// select a dummy constant here.
> > >>final RexNode zero = rexBuilder.makeExactLiteral(BigDecimal.ZERO);
> > >>preExprs = ImmutableList.of(Pair.of(zero, null));
> > >>  }
> > >>
> > >> 2. in RelBuilder:
> > >>
> > >>   // Some parts of the system can't handle rows with zero fields, so
> > >>  // pretend that one field is used.
> > >>  if (fieldsUsed.isEmpty()) {
> > >>r = ((Project) r).getInput();
> > >>  }
> > >>
> > >> They run in this order, and the 2nd overrides the former. The end
> > result is
> > >> that for query `SELECT COUNT(*) FROM foo`, the result of sql-to-rel
> > >> conversion is:
> > >>
> > >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >>LogicalTableScan(table=foo)
> > >>
> > >> instead of:
> > >>
> > >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >>  LogicalProject(DUMMY=[0])
> > >>LogicalTableScan(table=foo)
> > >>
> > >> In our implementation we push the projection to table scan. Without
> the
> > >> project, we fetch full rows, even though the aggregation uses no row.
> > >>
> > >> The code was introduced in
> > >> https://issues.apache.org/jira/browse/CALCITE-3763, but maybe it was
> > >> broken
> > >> later.
> > >>
> > >> Do you think this is an issue?
> > >>
> > >> Viliam
> > >>
> > >> --
> > >> This message contains confidential information and is intended only
> for
> > >> the
> > >> individuals named. If you are not the named addressee you should not
> > >> disseminate, distribute or copy this e-mail. Please notify the sender
> > >> immediately by e-mail if you have received this e-mail by mistake and
> > >> delete this e-mail from your system. E-mail transmission cannot be
> > >> guaranteed to be secure or error-free as information could be
> > intercepted,
> > >> corrupted, lost, destroyed, arrive late or incomplete, or contain
> > viruses.
> > >> The sender therefore does not accept liability for any errors or
> > omissions
> > >> in the contents of this message, which arise as a result of e-mail
> > >> transmi

Re: Projection for SELECT COUNT(*)

[Justin Swanhart]

Hi,

So what you are saying is when the TableScan gets a zero length record (no
projections means no record is projected so it has a length of zero) it
treats that as a request to read all columns instead of zero columns?  That
sounds like a bug in the TableScan.

On Wed, Jan 19, 2022 at 10:32 AM Viliam Durina  wrote:

> The issue here is not with zero-field records. The issue is that when doing
> `SELECT COUNT(*)`, the sql-to-rel conversion doesn't produce a projection
> with just a dummy constant to have one field, but there's no projection at
> all and all fields are read from the TableScan.
>
> Viliam
>
> On Wed, 19 Jan 2022 at 03:08, Julian Hyde  wrote:
>
> > As Stamatis said, we don’t have a consistent policy on zero-length
> > records. But in that thread I logged
> > https://issues.apache.org/jira/browse/CALCITE-4597 <
> > https://issues.apache.org/jira/browse/CALCITE-4597> to clarify the
> > situation. It would be great if someone worked on it.
> >
> > I see Viliam’s point that it makes physical optimization easier if there
> > is an explicit Project telling you which columns (if any) need to be read
> > from the TableScan. AggregateExtractProjectRule [1] may make it easier to
> > accomplish this. But in the usual case, when this rule is not enabled, I
> > don’t think we should create a Project.
> >
> > Julian
> >
> > [1]
> >
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53
> > <
> >
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53
> >
> >
> >
> > > On Jan 15, 2022, at 2:07 PM, Stamatis Zampetakis 
> > wrote:
> > >
> > > Hi Viliam,
> > >
> > > I don't see a problem with the current plan. It seems correct and more
> > > intuitive than the one with the DUMMY projection.
> > >
> > > LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >LogicalTableScan(table=foo)
> > >
> > > The code you cited in SqlToRelConverter seems an attempt to handle
> empty
> > > records/tuples that we are not handling very well in general [1].
> > > Doesn't seem related to performance as the use-case you mentioned.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1] https://lists.apache.org/thread/dtsz159x4nk3l9b3topgykqpsml024tv
> > >
> > > On Fri, Jan 14, 2022 at 12:57 PM Viliam Durina 
> > wrote:
> > >
> > >> I noticed this two pieces of code:
> > >>
> > >> 1. in SqlToRelConverter:
> > >>
> > >>  if (preExprs.size() == 0) {
> > >>// Special case for COUNT(*), where we can end up with no inputs
> > >>// at all.  The rest of the system doesn't like 0-tuples, so we
> > >>// select a dummy constant here.
> > >>final RexNode zero = rexBuilder.makeExactLiteral(BigDecimal.ZERO);
> > >>preExprs = ImmutableList.of(Pair.of(zero, null));
> > >>  }
> > >>
> > >> 2. in RelBuilder:
> > >>
> > >>   // Some parts of the system can't handle rows with zero fields, so
> > >>  // pretend that one field is used.
> > >>  if (fieldsUsed.isEmpty()) {
> > >>r = ((Project) r).getInput();
> > >>  }
> > >>
> > >> They run in this order, and the 2nd overrides the former. The end
> > result is
> > >> that for query `SELECT COUNT(*) FROM foo`, the result of sql-to-rel
> > >> conversion is:
> > >>
> > >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >>LogicalTableScan(table=foo)
> > >>
> > >> instead of:
> > >>
> > >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> > >>  LogicalProject(DUMMY=[0])
> > >>LogicalTableScan(table=foo)
> > >>
> > >> In our implementation we push the projection to table scan. Without
> the
> > >> project, we fetch full rows, even though the aggregation uses no row.
> > >>
> > >> The code was introduced in
> > >> https://issues.apache.org/jira/browse/CALCITE-3763, but maybe it was
> > >> broken
> > >> later.
> > >>
> > >> Do you think this is an issue?
> > >>
> > >> Viliam
> > >>
> > >> --
> > >> This message contains confidential information and is intended only
> for
> > >> the
> > >> individuals named. If you are not the named addressee you should not
> > >> disseminate, distribute or copy this e-mail. Please notify the sender
> > >> immediately by e-mail if you have received this e-mail by mistake and
> > >> delete this e-mail from your system. E-mail transmission cannot be
> > >> guaranteed to be secure or error-free as information could be
> > intercepted,
> > >> corrupted, lost, destroyed, arrive late or incomplete, or contain
> > viruses.
> > >> The sender therefore does not accept liability for any errors or
> > omissions
> > >> in the contents of this message, which arise as a result of e-mail
> > >> transmission. If verification is required, please request a hard-copy
> > >> version. -Hazelcast
> > >>
> >
> >
>
> --
> This message contains confidential information and is intended only for
> the
> individuals named. If you are not the named addr

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Jacques Nadeau]

Shoot, totally missed that this was using meetup.com. Figured the date/time
would be discussed here. In the future would be helpful to also mention
time/date on the mailing list.

On Wed, Jan 19, 2022, 8:43 AM Stamatis Zampetakis  wrote:

> Forgo the link to the event:
> https://www.meetup.com/Apache-Calcite/events/282836907/
>
> On Wed, Jan 19, 2022 at 5:42 PM Stamatis Zampetakis 
> wrote:
>
> > Small reminder: the meetup starts in ~15 minutes.
> >
> > Best,
> > Stamatis
> >
> > On Wed, Jan 12, 2022 at 11:30 PM Stamatis Zampetakis 
> > wrote:
> >
> >> Hi,
> >>
> >> I read the previous emails about the GraphQL project and it looks
> >> very interesting.
> >> @Gavin: Please send us a title, abstract, and duration of the talk.
> >> Don't hesitate to take more time if you need to. I think we still have a
> >> bit of margin.
> >>
> >> With this we can freeze the agenda for this meetup. If there is more
> >> interest we can immediately start discussing the next one.
> >>
> >> Although I appear as an organizer in the Apache Calcite Meetup group,
> >> this does not mean I am the only one deciding about the agenda or when a
> >> meetup should happen.
> >> If other committers/PMC members want to take a more active role in the
> >> organization I would be more than happy to add them as co-organizers in
> the
> >> group.
> >> Apart from that feedback & suggestions from anyone are always welcome.
> >>
> >> Best,
> >> Stamatis
> >>
> >> On Wed, Jan 12, 2022 at 8:36 PM Eugen Stan 
> >> wrote:
> >>
> >>> I would love to see GraphQL over Calcite for sure.
> >>>
> >>> On 12.01.2022 17:37, Gavin Ray wrote:
> >>> > If anyone is interested, I can spend 5-10 minutes briefly presenting
> >>> the
> >>> > work I've been doing on automatic GraphQL API generation from Calcite
> >>> data
> >>> > sources.
> >>> > GraphQL is still somewhat niche though, so I'll hold off on asking to
> >>> speak
> >>> > about it unless there are folks attending who are interested/using
> it.
> >>>
> >>>
> >>> --
> >>> Eugen Stan
> >>>
> >>> +40770 941 271  / https://www.netdava.com
> >>
> >>
>

Re: [DISCUSS] Refactoring tests

[Jacques Nadeau]

FYI, I'm trying to do a thorough review today (as much as possible with
patch this size).

On Wed, Jan 19, 2022, 4:36 AM Michael Mior  wrote:

> Thanks for doing this Julian! I haven't been able to do a detailed review,
> but from my skim of the PR, this looks like a nice improvement. I think
> it's always been a bit difficult for new Calcite developers to write good
> tests, especially for new modules. So anything that helps that experience
> is very welcome.
>
> --
> Michael Mior
> mm...@apache.org
>
>
> Le mer. 17 nov. 2021 à 01:15, Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com>
> a écrit :
>
> > Jacques>This sounds like it will mean we will need to make calcite-core
> > test artifacts available
> >
> > Test artifacts publication is yet another anti-pattern just like "base
> test
> > class".
> > This change has been discussed:
> > https://lists.apache.org/thread/fz96p94h016p11g777otqntjxg2oxgh1
> >
> > If you want to depend on a class from tests, consider moving it to
> /testkit
> > module:
> >
> >
> https://github.com/apache/calcite/tree/0899e6c157632ba1c5369a942cfe2be15fb4ed9f/testkit
> >
> > Jacques>We should think about the rules around Kotlin
> >
> > What happens in calcite-core/tests stays in calcite-core/tests :)
> >
> > It is reasonable to assume that testkit module would have dependencies,
> > and testkit would provide API that is usable from Java and other JVM
> > languages.
> >
> > In that regard, Kotlin dependency in testkit is not much different from
> > Quidem or commons-lang3.
> > Consumers might use Quidem if it fits just like they could use Kotlin if
> it
> > fits.
> >
> > Vladimir
> >
>

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Stamatis Zampetakis]

Forgo the link to the event:
https://www.meetup.com/Apache-Calcite/events/282836907/

On Wed, Jan 19, 2022 at 5:42 PM Stamatis Zampetakis 
wrote:

> Small reminder: the meetup starts in ~15 minutes.
>
> Best,
> Stamatis
>
> On Wed, Jan 12, 2022 at 11:30 PM Stamatis Zampetakis 
> wrote:
>
>> Hi,
>>
>> I read the previous emails about the GraphQL project and it looks
>> very interesting.
>> @Gavin: Please send us a title, abstract, and duration of the talk.
>> Don't hesitate to take more time if you need to. I think we still have a
>> bit of margin.
>>
>> With this we can freeze the agenda for this meetup. If there is more
>> interest we can immediately start discussing the next one.
>>
>> Although I appear as an organizer in the Apache Calcite Meetup group,
>> this does not mean I am the only one deciding about the agenda or when a
>> meetup should happen.
>> If other committers/PMC members want to take a more active role in the
>> organization I would be more than happy to add them as co-organizers in the
>> group.
>> Apart from that feedback & suggestions from anyone are always welcome.
>>
>> Best,
>> Stamatis
>>
>> On Wed, Jan 12, 2022 at 8:36 PM Eugen Stan 
>> wrote:
>>
>>> I would love to see GraphQL over Calcite for sure.
>>>
>>> On 12.01.2022 17:37, Gavin Ray wrote:
>>> > If anyone is interested, I can spend 5-10 minutes briefly presenting
>>> the
>>> > work I've been doing on automatic GraphQL API generation from Calcite
>>> data
>>> > sources.
>>> > GraphQL is still somewhat niche though, so I'll hold off on asking to
>>> speak
>>> > about it unless there are folks attending who are interested/using it.
>>>
>>>
>>> --
>>> Eugen Stan
>>>
>>> +40770 941 271  / https://www.netdava.com
>>
>>

Re: [DISCUSS] Apache Calcite Online Meetup January 2022

[Stamatis Zampetakis]

Small reminder: the meetup starts in ~15 minutes.

Best,
Stamatis

On Wed, Jan 12, 2022 at 11:30 PM Stamatis Zampetakis 
wrote:

> Hi,
>
> I read the previous emails about the GraphQL project and it looks
> very interesting.
> @Gavin: Please send us a title, abstract, and duration of the talk.
> Don't hesitate to take more time if you need to. I think we still have a
> bit of margin.
>
> With this we can freeze the agenda for this meetup. If there is more
> interest we can immediately start discussing the next one.
>
> Although I appear as an organizer in the Apache Calcite Meetup group, this
> does not mean I am the only one deciding about the agenda or when a meetup
> should happen.
> If other committers/PMC members want to take a more active role in the
> organization I would be more than happy to add them as co-organizers in the
> group.
> Apart from that feedback & suggestions from anyone are always welcome.
>
> Best,
> Stamatis
>
> On Wed, Jan 12, 2022 at 8:36 PM Eugen Stan  wrote:
>
>> I would love to see GraphQL over Calcite for sure.
>>
>> On 12.01.2022 17:37, Gavin Ray wrote:
>> > If anyone is interested, I can spend 5-10 minutes briefly presenting the
>> > work I've been doing on automatic GraphQL API generation from Calcite
>> data
>> > sources.
>> > GraphQL is still somewhat niche though, so I'll hold off on asking to
>> speak
>> > about it unless there are folks attending who are interested/using it.
>>
>>
>> --
>> Eugen Stan
>>
>> +40770 941 271  / https://www.netdava.com
>
>

Re: Projection for SELECT COUNT(*)

[Viliam Durina]

The issue here is not with zero-field records. The issue is that when doing
`SELECT COUNT(*)`, the sql-to-rel conversion doesn't produce a projection
with just a dummy constant to have one field, but there's no projection at
all and all fields are read from the TableScan.

Viliam

On Wed, 19 Jan 2022 at 03:08, Julian Hyde  wrote:

> As Stamatis said, we don’t have a consistent policy on zero-length
> records. But in that thread I logged
> https://issues.apache.org/jira/browse/CALCITE-4597 <
> https://issues.apache.org/jira/browse/CALCITE-4597> to clarify the
> situation. It would be great if someone worked on it.
>
> I see Viliam’s point that it makes physical optimization easier if there
> is an explicit Project telling you which columns (if any) need to be read
> from the TableScan. AggregateExtractProjectRule [1] may make it easier to
> accomplish this. But in the usual case, when this rule is not enabled, I
> don’t think we should create a Project.
>
> Julian
>
> [1]
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53
> <
> https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/AggregateExtractProjectRule.java#L53>
>
>
> > On Jan 15, 2022, at 2:07 PM, Stamatis Zampetakis 
> wrote:
> >
> > Hi Viliam,
> >
> > I don't see a problem with the current plan. It seems correct and more
> > intuitive than the one with the DUMMY projection.
> >
> > LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> >LogicalTableScan(table=foo)
> >
> > The code you cited in SqlToRelConverter seems an attempt to handle empty
> > records/tuples that we are not handling very well in general [1].
> > Doesn't seem related to performance as the use-case you mentioned.
> >
> > Best,
> > Stamatis
> >
> > [1] https://lists.apache.org/thread/dtsz159x4nk3l9b3topgykqpsml024tv
> >
> > On Fri, Jan 14, 2022 at 12:57 PM Viliam Durina 
> wrote:
> >
> >> I noticed this two pieces of code:
> >>
> >> 1. in SqlToRelConverter:
> >>
> >>  if (preExprs.size() == 0) {
> >>// Special case for COUNT(*), where we can end up with no inputs
> >>// at all.  The rest of the system doesn't like 0-tuples, so we
> >>// select a dummy constant here.
> >>final RexNode zero = rexBuilder.makeExactLiteral(BigDecimal.ZERO);
> >>preExprs = ImmutableList.of(Pair.of(zero, null));
> >>  }
> >>
> >> 2. in RelBuilder:
> >>
> >>   // Some parts of the system can't handle rows with zero fields, so
> >>  // pretend that one field is used.
> >>  if (fieldsUsed.isEmpty()) {
> >>r = ((Project) r).getInput();
> >>  }
> >>
> >> They run in this order, and the 2nd overrides the former. The end
> result is
> >> that for query `SELECT COUNT(*) FROM foo`, the result of sql-to-rel
> >> conversion is:
> >>
> >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> >>LogicalTableScan(table=foo)
> >>
> >> instead of:
> >>
> >> LogicalAggregate(group=[{}], EXPR$0=[COUNT($0)])
> >>  LogicalProject(DUMMY=[0])
> >>LogicalTableScan(table=foo)
> >>
> >> In our implementation we push the projection to table scan. Without the
> >> project, we fetch full rows, even though the aggregation uses no row.
> >>
> >> The code was introduced in
> >> https://issues.apache.org/jira/browse/CALCITE-3763, but maybe it was
> >> broken
> >> later.
> >>
> >> Do you think this is an issue?
> >>
> >> Viliam
> >>
> >> --
> >> This message contains confidential information and is intended only for
> >> the
> >> individuals named. If you are not the named addressee you should not
> >> disseminate, distribute or copy this e-mail. Please notify the sender
> >> immediately by e-mail if you have received this e-mail by mistake and
> >> delete this e-mail from your system. E-mail transmission cannot be
> >> guaranteed to be secure or error-free as information could be
> intercepted,
> >> corrupted, lost, destroyed, arrive late or incomplete, or contain
> viruses.
> >> The sender therefore does not accept liability for any errors or
> omissions
> >> in the contents of this message, which arise as a result of e-mail
> >> transmission. If verification is required, please request a hard-copy
> >> version. -Hazelcast
> >>
>
>

-- 
This message contains confidential information and is intended only for the 
individuals named. If you are not the named addressee you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately by e-mail if you have received this e-mail by mistake and 
delete this e-mail from your system. E-mail transmission cannot be 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. 
The sender therefore does not accept liability for any errors or omissions 
in the contents of this message, which arise as a result of e-mail 
transmission. If verification is required, p

Re: Apache Calcite - Usage of Querable table/Querable/RawQueryable/etc interfaces

[Gavin Ray]

My apologies, I didn't realize that the updated search wasn't public now
(it opened for me a few days ago)

Regular GH links:
https://github.com/search?q=rawqueryable+language%3AJava&type=code

Searching "CalciteConnection" across all repos, but removing
"apache/calcite" from the results:
https://github.com/search?q=CalciteConnection+-repo%3Aapache%2Fcalcite&type=code



On Tue, Jan 18, 2022 at 12:17 PM M Singh 
wrote:

>  Thanks Gavin for your pointers.  I am in the waiting list for the search
> tool you mentioned.  I will continue to search on google in the meanwhile.
> Mans
>
>
> On Tuesday, January 18, 2022, 08:44:31 AM EST, Gavin Ray <
> ray.gavi...@gmail.com> wrote:
>
>  Not to be dismissive, but one thing I've done that has been helpful for
> me,
> is to search all public repos for usage/examples of the thing I am trying
> to understand.
> For example, this "RawQueryable" does not appear to be utilized in any
> user-facing code on Github, so it may not be useful for you:
>
> https://cs.github.com/?q=rawqueryable%20language%3AJava&scopeName=All%20repos&scope=
>
> However, take something like "CalciteConnection" or "RelNode", try to
> remove the Calcite repo itself, and you'll get a lot of hits and usage
> examples:
>
> https://cs.github.com/?scopeName=All+repos&scope=&q=CalciteConnection+%28NOT+path%3A%22calcite%2F%22%29
>
> (I find that searching Scala repos usually gives more relevant results, due
> to lots of Java results being duplicates)
>
> This isn't nearly as good as getting a direct answer, but sometimes you
> might be able to find what you're looking for
>
>
>
>
> On Tue, Jan 18, 2022 at 5:39 AM M Singh 
> wrote:
>
> >  Hey Folks:
> > Just wanted to see if you have any pointers to help me understand the
> > queryable concepts.
> > Thanks
> >On Friday, January 14, 2022, 09:09:06 AM EST, M Singh
> >  wrote:
> >
> >  Hi Folks:
> > A few newbie questions -
> > 1. I wanted to find out what are the benefits of a queryable table (
> >
> https://calcite.apache.org/javadocAggregate/org/apache/calcite/schema/QueryableTable.html)
> and
> > what are the pros and cons of using it.
> > 2. What is the role of Querable interface (
> >
> https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/Queryable.html
> )
> > ?3. There is a getExpression methods in queryabletable (
> >
> https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/tree/Expression.html
> ),
> > and I wanted to find out if it is required and what are the pros/cons of
> > implementing it.
> > 4. What role does RawQueralbe play ? (
> >
> https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/RawQueryable.html
> ).
> > It has a method  getProvider method that returns QueryProvider.  What
> roles
> > does query provider play ?5. What role does ExtendedQuerable play ?
> > Please let me know if there are any basic
> > examples/blogs/documentation/etc.
> > Thanks for your help.
> > Mans
>

Re: [DISCUSS] Refactoring tests

[Michael Mior]

Thanks for doing this Julian! I haven't been able to do a detailed review,
but from my skim of the PR, this looks like a nice improvement. I think
it's always been a bit difficult for new Calcite developers to write good
tests, especially for new modules. So anything that helps that experience
is very welcome.

--
Michael Mior
mm...@apache.org


Le mer. 17 nov. 2021 à 01:15, Vladimir Sitnikov 
a écrit :

> Jacques>This sounds like it will mean we will need to make calcite-core
> test artifacts available
>
> Test artifacts publication is yet another anti-pattern just like "base test
> class".
> This change has been discussed:
> https://lists.apache.org/thread/fz96p94h016p11g777otqntjxg2oxgh1
>
> If you want to depend on a class from tests, consider moving it to /testkit
> module:
>
> https://github.com/apache/calcite/tree/0899e6c157632ba1c5369a942cfe2be15fb4ed9f/testkit
>
> Jacques>We should think about the rules around Kotlin
>
> What happens in calcite-core/tests stays in calcite-core/tests :)
>
> It is reasonable to assume that testkit module would have dependencies,
> and testkit would provide API that is usable from Java and other JVM
> languages.
>
> In that regard, Kotlin dependency in testkit is not much different from
> Quidem or commons-lang3.
> Consumers might use Quidem if it fits just like they could use Kotlin if it
> fits.
>
> Vladimir
>

Re: New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Yanjing Wang]

A simple example is converting *SELECT a, b FROM t WHERE c = 1 UNION
ALL SELECT a, b FROM t WHERE c = 2 *to *SELECT a, b FROM t WHERE c in (1,
2)*

Yanjing Wang  于2022年1月19日周三 18:35写道：

> Hi, community
>
> Here I recommend a new rule for converting UNION ALL sub plan to a single
> input with an OR filter, the following is its conversion diagram.
> [image: UnionAllToOrRule.jpg]
>
> The conversion prerequisites are
> 1. left filter range has no intersection with right filter range.
> 2. Project and Input Sub Tree must be identical.
>
> The rule will be used when Input Sub Tree is a computing-intensive or
> large IO operation.
>
> I don't know whether the community supports it or not, any suggestions
> will be appreciated.
>
>
>

New rule for Converting UNION ALL with same inputs but different filters to single input with OR FILTER

[Yanjing Wang]

Hi, community

Here I recommend a new rule for converting UNION ALL sub plan to a single
input with an OR filter, the following is its conversion diagram.
[image: UnionAllToOrRule.jpg]

The conversion prerequisites are
1. left filter range has no intersection with right filter range.
2. Project and Input Sub Tree must be identical.

The rule will be used when Input Sub Tree is a computing-intensive or large
IO operation.

I don't know whether the community supports it or not, any suggestions will
be appreciated.