date:20230308

[jira] [Created] (CALCITE-5566) agg can't be pushed down correctly

2023-03-08 Thread shampoo liu (Jira)

shampoo liu created CALCITE-5566:


 Summary: agg can't be pushed down correctly
 Key: CALCITE-5566
 URL: https://issues.apache.org/jira/browse/CALCITE-5566
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.33.0
Reporter: shampoo liu


{code:sql}
select count(*) from (select max("position_title") "position_title" from 
"foodmart"."employee" where "gender" like '%a%' group by "birth_date", 
"gender") where "position_title" like '%z%'
{code}
The plan generated from sql above is as this :
{code:java}
PLAN=EnumerableAggregate(group=[{}], EXPR$0=[COUNT()])
  JdbcToEnumerableConverter
JdbcFilter(condition=[LIKE($2, '%z%')])
  JdbcAggregate(group=[{8, 15}], position_title=[MAX($5)])
JdbcFilter(condition=[LIKE($15, '%a%')])
  JdbcTableScan(table=[[foodmart, employee]])
{code}

Expected:
{code:java}
PLAN=JdbcToEnumerableConverter
  JdbcAggregate(group=[{}], EXPR$0=[COUNT()])
JdbcFilter(condition=[LIKE($2, '%z%')])
  JdbcAggregate(group=[{8, 15}], position_title=[MAX($5)])
JdbcFilter(condition=[LIKE($15, '%a%')])
  JdbcTableScan(table=[[foodmart, employee]])
{code}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5567) Failed to apply plugin 'de.thetaphi.forbiddenapis' when running ./gradlew build

2023-03-08 Thread Matt P (Jira)

Matt P created CALCITE-5567:
---

 Summary: Failed to apply plugin 'de.thetaphi.forbiddenapis' when 
running ./gradlew build
 Key: CALCITE-5567
 URL: https://issues.apache.org/jira/browse/CALCITE-5567
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.33.0
 Environment: {{OS: arch linux}}

{{JDK: java-19-openjdk}}
Reporter: Matt P
 Attachments: build.gradle.kts, output.txt

Running {{./gradlew build}} on a fresh {{git clone}} fails with:

 
{code:java}
FAILURE: Build failed with an exception.
* Where:
Build file '/home/user/github/apache/calcite/build.gradle.kts' line: 513
* What went wrong:
Failed to apply plugin 'de.thetaphi.forbiddenapis'.
> Could not create plugin of type 'ForbiddenApisPlugin'.
   > Could not initialize class 
de.thetaphi.forbiddenapis.gradle.ForbiddenApisPlugin
{code}
The full output (output.txt) and {{build.gradle.kts}} files are attached.

This is similar to https://issues.apache.org/jira/browse/SOLR-14426 but running 
{{./gradlew clean}} results in the same error.

I've tried this on version 1.33 plus several older releases with similar 
results.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Big Query support (CALCITE-5543, CALCITE-5496)

2023-03-08 Thread Sarah Bienenstock

Hi
I work at a company called Privitar where we use Calcite for parsing,
adapting and executing queries with different dialects. If Calcite doesn’t
support some functionality that we need, we add in this support ourselves.
We are considering contributing these changes to the project.

We now have a large customer that requires Big Query support. I see that
there is active work to support parsing date and time (CALCITE-5543
) as well as an issue
to fix a limitation on timezone handling (CALCITE-5496
). I’m wondering if
these are actively being worked on and if there is a timeline in which I
can expect these to be contributed.

Thanks!
-- 

Sarah Bienenstock
Senior Software Engineer

sarah.bienenst...@privitar.com

www.privitar.com

Privitar, Israel
[image: Privitar logo]


Privacy is important to us. You can review our privacy policy here
. If you don’t want to
receive my emails please let me know. Or manage how we communicate with you
here .

-- 
This email is confidential, if you are not the intended recipient please 
delete it and notify us immediately by emailing the sender. You should not 
copy it or use it for any purpose nor disclose its contents to any other 
person. Privitar Limited is registered in England with registered number 
09305666. Registered office Salisbury House, Station Road, Cambridge, 
CB12LA.

Re: [QUESTION]: Bug Fix Release

2023-03-08 Thread James Turton

All of Drill's DATE_TRUNC unit tests pass when Drill uses
calcite-1.34.0-SNAPSHOT (and once we accommodate the new QUALIFY
clause). While we do now have an unrelated issue with DATE_DIFF which I
believe has resulted from the introduction of a three parameter
DATE_DIFF function in CALCITE-5469, I'm quite sure that we can resolve
this in Drill.

In summary I'm a +1 for this Calcite snapshot becoming an RC.

Thanks
James Turton

On 2023/03/07 00:11, Stamatis Zampetakis wrote:

Hey Charles,

Please test Drill with the latest calcite-1.34.0-SNAPSHOT [1] and if all is
good on your end I will prepare an RC for vote.

Best,
Stamatis

[1]
https://repository.apache.org/content/groups/snapshots/org/apache/calcite/calcite-core/1.34.0-SNAPSHOT/

On Sun, Mar 5, 2023 at 7:16 PM Charles Givre wrote:

Julian,
Now that Drill is on main Calcite instead of the fork, I'll commit that
the Drill community will do our best to try Drill with the RC candidates to
see if we can catch issues during the release cycle.
Thanks,
-- C

On Mar 5, 2023, at 12:20 PM, Julian Hyde wrote:

It was indeed a regression, but it didn’t break any of Calcite’s tests

and no one spoke up during the release vote. Mistakes are expensive to fix
after a release, cheaper during the release vote, and cheapest of all if
found by the test suite.

On Mar 5, 2023, at 6:33 AM, Charles Givre wrote:

That would be great! Again I’m only asking because this was a

regression. I really do appreciate it. Thanks!

Sent from my iPhone

On Mar 4, 2023, at 13:59, Stamatis Zampetakis

wrote:

If we get the 1.34.0 out a bit sooner than usual I guess this will be

good

enough for Drill. If the others agree I can try to prepare an RC during
next week. WDYT ?

Best,
Stamatis

On Sat, Mar 4, 2023, 6:13 PM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

The second option Benchao mentions is what Hive currently does as

well.

Best regards,
Alessandro

On Sat 4 Mar 2023, 13:19 Benchao Li, wrote:

Hi Charles,

Thank for reaching out!

IIRC, the idea of releasing bugfix version has been brought up in

the

past,

but I couldn't find the discussion (in Jira and dev ML).

I'd like to share my understanding why we chose not to release bug

fix

versions, please correct me if I'm wrong,
- Calcite has many bug fixes that span multi versions (even more

that 10

versions), then only keeping several (such as 3) bug fix releases

does

not

solve all these problems.
- Actually we usually do not distinguish too much between "bugfix"

and

"new

feature", so maintaining bug fix releases is not that easy.
- Calcite lacks reviewers and also release managers, only keeping

linear

releasing in rhythm could save us some efforts.

For regressions, I agree that this hurts downstream projects. For

such

cases, there are two approaches come into my mind:
- We can release a new version quickly than usual.
- The projects that need the fix/feature before our next scheduled

release,

they could copy these files into their projects, as we already did in
Flink[1]. They could remove these files once they adopt the new

release

Calcite.

I hope this helps.

[1]

https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/java/org/apache/calcite

Charles Givre 于2023年3月2日周四 06:22写道：

Hello Calcite Devs,
I wanted to thank everyone for the recent release of Calcite 1.33.

the PMC Chair for Apache Drill and we just released Drill 1.21[0]

which

now using the latest version of Calcite instead of our 2-3 year old

fork!

However, we encountered a small issue with Calcite 1.33 that does

not

affect just Drill. Specifically, there was a regression which was

caused

by CALCITE-5447[1] which effectively broke the DATE_TRUNC function.

The

bugfix has been fixed and merged in CALCITE-5522[2].

In any event, given that this function is fairly important and the

lengthy

release schedules of both Drill and Calcite, I wanted to ask whether

the

Calcite might consider doing a quick bugfix release with this and

any

other

regressions that may have popped up in 1.33 and have since been

fixed.

Thank you very much for all your work!
Best,
-- Charles

[0]:

https://github.com/apache/drill-site/blob/master/blog/_posts/en/2023-02-21-drill-1.21.0-released.md

[1]: https://issues.apache.org/jira/browse/CALCITE-5447
[2]: https://issues.apache.org/jira/browse/CALCITE-5522

Best,
Benchao Li

Re: [QUESTION]: Bug Fix Release

2023-03-08 Thread Tanner Clary

Hello,

With regards to the unrelated issue with DATE_DIFF, I authored CALCITE-5469
so perhaps if you want to open a new thread or post a comment on the case
itself, I would be happy to take a look.

Best,
Tanner

On Wed, Mar 8, 2023 at 8:42 AM James Turton  wrote:

> Hi
>
> All of Drill's DATE_TRUNC unit tests pass when Drill uses
> calcite-1.34.0-SNAPSHOT (and once we accommodate the new QUALIFY
> clause). While we do now have an unrelated issue with DATE_DIFF which I
> believe has resulted from the introduction of a three parameter
> DATE_DIFF function in CALCITE-5469, I'm quite sure that we can resolve
> this in Drill.
>
> In summary I'm a +1 for this Calcite snapshot becoming an RC.
>
> Thanks
> James Turton
>
>
> On 2023/03/07 00:11, Stamatis Zampetakis wrote:
> > Hey Charles,
> >
> > Please test Drill with the latest calcite-1.34.0-SNAPSHOT [1] and if all
> is
> > good on your end I will prepare an RC for vote.
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> https://repository.apache.org/content/groups/snapshots/org/apache/calcite/calcite-core/1.34.0-SNAPSHOT/
> >
> > On Sun, Mar 5, 2023 at 7:16 PM Charles Givre  wrote:
> >
> >> Julian,
> >> Now that Drill is on main Calcite instead of the fork, I'll commit that
> >> the Drill community will do our best to try Drill with the RC
> candidates to
> >> see if we can catch issues during the release cycle.
> >> Thanks,
> >> -- C
> >>
> >>
> >>> On Mar 5, 2023, at 12:20 PM, Julian Hyde 
> wrote:
> >>>
> >>> It was indeed a regression, but it didn’t break any of Calcite’s tests
> >> and no one spoke up during the release vote. Mistakes are expensive to
> fix
> >> after a release, cheaper during the release vote, and cheapest of all if
> >> found by the test suite.
>  On Mar 5, 2023, at 6:33 AM, Charles Givre  wrote:
> 
>  That would be great!  Again I’m only asking because this was a
> >> regression.   I really do appreciate it.  Thanks!
>  Sent from my iPhone
> 
> > On Mar 4, 2023, at 13:59, Stamatis Zampetakis 
> >> wrote:
> > If we get the 1.34.0 out a bit sooner than usual I guess this will
> be
> >> good
> > enough for Drill. If the others agree I can try to prepare an RC
> during
> > next week. WDYT ?
> >
> > Best,
> > Stamatis
> >
> >
> >> On Sat, Mar 4, 2023, 6:13 PM Alessandro Solimando <
> >> alessandro.solima...@gmail.com> wrote:
> >>
> >> The second option Benchao mentions is what Hive currently does as
> >> well.
> >> Best regards,
> >> Alessandro
> >>
>  On Sat 4 Mar 2023, 13:19 Benchao Li, 
> wrote:
> 
>  Hi Charles,
> 
>  Thank for reaching out!
> 
>  IIRC, the idea of releasing bugfix version has been brought up in
> >> the
> >> past,
> >>> but I couldn't find the discussion (in Jira and dev ML).
> >>>
> >>> I'd like to share my understanding why we chose not to release bug
> >> fix
> >>> versions, please correct me if I'm wrong,
> >>> - Calcite has many bug fixes that span multi versions (even more
> >> that 10
> >>> versions), then only keeping several (such as 3) bug fix releases
> >> does
> >> not
> >>> solve all these problems.
> >>> - Actually we usually do not distinguish too much between "bugfix"
> >> and
> >> "new
> >>> feature", so maintaining bug fix releases is not that easy.
> >>> - Calcite lacks reviewers and also release managers, only keeping
> >> linear
> >>> releasing in rhythm could save us some efforts.
> >>>
> >>> For regressions, I agree that this hurts downstream projects. For
> >> such
> >>> cases, there are two approaches come into my mind:
> >>> - We can release a new version quickly than usual.
> >>> - The projects that need the fix/feature before our next scheduled
> >> release,
> >>> they could copy these files into their projects, as we already did
> in
> >>> Flink[1]. They could remove these files once they adopt the new
> >> release
> >> of
> >>> Calcite.
> >>>
> >>> I hope this helps.
> >>>
> >>> [1]
> >>>
> >>>
> >>
> https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/java/org/apache/calcite
> >>>
> >>> Charles Givre  于2023年3月2日周四 06:22写道：
> >>>
>  Hello Calcite Devs,
>  I wanted to thank everyone for the recent release of Calcite 1.33.
> >> I
> >> am
>  the PMC Chair for Apache Drill and we just released Drill 1.21[0]
> >> which
> >>> is
>  now using the latest version of Calcite instead of our 2-3 year
> old
> >> fork!
>  However, we encountered a small issue with Calcite 1.33 that does
> >> not
>  affect just Drill.  Specifically, there was a regression which was
> >> caused
>  by CALCITE-5447[1] which effectively broke the DATE_TRUNC
> function.
> >> The
>  bugfix has been fixed and merged in CALCITE-5522[2].
> 
>

Re: [QUESTION]: Bug Fix Release

2023-03-08 Thread James Turton

Much appreciated. I only have so much know-how in this area but
CALCITE-5469 looks completely normal to me, just something that we need
to accommodate due to Drill having at some point settled on a
conflicting definition of DATE_DIFF that seems to resemble Hive and
MySQL's DATEDIFFs (no underscore) more than anything else.

We'll start a new thread if we can't simply take care of it ourselves.

On 2023/03/08 19:28, Tanner Clary wrote:

Hello,

With regards to the unrelated issue with DATE_DIFF, I authored CALCITE-5469
so perhaps if you want to open a new thread or post a comment on the case
itself, I would be happy to take a look.

Best,
Tanner

On Wed, Mar 8, 2023 at 8:42 AM James Turton wrote:

In summary I'm a +1 for this Calcite snapshot becoming an RC.

Thanks
James Turton

On 2023/03/07 00:11, Stamatis Zampetakis wrote:

Hey Charles,

Please test Drill with the latest calcite-1.34.0-SNAPSHOT [1] and if all

good on your end I will prepare an RC for vote.

Best,
Stamatis

[1]

https://repository.apache.org/content/groups/snapshots/org/apache/calcite/calcite-core/1.34.0-SNAPSHOT/

On Sun, Mar 5, 2023 at 7:16 PM Charles Givre wrote:

Julian,
Now that Drill is on main Calcite instead of the fork, I'll commit that
the Drill community will do our best to try Drill with the RC

candidates to

see if we can catch issues during the release cycle.
Thanks,
-- C

On Mar 5, 2023, at 12:20 PM, Julian Hyde

wrote:

It was indeed a regression, but it didn’t break any of Calcite’s tests

and no one spoke up during the release vote. Mistakes are expensive to

fix

after a release, cheaper during the release vote, and cheapest of all if
found by the test suite.

On Mar 5, 2023, at 6:33 AM, Charles Givre wrote:

That would be great! Again I’m only asking because this was a

regression. I really do appreciate it. Thanks!

Sent from my iPhone

On Mar 4, 2023, at 13:59, Stamatis Zampetakis

wrote:

If we get the 1.34.0 out a bit sooner than usual I guess this will

good

enough for Drill. If the others agree I can try to prepare an RC

during

next week. WDYT ?

Best,
Stamatis

On Sat, Mar 4, 2023, 6:13 PM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

The second option Benchao mentions is what Hive currently does as

well.

Best regards,
Alessandro

On Sat 4 Mar 2023, 13:19 Benchao Li,

wrote:

Hi Charles,

Thank for reaching out!

IIRC, the idea of releasing bugfix version has been brought up in

the

past,

but I couldn't find the discussion (in Jira and dev ML).

I'd like to share my understanding why we chose not to release bug

fix

versions, please correct me if I'm wrong,
- Calcite has many bug fixes that span multi versions (even more

that 10

versions), then only keeping several (such as 3) bug fix releases

does

not

solve all these problems.
- Actually we usually do not distinguish too much between "bugfix"

and

"new

feature", so maintaining bug fix releases is not that easy.
- Calcite lacks reviewers and also release managers, only keeping

linear

releasing in rhythm could save us some efforts.

For regressions, I agree that this hurts downstream projects. For

such

cases, there are two approaches come into my mind:
- We can release a new version quickly than usual.
- The projects that need the fix/feature before our next scheduled

release,

they could copy these files into their projects, as we already did

Flink[1]. They could remove these files once they adopt the new

release

Calcite.

I hope this helps.

[1]

https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/java/org/apache/calcite

Charles Givre 于2023年3月2日周四 06:22写道：

Hello Calcite Devs,
I wanted to thank everyone for the recent release of Calcite 1.33.

the PMC Chair for Apache Drill and we just released Drill 1.21[0]

which

now using the latest version of Calcite instead of our 2-3 year

old

fork!

However, we encountered a small issue with Calcite 1.33 that does

not

affect just Drill. Specifically, there was a regression which was

caused

by CALCITE-5447[1] which effectively broke the DATE_TRUNC

function.

The

bugfix has been fixed and merged in CALCITE-5522[2].

In any event, given that this function is fairly important and the

lengthy

release schedules of both Drill and Calcite, I wanted to ask

whether

the

Calcite might consider doing a quick bugfix release with this and

any

other

regressions that may have popped up in 1.33 and have since been

fixed.

Thank you very much for all your work!
Best,
-- Charles

[0]:

https://github.com/apache/drill-site/blob/master/blog/_posts

[jira] [Created] (CALCITE-5568) LogicalValues causes decorrelate to fail

2023-03-08 Thread libopeng (Jira)

libopeng created CALCITE-5568:
-

 Summary: LogicalValues causes decorrelate to fail
 Key: CALCITE-5568
 URL: https://issues.apache.org/jira/browse/CALCITE-5568
 Project: Calcite
  Issue Type: Bug
Reporter: libopeng


{code:java}
LogicalProject(sub_1=[$1])
  LogicalJoin(condition=[true], joinType=[left])
LogicalValues(tuples=[[{ 0 }]])

Re: Big Query support (CALCITE-5543, CALCITE-5496)

2023-03-08 Thread Stamatis Zampetakis

Hi Sarah,

I would suggest you add a comment under the respective JIRAs to see if the
contributor/reviewer is planning to push this forward; if not you can
assign this to you (or somebody else in your team) and start working on it.

Best,
Stamatis

On Wed, Mar 8, 2023 at 5:17 PM Sarah Bienenstock <
sarah.bienenst...@privitar.com> wrote:

> Hi
> I work at a company called Privitar where we use Calcite for parsing,
> adapting and executing queries with different dialects. If Calcite doesn’t
> support some functionality that we need, we add in this support ourselves.
> We are considering contributing these changes to the project.
>
> We now have a large customer that requires Big Query support. I see that
> there is active work to support parsing date and time (CALCITE-5543
> ) as well as an issue
> to fix a limitation on timezone handling (CALCITE-5496
> ). I’m wondering if
> these are actively being worked on and if there is a timeline in which I
> can expect these to be contributed.
>
> Thanks!
> --
>
> Sarah Bienenstock
> Senior Software Engineer
>
> sarah.bienenst...@privitar.com
>
> www.privitar.com
>
> Privitar, Israel
> [image: Privitar logo]
>
>
> Privacy is important to us. You can review our privacy policy here
> . If you don’t want to
> receive my emails please let me know. Or manage how we communicate with you
> here .
>
> --
> This email is confidential, if you are not the intended recipient please
> delete it and notify us immediately by emailing the sender. You should not
> copy it or use it for any purpose nor disclose its contents to any other
> person. Privitar Limited is registered in England with registered number
> 09305666. Registered office Salisbury House, Station Road, Cambridge,
> CB12LA.
>

[jira] [Created] (CALCITE-5566) agg can't be pushed down correctly

[jira] [Created] (CALCITE-5567) Failed to apply plugin 'de.thetaphi.forbiddenapis' when running ./gradlew build

Big Query support (CALCITE-5543, CALCITE-5496)

Re: [QUESTION]: Bug Fix Release

Re: [QUESTION]: Bug Fix Release

Re: [QUESTION]: Bug Fix Release

[jira] [Created] (CALCITE-5568) LogicalValues causes decorrelate to fail

Re: Big Query support (CALCITE-5543, CALCITE-5496)

8 matches

Site Navigation

Mail list logo

Footer information