[jira] [Created] (CALCITE-5344) Remove Travis CI configuration

[Francis Chuang (Jira)]

Francis Chuang created CALCITE-5344:
---

 Summary: Remove Travis CI configuration
 Key: CALCITE-5344
 URL: https://issues.apache.org/jira/browse/CALCITE-5344
 Project: Calcite
  Issue Type: Task
  Components: avatica, core
Reporter: Francis Chuang
Assignee: Francis Chuang
 Fix For: avatica-1.23.0, 1.33.0


Remove Travis-CI configuration as the ASF is turning off Travis-CI integration 
on 31 December 2022.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Migrating away from Travis-CI

[Benchao Li]

+1, thanks Francis for driving this.

Julian Hyde  于2022年10月25日周二 07:57写道：

> +1
>
> > On Oct 24, 2022, at 3:49 PM, Francis Chuang 
> wrote:
> >
> > Travis will no longer be available for ASF projects at the end of 2022.
> >
> > For Calcite, only the calcite and calcite-avatica repos use travis and
> those projects are also using Github Actions already. Perhaps in our case
> we will just need to remove the Travis configs from those repos.
> >
> > Francis
> >
> >  Forwarded Message 
> > Subject:  Migrating away from Travis-CI
> > Date: Mon, 24 Oct 2022 15:46:16 -0500
> > From: Drew Foulks 
> > Reply-To: us...@infra.apache.org
> > To:   annou...@infra.apache.org
> >
> >
> >
> > Greetings PMC Members!
> >
> > Infrastructure is moving away from Travis-CI at the end of 2022.
> >
> > If your project is not currently using Travis-CI, this email is not
> actionable.
> >
> > Projects using Travis-CI:
> >
> > Please take a look at
> https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations <
> https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations>
> >
> > and join the builds meetings and mailing list for project updates.
> >
> >
> > --
> > Cheers,
> >
> > Drew Foulks
> > ASF Infra
> >
> >
>
>

-- 

Best,
Benchao Li

Re: Migrating away from Travis-CI

[Julian Hyde]

+1

> On Oct 24, 2022, at 3:49 PM, Francis Chuang  wrote:
> 
> Travis will no longer be available for ASF projects at the end of 2022.
> 
> For Calcite, only the calcite and calcite-avatica repos use travis and those 
> projects are also using Github Actions already. Perhaps in our case we will 
> just need to remove the Travis configs from those repos.
> 
> Francis
> 
>  Forwarded Message 
> Subject:  Migrating away from Travis-CI
> Date: Mon, 24 Oct 2022 15:46:16 -0500
> From: Drew Foulks 
> Reply-To: us...@infra.apache.org
> To:   annou...@infra.apache.org
> 
> 
> 
> Greetings PMC Members!
> 
> Infrastructure is moving away from Travis-CI at the end of 2022.
> 
> If your project is not currently using Travis-CI, this email is not 
> actionable.
> 
> Projects using Travis-CI:
> 
> Please take a look at 
> https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations 
> 
> 
> and join the builds meetings and mailing list for project updates.
> 
> 
> -- 
> Cheers,
> 
> Drew Foulks
> ASF Infra
> 
> 

Fwd: Migrating away from Travis-CI

[Francis Chuang]

Travis will no longer be available for ASF projects at the end of 2022.

For Calcite, only the calcite and calcite-avatica repos use travis and 
those projects are also using Github Actions already. Perhaps in our 
case we will just need to remove the Travis configs from those repos.


Francis

 Forwarded Message 
Subject:Migrating away from Travis-CI
Date:   Mon, 24 Oct 2022 15:46:16 -0500
From:   Drew Foulks 
Reply-To:   us...@infra.apache.org
To: annou...@infra.apache.org



Greetings PMC Members!

Infrastructure is moving away from Travis-CI at the end of 2022.

If your project is not currently using Travis-CI, this email is not 
actionable.


Projects using Travis-CI:

Please take a look at 
https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations 



and join the builds meetings and mailing list for project updates.


--
Cheers,

Drew Foulks
ASF Infra

[jira] [Created] (CALCITE-5343) Type of division operator

[Mihai Budiu (Jira)]

Mihai Budiu created CALCITE-5343:


 Summary: Type of division operator
 Key: CALCITE-5343
 URL: https://issues.apache.org/jira/browse/CALCITE-5343
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Mihai Budiu


The Calcite type checker assigns a type of INTEGER (not nullable) to the DIVIDE 
RexCall in the following statement: `SELECT 1/0`. Some databases, such as 
MySQL, evaluate this expression to NULL.

This result is obtained even when using SqlConformanceEnum.MYSQL_5 for the 
parser and the validator.

My question is whether there is some other way to influence how type inference 
is performed for division.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

RE: Re: Using Calcite in a multi-tenant application

[Mike Albritton]

Thank you all for the helpful responses.  In my testing, I was using Julian’s 
suggested approach by creating a root schema for each tenant.  This allowed me 
to not have to force unique catalogs across tenants.  Thanks again for the 
information everyone.

Regards,
Mike 



On 2022/10/21 13:18:20 Gavin Ray wrote:
> It is a little bit difficult to explain if you aren't already pretty
> familiar with Calcite, but I've used Calcite for multi-tenant before.
> 
> What you do is hold open the single root CalciteConnection, and then create
> a sub-schema per tenant
> Then, you scope the current statement to a sub-schema based on the tenant
> name
> 
> So the schema layout might look something like this:
> 
>  CalciteConnection
>|
>  .getSchema()
>|
>   SchemaPlus (rootSchema)
>  /  \
> tenant-1-postgres-1 tenant-2-mysql-1
> 
> You can "navigate" or "fix" the current schema to the "tenant-X" schema
> using ".getSubSchema()" or JDBC ".setSchema()" on the Driver calls
> Then you can encode the tenant ID in IE, a JWT header claim, and decode it
> serverside, force it to be set prior to any statement being run, and the
> result is that SQL is securely scoped per-tenant.
> 
> I hope this helps and is understandable?
> Let me know if not.
> 
> 
> On Fri, Oct 21, 2022 at 5:51 AM Ruben Q L  wrote:
> 
> > > If you wanted to, you could write a merge sort algorithm that uses sort
> > runs on disk. But that’s not in Calcite currently.
> > Just to complete this info, there is a ticket logged for that feature [1],
> > but AFAIK there is nobody working on it at the moment.
> >
> > [1] https://issues.apache.org/jira/browse/CALCITE-4193
> >
> >
> > On Thu, Oct 20, 2022 at 9:01 PM Julian Hyde 
> > wrote:
> >
> > > First, remember that the Calcite "interface Schema" is quite low level.
> > It
> > > is basically a namespace that contains a set of tables and a set of
> > > sub-schemas. You can, and probably should, make it immutable.
> > >
> > > A Schema instance doesn’t know its own name or path. That allows you to
> > > “mount” it (as you would mount a filesystem) under several paths.
> > >
> > > This gives you a clue for how you could serve multiple tenants, each
> > > possibly seeing the same schema and tables (or possibly seeing different
> > > subsets). Just set a different root schema for each tenant’s connection.
> > I
> > > believe that’s analogous to how a container like Docker works.
> > >
> > > Calcite does not, by default, write anything to disk. The built-in
> > > adapters read from foreign wrappers (e.g. connecting to another JDBC
> > > database or Cassandra or MongoDB or CSV files) but then process data in
> > > memory.
> > >
> > > If you wanted to, you could write a merge sort algorithm that uses sort
> > > runs on disk. But that’s not in Calcite currently.
> > >
> > > Julian
> > >
> > >
> > > > On Oct 19, 2022, at 8:19 PM, JiaTao Tao  wrote:
> > > >
> > > > In fact, you can change calcite to use like HMS to manager schema
> > > >
> > > > Regards!
> > > >
> > > > Aron Tao
> > > >
> > > >
> > > > Mike Albritton  于2022年10月20日周四 07:16写道：
> > > >
> > > >> Hi all,
> > > >>
> > > >>
> > > >> I am considering using Calcite in a multi-tenant application that can
> > > run
> > > >> queries from different customers in the same process and have a few
> > > >> questions for those familiar with the code about this approach.
> > > >>
> > > >>
> > > >> Specifically, I would like to know if metadata is completely separated
> > > >> per-schema that is loaded or if it is ever shared between schemas?
> > > Also, I
> > > >> read in the docs somewhere that Calcite manages memory for cross
> > schema
> > > >> joins which seemed to suggest it may persist data to disk temporarily.
> > > Is
> > > >> this the case or is there somewhere I can learn more about how data
> > > >> persistence works?  Are there any other concerns that I should have
> > when
> > > >> running Calcite in a multi-tenant application?
> > > >>
> > > >>
> > > >> Finally, if you or someone you know may be interested in a short term
> > > >> contract to assist with a Calcite POC, let me know.
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Mike
> > > >>
> > >
> > >
> >
> 

Re: What is the difference between TableMacro and TableFunction?

[Jiajun Xie]

Sorry, there is a problem with the format of the last message. Here is
new version:


I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
are very similar. - From their descriptions, TableMacro is for "compile
time",  TableFunction is for “execution time".
- From an email[3], I know TableFunction can be seen as a lightweight
relational operator.

But some descriptions do not seem to strictly distinguish between the two
interfaces. For example, ``` */** Table function that implements a view. It
returns the operator* * * tree of the view's SQL query. */* public class
ViewTableMacro implements TableMacro ``` ViewTableMacro[4] implements
TableMacro, but the description is a Table function. Is the description
inaccurate? Is anyone familiar with their other differences? Thanks very
much. Refs: [1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
[2]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
[3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65 [4]
https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00

On Mon, 24 Oct 2022 at 19:43, Jiajun Xie  wrote:

> Hi, all:
>
> I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
> are very similar.
> - From their descriptions, TableMacro is for *"compile time", * TableFunction
> is for “*execution time". *
> *- From an email[3], I know T*ableFunction can be seen as a lightweight
> relational operator.
>
> *But s*ome descriptions do not seem to strictly distinguish between the
> two interfaces. For example,
> ```
> */** Table function that implements a view. It returns the operator*
> * * tree of the view's SQL query. */*
> public class ViewTableMacro implements TableMacro
> ```
> ViewTableMacro[3] implements TableMacro, but the description is a Table
> function. Is the description inaccurate?
>
> Is anyone familiar with their other differences?  Thanks very much.
>
> Refs:
> [1]
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
> [2]
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
> [3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65
> [4] https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00
>
>

What is the difference between TableMacro and TableFunction?

[Jiajun Xie]

Hi, all:

I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
are very similar.
- From their descriptions, TableMacro is for *"compile time", * TableFunction
is for “*execution time". *
*- From an email[3], I know T*ableFunction can be seen as a lightweight
relational operator.

*But s*ome descriptions do not seem to strictly distinguish between the two
interfaces. For example,
```
*/** Table function that implements a view. It returns the operator*
* * tree of the view's SQL query. */*
public class ViewTableMacro implements TableMacro
```
ViewTableMacro[3] implements TableMacro, but the description is a Table
function. Is the description inaccurate?

Is anyone familiar with their other differences?  Thanks very much.

Refs:
[1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
[2]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
[3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65
[4] https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00

Re: RelOptRulesTest.java and RelOptRulesTest.xml

[Benchao Li]

Thanks Alessandro for driving this.

It's great to hear this is being improved. I suffered from this problem
several times before.

Alessandro Solimando  于2022年10月24日周一
17:25写道：

> I have a working implementation in the PR 2948
>  for CALCITE-5340
> , it's covering the
> following test suites:
>
> - HepPlannerTest
> - RelOptRulesTest
> - SqlHintsConverterTest
> - SqlToRelConverterTest
> - TypeCoercionConverterTest
>
> The associated XML reference files will be updated in the context of this
> PR, and from now on the tests will fail upon discrepancies.
>
> Happy to hear your opinion if you have time.
>
> Best regards,
> Alessandro
>
> On Fri, 21 Oct 2022 at 10:42, Alessandro Solimando <
> alessandro.solima...@gmail.com> wrote:
>
> > Thank you guys for your input.
> >
> > Given that the official way to update the XML file (as per our doc) is to
> > automatically generate it, I think it should fail if the actual and
> > expected files differ, whatever the difference is.
> >
> > I will dig more into how DiffRepository works and make the test fail on
> > discrepancies in a subsequent PR.
> >
> > In the meantime, I have filed CALCITE-5340
> >  to track this.
> >
> > Best regards,
> > Alessandro
> >
> > On Fri, 21 Oct 2022 at 10:16, Stamatis Zampetakis 
> > wrote:
> >
> >> Thanks for taking care of this Alessandro!
> >>
> >> I believe the resources are still sorted as per [1] but it appears that
> >> there are still a few manual edits to the file.
> >>
> >> If we want to prevent such changes in the future it may be a good idea
> to
> >> add a post-class check In RelOptRulesTest (or somewhere in DiffRepo)
> >> checking that files are identical.
> >>
> >> Best,
> >> Stamatis
> >>
> >> [1] https://lists.apache.org/thread/j0fh9hhxdc9lc7n228kwj6xcs0z9hpb3
> >>
> >> On Thu, Oct 20, 2022 at 8:36 PM Julian Hyde 
> >> wrote:
> >>
> >> > Do you think that RelOptRulesTest should fail if the resources are not
> >> > sorted alphabetically? (I thought it did that at some point… I may be
> >> > wrong. Anyway, I am a fan of ‘fail fast’. The person who introduces
> the
> >> > error should be the person to fix it.)
> >> >
> >> > > On Oct 20, 2022, at 9:20 AM, Alessandro Solimando <
> >> > alessandro.solima...@gmail.com> wrote:
> >> > >
> >> > > Hello everyone,
> >> > > I have recently added tests to RelOptRulesTest.java, took the chance
> >> to
> >> > > update (maven instruction with gradle instructions, path has
> changed)
> >> and
> >> > > improved them a little bit.
> >> > >
> >> > > While testing the commands I have noticed that few PRs in the last
> >> months
> >> > > had tests added to RelOptRulesTest.java with a manual update of
> >> > > RelOptRulesTest.xml (possibly due to the stale instructions).
> >> > >
> >> > > Manual updates often suffer from formatting issues, stale/wrong SQL
> >> > > statements, different test order.
> >> > >
> >> > > Note that those discrepancies are not caught by running tests/CI,
> but
> >> > it's
> >> > > nonetheless problematic, especially because it forbids folks down
> the
> >> > line
> >> > > to re-generate the file automatically, because their PRs would carry
> >> the
> >> > > changes from past work.
> >> > >
> >> > > In https://github.com/apache/calcite/pull/2944 I will put back on
> >> track
> >> > > RelOptRulesTest.xml, please try to follow the procedure in the
> >> javadoc of
> >> > > RelOptRulesTest.java and let me know if we can improve them even
> >> further
> >> > in
> >> > > your opinion.
> >> > >
> >> > > Best regards,
> >> > > Alessandro
> >> >
> >> >
> >>
> >
>


-- 

Best,
Benchao Li

Re: RelOptRulesTest.java and RelOptRulesTest.xml

[Alessandro Solimando]

I have a working implementation in the PR 2948
 for CALCITE-5340
, it's covering the
following test suites:

- HepPlannerTest
- RelOptRulesTest
- SqlHintsConverterTest
- SqlToRelConverterTest
- TypeCoercionConverterTest

The associated XML reference files will be updated in the context of this
PR, and from now on the tests will fail upon discrepancies.

Happy to hear your opinion if you have time.

Best regards,
Alessandro

On Fri, 21 Oct 2022 at 10:42, Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Thank you guys for your input.
>
> Given that the official way to update the XML file (as per our doc) is to
> automatically generate it, I think it should fail if the actual and
> expected files differ, whatever the difference is.
>
> I will dig more into how DiffRepository works and make the test fail on
> discrepancies in a subsequent PR.
>
> In the meantime, I have filed CALCITE-5340
>  to track this.
>
> Best regards,
> Alessandro
>
> On Fri, 21 Oct 2022 at 10:16, Stamatis Zampetakis 
> wrote:
>
>> Thanks for taking care of this Alessandro!
>>
>> I believe the resources are still sorted as per [1] but it appears that
>> there are still a few manual edits to the file.
>>
>> If we want to prevent such changes in the future it may be a good idea to
>> add a post-class check In RelOptRulesTest (or somewhere in DiffRepo)
>> checking that files are identical.
>>
>> Best,
>> Stamatis
>>
>> [1] https://lists.apache.org/thread/j0fh9hhxdc9lc7n228kwj6xcs0z9hpb3
>>
>> On Thu, Oct 20, 2022 at 8:36 PM Julian Hyde 
>> wrote:
>>
>> > Do you think that RelOptRulesTest should fail if the resources are not
>> > sorted alphabetically? (I thought it did that at some point… I may be
>> > wrong. Anyway, I am a fan of ‘fail fast’. The person who introduces the
>> > error should be the person to fix it.)
>> >
>> > > On Oct 20, 2022, at 9:20 AM, Alessandro Solimando <
>> > alessandro.solima...@gmail.com> wrote:
>> > >
>> > > Hello everyone,
>> > > I have recently added tests to RelOptRulesTest.java, took the chance
>> to
>> > > update (maven instruction with gradle instructions, path has changed)
>> and
>> > > improved them a little bit.
>> > >
>> > > While testing the commands I have noticed that few PRs in the last
>> months
>> > > had tests added to RelOptRulesTest.java with a manual update of
>> > > RelOptRulesTest.xml (possibly due to the stale instructions).
>> > >
>> > > Manual updates often suffer from formatting issues, stale/wrong SQL
>> > > statements, different test order.
>> > >
>> > > Note that those discrepancies are not caught by running tests/CI, but
>> > it's
>> > > nonetheless problematic, especially because it forbids folks down the
>> > line
>> > > to re-generate the file automatically, because their PRs would carry
>> the
>> > > changes from past work.
>> > >
>> > > In https://github.com/apache/calcite/pull/2944 I will put back on
>> track
>> > > RelOptRulesTest.xml, please try to follow the procedure in the
>> javadoc of
>> > > RelOptRulesTest.java and let me know if we can improve them even
>> further
>> > in
>> > > your opinion.
>> > >
>> > > Best regards,
>> > > Alessandro
>> >
>> >
>>
>

apache calcite UDAF implementation

[张帅]

hi,
I am implementing a user-defined aggregate function, it needs input two
parameters like "AGG(column, 5)", the first is the agg column, the second
is
a number literal.
I implement it like this:
public class CalciteUDAF {
 public List init() {return new ArrayList()}
 public List add(List accumulator, Object o) {*accumulator.add(o)*}
 public Object result(List accumulator) {return }
}

I use schemaPlus.add("AGG",
AggregateFunctionImpl.create(CalciteUDAF.class)) to use it.

but how could I use the second parameter in this UDAF? could anyone give me
some advice?

Re: apache calcite UDAF implementation

[张帅]

get it, in add

张帅 <79834585...@gmail.com> 于2022年10月24日周一 09:39写道：

> hi,
> I am implementing a user-defined aggregate function, it needs input two
> parameters like "AGG(column, 5)", the first is the agg column, the second
> is
> a number literal.
> I implement it like this:
> public class CalciteUDAF {
>  public List init() {return new ArrayList()}
>  public List add(List accumulator, Object o) {*accumulator.add(o)*
> }
>  public Object result(List accumulator) {return }
> }
>
> I use schemaPlus.add("AGG",
> AggregateFunctionImpl.create(CalciteUDAF.class)) to use it.
>
> but how could I use the second parameter in this UDAF? could anyone give
> me some advice?
>