[jira] [Created] (CALCITE-3412) Response Error for FLOOR TIMESTAMP TO WEEK
huaicui created CALCITE-3412:
Summary: Response Error for FLOOR TIMESTAMP TO WEEK
Key: CALCITE-3412
URL: https://issues.apache.org/jira/browse/CALCITE-3412
Project: Calcite
Issue Type: Bug
Components: core
Environment: The start day of the week is not Sunday or Monday,
calcite use Tuesday to split a week, like this:
select FLOOR(CAST('2017-01-28' AS TIMESTAMP) TO WEEK);
The response is :
2017-01-26 00:00:00.0
!image-2019-10-15-13-33-34-896.png!
Reporter: huaicui
Attachments: image-2019-10-15-13-33-34-896.png
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] Automated security fixes via dependabot
I guess the missing bit is the bot that automatically creates JIRA for Dependabot issues. What if we create one? Vladimir
Re: [DISCUSS] Automated security fixes via dependabot
I've been using Dependabot on my fork of a few Apache repos (including the Calcite related ones [1][2][3]) for over a year now. I had to configure dependabot to point to these repositories. It is a nice reminder of all the different dependencies that can be updated. There is also a nice link to the changelog/release notes typically so you can find out what changed. I didn't realize that Github was going to start pushing PRs. Typically I've manually looked at the list of PRs and updated the dependencies accordingly after creating a Jira like CALCITE-2789 [4]. The PRs are a good sanity check if the change will be easy or not since the tests pass/fail based on the configured Travis jobs. After committing the change, I then go back to Github and either force dependabot to rebase the PRs or wait for dependabot to notice the update and it will autoclose the stale PRs. So I am in favor of using the automated PRs in some way at least to know what is out of date. I've seen some of the PRs do large jumps in dependency version (ie: 1.x to 2.x) so automerging probably isn't the best idea (unless there is a way to tag the PR as reviewed). I've been personally taking a monthly approach to dependencies for Apache Knox where I try to once a month update dependencies. This seems to me like a reasonable tradeoff of doing dependency updates frequently but not too frequently [5] and [6]. [1] https://github.com/risdenk/calcite/pulls [2] https://github.com/risdenk/calcite-avatica/pulls [3] https://github.com/risdenk/calcite-avatica-go/pulls [4] https://issues.apache.org/jira/browse/CALCITE-2789 [5] https://issues.apache.org/jira/browse/KNOX-2009 [6] https://issues.apache.org/jira/browse/KNOX-2049 Kevin Risden On Mon, Oct 14, 2019 at 6:40 AM Francis Chuang wrote: > +1 to squashing all these changes during a release. However, my only > concern is that if upgrading the dependencies during a release breaks > something in the codebase and turns into a larger change. I think in > this case, the change should not be squashed with the other commits and > be a standalone one. > > Francis > > On 13/10/2019 4:16 pm, Julian Hyde wrote: > > I’ve not looked at the PRs but they sound useful. Keeping software > secure these days is a moving target; we have to do work just to keep up. > All of our dependencies are doing that work too, and so we need to keep up > to date with them. > > > > I think it would be useful to have a task before each release to merge > in the ones that look important. > > > > If you are concerned that this will result in many “small” changes with > no JIRA case number, the person merging the PRs could squash them into a > single commit with its own JIRA case. (I’d use 'git cherry-pick … ; git > cherry-pick ; git rebase -i origin/master’) > > > > Julian > > > > > >> On Oct 12, 2019, at 1:01 PM, Muhammad Gelbana > wrote: > >> > >> Why would we not merge those PRs or even disable the whole thing ? > >> > >> > >> > >> On Fri, Oct 11, 2019 at 12:09 AM Francis Chuang < > francischu...@apache.org> > >> wrote: > >> > >>> Dependabot is a bot on Github that opens PRs to automatically upgrade > >>> out of date dependencies to fix security issues. Recently, Github > >>> acquired dependabot and is gradually enabling the bot on all > repositories. > >>> > >>> It just opened a PR to upgrade a few dependencies in the Avatica > >>> repository: https://github.com/apache/calcite-avatica/pull/114 > >>> > >>> I'd like to start some discussion as to how we should deal with these > >>> PRs. For some background, dependency upgrades should usually have a > jira > >>> issue number assigned, so that the change is fully trackable. We > >>> recently had some discussion regarding trivial fixes to documentation > >>> and the consensus was that changes to the code is not considered to be > >>> trivial and that an issue should be filed on jira. > >>> > >>> If we will not merge these PRs, I think it makes sense to ask infra to > >>> disable them. Having these open PRs and then closing them manually seem > >>> to generate a lot of noise. According to the documentation for > >>> dependabot [1] it appears that we can either opt out of having > >>> dependabot opening PRs completely or have it open PRs. There is no > >>> middle-ground where dependabot/Github sends members of the repo a > >>> notification for security issues, but do not open any PRs. > >>> > >>> What do you guys think? > >>> > >>> Francis > >>> > >>> [1] > >>> > https://help.github.com/en/articles/configuring-automated-security-fixes > >>> > > >
[jira] [Created] (CALCITE-3411) Incorrect code generated for BigDecimal ConstantExpression
Feng Zhu created CALCITE-3411:
-
Summary: Incorrect code generated for BigDecimal ConstantExpression
Key: CALCITE-3411
URL: https://issues.apache.org/jira/browse/CALCITE-3411
Project: Calcite
Issue Type: Bug
Affects Versions: 1.21.0
Reporter: Feng Zhu
Assignee: Feng Zhu
In current implementation
{code:java}
Expressions.constant("104", BigDecimal.class) -> new java.math.BigDecimal(104L)
Expressions.constant("1000", BigDecimal.class) -> new java.math.BigDecimal(1L,
-3){code}
However, in *java.math.BigDecimal*, there is no constructor:
{code:java}
java.math.BigDecimal(long exact, int scale) {code}
Instead we should call method
{code:java}
java.math.BigDecimal.valueOf(long exact, int scale){code}
h4. *Why this issue did not exposed before?*
*BlockBuilder* conducts optimizations that can bury this issue.
For example, when we disable the optimization.
_MaterializationTest#testAggregateMaterializationAggregateFuncs11_ will produce
the code below. However, it can't pass the compilation.
{code:java}
final long v = org.apache.calcite.runtime.SqlFunctions.floor(853763696000L,
((Long) new java.math.BigDecimal(1L, -3)).longValue());{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3410) Simplify RelOptRulesTest and HepPlannerTest by making test methods fluent
Danny Chen created CALCITE-3410: --- Summary: Simplify RelOptRulesTest and HepPlannerTest by making test methods fluent Key: CALCITE-3410 URL: https://issues.apache.org/jira/browse/CALCITE-3410 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Automated security fixes via dependabot
+1 to squashing all these changes during a release. However, my only concern is that if upgrading the dependencies during a release breaks something in the codebase and turns into a larger change. I think in this case, the change should not be squashed with the other commits and be a standalone one. Francis On 13/10/2019 4:16 pm, Julian Hyde wrote: I’ve not looked at the PRs but they sound useful. Keeping software secure these days is a moving target; we have to do work just to keep up. All of our dependencies are doing that work too, and so we need to keep up to date with them. I think it would be useful to have a task before each release to merge in the ones that look important. If you are concerned that this will result in many “small” changes with no JIRA case number, the person merging the PRs could squash them into a single commit with its own JIRA case. (I’d use 'git cherry-pick … ; git cherry-pick ; git rebase -i origin/master’) Julian On Oct 12, 2019, at 1:01 PM, Muhammad Gelbana wrote: Why would we not merge those PRs or even disable the whole thing ? On Fri, Oct 11, 2019 at 12:09 AM Francis Chuang wrote: Dependabot is a bot on Github that opens PRs to automatically upgrade out of date dependencies to fix security issues. Recently, Github acquired dependabot and is gradually enabling the bot on all repositories. It just opened a PR to upgrade a few dependencies in the Avatica repository: https://github.com/apache/calcite-avatica/pull/114 I'd like to start some discussion as to how we should deal with these PRs. For some background, dependency upgrades should usually have a jira issue number assigned, so that the change is fully trackable. We recently had some discussion regarding trivial fixes to documentation and the consensus was that changes to the code is not considered to be trivial and that an issue should be filed on jira. If we will not merge these PRs, I think it makes sense to ask infra to disable them. Having these open PRs and then closing them manually seem to generate a lot of noise. According to the documentation for dependabot [1] it appears that we can either opt out of having dependabot opening PRs completely or have it open PRs. There is no middle-ground where dependabot/Github sends members of the repo a notification for security issues, but do not open any PRs. What do you guys think? Francis [1] https://help.github.com/en/articles/configuring-automated-security-fixes
[jira] [Created] (CALCITE-3409) Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule
jin xing created CALCITE-3409: - Summary: Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule Key: CALCITE-3409 URL: https://issues.apache.org/jira/browse/CALCITE-3409 Project: Calcite Issue Type: Improvement Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3408) Add support for Enumerable Intersect/Minus all
Wang Yanlin created CALCITE-3408:
Summary: Add support for Enumerable Intersect/Minus all
Key: CALCITE-3408
URL: https://issues.apache.org/jira/browse/CALCITE-3408
Project: Calcite
Issue Type: New Feature
Reporter: Wang Yanlin
When running sql with calcite driver
{code:java}
final String sql = "select * from\n"
+ "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as
t(x, y))\n"
+ "except all\n"
+ "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x,
y))\n";
{code}
got
{code:java}
java.sql.SQLException: Error while executing SQL "explain plan for select * from
(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as t(x, y))
except all
(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))
": There are not enough rules to produce a node with desired properties:
convention=ENUMERABLE, sort=[].
Missing conversion is LogicalMinus[convention: NONE -> ENUMERABLE]
There is 1 empty subset: rel#27:Subset#4.ENUMERABLE.[], the relevant part of
the original plan is as follows
22:LogicalMinus(all=[true])
1:LogicalValues(subset=[rel#16:Subset#0.NONE.[]], tuples=[[{ 1, 'a' }, { 2,
'b' }, { 2, 'b' }, { 3, 'c' }]])
5:LogicalValues(subset=[rel#19:Subset#2.NONE.[]], tuples=[[{ 1, 'a' }, { 2,
'c' }, { 4, 'x' }]])
Root: rel#27:Subset#4.ENUMERABLE.[]
Original rel:
LogicalMinus(all=[true]): rowcount = 3.0, cumulative cost = {17.0 rows, 19.0
cpu, 0.0 io}, id = 14
LogicalProject(X=[$0], Y=[$1]): rowcount = 4.0, cumulative cost = {8.0 rows,
9.0 cpu, 0.0 io}, id = 9
LogicalValues(tuples=[[{ 1, 'a' }, { 2, 'b' }, { 2, 'b' }, { 3, 'c' }]]):
rowcount = 4.0, cumulative cost = {4.0 rows, 1.0 cpu, 0.0 io}, id = 1
LogicalProject(X=[$0], Y=[$1]): rowcount = 3.0, cumulative cost = {6.0 rows,
7.0 cpu, 0.0 io}, id = 12
LogicalValues(tuples=[[{ 1, 'a' }, { 2, 'c' }, { 4, 'x' }]]): rowcount =
3.0, cumulative cost = {3.0 rows, 1.0 cpu, 0.0 io}, id = 5
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects
I mean, testPushNonSimpleMappingProject [1] runs with RelFieldTrimmer enabled, which is done by [2] [1] https://github.com/apache/calcite/pull/1500/files#diff-3c834a32d46b821b5241e132f2ae6bfaR324 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L249 XING JIN 于2019年10月14日周一 下午4:03写道： > Yes, that's how testPushNonSimpleMappingProject [1] works > > [1] > https://github.com/apache/calcite/pull/1500/files#diff-3c834a32d46b821b5241e132f2ae6bfaR324 > > Danny Chan 于2019年10月14日周一 下午3:36写道： > >> > But why the final BindableTableScan is not pruned ? >> >> The RelFieldTrimmer default is turned off, you should open it explicitly. >> >> Best, >> Danny Chan >> 在 2019年10月14日 +0800 AM11:51，dev@calcite.apache.org，写道： >> > >> > BINDABLE_TABLE_SCAN_RULE >> >
Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects
Yes, that's how testPushNonSimpleMappingProject [1] works [1] https://github.com/apache/calcite/pull/1500/files#diff-3c834a32d46b821b5241e132f2ae6bfaR324 Danny Chan 于2019年10月14日周一 下午3:36写道： > > But why the final BindableTableScan is not pruned ? > > The RelFieldTrimmer default is turned off, you should open it explicitly. > > Best, > Danny Chan > 在 2019年10月14日 +0800 AM11:51，dev@calcite.apache.org，写道： > > > > BINDABLE_TABLE_SCAN_RULE >
Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects
> But why the final BindableTableScan is not pruned ? The RelFieldTrimmer default is turned off, you should open it explicitly. Best, Danny Chan 在 2019年10月14日 +0800 AM11:51，dev@calcite.apache.org，写道： > > BINDABLE_TABLE_SCAN_RULE
RE: Calcite - Model - MongoDB view
Hi, Neha Saini
What do you mean by MongoDB view, can you give more details, so we can give as
much help if we can.
Best,
Danny Chan
在 2019年10月14日 +0800 PM2:21，Neha Saini ，写道：
> Hi Team,
>
> Gentle reminder!
>
> Regards,
> Neha Saini
>
> From: Neha Saini
> Sent: Thursday, October 10, 2019 3:30 PM
> To: dev@calcite.apache.org
> Subject: Calcite - Model - MongoDB view
>
> Hi Team,
>
> Greetings for the day.
>
> I am working as a technical architect in HCL technologies.
> I am doing a PoC on Apache calcite.
>
> Using MongoDB adapter I want to use calcite sql parser on mongo db database.
> I am stuck at a point and need your help in the same.
>
> My data in Mongo DB has following structure:-
> {
> "_id": "tt0772249",
> "title": "Palaces of a Queen",
> "type": "movie",
> "year": 1967,
> "crew": [{
> "name": "Micael Calunga",
> "role": "actor"
> }, {
> "name": "Jordon Zonotti",
> "role": "actress"
> }
> ]
> }
>
> I want to create view with column title, year, crew.name
> But in modql.json , I am not able to MAP array of objects, like crew.name 0th
> element, or nth element.
>
> "tables": [
> {
> "name": "movie",
> "type": "view",
> "sql": "select cast(_MAP['title'] AS varchar(100)) AS title,
> cast(_MAP['year'] AS varchar(4)) AS year, cast(_MAP['crew.name'][0] AS
> varchar(20)) AS crewname from \"mongo_raw\".\"movie\""
> },
>
>
> In the GIThib code, we have zip example, but zip file is a flat json, but
> json I am using is embedded , hence not able to use the example.
>
> I would really appreciate some help.
>
> Thanks!
>
> Regards,
> Neha Saini
>
> ::DISCLAIMER::
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses
> in transmission. The e mail and its contents (with or without referred
> errors) shall therefore not attach any liability on the originator or HCL or
> its affiliates. Views or opinions, if any, presented in this email are solely
> those of the author and may not necessarily reflect the views or opinions of
> HCL or its affiliates. Any form of reproduction, dissemination, copying,
> disclosure, modification, distribution and / or publication of this message
> without the prior written consent of authorized representative of HCL is
> strictly prohibited. If you have received this email in error please delete
> it and notify the sender immediately. Before opening any email and/or
> attachments, please check them for viruses and other defects.
>
[jira] [Created] (CALCITE-3407) Add support for interpretering minus/intersect relational set operators
Wang Yanlin created CALCITE-3407:
Summary: Add support for interpretering minus/intersect relational
set operators
Key: CALCITE-3407
URL: https://issues.apache.org/jira/browse/CALCITE-3407
Project: Calcite
Issue Type: Improvement
Reporter: Wang Yanlin
Currently, for SetOp, only `union` is supported by Interpreter
add the test cases in InterpreterTest, and run, they will fail by throwing
exception
{code:java}
@Test public void testInterpretIntersect() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (1, 'a'), (1, 'a'), (2, 'b'), (3, 'c')) as
t(x, y))\n"
+ "intersect\n"
+ "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x,
y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[1, a]");
}
@Test public void testInterpretIntersectAll() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (1, 'a'), (1, 'a'), (2, 'b'), (3, 'c')) as
t(x, y))\n"
+ "intersect all\n"
+ "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x,
y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[1, a]", "[1, a]");
}
@Test public void testInterpretMinus() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as
t(x, y))\n"
+ "except\n"
+ "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x,
y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[2, b]", "[3, c]");
}
@Test public void testInterpretMinusAll() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as
t(x, y))\n"
+ "except all\n"
+ "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x,
y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[2, b]", "[2, b]", "[3, c]");
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
RE: Calcite - Model - MongoDB view
Hi Team,
Gentle reminder!
Regards,
Neha Saini
From: Neha Saini
Sent: Thursday, October 10, 2019 3:30 PM
To: dev@calcite.apache.org
Subject: Calcite - Model - MongoDB view
Hi Team,
Greetings for the day.
I am working as a technical architect in HCL technologies.
I am doing a PoC on Apache calcite.
Using MongoDB adapter I want to use calcite sql parser on mongo db database.
I am stuck at a point and need your help in the same.
My data in Mongo DB has following structure:-
{
"_id": "tt0772249",
"title": "Palaces of a Queen",
"type": "movie",
"year": 1967,
"crew": [{
"name": "Micael
Calunga",
"role": "actor"
}, {
"name": "Jordon
Zonotti",
"role":
"actress"
}
]
}
I want to create view with column title, year, crew.name
But in modql.json , I am not able to MAP array of objects, like crew.name 0th
element, or nth element.
"tables": [
{
"name": "movie",
"type": "view",
"sql": "select cast(_MAP['title'] AS varchar(100)) AS title,
cast(_MAP['year'] AS varchar(4)) AS year, cast(_MAP['crew.name'][0] AS
varchar(20)) AS crewname from \"mongo_raw\".\"movie\""
},
In the GIThib code, we have zip example, but zip file is a flat json, but json
I am using is embedded , hence not able to use the example.
I would really appreciate some help.
Thanks!
Regards,
Neha Saini
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended
for the named recipient(s) only. E-mail transmission is not guaranteed to be
secure or error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or may contain viruses in transmission.
The e mail and its contents (with or without referred errors) shall therefore
not attach any liability on the originator or HCL or its affiliates. Views or
opinions, if any, presented in this email are solely those of the author and
may not necessarily reflect the views or opinions of HCL or its affiliates. Any
form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written
consent of authorized representative of HCL is strictly prohibited. If you have
received this email in error please delete it and notify the sender
immediately. Before opening any email and/or attachments, please check them for
viruses and other defects.
