[jira] [Created] (CALCITE-3399) RelFieldTrimmer trim fields for UNION, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL
jin xing created CALCITE-3399: - Summary: RelFieldTrimmer trim fields for UNION, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL Key: CALCITE-3399 URL: https://issues.apache.org/jira/browse/CALCITE-3399 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing Assignee: jin xing RelFieldTrimmer#trimFields provides functionality to trim fields for UNION, UNION ALL, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL; But UNION ALL, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL works by checking duplication. Column pruning on inputs of SetOp might lead to different semantics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[DISCUSS] Automated security fixes via dependabot
Dependabot is a bot on Github that opens PRs to automatically upgrade out of date dependencies to fix security issues. Recently, Github acquired dependabot and is gradually enabling the bot on all repositories. It just opened a PR to upgrade a few dependencies in the Avatica repository: https://github.com/apache/calcite-avatica/pull/114 I'd like to start some discussion as to how we should deal with these PRs. For some background, dependency upgrades should usually have a jira issue number assigned, so that the change is fully trackable. We recently had some discussion regarding trivial fixes to documentation and the consensus was that changes to the code is not considered to be trivial and that an issue should be filed on jira. If we will not merge these PRs, I think it makes sense to ask infra to disable them. Having these open PRs and then closing them manually seem to generate a lot of noise. According to the documentation for dependabot [1] it appears that we can either opt out of having dependabot opening PRs completely or have it open PRs. There is no middle-ground where dependabot/Github sends members of the repo a notification for security issues, but do not open any PRs. What do you guys think? Francis [1] https://help.github.com/en/articles/configuring-automated-security-fixes
Re: [DISCUSS] Towards Avatica 1.16.0
Thanks, Josh! Very much appreciated! On 11/10/2019 2:35 am, Josh Elser wrote: I'll try to get CALCITE-3384 in. Will take a look through the reste here. A couple catch my eye. On 10/9/19 7:26 PM, Francis Chuang wrote: Hey everyone, It's been around 5 months since the last Avatica release. There's been a couple of commits to the code-base since and I'd like to use this as an opportunity to get a few more PRs in and make a release available for voting. I am happy to be release manager for this release. In terms of PRs I am hoping for the following to be merged for this release: - CALCITE-3384: Support Kerberos using SPNEGO over HTTPS - https://github.com/apache/calcite-avatica/pull/113 (Josh is reviewing) - CALCITE-333: Add pluggable frame size limits - https://issues.apache.org/jira/browse/CALCITE- (looking for a reviewer) - CALCITE-3199: Fix unixDateCeil and unixDateFloor - https://github.com/apache/calcite-avatica/pull/109 (looking for a reviewer) - CALCITE-3162: Reading arrays from Calcite through JdbcMeta throws exception - https://github.com/apache/calcite-avatica/pull/106 (looking for a reviewer) - CALCITE-3163: Mapping types in AbstractCursor does not adhere to JDBC spec - https://github.com/apache/calcite-avatica/pull/105 (looking for a reviewer) - CALCITE-3158: Move build system to Gradle - https://github.com/apache/calcite-avatica/pull/104 (Vladimir has put a lot of effort into this. Can we get some consensus as to whether this should to be merged?) - CALCITE-3078: Duplicate code for lastDay in Calcite and Avatica - https://github.com/apache/calcite-avatica/pull/98 (looking for a reviewer) - No Jira: Support UNIX timestamp to string date - https://github.com/apache/calcite-avatica/pull/96 (can someone confirm if these changes are valid or should the PR be closed?) - CALCITE-2704: Avoid using ISO-8859-1 to parse request in JSONHandler - https://github.com/apache/calcite-avatica/pull/85 (Someone posted some tests in the issue comments. Can someone please add the tests, review and merge?) - CALCITE-1806: Add Spark JDBC tests - https://github.com/apache/calcite-avatica/pull/28 (Kevin, any chance you can take another look at this?) If you have some spare cycles for Avatica, please pick up some of the PRs in this list, it would be very much appreciated! Francis
Re: [DISCUSS] Towards Avatica 1.16.0
I'll try to get CALCITE-3384 in. Will take a look through the reste here. A couple catch my eye. On 10/9/19 7:26 PM, Francis Chuang wrote: Hey everyone, It's been around 5 months since the last Avatica release. There's been a couple of commits to the code-base since and I'd like to use this as an opportunity to get a few more PRs in and make a release available for voting. I am happy to be release manager for this release. In terms of PRs I am hoping for the following to be merged for this release: - CALCITE-3384: Support Kerberos using SPNEGO over HTTPS - https://github.com/apache/calcite-avatica/pull/113 (Josh is reviewing) - CALCITE-333: Add pluggable frame size limits - https://issues.apache.org/jira/browse/CALCITE- (looking for a reviewer) - CALCITE-3199: Fix unixDateCeil and unixDateFloor - https://github.com/apache/calcite-avatica/pull/109 (looking for a reviewer) - CALCITE-3162: Reading arrays from Calcite through JdbcMeta throws exception - https://github.com/apache/calcite-avatica/pull/106 (looking for a reviewer) - CALCITE-3163: Mapping types in AbstractCursor does not adhere to JDBC spec - https://github.com/apache/calcite-avatica/pull/105 (looking for a reviewer) - CALCITE-3158: Move build system to Gradle - https://github.com/apache/calcite-avatica/pull/104 (Vladimir has put a lot of effort into this. Can we get some consensus as to whether this should to be merged?) - CALCITE-3078: Duplicate code for lastDay in Calcite and Avatica - https://github.com/apache/calcite-avatica/pull/98 (looking for a reviewer) - No Jira: Support UNIX timestamp to string date - https://github.com/apache/calcite-avatica/pull/96 (can someone confirm if these changes are valid or should the PR be closed?) - CALCITE-2704: Avoid using ISO-8859-1 to parse request in JSONHandler - https://github.com/apache/calcite-avatica/pull/85 (Someone posted some tests in the issue comments. Can someone please add the tests, review and merge?) - CALCITE-1806: Add Spark JDBC tests - https://github.com/apache/calcite-avatica/pull/28 (Kevin, any chance you can take another look at this?) If you have some spare cycles for Avatica, please pick up some of the PRs in this list, it would be very much appreciated! Francis
[jira] [Created] (CALCITE-3398) Release avatica 1.16.0
Francis Chuang created CALCITE-3398: --- Summary: Release avatica 1.16.0 Key: CALCITE-3398 URL: https://issues.apache.org/jira/browse/CALCITE-3398 Project: Calcite Issue Type: Improvement Components: avatica Reporter: Francis Chuang Assignee: Francis Chuang Fix For: avatica-1.16.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3397) AssertionError for interpreter multiset
Wang Yanlin created CALCITE-3397:
Summary: AssertionError for interpreter multiset
Key: CALCITE-3397
URL: https://issues.apache.org/jira/browse/CALCITE-3397
Project: Calcite
Issue Type: Bug
Reporter: Wang Yanlin
when interpretering sql
got,
{code:java}
java.lang.AssertionError: interpreter: no implementation for class
org.apache.calcite.rel.core.Collect
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:460)
at
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43)
at org.apache.calcite.rel.BiRel.childrenAccept(BiRel.java:46)
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447)
at
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43)
at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447)
at
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43)
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:405)
at
org.apache.calcite.interpreter.Interpreter.(Interpreter.java:88)
at
org.apache.calcite.test.InterpreterTest.testInterpretMultiset(InterpreterTest.java:127)
{code}
Reproduce this with test case
{code:java}
@Test public void testInterpretMultiset() throws Exception {
final String sql = "select multiset['a', 'b', 'c']";
SqlNode parse = planner.parse(sql);
SqlNode validate = planner.validate(parse);
RelNode convert = planner.rel(validate).project();
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[[a, b, c]]");
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3396) Materialization matching succeeds when query and view are both of UNION but have different 'all' property
jin xing created CALCITE-3396: - Summary: Materialization matching succeeds when query and view are both of UNION but have different 'all' property Key: CALCITE-3396 URL: https://issues.apache.org/jira/browse/CALCITE-3396 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Support Sql Hint for Calcite
Hi, Julian, do you think we can make some effort to push this feature into release 1.22 ? The Flink community kind of need this feature in the near future. The most controversial part is how to transfer the hints during planning phrase, my initial idea is: >To copy the hints automatedly in the method RelOptRuleCall#transformTo for the >new rel node from the >original rel node if both of them are all RelWithHint. I have added some simple planning test cases and it works as expected, but I’m not sure if this can fits all the corner cases(very probably not). The other part works great as the design doc describes. Best, Danny Chan 在 2019年9月5日 +0800 AM10:12，Julian Hyde ，写道： > Thanks for continuing to push on this! > > I don’t much like the MSSQL-style syntax for table hints. It adds a new use > of the WITH keyword that is unrelated to the use of WITH for common-table > expressions. Instead of > > select /*+ NO_HASH_JOIN, RESOURCE(mem='128mb', parallelism='24') */ > from > emp with (INDEX(idx1, idx2)) > join > dept with (PROPERTIES(k1='v1', k2='v2')) > on > emp.deptno = dept.deptno > > could we do > > select /*+ NO_HASH_JOIN, RESOURCE(mem='128mb', parallelism='24') */ > from > emp /*+ INDEX(idx1, idx2) */ > join > dept /*+ PROPERTIES(k1='v1', k2='v2’) */ > on > emp.deptno = dept.deptno > > > In some of the classes you have public fields of type ImmutableList. This > makes it difficult to coexist in an environment that uses a different version > of Guava, or shades Guava. Probably better to make them of type List. (You > don’t need to change ImmutableBitSet; it’s not a Guava class.) > > There is one argument where a List is assigned to an > ImmutableBitSet. Make it an Iterable, and people can pass in an > existing ImmutableBitSet without copying. > > Julian > > > > On Sep 3, 2019, at 1:18 AM, Danny Chan wrote: > > > > Hi, fellows, I’m here again :) > > > > This time I want to have a full discussion about CALCITE-482, which is an > > issue fired at 27/Nov/14 by Vladimir Sitnikov. > > > > Almost every sql vendor supports sql hint for their production version DB > > engines, so it would be nice if we have a framework in Calcite to support > > this, so that the engines that built based on CALCITE would have the > > ability to extend and have their custom sql hint implementations. > > > > In April this year I have fired an initial discussion about this topic[1], > > and I’m glad that we have some agreements for the design. > > > > Recently I have fired a PR[3] and write a design doc[2] mostly based on the > > discussion[1], so feel free to give some feedback here so we can make the > > hint framework more flexible. > > > > Any suggestions are appreciated. > > > > > > [1] > > https://lists.apache.org/list.html?dev@calcite.apache.org:dfr=2019-4-1|dto=2019-4-30:How%20to%20traverse > > [2] > > https://docs.google.com/document/d/1mykz-w2t1Yw7CH6NjUWpWqCAf_6YNKxSc59gXafrNCs/edit?usp=sharing > > [3] https://github.com/apache/calcite/pull/1354 > > > > Best, > > Danny Chan >
