[jira] [Commented] (CALCITE-3160) Failed to materialize when the aggregate function uses group key
[ https://issues.apache.org/jira/browse/CALCITE-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876680#comment-16876680 ] Feng Zhu commented on CALCITE-3160: --- I have also noticed such kind of issues. (1)When sum(empid), AggregateToAggregateUnifyRule will be matched. However, it uses "!=" to compare Projects in query and target. {code:java} if (query.getInput() != target.getInput()) { return null; } {code} But for the Projects matched in TrivialRule, the result still use that in query without replacing it into target. Consequently, in the next iteration, AggregateToAggregateUnifyRule cannot reuse the MV. {code:java} public UnifyResult apply(UnifyRuleCall call) { if (call.query.equals(call.target)) { return call.result(call.query); } return null; } {code} (2)When sum(salary), ProjectToProjectUnifyRule will replace child Projects and AggregateOnProjectToAggregateUnifyRule can reuse the MV. Therefore, I think we need to fix the logic in TrivialRule as *_call.result(call.result)_* or unify the condition as "_*!query.getInput.equals(target.getInput)*_". We also need to check other rules. > Failed to materialize when the aggregate function uses group key > > > Key: CALCITE-3160 > URL: https://issues.apache.org/jira/browse/CALCITE-3160 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Repro: > {code:java} > @Test public void testAggregateGroupSetsRollUp() { > checkMaterialize( > "select \"empid\", \"deptno\", count(*) as c, sum(\"empid\") as s > from \"emps\" " > + "group by \"empid\", \"deptno\"", > "select count(*) + 1 as c, \"deptno\" from \"emps\" group by > cube(\"empid\",\"deptno\")", > HR_FKUK_MODEL, > CalciteAssert.checkResultContains( > "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], " > + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n" > + " EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, > {0}, {1}, {}]], agg#0=[$SUM0($2)])\n" > + "EnumerableTableScan(table=[[hr, m0]])")); > } > {code} > Note that if we change sum(\"empid\") to sum("salary"), it can materialize > successfully. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876677#comment-16876677 ] Danny Chan commented on CALCITE-3166: - Thanks [~julianhyde], i'm ok with this Context unwrap, cause it seems like a code refactoring. > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876634#comment-16876634 ] Julian Hyde edited comment on CALCITE-3166 at 7/2/19 3:25 AM: -- {quote}Does this mean we should always make a {{RelBuilder.Config}} instance into the {{Context}} now ? This api seems not that concise to use.{quote} The constructor for {{RelBuilder}} takes a {{Context}}, and i don't want to add other constructors. {{Context}} is capable of wrapping any other objects that we want to pass in - such as {{RelNode}} factories and a {{RexSimplifier}}. I don't expect people to need to customize {{RelBuilder.Config}} very often, so the API doesn't need to be concise. The main use case is to disable features because of bugs. We shouldn't be putting features into {{RelBuilder}} that are not globally applicable. If there are optional features - e.g. whether to decorrelate a sub-query - they are better packaged as rules. was (Author: julianhyde): {quote}Does this mean we should always make a {{RelBuilder.Config}} instance into the {{Context}} now ? This api seems not that concise to use.{quote} The constructor for {{RelBuilder}} takes a {{Context}}, and i don't want to add other constructors. I don't expect people to need to customize {{RelBuilder.Config}} very often, so the API doesn't need to be concise. The main use case is to disable features because of bugs. We shouldn't be putting features into {{RelBuilder}} that are not globally applicable. If there are optional features - e.g. whether to decorrelate a sub-query - they are better packaged as rules. > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876634#comment-16876634 ] Julian Hyde commented on CALCITE-3166: -- {quote}Does this mean we should always make a {{RelBuilder.Config}} instance into the {{Context}} now ? This api seems not that concise to use.{quote} The constructor for {{RelBuilder}} takes a {{Context}}, and i don't want to add other constructors. I don't expect people to need to customize {{RelBuilder.Config}} very often, so the API doesn't need to be concise. The main use case is to disable features because of bugs. We shouldn't be putting features into {{RelBuilder}} that are not globally applicable. If there are optional features - e.g. whether to decorrelate a sub-query - they are better packaged as rules. > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3152) Unify throws in sql parser
[ https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876604#comment-16876604 ] Danny Chan commented on CALCITE-3152: - I'm gonna merge this PR if there are no more comments in 24 hours. > Unify throws in sql parser > -- > > Key: CALCITE-3152 > URL: https://issues.apache.org/jira/browse/CALCITE-3152 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Copied from DEV mail, > Now our parser has 3 kinds of throws behavior: > > [1] Use JavaCC generateParseException > [2] Use SqlUtil.newContextException > [3] Use JavaCC ParseExecption directly > > For [1] and [2] there is a position info in the exception message, a throw > may like: > {code:java} > From line 1, column 15 to line 1, column 26: > {code} > But for 3, we only have the error message without position info, which is not > that user friendly when the sql text is huge (there are 10 occurance for 3 in > our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all > the cases, the 2 can totally replace 3. > > [1] > [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494] > [2] > [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386] > [3] > [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876603#comment-16876603 ] Danny Chan commented on CALCITE-3166: - Reviewed and left some comments. > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-3166: - Fix Version/s: 1.21.0 > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876579#comment-16876579 ] Julian Hyde commented on CALCITE-3166: -- Can someone please review https://github.com/julianhyde/calcite/tree/3166-relBuilder-config. > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-3166: - Labels: pull-request-available (was: ) > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-3166) Make RelBuilder configurable
[ https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde reassigned CALCITE-3166: Assignee: Julian Hyde > Make RelBuilder configurable > > > Key: CALCITE-3166 > URL: https://issues.apache.org/jira/browse/CALCITE-3166 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Make {{RelBuilder}} configurable, so that particular optimizations can easily > be turned off. > I propose to add a class {{RelBuilder.Config}}, which is immutable and has a > public final field for each configuration property; also a class > {{RelBuilder.ConfigBuilder}} to create a config. > {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by > calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3166) Make RelBuilder configurable
Julian Hyde created CALCITE-3166: Summary: Make RelBuilder configurable Key: CALCITE-3166 URL: https://issues.apache.org/jira/browse/CALCITE-3166 Project: Calcite Issue Type: Bug Reporter: Julian Hyde Make {{RelBuilder}} configurable, so that particular optimizations can easily be turned off. I propose to add a class {{RelBuilder.Config}}, which is immutable and has a public final field for each configuration property; also a class {{RelBuilder.ConfigBuilder}} to create a config. {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3144) Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x END)" to "SUM(x) FILTER (WHERE b)"
[ https://issues.apache.org/jira/browse/CALCITE-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-3144: - Fix Version/s: 1.21.0 > Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x > END)" to "SUM(x) FILTER (WHERE b)" > -- > > Key: CALCITE-3144 > URL: https://issues.apache.org/jira/browse/CALCITE-3144 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.21.0 > > > Add a rule that converts "SUM(CASE WHEN b THEN x END)" to "SUM\(x) FILTER > (WHERE b)". > Druid added {{CaseFilteredAggregatorRule}} in > https://github.com/apache/incubator-druid/pull/4360. > Maybe {{AggregateCaseToFilterRule}} is a slightly better name. Or maybe this > transform could be done in {{RelBuilder.aggregate}}, and we wouldn't need a > rule. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3144) Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x END)" to "SUM(x) FILTER (WHERE b)"
[ https://issues.apache.org/jira/browse/CALCITE-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876551#comment-16876551 ] Julian Hyde commented on CALCITE-3144: -- Can someone please review [https://github.com/julianhyde/calcite/tree/3144-aggregate-case-to-filter]. > Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x > END)" to "SUM(x) FILTER (WHERE b)" > -- > > Key: CALCITE-3144 > URL: https://issues.apache.org/jira/browse/CALCITE-3144 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Add a rule that converts "SUM(CASE WHEN b THEN x END)" to "SUM\(x) FILTER > (WHERE b)". > Druid added {{CaseFilteredAggregatorRule}} in > https://github.com/apache/incubator-druid/pull/4360. > Maybe {{AggregateCaseToFilterRule}} is a slightly better name. Or maybe this > transform could be done in {{RelBuilder.aggregate}}, and we wouldn't need a > rule. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3144) Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x END)" to "SUM(x) FILTER (WHERE b)"
[ https://issues.apache.org/jira/browse/CALCITE-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876495#comment-16876495 ] Gian Merlino commented on CALCITE-3144: --- > Gian Merlino, Do you recall the reason for this line? I don't think it's > necessary, because I think the rewrite is valid even if there are GROUPING > SETS etc. I don't recall, but, Druid doesn't currently support GROUPING SETS so the thought process may have been "well, let's not worry about whether or not this is valid for GROUPING SETS or not". > Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x > END)" to "SUM(x) FILTER (WHERE b)" > -- > > Key: CALCITE-3144 > URL: https://issues.apache.org/jira/browse/CALCITE-3144 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Add a rule that converts "SUM(CASE WHEN b THEN x END)" to "SUM\(x) FILTER > (WHERE b)". > Druid added {{CaseFilteredAggregatorRule}} in > https://github.com/apache/incubator-druid/pull/4360. > Maybe {{AggregateCaseToFilterRule}} is a slightly better name. Or maybe this > transform could be done in {{RelBuilder.aggregate}}, and we wouldn't need a > rule. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-3144) Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x END)" to "SUM(x) FILTER (WHERE b)"
[ https://issues.apache.org/jira/browse/CALCITE-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde reassigned CALCITE-3144: Assignee: Julian Hyde > Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x > END)" to "SUM(x) FILTER (WHERE b)" > -- > > Key: CALCITE-3144 > URL: https://issues.apache.org/jira/browse/CALCITE-3144 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > > Add a rule that converts "SUM(CASE WHEN b THEN x END)" to "SUM\(x) FILTER > (WHERE b)". > Druid added {{CaseFilteredAggregatorRule}} in > https://github.com/apache/incubator-druid/pull/4360. > Maybe {{AggregateCaseToFilterRule}} is a slightly better name. Or maybe this > transform could be done in {{RelBuilder.aggregate}}, and we wouldn't need a > rule. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3144) Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x END)" to "SUM(x) FILTER (WHERE b)"
[ https://issues.apache.org/jira/browse/CALCITE-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876418#comment-16876418 ] Julian Hyde commented on CALCITE-3144: -- [~gian], Do you recall the reason for this line? I don't think it's necessary, because I think the rewrite is valid even if there are {{GROUPING SETS}} etc. {code:java} if (aggregate.getGroupSets().size() != 1) { return false; } {code} > Add rule, CaseFilteredAggregatorRule, that converts "SUM(CASE WHEN b THEN x > END)" to "SUM(x) FILTER (WHERE b)" > -- > > Key: CALCITE-3144 > URL: https://issues.apache.org/jira/browse/CALCITE-3144 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Priority: Major > > Add a rule that converts "SUM(CASE WHEN b THEN x END)" to "SUM\(x) FILTER > (WHERE b)". > Druid added {{CaseFilteredAggregatorRule}} in > https://github.com/apache/incubator-druid/pull/4360. > Maybe {{AggregateCaseToFilterRule}} is a slightly better name. Or maybe this > transform could be done in {{RelBuilder.aggregate}}, and we wouldn't need a > rule. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3159) Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
[ https://issues.apache.org/jira/browse/CALCITE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876409#comment-16876409 ] Julian Hyde edited comment on CALCITE-3159 at 7/1/19 6:26 PM: -- I agree. Maybe add a method {{Optionality getDistinctOptionality()}} in {{SqlAggFunction}}, default {{OPTIONAL}}, and returning {{IGNORED}} for {{MIN}}, {{MAX}}, {{BIT_OR}}, {{BIT_AND}}. And in {{RelBuilder.aggregateCall}}, set {{distinct}} to {{false}} if the function has a value {{IGNORED}}. was (Author: julianhyde): I agree. Maybe add a method {{Optionality getDistinctOptionality()}} in {{SqlAggFunction}}, default {{OPTIONAL}}. , but set to {{IGNORED}} for {{MIN}}, {{MAX}}, {{BIT_OR}}, {{BIT_AND}}. And in {{RelBuilder.aggregateCall}}, set {{distinct}} to {{false}} if the function has a value {{IGNORED}}. > Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions > -- > > Key: CALCITE-3159 > URL: https://issues.apache.org/jira/browse/CALCITE-3159 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > > For the following query: > {code:java} > select a, min(distinct b), bit_or(distinct c) from foo group by a; > {code} > Currently Calcite still preserve the distinct for these aggregate functions, > but DISTINCT is not meaningful with MIN/MAX and is available for ISO > compatibility only. We can safely remove distinct and get more optimization > opportunities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3159) Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions
[ https://issues.apache.org/jira/browse/CALCITE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876409#comment-16876409 ] Julian Hyde commented on CALCITE-3159: -- I agree. Maybe add a method {{Optionality getDistinctOptionality()}} in {{SqlAggFunction}}, default {{OPTIONAL}}. , but set to {{IGNORED}} for {{MIN}}, {{MAX}}, {{BIT_OR}}, {{BIT_AND}}. And in {{RelBuilder.aggregateCall}}, set {{distinct}} to {{false}} if the function has a value {{IGNORED}}. > Distinct can be removed for MIN/MAX/BIT_OR/BIT_AND aggregate functions > -- > > Key: CALCITE-3159 > URL: https://issues.apache.org/jira/browse/CALCITE-3159 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > > For the following query: > {code:java} > select a, min(distinct b), bit_or(distinct c) from foo group by a; > {code} > Currently Calcite still preserve the distinct for these aggregate functions, > but DISTINCT is not meaningful with MIN/MAX and is available for ISO > compatibility only. We can safely remove distinct and get more optimization > opportunities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE
[ https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876391#comment-16876391 ] Julian Hyde commented on CALCITE-1935: -- It's not very important that you squash; it's more important that you rebase onto (relatively recent) master branch. When the PR is ready the committer can squash. The committer might choose to keep each author's contributions as a separate commit, or might combine commits and make sure that everyone gets credit. > Reference implementation for MATCH_RECOGNIZE > > > Key: CALCITE-1935 > URL: https://issues.apache.org/jira/browse/CALCITE-1935 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Priority: Major > Labels: match > > We now have comprehensive support for parsing and validating MATCH_RECOGNIZE > queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know > the purpose of this work is to do CEP within Flink, but a reference > implementation that works on non-streaming data would be valuable. > I propose that we add a class EnumerableMatch that can generate Java code to > evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be > efficient. I don't mind if it (say) buffers all the data in memory and makes > O(n ^ 3) passes over it. People can make it more efficient over time. > When we have a reference implementation, people can start playing with this > feature. And we can start building a corpus of data sets, queries, and their > expected result. The Flink implementation will be able to test against those > same queries, and should give the same results, even though Flink will be > reading streaming data. > Let's create {{match.iq}} with the following query based on > https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1: > {code} > !set outputformat mysql > !use match > SELECT * > FROM sales_history MATCH_RECOGNIZE ( > PARTITION BY product > ORDER BY tstamp > MEASURES STRT.tstamp AS start_tstamp, >LAST(UP.tstamp) AS peak_tstamp, >LAST(DOWN.tstamp) AS end_tstamp, >MATCH_NUMBER() AS mno > ONE ROW PER MATCH > AFTER MATCH SKIP TO LAST DOWN > PATTERN (STRT UP+ FLAT* DOWN+) > DEFINE >UP AS UP.units_sold > PREV(UP.units_sold), >FLAT AS FLAT.units_sold = PREV(FLAT.units_sold), >DOWN AS DOWN.units_sold < PREV(DOWN.units_sold) >) MR > ORDER BY MR.product, MR.start_tstamp; > PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO > -- --- --- --- -- > TWINKIES 01-OCT-2014 03-OCT-2014 06-OCT-2014 1 > TWINKIES 06-OCT-2014 08-OCT-2014 09-OCT-2014 2 > TWINKIES 09-OCT-2014 13-OCT-2014 16-OCT-2014 3 > TWINKIES 16-OCT-2014 18-OCT-2014 20-OCT-2014 4 > 4 rows selected. > !ok > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3165) Project#accept(RexShuttle shuttle) does not update rowType
[ https://issues.apache.org/jira/browse/CALCITE-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876380#comment-16876380 ] Haisheng Yuan commented on CALCITE-3165: Yup, a test case might be helpful. > Project#accept(RexShuttle shuttle) does not update rowType > -- > > Key: CALCITE-3165 > URL: https://issues.apache.org/jira/browse/CALCITE-3165 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > > {{Project#accept(RexShuttle shuttle)}} can return a copy of the original > Project, if the list of expressions is modified by the shuttle. However, this > Project copy will maintain the original rowType, which might be wrong because > this type is based on the expressions list (which was modified by the shuttle) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3165) Project#accept(RexShuttle shuttle) does not update rowType
[ https://issues.apache.org/jira/browse/CALCITE-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876315#comment-16876315 ] Ruben Quesada Lopez edited comment on CALCITE-3165 at 7/1/19 4:15 PM: -- Currently, {{Project#accept(RexShuttle shuttle)}} is implemented as: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } return copy(traitSet, getInput(), exps, rowType); } {code} Thus, the original rowType is always kept, even though we know that the list of expressions has been changed, so we might have potentially a different rowType. The idea would be doing something in the line of: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } final RelDataType newRowType = RexUtil.createStructType( getInput().getCluster().getTypeFactory(), exps, rowType.getFieldNames(), SqlValidatorUtil.F_SUGGESTER); return copy(traitSet, getInput(), exps, newRowType); } {code} I'll try to provide a unit test to show the problem. was (Author: rubenql): Currently, Project#accept(RexShuttle shuttle) does: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } return copy(traitSet, getInput(), exps, rowType); } {code} Thus, the original rowType is always kept, even though we know that the list of expressions has been changed, so we might have potentially a different rowType. The idea would be doing something in the line of: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } final RelDataType newRowType = RexUtil.createStructType( getInput().getCluster().getTypeFactory(), exps, rowType.getFieldNames(), SqlValidatorUtil.F_SUGGESTER); return copy(traitSet, getInput(), exps, newRowType); } {code} I'll try to provide a unit test to show the problem. > Project#accept(RexShuttle shuttle) does not update rowType > -- > > Key: CALCITE-3165 > URL: https://issues.apache.org/jira/browse/CALCITE-3165 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > > {{Project#accept(RexShuttle shuttle)}} can return a copy of the original > Project, if the list of expressions is modified by the shuttle. However, this > Project copy will maintain the original rowType, which might be wrong because > this type is based on the expressions list (which was modified by the shuttle) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3165) Project#accept(RexShuttle shuttle) does not update rowType
[ https://issues.apache.org/jira/browse/CALCITE-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876315#comment-16876315 ] Ruben Quesada Lopez commented on CALCITE-3165: -- Currently, Project#accept(RexShuttle shuttle) does: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } return copy(traitSet, getInput(), exps, rowType); } {code} Thus, the original rowType is always kept, even though we know that the list of expressions has been changed, so we might have potentially a different rowType. The idea would be doing something in the line of: {code} public RelNode accept(RexShuttle shuttle) { List exps = shuttle.apply(this.exps); if (this.exps == exps) { return this; } final RelDataType newRowType = RexUtil.createStructType( getInput().getCluster().getTypeFactory(), exps, rowType.getFieldNames(), SqlValidatorUtil.F_SUGGESTER); return copy(traitSet, getInput(), exps, newRowType); } {code} I'll try to provide a unit test to show the problem. > Project#accept(RexShuttle shuttle) does not update rowType > -- > > Key: CALCITE-3165 > URL: https://issues.apache.org/jira/browse/CALCITE-3165 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > > {{Project#accept(RexShuttle shuttle)}} can return a copy of the original > Project, if the list of expressions is modified by the shuttle. However, this > Project copy will maintain the original rowType, which might be wrong because > this type is based on the expressions list (which was modified by the shuttle) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3165) Project#accept(RexShuttle shuttle) does not update rowType
Ruben Quesada Lopez created CALCITE-3165: Summary: Project#accept(RexShuttle shuttle) does not update rowType Key: CALCITE-3165 URL: https://issues.apache.org/jira/browse/CALCITE-3165 Project: Calcite Issue Type: Bug Components: core Reporter: Ruben Quesada Lopez Assignee: Ruben Quesada Lopez {{Project#accept(RexShuttle shuttle)}} can return a copy of the original Project, if the list of expressions is modified by the shuttle. However, this Project copy will maintain the original rowType, which might be wrong because this type is based on the expressions list (which was modified by the shuttle) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3125) Remove completely CorrelateJoinType
[ https://issues.apache.org/jira/browse/CALCITE-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Quesada Lopez resolved CALCITE-3125. -- Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/64a61cda18ada294e74ed6841d714d067bf0cb4e > Remove completely CorrelateJoinType > --- > > Key: CALCITE-3125 > URL: https://issues.apache.org/jira/browse/CALCITE-3125 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > CorrelateJoinType was deprecated in 1.20 via CALCITE-3119. > It shall be kept as deprecated for one minor release (1.20); and then remove > completely for 1.21. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3164) Averaging an all-nulls values after grouping produce NaN instead of NULL
Muhammad Gelbana created CALCITE-3164: - Summary: Averaging an all-nulls values after grouping produce NaN instead of NULL Key: CALCITE-3164 URL: https://issues.apache.org/jira/browse/CALCITE-3164 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Muhammad Gelbana {code:sql} -- Values are a single tuple SELECT C1, avg(C2) FROM (VALUES('X', NULL::INT)) T (C1, C2) GROUP BY C1 -- Values are more than a single tuple SELECT C1, avg(C2) FROM (VALUES('X', NULL::INT), ('X', NULL::INT)) T (C1, C2) GROUP BY C1 {code} Those queries return {{NaN}} while it's expected to return {{NULL}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3125) Remove completely CorrelateJoinType
[ https://issues.apache.org/jira/browse/CALCITE-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3125: Labels: pull-request-available (was: ) > Remove completely CorrelateJoinType > --- > > Key: CALCITE-3125 > URL: https://issues.apache.org/jira/browse/CALCITE-3125 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > > CorrelateJoinType was deprecated in 1.20 via CALCITE-3119. > It shall be kept as deprecated for one minor release (1.20); and then remove > completely for 1.21. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3133) Remove completely SemiJoinType
[ https://issues.apache.org/jira/browse/CALCITE-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Quesada Lopez resolved CALCITE-3133. -- Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/ceb13a134502b950ce1ce1c25d9267aa51ca14b5 > Remove completely SemiJoinType > -- > > Key: CALCITE-3133 > URL: https://issues.apache.org/jira/browse/CALCITE-3133 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > SemiJoinType was deprecated in 1.20 via CALCITE-2969. > It shall be kept as deprecated for one minor release (1.20); and then remove > completely for 1.21. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3163) Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specifications
[ https://issues.apache.org/jira/browse/CALCITE-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3163: -- Description: This is an issue I found while working on CALCITE-3162: According to my understanding, the method _AbstractCursor#convertValue()_, used for converting the component types of ARRAYs, does not adhere to what is specified in the [JDBC specification (Appendix B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] regarding mapping of types: * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a _float_. * The implementation is currently missing support for JDBC type REAL (7), which should convert to _float_ or _Float_ respectively This issue will cause an IllegalStateException to be thrown in clients using the Avatica Driver if they encounter REAL datatype in arrays. was: This is an issue I found while working on CALCITE-3162: According to my understanding, the method _AbstractCursor#convertValue()_, used for converting the component types of ARRAYs, does not adhere to what is specified in the [JDBC specification (Appendix B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] regarding mapping of types: * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a _float_. * The implementation is currently missing support for JDBC type REAL (7), which should convert to _float_ or _Float_ respectively This issue will cause an IllegalStateException to be thrown in clients using the Avatica Driver, if they encounter REAL datatype in arrays. > Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC > specifications > > > Key: CALCITE-3163 > URL: https://issues.apache.org/jira/browse/CALCITE-3163 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 >Reporter: Ralph Gasser >Priority: Major > > This is an issue I found while working on CALCITE-3162: According to my > understanding, the method _AbstractCursor#convertValue()_, used for > converting the component types of ARRAYs, does not adhere to what is > specified in the [JDBC specification (Appendix > B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] > regarding mapping of types: > * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java > _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a > _float_. > * The implementation is currently missing support for JDBC type REAL (7), > which should convert to _float_ or _Float_ respectively > This issue will cause an IllegalStateException to be thrown in clients using > the Avatica Driver if they encounter REAL datatype in arrays. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3163) Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specifications
[ https://issues.apache.org/jira/browse/CALCITE-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3163: -- Description: This is an issue I found while working on CALCITE-3162: According to my understanding, the method _AbstractCursor#convertValue()_, used for converting the component types of ARRAYs, does not adhere to what is specified in the [JDBC specification (Appendix B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] regarding mapping of types: * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a _float_. * The implementation is currently missing support for JDBC type REAL (7), which should convert to _float_ or _Float_ respectively This issue will cause an IllegalStateException to be thrown in clients using the Avatica Driver, if they encounter REAL datatype in arrays. was: This is an issue I found while working on CALCITE-3162: According to my understanding, the method _AbstractCursor#convertValue()_, used for converting the component types of ARRAYs, does not adhere to what is specified in the [JDBC specification|[https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea]] (Appendix B) regarding mapping of types: * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a _float_. * The implementation is currently missing support for JDBC type REAL (7), which should convert to _float_ or _Float_ respectively This issue will cause an IllegalStateException to be thrown in clients using the Avatica Driver, if they encounter REAL datatype in arrays. > Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC > specifications > > > Key: CALCITE-3163 > URL: https://issues.apache.org/jira/browse/CALCITE-3163 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 >Reporter: Ralph Gasser >Priority: Major > > This is an issue I found while working on CALCITE-3162: According to my > understanding, the method _AbstractCursor#convertValue()_, used for > converting the component types of ARRAYs, does not adhere to what is > specified in the [JDBC specification (Appendix > B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] > regarding mapping of types: > * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java > _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a > _float_. > * The implementation is currently missing support for JDBC type REAL (7), > which should convert to _float_ or _Float_ respectively > This issue will cause an IllegalStateException to be thrown in clients using > the Avatica Driver, if they encounter REAL datatype in arrays. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3163) Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specifications
[ https://issues.apache.org/jira/browse/CALCITE-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3163: -- Summary: Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specifications (was: Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specification) > Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC > specifications > > > Key: CALCITE-3163 > URL: https://issues.apache.org/jira/browse/CALCITE-3163 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 >Reporter: Ralph Gasser >Priority: Major > > This is an issue I found while working on CALCITE-3162: According to my > understanding, the method _AbstractCursor#convertValue()_, used for > converting the component types of ARRAYs, does not adhere to what is > specified in the [JDBC > specification|[https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea]] > (Appendix B) regarding mapping of types: > * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java > _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a > _float_. > * The implementation is currently missing support for JDBC type REAL (7), > which should convert to _float_ or _Float_ respectively > This issue will cause an IllegalStateException to be thrown in clients using > the Avatica Driver, if they encounter REAL datatype in arrays. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3162) Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException
[ https://issues.apache.org/jira/browse/CALCITE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3162: -- Summary: Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException (was: Reading Arrays from Calcite using Avatica generates error) > Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException > -- > > Key: CALCITE-3162 > URL: https://issues.apache.org/jira/browse/CALCITE-3162 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 > Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment > Zulu11.29+3-CA (build 11.0.2+7-LTS) > Issue occurs with both Apache Calcite 1.19.0 & 1.20.0. >Reporter: Ralph Gasser >Priority: Major > > I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a > custom data store and in addition, I'm using _Apache Avatica_ to expose the > entire functionality via JDBC for an arbitrary, potential remote client to > use. > We're working a lot with Array types, since we're using our backend to store > high-dimensional vectors. However, it seems that currently, Apache Avatica > has troubles handling such arrays. > Take the following test-case that reproduces the problem pretty well. > {code:java} > @Test > public void test() throws Exception { > HttpServer server = null; > try { > Class.forName("org.apache.calcite.jdbc.Driver"); > server = new HttpServer.Builder<>() > .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", > newProperties())),Driver.Serialization.PROTOBUF) > .withPort(1234) > .build(); > server.start(); > final Connection connection = > DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); > final Statement stmt = connection.createStatement(); > final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, > 3.0, 2.0]"); > resultSet.next(); > resultSet.getArray(1); > } catch (Exception e) { > System.out.println(e.getMessage()); > } finally { > if (server != null) { >server.stop(); > } > } > } > {code} > Executing the statement will throw an AvaticaSqlException: > {code:java} > org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error > while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: > RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> > SQLException: invalid column ordinal: 2 > {code} > The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ > class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. > I am actually testing a potential fix but first I wanted to make sure, that > there is nothing wrong with the way I'm using the two components. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3163) Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specification
Ralph Gasser created CALCITE-3163: - Summary: Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specification Key: CALCITE-3163 URL: https://issues.apache.org/jira/browse/CALCITE-3163 Project: Calcite Issue Type: Bug Components: avatica Affects Versions: avatica-1.15.0 Reporter: Ralph Gasser This is an issue I found while working on CALCITE-3162: According to my understanding, the method _AbstractCursor#convertValue()_, used for converting the component types of ARRAYs, does not adhere to what is specified in the [JDBC specification|[https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea]] (Appendix B) regarding mapping of types: * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a _float_. * The implementation is currently missing support for JDBC type REAL (7), which should convert to _float_ or _Float_ respectively This issue will cause an IllegalStateException to be thrown in clients using the Avatica Driver, if they encounter REAL datatype in arrays. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3162) Reading Arrays from Calcite using Avatica generates error
[ https://issues.apache.org/jira/browse/CALCITE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3162: -- Description: I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a custom data store and in addition, I'm using _Apache Avatica_ to expose the entire functionality via JDBC for an arbitrary, potential remote client to use. We're working a lot with Array types, since we're using our backend to store high-dimensional vectors. However, it seems that currently, Apache Avatica has troubles handling such arrays. Take the following test-case that reproduces the problem pretty well. {code:java} @Test public void test() throws Exception { HttpServer server = null; try { Class.forName("org.apache.calcite.jdbc.Driver"); server = new HttpServer.Builder<>() .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", newProperties())),Driver.Serialization.PROTOBUF) .withPort(1234) .build(); server.start(); final Connection connection = DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); final Statement stmt = connection.createStatement(); final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, 3.0, 2.0]"); resultSet.next(); resultSet.getArray(1); } catch (Exception e) { System.out.println(e.getMessage()); } finally { if (server != null) { server.stop(); } } } {code} Executing the statement will throw an AvaticaSqlException: {code:java} org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> SQLException: invalid column ordinal: 2 {code} The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. I am actually testing a potential fix but first I wanted to make sure, that there is nothing wrong with the way I'm using the two components. was: I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a custom data store and in addition, I'm using _Apache Avatica_ to expose the entire functionality via JDBC for an arbitrary, potential remote client to use. We're working a lot with Array types, since we're using our backend to store high-dimensional vectors. However, it seems that currently, Apache Avatica has troubles handling such arrays. Take the following test-case that reproduces the problem pretty well. {code:java} @Test public void test() throws Exception { HttpServer server = null; try { Class.forName("org.apache.calcite.jdbc.Driver"); server = new HttpServer.Builder<>() .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", newProperties())),Driver.Serialization.PROTOBUF) .withPort(1234) .build(); server.start(); final Connection connection = DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); final Statement stmt = connection.createStatement(); final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, 3.0, 2.0]"); resultSet.getArray(1); } catch (Exception e) { System.out.println(e.getMessage()); } finally { if (server != null) { server.stop(); } } } {code} Executing the statement will throw an AvaticaSqlException: {code:java} org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> SQLException: invalid column ordinal: 2 {code} The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. I am actually testing a potential fix but first I wanted to make sure, that there is nothing wrong with the way I'm using the two components. > Reading Arrays from Calcite using Avatica generates error > - > > Key: CALCITE-3162 > URL: https://issues.apache.org/jira/browse/CALCITE-3162 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 > Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment > Zulu11.29+3-CA (build 11.0.2+7-LTS) > Issue occurs with both Apache Calcite 1.19.0 & 1.20.0. >Reporter: Ralph Gasser >Priority: Major > > I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a > custom data store and in addition, I'm using _Apache Avatica_ to expose the > entire functionality via JDBC for an arbitrary,
[jira] [Closed] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Quesada Lopez closed CALCITE-3161. Resolution: Duplicate Closing as duplicated of CALCITE-3133 > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3133) Remove completely SemiJoinType
[ https://issues.apache.org/jira/browse/CALCITE-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3133: Labels: pull-request-available (was: ) > Remove completely SemiJoinType > -- > > Key: CALCITE-3133 > URL: https://issues.apache.org/jira/browse/CALCITE-3133 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > > SemiJoinType was deprecated in 1.20 via CALCITE-2969. > It shall be kept as deprecated for one minor release (1.20); and then remove > completely for 1.21. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876067#comment-16876067 ] Ruben Quesada Lopez commented on CALCITE-3161: -- Ok, [~danny0405], I'll start working on CALCITE-3133 and flag this one as duplicated. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3162) Reading Arrays from Calcite using Avatica generates error
[ https://issues.apache.org/jira/browse/CALCITE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Gasser updated CALCITE-3162: -- Description: I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a custom data store and in addition, I'm using _Apache Avatica_ to expose the entire functionality via JDBC for an arbitrary, potential remote client to use. We're working a lot with Array types, since we're using our backend to store high-dimensional vectors. However, it seems that currently, Apache Avatica has troubles handling such arrays. Take the following test-case that reproduces the problem pretty well. {code:java} @Test public void test() throws Exception { HttpServer server = null; try { Class.forName("org.apache.calcite.jdbc.Driver"); server = new HttpServer.Builder<>() .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", newProperties())),Driver.Serialization.PROTOBUF) .withPort(1234) .build(); server.start(); final Connection connection = DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); final Statement stmt = connection.createStatement(); final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, 3.0, 2.0]"); resultSet.getArray(1); } catch (Exception e) { System.out.println(e.getMessage()); } finally { if (server != null) { server.stop(); } } } {code} Executing the statement will throw an AvaticaSqlException: {code:java} org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> SQLException: invalid column ordinal: 2 {code} The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. I am actually testing a potential fix but first I wanted to make sure, that there is nothing wrong with the way I'm using the two components. was: I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a custom data store and in addition, I'm using _Apache Avatica_ to expose the entire functionality via JDBC for an arbitrary, potential remote client to use. We're working a lot with Array types, since we're using our backend to store high-dimensional vectors. However, it seems that currently, Apache Avatica has troubles handling such arrays. Take the following test-case that reproduces the problem pretty well. {code:java} @Test public void test() throws Exception { HttpServer server = null; try { Class.forName("org.apache.calcite.jdbc.Driver"); server = new HttpServer.Builder<>() .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", newProperties())),Driver.Serialization.PROTOBUF) .withPort(1234) .build(); server.start(); final Connection connection = DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); final Statement stmt = connection.createStatement(); final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, 3.0, 2.0]"); resultSet.getArray(1); } catch (Exception e) { System.out.println(e.getMessage()); } finally { if (server != null) { server.stop(); } } } {code} Executing the statement will throw an AvaticaSqlException: {code:java} org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> SQLException: invalid column ordinal: 2 {code} The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. I am actually testing a potential fix but first I wanted to make sure, that there is nothing wrong with the way I'm using the two components. > Reading Arrays from Calcite using Avatica generates error > - > > Key: CALCITE-3162 > URL: https://issues.apache.org/jira/browse/CALCITE-3162 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 > Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment > Zulu11.29+3-CA (build 11.0.2+7-LTS) > Issue occurs with both Apache Calcite 1.19.0 & 1.20.0. >Reporter: Ralph Gasser >Priority: Major > > I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a >
[jira] [Commented] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876064#comment-16876064 ] Danny Chan commented on CALCITE-3161: - [~rubenql] Oops, i'm so sorry that i missed your JIRA issue, go ahead with your work, and maybe you can just mark this issue duplicated. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3162) Reading Arrays from Calcite using Avatica generates error
Ralph Gasser created CALCITE-3162: - Summary: Reading Arrays from Calcite using Avatica generates error Key: CALCITE-3162 URL: https://issues.apache.org/jira/browse/CALCITE-3162 Project: Calcite Issue Type: Bug Components: avatica Affects Versions: avatica-1.15.0 Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment Zulu11.29+3-CA (build 11.0.2+7-LTS) Issue occurs with both Apache Calcite 1.19.0 & 1.20.0. Reporter: Ralph Gasser I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a custom data store and in addition, I'm using _Apache Avatica_ to expose the entire functionality via JDBC for an arbitrary, potential remote client to use. We're working a lot with Array types, since we're using our backend to store high-dimensional vectors. However, it seems that currently, Apache Avatica has troubles handling such arrays. Take the following test-case that reproduces the problem pretty well. {code:java} @Test public void test() throws Exception { HttpServer server = null; try { Class.forName("org.apache.calcite.jdbc.Driver"); server = new HttpServer.Builder<>() .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:", newProperties())),Driver.Serialization.PROTOBUF) .withPort(1234) .build(); server.start(); final Connection connection = DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;); final Statement stmt = connection.createStatement(); final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0, 3.0, 2.0]"); resultSet.getArray(1); } catch (Exception e) { System.out.println(e.getMessage()); } finally { if (server != null) { server.stop(); } } } {code} Executing the statement will throw an AvaticaSqlException: {code:java} org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error: RuntimeException: java.sql.SQLException: invalid column ordinal: 2 -> SQLException: invalid column ordinal: 2 {code} The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_ class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method. I am actually testing a potential fix but first I wanted to make sure, that there is nothing wrong with the way I'm using the two components. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876042#comment-16876042 ] Ruben Quesada Lopez edited comment on CALCITE-3161 at 7/1/19 9:28 AM: -- [~danny0405], sorry, I had already created CALCITE-3133 "Remove completely SemiJoinType" for this purpose, but I have not started yet. If you are going to work on this right now, I can close my ticket as duplicated and we keep the current one; or you can close the current one and work on CALCITE-3133, whatever you prefer. was (Author: rubenql): [~danny0405], sorry, I had already created CALCITE-3133 "Remove completely SemiJoinType" for this purpose, but I have not started yet. If you are going to work on this right now I can close my ticket as duplicated and we keep the current one, or you can close the current one and work on CALCITE-3133, whatever you prefer. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876042#comment-16876042 ] Ruben Quesada Lopez edited comment on CALCITE-3161 at 7/1/19 9:28 AM: -- [~danny0405], sorry, I had already created CALCITE-3133 "Remove completely SemiJoinType" for this purpose, but I have not started yet. If you are going to work on this right now I can close my ticket as duplicated and we keep the current one, or you can close the current one and work on CALCITE-3133, whatever you prefer. was (Author: rubenql): [~danny0405], sorry, I had already created CALCITE-3133 "Remove completely SemiJoinType" for this purpose, but if you are going to work on this right now I can close my ticket as duplicated and we keep the current one. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876042#comment-16876042 ] Ruben Quesada Lopez commented on CALCITE-3161: -- [~danny0405], sorry, I had already created CALCITE-3133 "Remove completely SemiJoinType" for this purpose, but if you are going to work on this right now I can close my ticket as duplicated and we keep the current one. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
Danny Chan created CALCITE-3161: --- Summary: Remove deprecated SemiJoinType usage completely Key: CALCITE-3161 URL: https://issues.apache.org/jira/browse/CALCITE-3161 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.21.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3126) Remove deprecated SemiJoin usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3126. - Resolution: Fixed Fixed in [79a4ec6|https://github.com/apache/calcite/commit/79a4ec63adfa83c6920b9ece5042f65a322ff80f] ! > Remove deprecated SemiJoin usage completely > --- > > Key: CALCITE-3126 > URL: https://issues.apache.org/jira/browse/CALCITE-3126 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As a following of CALCITE-2969, Julian Hyde has keep some deprecated usage of > SemiJoin class while supress the warnings[1], they are expected to be removed > from version 1.21 > [1] > [https://github.com/apache/calcite/commit/ef0c4c6ddb7553e2e858259007d8ca1cb3692101] -- This message was sent by Atlassian JIRA (v7.6.3#76005)