[jira] [Created] (CALCITE-1544) AggregateJoinTransposeRule transform failed
Kurt Young created CALCITE-1544:
---
Summary: AggregateJoinTransposeRule transform failed
Key: CALCITE-1544
URL: https://issues.apache.org/jira/browse/CALCITE-1544
Project: Calcite
Issue Type: Bug
Reporter: Kurt Young
Assignee: Julian Hyde
Here is the sql to reproduce the failure:
{code}
select e.deptno
from sales.emp as e join sales.dept as d on e.deptno = d.deptno
group by e.deptno
{code}
After push the aggregator past join, the RowType seems been changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1543) Decorrelator assertion fail
[
https://issues.apache.org/jira/browse/CALCITE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753545#comment-15753545
]
Kurt Young commented on CALCITE-1543:
-
[~julianhyde] I used this query to reproduce:
{code}
select sum(l_extendedprice)
from lineitem, part
where
p_partkey = l_partkey
and l_quantity > (
select avg(l_quantity)
from lineitem
where l_partkey = p_partkey
)
{code}
There are two Aggregates in the operator tree, one with avg(l_quantity) will be
affected by the correlation, but the outer aggregation sum(l_extendedprice) is
not.
> Decorrelator assertion fail
> ---
>
> Key: CALCITE-1543
> URL: https://issues.apache.org/jira/browse/CALCITE-1543
> Project: Calcite
> Issue Type: Bug
>Reporter: Kurt Young
>Assignee: Julian Hyde
>
> Caused by: java.lang.AssertionError at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:456)
> The code is:
> {code}
> final Frame frame = getInvoke(oldInput, rel);
> if (frame == null) {
> // If input has not been rewritten, do not rewrite this rel.
> return null;
> }
> assert !frame.corVarOutputPos.isEmpty();
> {code}
> Seems like it's no need to assert the corVarOutputPos because in some cases,
> the correlate has been fully rewritten.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1543) Decorrelator assertion fail
[
https://issues.apache.org/jira/browse/CALCITE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753376#comment-15753376
]
Julian Hyde commented on CALCITE-1543:
--
Can you provide a test case for this?
> Decorrelator assertion fail
> ---
>
> Key: CALCITE-1543
> URL: https://issues.apache.org/jira/browse/CALCITE-1543
> Project: Calcite
> Issue Type: Bug
>Reporter: Kurt Young
>Assignee: Julian Hyde
>
> Caused by: java.lang.AssertionError at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:456)
> The code is:
> {code}
> final Frame frame = getInvoke(oldInput, rel);
> if (frame == null) {
> // If input has not been rewritten, do not rewrite this rel.
> return null;
> }
> assert !frame.corVarOutputPos.isEmpty();
> {code}
> Seems like it's no need to assert the corVarOutputPos because in some cases,
> the correlate has been fully rewritten.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1543) Decorrelator assertion fail
Kurt Young created CALCITE-1543:
---
Summary: Decorrelator assertion fail
Key: CALCITE-1543
URL: https://issues.apache.org/jira/browse/CALCITE-1543
Project: Calcite
Issue Type: Bug
Reporter: Kurt Young
Assignee: Julian Hyde
Caused by: java.lang.AssertionError at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:456)
The code is:
{code}
final Frame frame = getInvoke(oldInput, rel);
if (frame == null) {
// If input has not been rewritten, do not rewrite this rel.
return null;
}
assert !frame.corVarOutputPos.isEmpty();
{code}
Seems like it's no need to assert the corVarOutputPos because in some cases,
the correlate has been fully rewritten.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1542) Update RelShuttle interface to work on Filter, Join etc instead of Logical rel nodes
Vineet Garg created CALCITE-1542: Summary: Update RelShuttle interface to work on Filter, Join etc instead of Logical rel nodes Key: CALCITE-1542 URL: https://issues.apache.org/jira/browse/CALCITE-1542 Project: Calcite Issue Type: Task Reporter: Vineet Garg Assignee: Julian Hyde Currently RelShuttle interface works on Logical rel nodes (e.g. LogicalFilter etc). With CALCITE-1541's change on RelDecorrelator RelShuttle will need to be updated as well. Logging in a separate JIRA for it since Julian pointed out that this will be a breaking change. If projects have sub-classed RelShuttle and have overridden methods without using @Override they will break with no warning. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1541) Update RelDecorrelator to work on Filter, Join, Project etc
Vineet Garg created CALCITE-1541: Summary: Update RelDecorrelator to work on Filter, Join, Project etc Key: CALCITE-1541 URL: https://issues.apache.org/jira/browse/CALCITE-1541 Project: Calcite Issue Type: Task Components: core Reporter: Vineet Garg Assignee: Julian Hyde Reldecorrelator’s logic including all rules implemented within it are written to take LogicalJoin, LogicalFilter, LogicalProject etc Logical rel nodes. Since Logical operators are final that makes extending RelDecorrelator very difficult. It makes more sense to have RelDecorrelator operate on Join, Filter etc As Julian pointed out with this change RelDecorrelator will need to use RelNode factories (ideally a RelBuilder) so that it can create RelNodes of the appropriate sub-type -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1499) Exclude VolcanoPlanner's "originalRoot" from default planning process
[
https://issues.apache.org/jira/browse/CALCITE-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752981#comment-15752981
]
Julian Hyde commented on CALCITE-1499:
--
The grand plan is that there are no {{addXxx}} methods on {{RelOptPlanner}}
because everything is immutable. A planner would be created with lists of rules
and trait definitions and it would immediately copy them into immutable lists.
I think we should drive towards that plan as soon as possible, and not worry
too much about fixing the existing behavior.
> Exclude VolcanoPlanner's "originalRoot" from default planning process
> -
>
> Key: CALCITE-1499
> URL: https://issues.apache.org/jira/browse/CALCITE-1499
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.10.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>
> The Calcite compilation framework runs a series of Programs for query
> planning. The default programs consist of some pre-processing HepPrograms,
> e.g., de-correlation, field-trimming, etc., the volcano planning program,
> and some post-processing HepPrograms. In {{Prepare.optimize()}},
> {{planner.setRoot()}} is called before running the programs. As a result, the
> original rel from sql-to-rel conversion becomes the "originalRoot" in the
> VolcanoPlanner, and the new rel from the pre-processing programs becomes the
> new "root". In some cases, we would only want to run volcano planning on the
> new root based on the assumption that the new root is the desired form after
> pre-processing. And if we have an option to turn off the planning of the
> original root, the planning space can be significantly reduced.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1527) Support DML in the JDBC adapter
[
https://issues.apache.org/jira/browse/CALCITE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752963#comment-15752963
]
Julian Hyde commented on CALCITE-1527:
--
[~tzolov], The test is flapping on JDK 1.8 (stable on 1.7, for some reason). I
get intermittent failures like this:
{noformat}
Tests run: 29, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.083 sec <<<
FAILURE! - in org.apache.calcite.test.JdbcAdapterTest
testTableModifyDelete(org.apache.calcite.test.JdbcAdapterTest) Time elapsed:
0.097 sec <<< FAILURE!
java.lang.AssertionError:
Expected: is <1>
but: was <2>
at
org.apache.calcite.test.JdbcAdapterTest.testTableModifyDelete(JdbcAdapterTest.java:763)
{noformat}
Can you take a look, please?
> Support DML in the JDBC adapter
> ---
>
> Key: CALCITE-1527
> URL: https://issues.apache.org/jira/browse/CALCITE-1527
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Reporter: Christian Tzolov
>Assignee: Julian Hyde
> Fix For: 1.11.0
>
>
> Currently the JDBC adapter does not support the DML operations: *INSERT*,
> *DELETE* and *UPDATE*.
> Solution needs to convert the parsed *Modify* and *Values* RelNodes into
> *JdbcTableModify*, *JdbcValues* ... such and then in turn into corresponding
> SqlInsert, SqlUpdate and SqlDelete.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reassigned CALCITE-1414:
Assignee: Julian Feinauer (was: Julian Hyde)
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
> Labels: newbie
> Fix For: 1.11.0
>
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-1414:
-
Fix Version/s: 1.11.0
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
> Labels: newbie
> Fix For: 1.11.0
>
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752954#comment-15752954
]
Julian Hyde commented on CALCITE-1414:
--
[~julian.feinauer], Looks good. A few comments:
* Can you please rename it to {{RAND_INTEGER}}? This makes it consistent with
{{RAND}} (which is in the ODBC spec and is therefore somewhat standard) but
makes it clear that this is intended to produce integers, not doubles, and does
not accept a seed argument. (Sorry to change my mind - I know I originally
suggested naming it {{RANDOM}}.)
* In {{SqlStdOperatorTable}} can you rename {{RANDOM_FUNCTION}} to
{{RAND_INTEGER}} to be consistent with other fields, which are just the
function name.
* Can you add {{RAND_INTEGER\(numeric\)}} to reference.md.
When you've fixed these, just add a commit to PR 339 and add a comment to this
case; no need to create a new PR.
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
> Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-1414:
-
Description:
Add a RANDOM function.
It is not standard SQL, but most DBMSs have one, and it be useful.
It will also allow us to better test non-deterministic expressions.
I propose that:
* {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
* There is no facility to specify a seed at this time.
* The random number generator is randomly seeded at query start time based on
(say) the method used by Java's {{new Random()}}; if you run the query twice,
you will (probably) not get the same results.
* If you call the function twice in the same row, it will (probably) return
different results.
was:
Add a RANDOM function.
It is not standard SQL, but most DBMSs have one, and it be useful.
It will also allow us to better test non-deterministic expressions.
I propose that:
* {{RANDOM(n)}} returns an integer between 0 and n - 1 inclusive.
* There is no facility to specify a seed at this time.
* The random number generator is randomly seeded at query start time based on
(say) the method used by Java's {{new Random()}}; if you run the query twice,
you will (probably) not get the same results.
* If you call the function twice in the same row, it will (probably) return
different results.
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
> Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1539) Enable proxy access to Avatica server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/CALCITE-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752332#comment-15752332 ] Josh Elser commented on CALCITE-1539: - bq. I am happy to help out . But I may take me a while to get the env to provide a patch out No worries! I know this takes a bit of setup to get working. Please do ask questions if you need help setting something up. This would be a good area for improvement too :) > Enable proxy access to Avatica server for third party on behalf of end users > > > Key: CALCITE-1539 > URL: https://issues.apache.org/jira/browse/CALCITE-1539 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jerry He >Assignee: Josh Elser > > We want to enable proxy access to Avatica server from an end user, but the > end user comes in via a third party impersonation. For example, Knox and Hue. > The Knox server user conveys the end user to Avatica. > Similar things have been done for HBase Rest Sever HBASE-9866 and Hive Server > HIVE-5155 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1539) Enable proxy access to Avatica server for third party on behalf of end users
[
https://issues.apache.org/jira/browse/CALCITE-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752313#comment-15752313
]
Jerry He commented on CALCITE-1539:
---
bq. Do you know if Hue works in the same manner?
Yes, AFAIK for HBase case it goes through HBase Thrift server using the doAs.
bq. The trick here in Avatica is that we have to not think in "terms of
Hadoop". That is not in the picture at all.
Yes. Looking at the portion of the code, could we add another API in the
DoAsRemoteUserCallback interface, like
{code}
public T doAsRemoteUser(String remoteUserName, String
remoteEffectiveUserName, String remoteAddress, final Callable action)
{code}
Avatica fills it out and the implementation decides what to do.
I am happy to help out :-). But I may take me a while to get the env to
provide a patch out :-(
> Enable proxy access to Avatica server for third party on behalf of end users
>
>
> Key: CALCITE-1539
> URL: https://issues.apache.org/jira/browse/CALCITE-1539
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Jerry He
>Assignee: Josh Elser
>
> We want to enable proxy access to Avatica server from an end user, but the
> end user comes in via a third party impersonation. For example, Knox and Hue.
> The Knox server user conveys the end user to Avatica.
> Similar things have been done for HBase Rest Sever HBASE-9866 and Hive Server
> HIVE-5155
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1540) window function does not support multiple partition columns
[
https://issues.apache.org/jira/browse/CALCITE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma updated CALCITE-1540:
Attachment: 0001-CALCITE-1540-window-function-does-not-support-multip.patch
patch attached. Since only no-parameter constructor exists for
SyntheticRecordType:
{quote}
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor#implementRoot
// Here a constructor without parameter is used because the generated
// code could cause error if number of fields is too large.
{quote}
I have to manually assign each field value.
Had little experience on modifying Calcite's source code, please help to review.
BTW, where should I add more test case queries? I assume there should be a list
of test queries to run on both Calcite and another DB, so that execution
correctness can be ensured via comparing.
> window function does not support multiple partition columns
> ---
>
> Key: CALCITE-1540
> URL: https://issues.apache.org/jira/browse/CALCITE-1540
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: hongbin ma
>Assignee: Julian Hyde
> Attachments:
> 0001-CALCITE-1540-window-function-does-not-support-multip.patch
>
>
> When I add multiple partition columns to a window function:
> {code:xml}
> @Test public void testSelect() throws SQLException {
> checkSql("model", "select NAME,DEPTNO,count(*) over (partition by
> NAME,DEPTNO) from EMPS");
> }
> {code}
> Following exception is thrown:
> java.sql.SQLException: Error while executing SQL "select NAME,DEPTNO,count(*)
> over (partition by NAME,DEPTNO) from EMPS": Error while compiling generated
> Java code:
> However the document from calcite.apache.org/docs/reference.html states
> multiple partition columns should be allowed:
> {code}
> windowSpec:
> [ windowName ]
> '('
> [ ORDER BY orderItem [, orderItem ]* ]
> [ PARTITION BY expression [, expression ]* ]
> [
> RANGE numericOrIntervalExpression { PRECEDING | FOLLOWING }
> | ROWS numericExpression { PRECEDING | FOLLOWING }
> ]
> ')'
> {code}
> After searching it seems no one is reporting the same issue. I'm opening this
> JIAR as a placeholder, will try to fix this issue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751039#comment-15751039
]
Julian Feinauer edited comment on CALCITE-1414 at 12/15/16 11:03 AM:
-
I added a pull request: https://github.com/apache/calcite/pull/339 with my
suggestion how to implement this function.
I had to resubmit the PR https://github.com/apache/calcite/pull/338 due to a
problem with the CI / Validator.
Sorry for that.
was (Author: julian.feinauer):
I added a pull request: https://github.com/apache/calcite/pull/338 with my
suggestion how to implement this function.
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
> Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1414) Add a RANDOM function
[
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751039#comment-15751039
]
Julian Feinauer commented on CALCITE-1414:
--
I added a pull request: https://github.com/apache/calcite/pull/338 with my
suggestion how to implement this function.
> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
> Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> * {{RANDOM(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on
> (say) the method used by Java's {{new Random()}}; if you run the query twice,
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return
> different results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
