[jira] [Commented] (CALCITE-2298) Correlated SubQuery in Having generates error plan when correlated fields does not exist
[
https://issues.apache.org/jira/browse/CALCITE-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478486#comment-16478486
]
godfrey he commented on CALCITE-2298:
-
[~julianhyde], do you have any more suggestions ? :)
> Correlated SubQuery in Having generates error plan when correlated fields
> does not exist
>
>
> Key: CALCITE-2298
> URL: https://issues.apache.org/jira/browse/CALCITE-2298
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: godfrey he
>Assignee: Julian Hyde
>Priority: Major
>
> {code}
> @Test public void testDecorrelateWithUnresolvedField() throws Exception {
> final String sql = "select ename\n"
>+ "from sales.emp\n"
>+ "group by ename, deptno\n"
>+ "having max(sal) <= \n"
>+ " (select max(sal) from sales.emp_b where emp.job =
> emp_b.job group by ename)";
> checkSubQuery(sql).withLateDecorrelation(true).check();
> }
> {code}
> for now, we will get the following plan:
> {code}
> LogicalProject(ENAME=[$0])
> LogicalFilter(condition=[<=($2, $SCALAR_QUERY({
> LogicalProject(EXPR$0=[$1])
> LogicalAggregate(group=[{0}], EXPR$0=[MAX($1)])
> LogicalProject(ENAME=[$1], SAL=[$5])
> LogicalFilter(condition=[=($cor0.JOB, $2)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP_B]])
> }))])
> LogicalAggregate(group=[{0, 1}], agg#0=[MAX($2)])
> LogicalProject(ENAME=[$1], DEPTNO=[$7], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> However emp.job is not the grouping fields or the aggCall result.
> the expected result is throwing an Exception, like:
> {code}
> org.apache.calcite.runtime.CalciteContextException: From line 5, column 47 to
> line 5, column 50: Column 'JOB' not found in 'LogicalAggregate(group=[{0,
> 1}], agg#0=[MAX($2)])'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478471#comment-16478471 ] Subbarao commented on CALCITE-2313: --- But i am getting all system functions and string functions except user defined functions please give me information as soon as possible with relevent code.How to retrive all function names agter connecting calcite connection by using databasemetadata properties > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subbarao updated CALCITE-2313: -- Attachment: (was: CalciteWithOracle.java) > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478456#comment-16478456 ] Subbarao commented on CALCITE-2313: --- how to get all function names in calcite by using databasemetadata > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > Attachments: CalciteWithOracle.java > > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478449#comment-16478449 ] Subbarao commented on CALCITE-2313: --- Then how to get all function names in oracle database using calcite > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > Attachments: CalciteWithOracle.java > > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2312) Support Partition By in sql select statement
[
https://issues.apache.org/jira/browse/CALCITE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478405#comment-16478405
]
Fei Xu commented on CALCITE-2312:
-
Thanks for the comment.
So, may be I can start from SqlCreateTable, add PARTITION BY strategy, just
like adding PRIMARY KEY.
> Support Partition By in sql select statement
>
>
> Key: CALCITE-2312
> URL: https://issues.apache.org/jira/browse/CALCITE-2312
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Fei Xu
>Assignee: Julian Hyde
>Priority: Major
>
> I noticed that calcite already have an Exchange RelNode represents
> distribution on RelNode's input, but no sql clause support.
>
> We are use calcite building SQL layer for out streaming platform. and in
> streaming computation, data shuffle is a very important function, not only
> for our engine, but also for our users.
> For example, In stream-join-table, the engine will load the table data to a
> cache at runtime, and stream join table is actually stream look up table.
> If the stream data could partition by hash or range, before look up table. It
> will be cache friendly cause particular stream data look up particular table
> data.
>
> So I consider do some extensions on sql clause to support Exchange, e.g.
> {code:java}
> // shuffle data using hash_distribution
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY o.productId
> // shuffle data to a singleton node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY SINGLETON
> // shuffle data to all nodes
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY BROADCAST
> // shuffle data to random node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANDOM
> // shuffle data using round-robin policy
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY ROUND_ROBIN
> // shuffle data using range policy
> // Current I'm not sure about the appropriate clause to represents range
> // shuffle, so it is just an demo.
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANGE o.productId, 0, 4096
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2312) Support Partition By in sql select statement
[
https://issues.apache.org/jira/browse/CALCITE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478083#comment-16478083
]
Julian Hyde commented on CALCITE-2312:
--
Hive added PARTITION BY to their SQL and I think they lived to regret it. It's
not good to surface physical stuff (the "how") in logical SQL (the "what").
So, I wouldn't be keen to add this to Calcite's core SQL. You could do it as an
extension of course.
But better might be to do CREATE STREAM that creates a stream with a
partitioning strategy built in. Or, if it is an intermediate result in the
query, let the optimizer figure out what partitioning strategy to use.
Optimizers figure out partitioning and sorting for hash joins and merge joins
today.
> Support Partition By in sql select statement
>
>
> Key: CALCITE-2312
> URL: https://issues.apache.org/jira/browse/CALCITE-2312
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Fei Xu
>Assignee: Julian Hyde
>Priority: Major
>
> I noticed that calcite already have an Exchange RelNode represents
> distribution on RelNode's input, but no sql clause support.
>
> We are use calcite building SQL layer for out streaming platform. and in
> streaming computation, data shuffle is a very important function, not only
> for our engine, but also for our users.
> For example, In stream-join-table, the engine will load the table data to a
> cache at runtime, and stream join table is actually stream look up table.
> If the stream data could partition by hash or range, before look up table. It
> will be cache friendly cause particular stream data look up particular table
> data.
>
> So I consider do some extensions on sql clause to support Exchange, e.g.
> {code:java}
> // shuffle data using hash_distribution
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY o.productId
> // shuffle data to a singleton node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY SINGLETON
> // shuffle data to all nodes
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY BROADCAST
> // shuffle data to random node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANDOM
> // shuffle data using round-robin policy
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY ROUND_ROBIN
> // shuffle data using range policy
> // Current I'm not sure about the appropriate clause to represents range
> // shuffle, so it is just an demo.
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANGE o.productId, 0, 4096
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478081#comment-16478081 ] Julian Hyde commented on CALCITE-2313: -- This is intended behavior. There are no functions defined in that schema. > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > Attachments: CalciteWithOracle.java > > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2314) Verify RexNode transformations by evaluating before and after expressions against sample values
Julian Hyde created CALCITE-2314:
Summary: Verify RexNode transformations by evaluating before and
after expressions against sample values
Key: CALCITE-2314
URL: https://issues.apache.org/jira/browse/CALCITE-2314
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde
Verify transformations of {{RexNode}} expressions (such as simplification, see
{{class RexSimplify}}) by evaluating before and after expressions against
sample values.
Given an expression, say {{($0 < $1) IS NOT NULL}} we can bind $0 and $1 to
values (0, 1, null) and evaluate the 9 combinations. We can also evaluate the
simplified expression {{$0 IS NOT NULL AND $1 IS NOT NULL}} and see whether it
gives the same value for each of the combinations.
We can add this checking to existing tests with little effort. It will improve
confidence that we handle 3-valued logic correctly.
As part of this task, we will create an interpreter {{class RexInterpreter}}
that evaluates RexNode expressions for common operators without generating
code. It is not very efficient and does not cover very many types, functions
and operators, but should be easy for people to extend to add a few more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477948#comment-16477948 ] ASF GitHub Bot commented on CALCITE-2284: - Github user aaraujo commented on the issue: https://github.com/apache/calcite-avatica/pull/46 Thanks for the review @joshelser. Stand by for a subsequent PQS patch. > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477895#comment-16477895 ] ASF GitHub Bot commented on CALCITE-2284: - Github user asfgit closed the pull request at: https://github.com/apache/calcite-avatica/pull/46 > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved CALCITE-2284. - Resolution: Fixed Fixed in https://git-wip-us.apache.org/repos/asf?p=calcite-avatica.git;a=commit;h=612b80cb2e2dff7913a4d29a252a4c0db28d2095 > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated CALCITE-2284: Affects Version/s: (was: 1.11.0) > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated CALCITE-2284: Fix Version/s: avatica-1.12.0 > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned CALCITE-2284: --- Assignee: Alex Araujo > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Alex Araujo >Assignee: Alex Araujo >Priority: Critical > Fix For: avatica-1.12.0 > > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2284) Allow Jetty Server to be customized before startup
[ https://issues.apache.org/jira/browse/CALCITE-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477449#comment-16477449 ] ASF GitHub Bot commented on CALCITE-2284: - Github user joshelser commented on the issue: https://github.com/apache/calcite-avatica/pull/46 > Java seems to provide backwards compat for the raw API by adding additional methods to class files during compile time. This is consistent with my local testing. Good enough for me. Let me take another quick look, but I think this is ready to be merged. Thanks, @aaraujo!! > Allow Jetty Server to be customized before startup > -- > > Key: CALCITE-2284 > URL: https://issues.apache.org/jira/browse/CALCITE-2284 > Project: Calcite > Issue Type: Improvement > Components: avatica >Affects Versions: 1.11.0 >Reporter: Alex Araujo >Priority: Critical > Attachments: CALCITE-2284.patch > > > Avatica server options currently allow users to enable TLS with file based > key stores. In order to support additional options (in memory keystores, > dynamically loaded keys/certificates, etc.) Avatica should provide a callback > to customize the embedded Jetty server before starting it up. > > This would be similar to how Spring Boot allows developers to customize the > embedded server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2294) Allow customization for AvaticaServerConfiguration for plugging new authentication mechanisms
[
https://issues.apache.org/jira/browse/CALCITE-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477443#comment-16477443
]
Josh Elser commented on CALCITE-2294:
-
{quote}Also, the builder method {{withAuthentication}} makes assumptions around
{{loginServiceRealm}} and other parameters which should not be the case. Should
we refactor it as well?
{quote}
What's the assumption you're concerned with? Happy to entertain changes, just
not sure what the issue is.
{quote}The {{HTTPServer}} class provides a static builder class called as
{{Builder}}, however still exposes the constructor as public. This seems
against the builder pattern and breaks expectations. Can I change the
constructors to private?
{quote}
What expectations is it breaking? I'm not seeing a functional difference
between a method {{newBuilder()}} on HttpServer versus the public no-args
constructor on {{Builder}}.
> Allow customization for AvaticaServerConfiguration for plugging new
> authentication mechanisms
> -
>
> Key: CALCITE-2294
> URL: https://issues.apache.org/jira/browse/CALCITE-2294
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Karan Mehta
>Priority: Major
>
> {{AvaticaServerConfiguration}} is currently only created if authentication
> mechanism such as {{BASIC, DIGEST or SPNEGO}} is provided. We can change it
> to a builder pattern to create this object and provide a way for users to
> plugin their own security configuration.
> An example here can be using it for custom config that supports MTLS.
> Thanks [~alexaraujo] for suggesting this approach.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2313) Showing Empty Result
[ https://issues.apache.org/jira/browse/CALCITE-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subbarao updated CALCITE-2313: -- Priority: Major (was: Minor) > Showing Empty Result > > > Key: CALCITE-2313 > URL: https://issues.apache.org/jira/browse/CALCITE-2313 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.15.0 >Reporter: Subbarao >Assignee: Julian Hyde >Priority: Major > Attachments: CalciteWithOracle.java > > > While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2313) Showing Empty Result
Subbarao created CALCITE-2313: - Summary: Showing Empty Result Key: CALCITE-2313 URL: https://issues.apache.org/jira/browse/CALCITE-2313 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.15.0 Reporter: Subbarao Assignee: Julian Hyde Attachments: CalciteWithOracle.java While i am executing getFunctionNames() in Schema it will be showing Empty [ ] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2312) Support Partition By in sql select statement
Fei Xu created CALCITE-2312:
---
Summary: Support Partition By in sql select statement
Key: CALCITE-2312
URL: https://issues.apache.org/jira/browse/CALCITE-2312
Project: Calcite
Issue Type: New Feature
Components: core
Reporter: Fei Xu
Assignee: Julian Hyde
I noticed that calcite already have an Exchange RelNode represents distribution
on RelNode's input, but no sql clause support.
We are use calcite building SQL layer for out streaming platform. and in
streaming computation, data shuffle is a very important function, not only for
our engine, but also for our users.
For example, In stream-join-table, the engine will load the table data to a
cache at runtime, and stream join table is actually stream look up table.
If the stream data could partition by hash or range, before look up table. It
will be cache friendly cause particular stream data look up particular table
data.
So I consider do some extensions on sql clause to support Exchange, e.g.
{code:java}
// shuffle data using hash_distribution
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY o.productId
// shuffle data to a singleton node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY SINGLETON
// shuffle data to all nodes
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY BROADCAST
// shuffle data to random node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANDOM
// shuffle data using round-robin policy
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY ROUND_ROBIN
// shuffle data using range policy
// Current I'm not sure about the appropriate clause to represents range
// shuffle, so it is just an demo.
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANGE o.productId, 0, 4096
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2312) Support Partition By in sql select statement
[
https://issues.apache.org/jira/browse/CALCITE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fei Xu updated CALCITE-2312:
Description:
I noticed that calcite already have an Exchange RelNode represents distribution
on RelNode's input, but no sql clause support.
We are use calcite building SQL layer for out streaming platform. and in
streaming computation, data shuffle is a very important function, not only for
our engine, but also for our users.
For example, In stream-join-table, the engine will load the table data to a
cache at runtime, and stream join table is actually stream look up table.
If the stream data could partition by hash or range, before look up table. It
will be cache friendly cause particular stream data look up particular table
data.
So I consider do some extensions on sql clause to support Exchange, e.g.
{code:java}
// shuffle data using hash_distribution
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY o.productId
// shuffle data to a singleton node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY SINGLETON
// shuffle data to all nodes
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY BROADCAST
// shuffle data to random node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANDOM
// shuffle data using round-robin policy
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY ROUND_ROBIN
// shuffle data using range policy
// Current I'm not sure about the appropriate clause to represents range
// shuffle, so it is just an demo.
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANGE o.productId, 0, 4096
{code}
was:
I noticed that calcite already have an Exchange RelNode represents distribution
on RelNode's input, but no sql clause support.
We are use calcite building SQL layer for out streaming platform. and in
streaming computation, data shuffle is a very important function, not only for
our engine, but also for our users.
For example, In stream-join-table, the engine will load the table data to a
cache at runtime, and stream join table is actually stream look up table.
If the stream data could partition by hash or range, before look up table. It
will be cache friendly cause particular stream data look up particular table
data.
So I consider do some extensions on sql clause to support Exchange, e.g.
{code:java}
// shuffle data using hash_distribution
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY o.productId
// shuffle data to a singleton node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY SINGLETON
// shuffle data to all nodes
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY BROADCAST
// shuffle data to random node
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANDOM
// shuffle data using round-robin policy
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY ROUND_ROBIN
// shuffle data using range policy
// Current I'm not sure about the appropriate clause to represents range
// shuffle, so it is just an demo.
INSERT INTO output
SELECT
*
FROM orders o
PARTITION BY RANGE o.productId, 0, 4096
{code}
> Support Partition By in sql select statement
>
>
> Key: CALCITE-2312
> URL: https://issues.apache.org/jira/browse/CALCITE-2312
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: Fei Xu
>Assignee: Julian Hyde
>Priority: Major
>
> I noticed that calcite already have an Exchange RelNode represents
> distribution on RelNode's input, but no sql clause support.
>
> We are use calcite building SQL layer for out streaming platform. and in
> streaming computation, data shuffle is a very important function, not only
> for our engine, but also for our users.
> For example, In stream-join-table, the engine will load the table data to a
> cache at runtime, and stream join table is actually stream look up table.
> If the stream data could partition by hash or range, before look up table. It
> will be cache friendly cause particular stream data look up particular table
> data.
>
> So I consider do some extensions on sql clause to support Exchange, e.g.
> {code:java}
> // shuffle data using hash_distribution
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY o.productId
> // shuffle data to a singleton node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY SINGLETON
> // shuffle data to all nodes
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY BROADCAST
> // shuffle data to random node
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY RANDOM
> // shuffle data using round-robin policy
> INSERT INTO output
> SELECT
> *
> FROM orders o
> PARTITION BY ROUND_ROBIN
> // shuffle data using range policy
> // Current I'm not sure about the appropriate
[jira] [Commented] (CALCITE-873) Prevent sort when ORDER BY not necessary due to equality constraints
[
https://issues.apache.org/jira/browse/CALCITE-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476971#comment-16476971
]
Atri Sharma commented on CALCITE-873:
-
[~julianhyde] I am reviving this PR with running the added tests and have also
added the rule to CalcitePrepareImpl. The tests pass successfully. Please see
and let me know. Thanks
> Prevent sort when ORDER BY not necessary due to equality constraints
>
>
> Key: CALCITE-873
> URL: https://issues.apache.org/jira/browse/CALCITE-873
> Project: Calcite
> Issue Type: Improvement
>Reporter: James Taylor
>Assignee: Atri Sharma
>Priority: Major
>
> We're working on an optimization in Phoenix to optimize away an ORDER BY when
> it is known based on equality expressions in the WHERE clause that it is not
> necessary (PHOENIX-2194). It'd be great if Calcite could do that as well.
> Here's a example, given the following schema:
> {code}
> CREATE TABLE T (
> K1 VARCHAR,
> K2 VARCHAR,
> K3 VARCHAR,
> CONSTRAINT pk PRIMARY KEY (K1, K2, K3));
> {code}
> In the following queries, no sort is necessary:
> {code}
> SELECT * FROM T WHERE K1='A' ORDER BY K2,K3;
> SELECT * FROM T WHERE K2='B' ORDER BY K1,K3;
> SELECT * FROM T WHERE K1='A' AND K2='B' ORDER BY K3;
> {code}
> There are also some edge cases where a function may be known to select a
> *prefix* of the column value where it's still ok to not sort:
> {code}
> SELECT * FROM T WHERE K1='A' AND SUBSTR(K2,1,3)='ABC' ORDER BY K2;
> {code}
> But if another column is included in the ORDER BY after the prefixing, a sort
> would still be necessary:
> {code}
> SELECT * FROM T WHERE K1='A' AND SUBSTR(K2,1,3)='ABC' ORDER BY K2,K3;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
