[jira] [Created] (CALCITE-2694) Improve type conversion with respect to charset and collation
Ted Xu created CALCITE-2694: --- Summary: Improve type conversion with respect to charset and collation Key: CALCITE-2694 URL: https://issues.apache.org/jira/browse/CALCITE-2694 Project: Calcite Issue Type: Sub-task Reporter: Ted Xu Assignee: Julian Hyde Type conversion need to improved both in correctness and efficiency w.r.t. charset and collation. The changes include: # RexSimplify # RexDataTypeFactory#leastRestrictive -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2691) Improve character set and collate support
Ted Xu created CALCITE-2691: --- Summary: Improve character set and collate support Key: CALCITE-2691 URL: https://issues.apache.org/jira/browse/CALCITE-2691 Project: Calcite Issue Type: Improvement Reporter: Ted Xu Assignee: Julian Hyde Attachments: Charset Full Support in Calcite.pdf Current Calcite implementation lacks full support on charset and collate. This is an umbrella to track all related changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2693) Allow literal creation with charsets other than LATIN1 and UTF16
Ted Xu created CALCITE-2693: --- Summary: Allow literal creation with charsets other than LATIN1 and UTF16 Key: CALCITE-2693 URL: https://issues.apache.org/jira/browse/CALCITE-2693 Project: Calcite Issue Type: Sub-task Reporter: Ted Xu Assignee: Julian Hyde -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2691) Improve character set and collation support
[ https://issues.apache.org/jira/browse/CALCITE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-2691: Description: Current Calcite implementation lacks full support on charset and collation. This is an umbrella to track all related changes. (was: Current Calcite implementation lacks full support on charset and collate. This is an umbrella to track all related changes.) > Improve character set and collation support > --- > > Key: CALCITE-2691 > URL: https://issues.apache.org/jira/browse/CALCITE-2691 > Project: Calcite > Issue Type: Improvement >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > Attachments: Charset Full Support in Calcite.pdf > > > Current Calcite implementation lacks full support on charset and collation. > This is an umbrella to track all related changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2692) Introduce default charset and collation to type system interface
[ https://issues.apache.org/jira/browse/CALCITE-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-2692: Summary: Introduce default charset and collation to type system interface (was: Introduce default charset and collate to type system interface) > Introduce default charset and collation to type system interface > > > Key: CALCITE-2692 > URL: https://issues.apache.org/jira/browse/CALCITE-2692 > Project: Calcite > Issue Type: Sub-task >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > Introduce getDefaultCharset and getDefaultCollation to RelTypeSystem, from > which RelTypeFactory could create character type with reference to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2692) Introduce default charset and collate to type system interface
Ted Xu created CALCITE-2692: --- Summary: Introduce default charset and collate to type system interface Key: CALCITE-2692 URL: https://issues.apache.org/jira/browse/CALCITE-2692 Project: Calcite Issue Type: Sub-task Reporter: Ted Xu Assignee: Julian Hyde Introduce getDefaultCharset and getDefaultCollation to RelTypeSystem, from which RelTypeFactory could create character type with reference to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2691) Improve character set and collation support
[ https://issues.apache.org/jira/browse/CALCITE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-2691: Summary: Improve character set and collation support (was: Improve character set and collate support) > Improve character set and collation support > --- > > Key: CALCITE-2691 > URL: https://issues.apache.org/jira/browse/CALCITE-2691 > Project: Calcite > Issue Type: Improvement >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > Attachments: Charset Full Support in Calcite.pdf > > > Current Calcite implementation lacks full support on charset and collate. > This is an umbrella to track all related changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing unnecessary charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682246#comment-16682246 ] Ted Xu commented on CALCITE-2619: - Thanks [~julianhyde], I opened another PR [https://github.com/apache/calcite/pull/911]. > Reduce string literal creation cost by removing unnecessary charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2619) Reduce string literal creation cost by removing unnecessary charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-2619: Summary: Reduce string literal creation cost by removing unnecessary charset check (was: Reduce string literal creation cost by removing charset check) > Reduce string literal creation cost by removing unnecessary charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16680757#comment-16680757 ] Ted Xu commented on CALCITE-2619: - [~julianhyde] sorry for the late reply, I was already working on this issue. However, the change is a bit larger than what I expected. I'd like to raise some more issue before I submit the patch: 1. I think the original verification of charset can only tell a Unicode string is LATIN1 encoded or not, since 'value' of NlsString is a Java String. I would change the 'value' type from String to byte[]. 2. The payload of NlsString is a byte[] but we still need to cache an encoded String to reduce encoding cost. I would also like to have a method 'getValueBytes() : byte[]' if someone need to skip encoding entirely. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2646) Support nullable struct type
[
https://issues.apache.org/jira/browse/CALCITE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Xu resolved CALCITE-2646.
-
Resolution: Duplicate
> Support nullable struct type
>
>
> Key: CALCITE-2646
> URL: https://issues.apache.org/jira/browse/CALCITE-2646
> Project: Calcite
> Issue Type: Bug
>Reporter: Ted Xu
>Assignee: Julian Hyde
>Priority: Major
>
> The struct type (or RelRecordType) is hard coded NOT nullable, which lead
> exception thrown when invoking:
> {{RexBuilder.makeLiteral(null, structType);}}
> The exception looks like following:
> java.lang.IllegalArgumentException
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
> at org.apache.calcite.rex.RexLiteral.(RexLiteral.java:217)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2646) Support nullable struct type
[
https://issues.apache.org/jira/browse/CALCITE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Xu updated CALCITE-2646:
Issue Type: Bug (was: Improvement)
> Support nullable struct type
>
>
> Key: CALCITE-2646
> URL: https://issues.apache.org/jira/browse/CALCITE-2646
> Project: Calcite
> Issue Type: Bug
>Reporter: Ted Xu
>Assignee: Julian Hyde
>Priority: Major
>
> The struct type (or RelRecordType) is hard coded NOT nullable, which lead
> exception thrown when invoking:
> {{RexBuilder.makeLiteral(null, structType);}}
> The exception looks like following:
> java.lang.IllegalArgumentException
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
> at org.apache.calcite.rex.RexLiteral.(RexLiteral.java:217)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2646) Support nullable struct type
Ted Xu created CALCITE-2646:
---
Summary: Support nullable struct type
Key: CALCITE-2646
URL: https://issues.apache.org/jira/browse/CALCITE-2646
Project: Calcite
Issue Type: Improvement
Reporter: Ted Xu
Assignee: Julian Hyde
The struct type (or RelRecordType) is hard coded NOT nullable, which lead
exception thrown when invoking:
{{RexBuilder.makeLiteral(null, structType);}}
The exception looks like following:
java.lang.IllegalArgumentException
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
at org.apache.calcite.rex.RexLiteral.(RexLiteral.java:217)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659943#comment-16659943 ] Ted Xu commented on CALCITE-2619: - Thanks [~lemire] for the test and blog post. We should move forward by adopting Guava. Daniel, are you going to give a patch in Calcite? Or else I'd like to contribute. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649663#comment-16649663 ] Ted Xu commented on CALCITE-2619: - Great! Thanks [~julianhyde] and [~lemire]! [~lemire] I'm assigning this Jira to you, please let me know if there is anything I can help. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Ted Xu >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu reassigned CALCITE-2619: --- Assignee: Julian Hyde (was: Ted Xu) > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647315#comment-16647315 ] Ted Xu edited comment on CALCITE-2619 at 10/12/18 3:03 AM: --- As for the cost distribution, I did a quick test (string concat of random 10 characters): ||Name||CPU time||Invocations|| |org.apache.calcite.sql.SqlUtil.translateCharacterSetName(String)|10.7s (0.1%)|16,089| |java.nio.charset.CharsetEncoder.encode(java.nio.CharBuffer, java.nio.ByteBuffer, boolean)|1.374s (7.1%)|16,089 | Charset.forName has its own cache so the cost can be ignored. As for the improvements mentioned above: # Caching values been checked: we've considered the exact way, but looking up a string value from cache is still very expensive, not to mention the memory overhead of the cache. # Skip common charset verification: [~julianhyde] can you elaborate more about this one? However, in CJK (China, Japan, Korea) countries UTF-8 is commonly adopted. We use UTF-8 as our default charset. # Skip copying verification: copy of NlsString changes the value, skip verification is still unsafe. was (Author: tedxu): As for the cost distribution, I did a quick test: ||Name||CPU time||Invocations|| |org.apache.calcite.sql.SqlUtil.translateCharacterSetName(String)|10.7s (0.1%)|16,089| |java.nio.charset.CharsetEncoder.encode(java.nio.CharBuffer, java.nio.ByteBuffer, boolean)|1.374s (7.1%)|16,089 | Charset.forName has its own cache so the cost can be ignored. As for the improvements mentioned above: # Caching values been checked: we've considered the exact way, but looking up a string value from cache is still very expensive, not to mention the memory overhead of the cache. # Skip common charset verification: [~julianhyde] can you elaborate more about this one? However, in CJK (China, Japan, Korea) countries UTF-8 is commonly adopted. We use UTF-8 as our default charset. # Skip copying verification: copy of NlsString changes the value, skip verification is still unsafe. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Ted Xu >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647315#comment-16647315 ] Ted Xu commented on CALCITE-2619: - As for the cost distribution, I did a quick test: ||Name||CPU time||Invocations|| |org.apache.calcite.sql.SqlUtil.translateCharacterSetName(String)|10.7s (0.1%)|16,089| |java.nio.charset.CharsetEncoder.encode(java.nio.CharBuffer, java.nio.ByteBuffer, boolean)|1.374s (7.1%)|16,089 | Charset.forName has its own cache so the cost can be ignored. As for the improvements mentioned above: # Caching values been checked: we've considered the exact way, but looking up a string value from cache is still very expensive, not to mention the memory overhead of the cache. # Skip common charset verification: [~julianhyde] can you elaborate more about this one? However, in CJK (China, Japan, Korea) countries UTF-8 is commonly adopted. We use UTF-8 as our default charset. # Skip copying verification: copy of NlsString changes the value, skip verification is still unsafe. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Ted Xu >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu reassigned CALCITE-2619: --- Assignee: Ted Xu (was: Julian Hyde) > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Ted Xu >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2619) Reduce string literal creation cost by removing charset check
[ https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646165#comment-16646165 ] Ted Xu commented on CALCITE-2619: - Thanks [~julianhyde], I created a pull request at [https://github.com/apache/calcite/pull/883], supporting optional unsafe creation of NlsString. > Reduce string literal creation cost by removing charset check > - > > Key: CALCITE-2619 > URL: https://issues.apache.org/jira/browse/CALCITE-2619 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Major > > The cost of creating NlsString is very high, due to its charset check. In > some cases, e.g., expression evaluate because of Partition Prune, the > NlsString creation costs 40%+ of total executor's overhead. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2619) Reduce string literal creation cost by removing charset check
Ted Xu created CALCITE-2619: --- Summary: Reduce string literal creation cost by removing charset check Key: CALCITE-2619 URL: https://issues.apache.org/jira/browse/CALCITE-2619 Project: Calcite Issue Type: Improvement Components: core Reporter: Ted Xu Assignee: Julian Hyde The cost of creating NlsString is very high, due to its charset check. In some cases, e.g., expression evaluate because of Partition Prune, the NlsString creation costs 40%+ of total executor's overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2616) Can't create Unicode literal by RelBuilder
[
https://issues.apache.org/jira/browse/CALCITE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644966#comment-16644966
]
Ted Xu commented on CALCITE-2616:
-
We have encountered the same issue here. The root cause should be some hard
coded charset in SqlUtil#translateCharacterSetName, which is invoked by
NlsString.
In fact, from our profiling result, the cost of checking charset correctness in
NlsString is very high. Considering it is not a critical path, we should remove
that piece of code. [~julianhyde]
> Can't create Unicode literal by RelBuilder
> --
>
> Key: CALCITE-2616
> URL: https://issues.apache.org/jira/browse/CALCITE-2616
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Anton Haidai
>Assignee: Julian Hyde
>Priority: Major
>
> Test in RelBuilderTest to reproduce the issue:
> {code:java}
> @Test public void testScanWithFilterByUnicodeValue() {
> final RelBuilder builder = RelBuilder.create(config().build());
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.call(SqlStdOperatorTable.EQUALS,
> builder.field("ENAME"),
> builder.literal("Петро ピーター")
> )
> )
> .build();
> }
> {code}
> Result:
> org.apache.calcite.runtime.CalciteException: Failed to encode 'Петро ピーター' in
> character set 'ISO-8859-1'
> Possible workaround: create saffron.properties with the following property
> saffron.default.charset=UTF-16LE
> But UTF-8 will not work as a value of this property, see
> SqlUtil.translateCharacterSetName
> Related code:
> * SqlUtil.translateCharacterSetName(charsetName)
> * RelDataTypeFactoryImpl.getDefaultCharset()
> * SaffronProperties
> Could it be considered to switch defaults from "ISO-8859-1" to "UTF-8" in
> SaffronProperties?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1934) Make VolcanoPlanner reentrant
[ https://issues.apache.org/jira/browse/CALCITE-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124512#comment-16124512 ] Ted Xu commented on CALCITE-1934: - Thanks [~julianhyde]! CALCITE-1536 should fix my issue, it is a better way to me. What do you plan on CALCITE-1536? It looks like a major refactor and brings incompatibility. > Make VolcanoPlanner reentrant > - > > Key: CALCITE-1934 > URL: https://issues.apache.org/jira/browse/CALCITE-1934 > Project: Calcite > Issue Type: New Feature >Reporter: Ted Xu >Assignee: Julian Hyde > > VolcanoPlanner should be reentrant so as to do an additional round of > optimize (possibly with different set of rules) without recompile RelNodes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1933) Cost estimation for expression evaluate
[
https://issues.apache.org/jira/browse/CALCITE-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124507#comment-16124507
]
Ted Xu commented on CALCITE-1933:
-
Thanks [~julianhyde] for the reply.
You're right, expression cost can be provided as metadata. In fact, I was
proposing to use Schema because we treat expression cost as SqlOperator
attribute _in our own implementation_. This makes users' hint (settings in
metadata service or annotations on UDF code) injection more convenient. Here we
say expression cost we mean _expression complexity_, the underlying concept
includes:
1. Certain cost of a call to given SqlOperator is a constant
2. The cost is irrelevant to the input data
On second thought, I found above assumptions was too simple. The cost or
complexity of expression may depend on:
1. Data types (e.g., cost of cast from string to double and integer to double
is different)
2. Input data value, distribution, ordering, etc. (e.g., like '%foobar%' and
like '%foo%bar%' is different)
3. Execution engine implementation details, like caching\[1\], code generation
and vectorized execution.
If we treat expression cost as metadata, the interface could be simplified as
follows:
{code:java}
interface Handler extends MetadataHandler {
Double getExpressionComplexity(RelNode r, RelMetadataQuery mq,
RexNode expression, RexNode predicate);
}
{code}
It looks not necessarily like a 'core' functionality and makes no much
difference to NonCummulativeCost itself. But still it would help estimating
filter cost.
I'm not sure if I explained my idea clearly. What do you think? Shall we
introduce such a metadata?
\[1\] Stonebraker, M., & Hellerstein, J. M. (n.d.). Optimizing Predicate
Migration : Queries with Expensive Predicates, 267–276.
> Cost estimation for expression evaluate
> ---
>
> Key: CALCITE-1933
> URL: https://issues.apache.org/jira/browse/CALCITE-1933
> Project: Calcite
> Issue Type: New Feature
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> Expression evaluate performance is different one to another based on its
> operator type and underlying execution engine. For example, a regex pattern
> matching (e.g., col like 'foo%bar%') may requires 100 times more cost
> compares to a simple comparison (e.g., col > 3). Currently we don't have this
> kind of differentiation.
> Expression cost will grant following applications:
> 1. Improved predicate push down: predicates can be pushed down to the right
> position based on row count and its own cost.
> 2. Predicate ordering rule: predicate with different order, e.g.,
> {code:sql}
> col1 like 'foo%bar%' and col2 > 3
> col2 > 3 and col1 like 'foo%bar%'
> {code}
>
> may have different performance, introducing a predicate ordering rule will
> help find a better one.
> The cost of each RexCall / RexAggCall is provided by Schema, stored in an
> external metadata service.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-1934) Make VolcanoPlanner reentrant
Ted Xu created CALCITE-1934: --- Summary: Make VolcanoPlanner reentrant Key: CALCITE-1934 URL: https://issues.apache.org/jira/browse/CALCITE-1934 Project: Calcite Issue Type: New Feature Reporter: Ted Xu Assignee: Julian Hyde VolcanoPlanner should be reentrant so as to do an additional round of optimize (possibly with different set of rules) without recompile RelNodes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1933) Cost estimation for expression evaluate
Ted Xu created CALCITE-1933:
---
Summary: Cost estimation for expression evaluate
Key: CALCITE-1933
URL: https://issues.apache.org/jira/browse/CALCITE-1933
Project: Calcite
Issue Type: New Feature
Reporter: Ted Xu
Assignee: Julian Hyde
Expression evaluate performance is different one to another based on its
operator type and underlying execution engine. For example, a regex pattern
matching (e.g., col like 'foo%bar%') may requires 100 times more cost compares
to a simple comparison (e.g., col > 3). Currently we don't have this kind of
differentiation.
Expression cost will grant following applications:
1. Improved predicate push down: predicates can be pushed down to the right
position based on row count and its own cost.
2. Predicate ordering rule: predicate with different order, e.g.,
{code:sql}
col1 like 'foo%bar%' and col2 > 3
col2 > 3 and col1 like 'foo%bar%'
{code}
may have different performance, introducing a predicate ordering rule will help
find a better one.
The cost of each RexCall / RexAggCall is provided by Schema, stored in an
external metadata service.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1925) JaninoRelMetadataProvider to cache null value
[ https://issues.apache.org/jira/browse/CALCITE-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119264#comment-16119264 ] Ted Xu commented on CALCITE-1925: - Thanks [~julianhyde]! > JaninoRelMetadataProvider to cache null value > - > > Key: CALCITE-1925 > URL: https://issues.apache.org/jira/browse/CALCITE-1925 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.13.0 >Reporter: Ted Xu >Assignee: Julian Hyde > Fix For: 1.14.0 > > > JaninoRelMetadataProvider should cache null value to improve performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1925) JaninoRelMetadataProvider to cache null value
[ https://issues.apache.org/jira/browse/CALCITE-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116370#comment-16116370 ] Ted Xu commented on CALCITE-1925: - It's a bit hard to create a unit test. [~julianhyde] could you please have a look at my pull request? > JaninoRelMetadataProvider to cache null value > - > > Key: CALCITE-1925 > URL: https://issues.apache.org/jira/browse/CALCITE-1925 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.13.0 >Reporter: Ted Xu >Assignee: Julian Hyde > > JaninoRelMetadataProvider should cache null value to improve performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1925) JaninoRelMetadataProvider to cache null value
[ https://issues.apache.org/jira/browse/CALCITE-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114118#comment-16114118 ] Ted Xu commented on CALCITE-1925: - Here https://github.com/apache/calcite/pull/507 is a rough showcase. I'll try if I can create a test. > JaninoRelMetadataProvider to cache null value > - > > Key: CALCITE-1925 > URL: https://issues.apache.org/jira/browse/CALCITE-1925 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.13.0 >Reporter: Ted Xu >Assignee: Julian Hyde > > JaninoRelMetadataProvider should cache null value to improve performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1925) JaninoRelMetadataProvider to cache null value
Ted Xu created CALCITE-1925: --- Summary: JaninoRelMetadataProvider to cache null value Key: CALCITE-1925 URL: https://issues.apache.org/jira/browse/CALCITE-1925 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.13.0 Reporter: Ted Xu Assignee: Julian Hyde JaninoRelMetadataProvider should cache null value to improve performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1920) Add IS_NOT_NULL predicate on not-nullable equi-join keys
[
https://issues.apache.org/jira/browse/CALCITE-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112147#comment-16112147
]
Ted Xu commented on CALCITE-1920:
-
Looks like CALCITE-1526 already fixed this one.
> Add IS_NOT_NULL predicate on not-nullable equi-join keys
>
>
> Key: CALCITE-1920
> URL: https://issues.apache.org/jira/browse/CALCITE-1920
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.13.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> In equi-joins, if the join keys are not-nullable according to join type, we
> can assign a IS_NOT_NULL predicate to reduce input. For example, query:
> {code:sql}
> select * from src1 join src2 on src1.id = src2.id
> {code}
> can be rewrite to
> {code:sql}
> select * from
> (select * from src1 where src1.id is not null) src1
> join
> (select * from src2 where src2.id is not null) src2
> on src1.id = src2.id
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (CALCITE-1920) Add IS_NOT_NULL predicate on not-nullable equi-join keys
[
https://issues.apache.org/jira/browse/CALCITE-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Xu closed CALCITE-1920.
---
Resolution: Duplicate
> Add IS_NOT_NULL predicate on not-nullable equi-join keys
>
>
> Key: CALCITE-1920
> URL: https://issues.apache.org/jira/browse/CALCITE-1920
> Project: Calcite
> Issue Type: Improvement
>Affects Versions: 1.13.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> In equi-joins, if the join keys are not-nullable according to join type, we
> can assign a IS_NOT_NULL predicate to reduce input. For example, query:
> {code:sql}
> select * from src1 join src2 on src1.id = src2.id
> {code}
> can be rewrite to
> {code:sql}
> select * from
> (select * from src1 where src1.id is not null) src1
> join
> (select * from src2 where src2.id is not null) src2
> on src1.id = src2.id
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-1920) Add IS_NOT_NULL predicate on not-nullable equi-join keys
Ted Xu created CALCITE-1920:
---
Summary: Add IS_NOT_NULL predicate on not-nullable equi-join keys
Key: CALCITE-1920
URL: https://issues.apache.org/jira/browse/CALCITE-1920
Project: Calcite
Issue Type: Improvement
Affects Versions: 1.13.0
Reporter: Ted Xu
Assignee: Julian Hyde
In equi-joins, if the join keys are not-nullable according to join type, we can
assign a IS_NOT_NULL predicate to reduce input. For example, query:
{code:sql}
select * from src1 join src2 on src1.id = src2.id
{code}
can be rewrite to
{code:sql}
select * from
(select * from src1 where src1.id is not null) src1
join
(select * from src2 where src2.id is not null) src2
on src1.id = src2.id
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1883) HepPlanner should force garbage collect whenever a root registered
[ https://issues.apache.org/jira/browse/CALCITE-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090938#comment-16090938 ] Ted Xu commented on CALCITE-1883: - Thanks [~julianhyde], the pull request is updated. That is actually me, sorry for my misconfiguring of git properties. > HepPlanner should force garbage collect whenever a root registered > -- > > Key: CALCITE-1883 > URL: https://issues.apache.org/jira/browse/CALCITE-1883 > Project: Calcite > Issue Type: Bug >Reporter: Ted Xu >Assignee: Julian Hyde > > Currently HepPlanner#collectGarbage() will skip if there is no new transform > since last GC. If HepPlanner is reused, it is common case that no transform > is applied after last cheapest plan is built while before next round of > optimize is kicked off. > This may break HepPlanner because the garbage is probably not a graph after > HepPlanner#buildFinalPlan . > I'll try if I can create a reproduce case. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1883) HepPlanner should force garbage collect whenever a root registered
[ https://issues.apache.org/jira/browse/CALCITE-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088379#comment-16088379 ] Ted Xu commented on CALCITE-1883: - [~julianhyde] I've issued a pull request here: https://github.com/apache/calcite/pull/497 I'm not sure if the fix (leveraging the idea of number of transformations) is good enough, but you can reproduce the bug from that test case. > HepPlanner should force garbage collect whenever a root registered > -- > > Key: CALCITE-1883 > URL: https://issues.apache.org/jira/browse/CALCITE-1883 > Project: Calcite > Issue Type: Bug >Reporter: Ted Xu >Assignee: Julian Hyde > > Currently HepPlanner#collectGarbage() will skip if there is no new transform > since last GC. If HepPlanner is reused, it is common case that no transform > is applied after last cheapest plan is built while before next round of > optimize is kicked off. > This may break HepPlanner because the garbage is probably not a graph after > HepPlanner#buildFinalPlan . > I'll try if I can create a reproduce case. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1883) HepPlanner should force garbage collect whenever a root registered
Ted Xu created CALCITE-1883: --- Summary: HepPlanner should force garbage collect whenever a root registered Key: CALCITE-1883 URL: https://issues.apache.org/jira/browse/CALCITE-1883 Project: Calcite Issue Type: Bug Reporter: Ted Xu Assignee: Julian Hyde Currently HepPlanner#collectGarbage() will skip if there is no new transform since last GC. If HepPlanner is reused, it is common case that no transform is applied after last cheapest plan is built while before next round of optimize is kicked off. This may break HepPlanner because the garbage is probably not a graph after HepPlanner#buildFinalPlan . I'll try if I can create a reproduce case. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1846) Metadata pulled up predicates should skip non-deterministic calls
[ https://issues.apache.org/jira/browse/CALCITE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055035#comment-16055035 ] Ted Xu commented on CALCITE-1846: - Thanks [~jcamachorodriguez]! > Metadata pulled up predicates should skip non-deterministic calls > - > > Key: CALCITE-1846 > URL: https://issues.apache.org/jira/browse/CALCITE-1846 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ted Xu >Assignee: Ted Xu > Fix For: 1.13.0 > > > Metadata MdPredicates should ignore undeterministic calls, or else there > would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1846) Metadata pulled up predicates should skip non-deterministic calls
[ https://issues.apache.org/jira/browse/CALCITE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053460#comment-16053460 ] Ted Xu commented on CALCITE-1846: - [~jcamachorodriguez] could you please have a look? The failing CI build seems not related. > Metadata pulled up predicates should skip non-deterministic calls > - > > Key: CALCITE-1846 > URL: https://issues.apache.org/jira/browse/CALCITE-1846 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > > Metadata MdPredicates should ignore undeterministic calls, or else there > would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CALCITE-1846) Metadata pulled up predicates should skip non-deterministic calls
[ https://issues.apache.org/jira/browse/CALCITE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-1846: Summary: Metadata pulled up predicates should skip non-deterministic calls (was: Metadata pulled up predicates should skip undeterministic calls) > Metadata pulled up predicates should skip non-deterministic calls > - > > Key: CALCITE-1846 > URL: https://issues.apache.org/jira/browse/CALCITE-1846 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > > Metadata MdPredicates should ignore undeterministic calls, or else there > would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1846) Metadata pulled up predicates should skip undeterministic calls
[ https://issues.apache.org/jira/browse/CALCITE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053058#comment-16053058 ] Ted Xu commented on CALCITE-1846: - Thanks [~julianhyde]. Pull request updated https://github.com/apache/calcite/pull/475 . > Metadata pulled up predicates should skip undeterministic calls > --- > > Key: CALCITE-1846 > URL: https://issues.apache.org/jira/browse/CALCITE-1846 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > > Metadata MdPredicates should ignore undeterministic calls, or else there > would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1846) Metadata pulled up predicates should skip undeterministic calls
[ https://issues.apache.org/jira/browse/CALCITE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052735#comment-16052735 ] Ted Xu commented on CALCITE-1846: - Hi guys, I've issued a pull request https://github.com/apache/calcite/pull/475 featuring this bug, someone please have a look, thanks! > Metadata pulled up predicates should skip undeterministic calls > --- > > Key: CALCITE-1846 > URL: https://issues.apache.org/jira/browse/CALCITE-1846 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > > Metadata MdPredicates should ignore undeterministic calls, or else there > would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1846) Metadata pulled up predicates should skip undeterministic calls
Ted Xu created CALCITE-1846: --- Summary: Metadata pulled up predicates should skip undeterministic calls Key: CALCITE-1846 URL: https://issues.apache.org/jira/browse/CALCITE-1846 Project: Calcite Issue Type: Bug Components: core Reporter: Ted Xu Assignee: Julian Hyde Metadata MdPredicates should ignore undeterministic calls, or else there would be unexpected result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1337) Lazy evaluate RexCall digests
[ https://issues.apache.org/jira/browse/CALCITE-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403437#comment-15403437 ] Ted Xu commented on CALCITE-1337: - Your tests added accordingly. > Lazy evaluate RexCall digests > - > > Key: CALCITE-1337 > URL: https://issues.apache.org/jira/browse/CALCITE-1337 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > Fix For: 1.8.0 > > Attachments: p.patch > > > Currently RexCall compute digests eagerly in its constructor, also, it > compute digest every time when toString invoked. It may cause performance > issue when the RexCall tree is very large. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1337) Lazy evaluate RexCall digests
[ https://issues.apache.org/jira/browse/CALCITE-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403302#comment-15403302 ] Ted Xu commented on CALCITE-1337: - Thanks [~julianhyde], rope looks a good solution for your case! I will have a look into that. I'm not addressing the duplicated digest problem right now. We may fix the duplicated digest computation first. > Lazy evaluate RexCall digests > - > > Key: CALCITE-1337 > URL: https://issues.apache.org/jira/browse/CALCITE-1337 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > Fix For: 1.8.0 > > > Currently RexCall compute digests eagerly in its constructor, also, it > compute digest every time when toString invoked. It may cause performance > issue when the RexCall tree is very large. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1337) Lazy evaluate RexCall digests
[ https://issues.apache.org/jira/browse/CALCITE-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403293#comment-15403293 ] Ted Xu commented on CALCITE-1337: - Hi [~julianhyde] please have a look on my patch https://github.com/apache/calcite/pull/264, thanks! > Lazy evaluate RexCall digests > - > > Key: CALCITE-1337 > URL: https://issues.apache.org/jira/browse/CALCITE-1337 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ted Xu >Assignee: Julian Hyde > Fix For: 1.8.0 > > > Currently RexCall compute digests eagerly in its constructor, also, it > compute digest every time when toString invoked. It may cause performance > issue when the RexCall tree is very large. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1337) Lazy evaluate RexCall digests
Ted Xu created CALCITE-1337: --- Summary: Lazy evaluate RexCall digests Key: CALCITE-1337 URL: https://issues.apache.org/jira/browse/CALCITE-1337 Project: Calcite Issue Type: Improvement Components: core Reporter: Ted Xu Assignee: Julian Hyde Fix For: 1.8.0 Currently RexCall compute digests eagerly in its constructor, also, it compute digest every time when toString invoked. It may cause performance issue when the RexCall tree is very large. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CALCITE-1299) Reference type of Pulled Up Predicates is not handled properly
[
https://issues.apache.org/jira/browse/CALCITE-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Xu updated CALCITE-1299:
Description:
RelMdPredicates is not handling reference types well. Noticing the following
query:
{code:title=query.sql|borderStyle=solid}
SELECT * FROM
(SELECT * FROM src) o JOIN (SELECT "foo" as key, value FROM src1) o1
ON o.key = o1.key
{code}
The right side predicate is [key -> "foo"] with key type VARCHAR NOT NULL, the
very type is not matching the input reference from the left side (which is
VARCHAR), causing error when applying JoinPushTransitivePredicatesRule.
was:
RelMdPredicates is not handling reference types well. Noticing the following
query:
{code:title=query.sql|borderStyle=solid}
SELECT * FROM
(SELECT * FROM src) o JOIN (SELECT "foo" as key, value FROM src1) o1
ON o.key = o1.key
{code}
The right side predicate is [key -> "foo"] with key type VARCHAR NOT NULL, the
very type is not matching the input reference from the left side, causing error
when applying JoinPushTransitivePredicatesRule.
> Reference type of Pulled Up Predicates is not handled properly
> --
>
> Key: CALCITE-1299
> URL: https://issues.apache.org/jira/browse/CALCITE-1299
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: next
>Reporter: Ted Xu
>Assignee: Julian Hyde
>Priority: Critical
> Fix For: 1.9.0
>
>
> RelMdPredicates is not handling reference types well. Noticing the following
> query:
> {code:title=query.sql|borderStyle=solid}
> SELECT * FROM
> (SELECT * FROM src) o JOIN (SELECT "foo" as key, value FROM src1) o1
> ON o.key = o1.key
> {code}
> The right side predicate is [key -> "foo"] with key type VARCHAR NOT NULL,
> the very type is not matching the input reference from the left side (which
> is VARCHAR), causing error when applying JoinPushTransitivePredicatesRule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1299) Reference type of Pulled Up Predicates is not handled properly
[
https://issues.apache.org/jira/browse/CALCITE-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341322#comment-15341322
]
Ted Xu commented on CALCITE-1299:
-
I'd like to take this one.
The root cause seems to be in RexPermuteInputsShuttle. I'll submit a patch when
I'm ready.
> Reference type of Pulled Up Predicates is not handled properly
> --
>
> Key: CALCITE-1299
> URL: https://issues.apache.org/jira/browse/CALCITE-1299
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: next
>Reporter: Ted Xu
>Assignee: Julian Hyde
>Priority: Critical
> Fix For: 1.9.0
>
>
> RelMdPredicates is not handling reference types well. Noticing the following
> query:
> {code:title=query.sql|borderStyle=solid}
> SELECT * FROM
> (SELECT * FROM src) o JOIN (SELECT "foo" as key, value FROM src1) o1
> ON o.key = o1.key
> {code}
> The right side predicate is [key -> "foo"] with key type VARCHAR NOT NULL,
> the very type is not matching the input reference from the left side, causing
> error when applying JoinPushTransitivePredicatesRule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1299) Reference type of Pulled Up Predicates is not handled properly
Ted Xu created CALCITE-1299:
---
Summary: Reference type of Pulled Up Predicates is not handled
properly
Key: CALCITE-1299
URL: https://issues.apache.org/jira/browse/CALCITE-1299
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: next
Reporter: Ted Xu
Assignee: Julian Hyde
Priority: Critical
Fix For: 1.9.0
RelMdPredicates is not handling reference types well. Noticing the following
query:
{code:title=query.sql|borderStyle=solid}
SELECT * FROM
(SELECT * FROM src) o JOIN (SELECT "foo" as key, value FROM src1) o1
ON o.key = o1.key
{code}
The right side predicate is [key -> "foo"] with key type VARCHAR NOT NULL, the
very type is not matching the input reference from the left side, causing error
when applying JoinPushTransitivePredicatesRule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1076) Metadata distribution is broken due to last refactor
[
https://issues.apache.org/jira/browse/CALCITE-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133744#comment-15133744
]
Ted Xu commented on CALCITE-1076:
-
Thanks [~julianhyde].
The pull request is updated.
> Metadata distribution is broken due to last refactor
>
>
> Key: CALCITE-1076
> URL: https://issues.apache.org/jira/browse/CALCITE-1076
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.7.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> Metadata distribution is broken due to last refactor.
> Calls of RelMdDistribution will get:
> {code}
> java.lang.AssertionError: are your methods named wrong?
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1076) Metadata distribution is broken due to last refactor
[
https://issues.apache.org/jira/browse/CALCITE-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133559#comment-15133559
]
Ted Xu commented on CALCITE-1076:
-
Thanks [~julianhyde], that's exactly what I'm expecting, but I'm not sure if
adding {{RelMdDistribution}} into default metadata provider could have other
impact.
I'm added {{RelMdDistribution}} into default metadata provider, the updated
pull request is https://github.com/apache/calcite/pull/194. We can also add
{{RelMdDistribution}} into RelMetadataTest only, if you feel the former way is
too intrusive.
> Metadata distribution is broken due to last refactor
>
>
> Key: CALCITE-1076
> URL: https://issues.apache.org/jira/browse/CALCITE-1076
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.7.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> Metadata distribution is broken due to last refactor.
> Calls of RelMdDistribution will get:
> {code}
> java.lang.AssertionError: are your methods named wrong?
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1076) Metadata distribution is broken due to last refactor
[
https://issues.apache.org/jira/browse/CALCITE-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131796#comment-15131796
]
Ted Xu commented on CALCITE-1076:
-
[~julianhyde] thanks for looking into this.
{{RelDistribute}} can be a pluggable metadata, however, it is used in Logical
RelNodes like {{LogicalFilter}}, which is deeply coupled with multiple rules
and utilities. It is highly recommended that we fix this bug in calcite-core.
> Metadata distribution is broken due to last refactor
>
>
> Key: CALCITE-1076
> URL: https://issues.apache.org/jira/browse/CALCITE-1076
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.7.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> Metadata distribution is broken due to last refactor.
> Calls of RelMdDistribution will get:
> {code}
> java.lang.AssertionError: are your methods named wrong?
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1076) Metadata distribution is broken due to last refactor
[
https://issues.apache.org/jira/browse/CALCITE-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131704#comment-15131704
]
Ted Xu commented on CALCITE-1076:
-
I created a pull request at https://github.com/apache/calcite/pull/193.
Julian please kindly review it.
> Metadata distribution is broken due to last refactor
>
>
> Key: CALCITE-1076
> URL: https://issues.apache.org/jira/browse/CALCITE-1076
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.7.0
>Reporter: Ted Xu
>Assignee: Julian Hyde
>
> Metadata distribution is broken due to last refactor.
> Calls of RelMdDistribution will get:
> {code}
> java.lang.AssertionError: are your methods named wrong?
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1076) Metadata distribution is broken due to last refactor
Ted Xu created CALCITE-1076:
---
Summary: Metadata distribution is broken due to last refactor
Key: CALCITE-1076
URL: https://issues.apache.org/jira/browse/CALCITE-1076
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.7.0
Reporter: Ted Xu
Assignee: Julian Hyde
Metadata distribution is broken due to last refactor.
Calls of RelMdDistribution will get:
{code}
java.lang.AssertionError: are your methods named wrong?
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1043) RexOptUtil does not support function table other than SqlStdOperatorTable
[ https://issues.apache.org/jira/browse/CALCITE-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Xu updated CALCITE-1043: Attachment: calcite-1043.patch Uploaded an initial patch to check operator SqlKind rather than compare attributes of SqlStdOperatorTable. > RexOptUtil does not support function table other than SqlStdOperatorTable > - > > Key: CALCITE-1043 > URL: https://issues.apache.org/jira/browse/CALCITE-1043 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.6.0 >Reporter: Ted Xu >Assignee: Julian Hyde >Priority: Critical > Fix For: 1.6.0 > > Attachments: calcite-1043.patch > > > RexOptUtil currently doesn't support function table other than > SqlStdOperatorTable, which prevents third-party function implementations to > reuse RexOptUtil. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1043) RexOptUtil does not support function table other than SqlStdOperatorTable
Ted Xu created CALCITE-1043: --- Summary: RexOptUtil does not support function table other than SqlStdOperatorTable Key: CALCITE-1043 URL: https://issues.apache.org/jira/browse/CALCITE-1043 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.6.0 Reporter: Ted Xu Assignee: Julian Hyde Priority: Critical Fix For: 1.6.0 RexOptUtil currently doesn't support function table other than SqlStdOperatorTable, which prevents third-party function implementations to reuse RexOptUtil. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
