[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193104#comment-17193104
]
Julian Hyde commented on CALCITE-4173:
--
CALCITE-4237 is now fixed.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192341#comment-17192341
]
Stamatis Zampetakis commented on CALCITE-4173:
--
[~vladimirsitnikov] Given the timing I guess we can push the fix in a separate
commit under CALCITE-4237.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190922#comment-17190922
]
Vladimir Sitnikov commented on CALCITE-4173:
@julianhyde , sorry for pinging you, however, the change somehow throws
AssertionError in a couple of slow tests:
https://github.com/apache/calcite/actions/runs/239955332
I wonder if you would like to force-push the fix or if the fix could be added
as an extra commit.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188218#comment-17188218
]
Ruben Q L commented on CALCITE-4173:
Ok, thanks for the clarification, [~julianhyde]
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187930#comment-17187930
]
Julian Hyde commented on CALCITE-4173:
--
[~rubenql], Short answer is no. This change does not give you what you want, or
make it any easier to implement.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187780#comment-17187780
]
Ruben Q L commented on CALCITE-4173:
I have a subsidiary question about this evolution: with this new feature, will
we be able to support dynamic parameters inside IN expressions?
{{SELECT FROM ... WHERE x IN (?)}}
If I am not mistaken this is currently not supported.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4173) Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
[
https://issues.apache.org/jira/browse/CALCITE-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17182088#comment-17182088
]
Julian Hyde commented on CALCITE-4173:
--
A review comment on CALCITE-4159 that I missed: in
[RelOptPredicateList.isEffectivelyNotNull|https://github.com/apache/calcite/blob/9ca61f020d134032dac8167b83bf431f078e33c5/core/src/main/java/org/apache/calcite/plan/RelOptPredicateList.java#L219],
make sure that it works for {{10 > ref}} as it does for {{10 < ref}}.
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall
> -
>
> Key: CALCITE-4173
> URL: https://issues.apache.org/jira/browse/CALCITE-4173
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>
> Add internal SEARCH operator and Sarg literal, replacing use of IN in RexCall.
> Recently we started to allow IN in RexCalls to represent lists of constant
> values; previously it had only been allowed in SqlCall. The use is confusing,
> because a RexSubQuery is a sub-class of RexCall that may also use IN as its
> operator.
> More important, we would like to be able to represent more general search
> arguments in a single RexCall. Examples:
> * {{x BETWEEN 3 AND 10}}
> * {{x > 3 AND x <= 10}}
> * {{x NOT BETWEEN 3 AND 10}}
> * {{x IS NULL OR x BETWEEN 3 AND 100 AND x != 50 OR x IN (200, 300, 400)}}
> * {{x > 5 AND x < 10 AND x IN (3, 8, 10, 20)}}
> All of these can be converted to sets of ranges, where each range has a lower
> or upper bound (or both), bounds can be open or closed, plus a flag to say
> whether NULL is an allowed value. Guava's RangeSet is an efficient
> implementation of range sets.
> This change would create a new class {{Sarg}} to represent a range set as a
> literal. The new internal SEARCH operator tests whether an operand belongs to
> the range set.
> A RexCall to SEARCH is converted back to SQL, typically an IN or OR.
> This change would obsolete the use of IN in {{RexCall}} to represent a fixed
> list of values. (This is a breaking change, but when we first allowed IN in
> {{RexCall}}, in CALCITE-2444, it was only intended to be for
> {{RelToSqlConverter}}. Most people still believe that [IN is not
> allowed|https://lists.apache.org/thread.html/e1c5d56ecca7c1bc3608344ceac9b209bb8100fbca1c1928feb9cce7%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
