[jira] [Commented] (CALCITE-5981) TIMESTAMPDIFF function returns incorrect result
[
https://issues.apache.org/jira/browse/CALCITE-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762551#comment-17762551
]
Tanner Clary commented on CALCITE-5981:
---
I have done some work in the TimestampDiffConvertlet. It is pretty messy and
whether it's apart of this issue or a separate one, I would love to help clean
it up/help with the review process. I only have experience with BigQuery's
TIMESTAMP_DIFF semantics but that might be relevant, too. Just let me know, and
thanks for filing this issue.
> TIMESTAMPDIFF function returns incorrect result
>
>
> Key: CALCITE-5981
> URL: https://issues.apache.org/jira/browse/CALCITE-5981
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> The following test fails, when added to SqlOperatorTest:
> {code:java}
> f.checkScalar("timestampdiff(month, DATE '2004-02-29', DATE '2005-02-28')",
> "11", "INTEGER NOT NULL");
> {code}
> The result returned by the expression is 12. However, MySQL returns 11.
> The semantics of this function is not described clearly in the documentation,
> but according to several prior issues [1] [2] [3] the intended semantics
> should be the same as in MySQL.
> A corresponding MySQL test: [4]
> The implementation seems to be in
> StandardConvertletTable.TimestampDiffConvertlet.
> [1] https://issues.apache.org/jira/browse/CALCITE-1827,
> [2] https://issues.apache.org/jira/browse/CALCITE-3529,
> [3] https://issues.apache.org/jira/browse/CALCITE-1124.
> [4]
> [https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/mysql-test/r/func_time.result#L1151]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5981) TIMESTAMPDIFF function returns incorrect result
Mihai Budiu created CALCITE-5981:
Summary: TIMESTAMPDIFF function returns incorrect result
Key: CALCITE-5981
URL: https://issues.apache.org/jira/browse/CALCITE-5981
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test fails, when added to SqlOperatorTest:
{code:java}
f.checkScalar("timestampdiff(month, DATE '2004-02-29', DATE '2005-02-28')",
"11", "INTEGER NOT NULL");
{code}
The result returned by the expression is 12. However, MySQL returns 11.
The semantics of this function is not described clearly in the documentation,
but according to several prior issues [1] [2] [3] the intended semantics should
be the same as in MySQL.
A corresponding MySQL test: [4]
The implementation seems to be in
StandardConvertletTable.TimestampDiffConvertlet.
[1] https://issues.apache.org/jira/browse/CALCITE-1827,
[2] https://issues.apache.org/jira/browse/CALCITE-3529,
[3] https://issues.apache.org/jira/browse/CALCITE-1124.
[4]
[https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/mysql-test/r/func_time.result#L1151]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762533#comment-17762533
]
Julian Hyde commented on CALCITE-5919:
--
[~mbudiu], But also: create links. Click the 'More' menu, then 'Link', and 'is
related to'.
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762530#comment-17762530
]
Mihai Budiu edited comment on CALCITE-5919 at 9/7/23 1:06 AM:
--
This is also related to CALCITE-3529
was (Author: JIRAUSER295926):
This is also related to https://issues.apache.org/jira/browse/CALCITE-3529
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762528#comment-17762528
]
Mihai Budiu edited comment on CALCITE-5919 at 9/7/23 1:06 AM:
--
This is related to CALCITE-3530
was (Author: JIRAUSER295926):
This is related to https://issues.apache.org/jira/browse/CALCITE-3530
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762531#comment-17762531
]
Julian Hyde commented on CALCITE-5919:
--
Please make Jira links, so the links can be found from the other end. Also,
just paste the case id, e.g. CALCITE-3529, so that it will change when the
referenced case is closed.
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762530#comment-17762530
]
Mihai Budiu commented on CALCITE-5919:
--
This is also related to https://issues.apache.org/jira/browse/CALCITE-3529
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-3530) TIMESTAMPADD/TIMESTAMPDIFF with microsecond/nanosecond unit lost precision
[ https://issues.apache.org/jira/browse/CALCITE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762529#comment-17762529 ] Mihai Budiu commented on CALCITE-3530: -- This is related to https://issues.apache.org/jira/browse/CALCITE-5919?filter=-2 The solution is probably to use a Decimal data type, that would work for arbitrary precision. > TIMESTAMPADD/TIMESTAMPDIFF with microsecond/nanosecond unit lost precision > -- > > Key: CALCITE-3530 > URL: https://issues.apache.org/jira/browse/CALCITE-3530 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Zhenghua Gao >Priority: Minor > > Since the TimestampAddConvertlet and TimestampDiffConvertlet tread TIMESTAMP > as long (with millisecond precision), they lost precision even if the > downstream can support microsecond or nanosecond. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble edited comment on CALCITE-5636 at 9/7/23 12:58 AM:
--
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
couple bugs at play here.
Anyway, the reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
[A few lines
down|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4659]
in {{validateSelectList()}}, we invoke {{validateExpr()}} on that expanded
node, and this eventually tries to derive the type of of that {{CAST}}
expression, which results in visiting the inserted {{TIMESTAMP(0)}} expression
(which is a {{SqlDataTypeSpec}})
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6657],
and in so doing switches it from {{TIMESTAMP}} to {{TIMESTAMP WITH LOCAL TIME
ZONE}}, because it thinks that the {{CAST}} expression was written by the user,
and must therefore be translated according to the catalog reader.
I'm still thinking about how to handle this. My [original
PR|https://github.com/apache/calcite/pull/3144] tried to handle it by
consulting the type map when the {{CAST}} is inserted during coercion, but what
we probably actually want is for the type map to be consulted during validation
IFF the {{SqlDataTypeSpec}} was actually written by the user, as opposed to
being inserted during expansion. Perhaps we should introduce a boolean flag to
{{SqlDataTypeSpec}} that indicates that the type is final?
was (Author: wnoble):
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
[jira] [Commented] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
[
https://issues.apache.org/jira/browse/CALCITE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762528#comment-17762528
]
Mihai Budiu commented on CALCITE-5919:
--
This is related to https://issues.apache.org/jira/browse/CALCITE-3530
> Compile-time implementation of EXTRACT ignores sub-millisecond values
> -
>
> Key: CALCITE-5919
> URL: https://issues.apache.org/jira/browse/CALCITE-5919
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the
> compile-time evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
> '13:30:25.575401') will produce a result up to milliseconds only,
> irrespective of the type system provided. (This query will evaluate to
> 25575000 instead of 25575401).
> The bug is in RexImpTable.ExtractImplementor, here:
> {code:java}
> case MILLISECOND:
> case MICROSECOND:
> case NANOSECOND:
> if (sqlTypeName == SqlTypeName.DATE) {
> return Expressions.constant(0L);
> }
> operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
> !isIntervalType); // << BUG
> return Expressions.multiply(
> operand, Expressions.constant((long) (1 /
> unit.multiplier.doubleValue(;
> {code}
> The mod operation uses a multiplier for MINUTE that is expressed in
> milliseconds, so it always truncates away values below milliseconds. The
> multiplier seems to assume that the type system precision for TIME is set to
> 3, which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble edited comment on CALCITE-5636 at 9/7/23 12:54 AM:
--
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
couple bugs at play here.
Anyway, the reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
[A few lines
down|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4659]
in {{validateSelectList()}}, we invoke {{validateExpr()}} on that expanded
node, and this eventually tries to derive the type of of that {{CAST}}
expression, which results in visiting the inserted {{TIMESTAMP(0)}} expression
(which is a {{SqlDataTypeSpec}})
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6657],
and in so doing switches it from {{TIMESTAMP}} to {{TIMESTAMP WITH LOCAL TIME
ZONE}}, because it thinks that the {{CAST}} expression was written by the user,
and must therefore be translated according to the catalog reader.
I'm still thinking about how to handle this.
was (Author: wnoble):
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
[jira] [Comment Edited] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble edited comment on CALCITE-5636 at 9/7/23 12:52 AM:
--
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
couple bugs at play here.
Anyway, the reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
[A few lines
down|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4659]
in {{validateSelectList()}}, we invoke {{validateExpr()}} on that expanded
node, and this eventually tries to derive the type of of that {{CAST}}
expression, which results in visiting the inserted {{SqlDataTypeSpec}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6657],
and in so doing switches it from {{TIMESTAMP}} to {{TIMESTAMP WITH LOCAL TIME
ZONE}}, because it thinks that the {{CAST}} expression was written by the user,
and must therefore be translated according to the catalog reader.
I'm still thinking about how to handle this.
was (Author: wnoble):
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
couple bugs at play here.
The reason for
[jira] [Comment Edited] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble edited comment on CALCITE-5636 at 9/7/23 12:48 AM:
--
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops. There may actually be a
couple bugs at play here.
The reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
[A few lines
down|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4659]
in {{validateSelectList()}}, we invoke {{validateExpr()}} on that expanded
node, and this eventually tries to derive the type of of that {{CAST}}
expression, which results in visiting the inserted {{SqlDataTypeSpec}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6657],
and in so doing switches it from {{TIMESTAMP}} to {{TIMESTAMP WITH LOCAL TIME
ZONE}}, because it thinks that the {{CAST}} expression was written by the user,
and must therefore be translated according to the catalog reader.
I'm still thinking about how to handle this.
was (Author: wnoble):
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops.
The reason for the discrepancy can be seen in
[jira] [Comment Edited] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble edited comment on CALCITE-5636 at 9/7/23 12:45 AM:
--
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops.
The reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
[A few lines
down|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4659]
in {{validateSelectList()}}, we invoke {{validateExpr()}} on that expanded
node, and this eventually tries to derive the type of of that {{CAST}}
expression, which results in visiting the inserted {{SqlDataTypeSpec}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6657],
and in so doing switches it from {{TIMESTAMP}} to {{TIMESTAMP WITH LOCAL TIME
ZONE}}, because it thinks that the {{CAST}} expression was written by the user,
and must therefore be translated according to the catalog reader.
I'm still thinking about how to handle this.
was (Author: wnoble):
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops.
The reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes
[jira] [Commented] (CALCITE-5636) Consult type mappings when coercing types
[
https://issues.apache.org/jira/browse/CALCITE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762525#comment-17762525
]
Will Noble commented on CALCITE-5636:
-
Finally getting back to this. Hopefully this is a much better description of
the problem. It occurs when using BigQuery's catalog type map (where
{{TIMESTAMP}} maps to {{TIMESTAMP WITH LOCAL TIME ZONE}}) and
{{SqlToRelConverter}} encounters a query like this:
{code:sql}
SELECT TIMESTAMP_ADD(DATE(1900, 1, 1), INTERVAL (1) DAY) AS `ts`
{code}
The parsed {{SqlNode}} has validated type {{TIMESTAMP(0) NOT NULL}}, and it
ends up converting into the following relational expression:
{code:java}
LogicalProject(ts=[+(CAST(DATE(1900, 1, 1)):TIMESTAMP_WITH_LOCAL_TIME_ZONE(0)
NOT NULL, *(8640:INTERVAL DAY, 1))])
{code}
which of course has converted type {{TIMESTAMP_WITH_LOCAL_TIME_ZONE(0) NOT
NULL}}, so the type of the {{SqlNode}} does not match the type of the
{{{}RelNode{}}}, and it triggers [this
exception|https://github.com/apache/calcite/blob/50c3edfc3d6630528ab51fe836bd50df82cc7db8/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L489].
Btw, despite the fact that a BQ {{TIMESTAMP}} is really a timestamp w/ ltz, the
{{TIMESTAMP_ADD()}} function actually returns a {{DATETIME}} object (ISO
timestamp) when the first argument is a {{DATE}}, so the validated type of the
{{SqlNode}} is actually correct, as verified by manually running the query
against BQ. My original PR was mistaken in believing that it should be
timestamp w/ ltz, as the name seems to imply. Whoops.
The reason for the discrepancy can be seen in
{{SqlValidatorImpl.validateSelectList()}}. This invokes {{expandSelectItem()}}
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L4638],
which eventually tries to derive the type of the {{TIMESTAMP_ADD()}} call
[here|https://github.com/apache/calcite/blob/bcf6bd8577b25c563b1c597c70704594a18ca1a3/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6555],
which must validate the operands in order to derive the type of the function
call, and in so doing coerces the type of the first operand by casting it to an
ISO timestamp. All of this results in a value in {{expandedSelectItems}} that
looks like this:
{code:sql}
TIMESTAMP_ADD(CAST(DATE(1900, 1, 1) AS TIMESTAMP(0)), INTERVAL 1 DAY) AS `ts`
{code}
A few lines down,
> Consult type mappings when coercing types
> -
>
> Key: CALCITE-5636
> URL: https://issues.apache.org/jira/browse/CALCITE-5636
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Will Noble
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This is another offshoot of CALCITE-5346.
> In
> [{{AbstractTypeCoercion.implicitCast}}|https://github.com/apache/calcite/blob/2dba40e7a0a5651eac5a30d9e0a72f178bd9bff2/core/src/main/java/org/apache/calcite/sql/validate/implicit/AbstractTypeCoercion.java#L713],
> Calcite may insert {{CAST}} invocations disregarding the root schema type
> map. This happens during the validation phase, so it's easy enough to read
> the type map directly in the {{AbstractTypeCoercion}} utility, although it's
> hard to say if this is an optimal solution. It may make sense to incorporate
> the type map into the {{RelDataTypeSystem}} which is more widely available in
> non-validation phases of SQL processing pipelines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{{}replacement{}}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example (added one space between \ \ to override md formatting):
{{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
|result|
|aXc|
was:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{{}replacement{}}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example (added one space between \ \ to override md formatting):
{{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
|result|
|aX|
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> *{{REGEXP_REPLACE(value, regexp, replacement)}}*
> Returns a STRING where all substrings of {{value}} that match regular
> expression {{regexp}} are replaced with {{{}replacement{}}}.
> backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
> argument to insert text matching the corresponding parenthesized group in the
> {{regexp}} pattern.
> Example (added one space between \ \ to override md formatting):
> {{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
> |result|
> |aXc|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example (added one space between \ \ to override md formatting):
{{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
|result|
|aX|
was:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example:
{{SELECT REGEXP_REPLACE('abc', 'b(.)', 'X\\1') as result;}}
|result|
|aXc|
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> *{{REGEXP_REPLACE(value, regexp, replacement)}}*
> Returns a STRING where all substrings of {{value}} that match regular
> expression {{regexp}} are replaced with {{replacement}}.
> backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
> argument to insert text matching the corresponding parenthesized group in the
> {{regexp}} pattern.
> Example (added one space between \ \ to override md formatting):
> {{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
> |result|
> |aX|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{{}replacement{}}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example (added one space between \ \ to override md formatting):
{{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
|result|
|aX|
was:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example (added one space between \ \ to override md formatting):
{{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
|result|
|aX|
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> *{{REGEXP_REPLACE(value, regexp, replacement)}}*
> Returns a STRING where all substrings of {{value}} that match regular
> expression {{regexp}} are replaced with {{{}replacement{}}}.
> backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
> argument to insert text matching the corresponding parenthesized group in the
> {{regexp}} pattern.
> Example (added one space between \ \ to override md formatting):
> {{SELECT REGEXP_REPLACE("abc'", "b(.)", "X\ \1") as result;}}
> |result|
> |aX|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern.
Example:
{{SELECT REGEXP_REPLACE('abc', 'b(.)', 'X\\1') as result;}}
|result|
|aXc|
was:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> *{{REGEXP_REPLACE(value, regexp, replacement)}}*
> Returns a STRING where all substrings of {{value}} that match regular
> expression {{regexp}} are replaced with {{replacement}}.
> backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
> argument to insert text matching the corresponding parenthesized group in the
> {{regexp}} pattern.
> Example:
> {{SELECT REGEXP_REPLACE('abc', 'b(.)', 'X\\1') as result;}}
> |result|
> |aXc|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762489#comment-17762489
]
Julian Hyde commented on CALCITE-5978:
--
Thanks! (And thanks for updating CALCITE-5979 without being asked.)
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}*
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{{}regexp{}}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
> Example:
> {{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
> |result|
> |4|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5975) PERCENTILE_CONT function throws UnsupportedOperationException in RelBuilder.aggregateCall
[
https://issues.apache.org/jira/browse/CALCITE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-5975:
-
Summary: PERCENTILE_CONT function throws UnsupportedOperationException in
RelBuilder.aggregateCall (was: percentile_cont function throws
UnsupportedOperationException)
> PERCENTILE_CONT function throws UnsupportedOperationException in
> RelBuilder.aggregateCall
> -
>
> Key: CALCITE-5975
> URL: https://issues.apache.org/jira/browse/CALCITE-5975
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.35.0
>Reporter: wqn
>Priority: Major
>
> execute a sql like
> {code:java}
> SELECT percentile_cont(0.0005) within group (ORDER BY aa DESC) from
> table{code}
> it throws UnsupportedOperationException
> {code:java}
> Caused by: java.lang.UnsupportedOperationException: null at
> org.apache.calcite.sql.SqlOperatorBinding.getCollationType(SqlOperatorBinding.java:244)
> at
> org.apache.calcite.sql.type.ReturnTypes.lambda$static$25(ReturnTypes.java:1285)
> at
> org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:533)
> at org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198)
> at
> org.apache.calcite.tools.RelBuilder$AggCallImpl.aggregateCall(RelBuilder.java:4173)
> at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2424)
> at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:2348)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:1102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532) at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:225)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:494)
> at sun.reflect.GeneratedMethodAccessor1005.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:225)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:824)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5975) percentile_cont function throws UnsupportedOperationException
[
https://issues.apache.org/jira/browse/CALCITE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762487#comment-17762487
]
Julian Hyde commented on CALCITE-5975:
--
I don't think that {{PERCENTILE_CONT}} is fully implemented yet. You can
validate a query with it, but you cannot yet execute queries. See CALCITE-4666.
However, your case would occur if someone tried to create a call to
{{PERCENTILE_CONT}} via {{{}RelBuilder{}}}, even if they don't try to execute
the query. That is something that should be currently working, but clearly it
isn't.
> percentile_cont function throws UnsupportedOperationException
> -
>
> Key: CALCITE-5975
> URL: https://issues.apache.org/jira/browse/CALCITE-5975
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.35.0
>Reporter: wqn
>Priority: Major
>
> execute a sql like
> {code:java}
> SELECT percentile_cont(0.0005) within group (ORDER BY aa DESC) from
> table{code}
> it throws UnsupportedOperationException
> {code:java}
> Caused by: java.lang.UnsupportedOperationException: null at
> org.apache.calcite.sql.SqlOperatorBinding.getCollationType(SqlOperatorBinding.java:244)
> at
> org.apache.calcite.sql.type.ReturnTypes.lambda$static$25(ReturnTypes.java:1285)
> at
> org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:533)
> at org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198)
> at
> org.apache.calcite.tools.RelBuilder$AggCallImpl.aggregateCall(RelBuilder.java:4173)
> at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2424)
> at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:2348)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:1102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532) at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:225)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:494)
> at sun.reflect.GeneratedMethodAccessor1005.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:225)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:824)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:283)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5974) Es index mapping set dynamic_templates without properties causing bug
[
https://issues.apache.org/jira/browse/CALCITE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762486#comment-17762486
]
Julian Hyde commented on CALCITE-5974:
--
In the summary, "Es" isn't very useful for search. And you say there is a bug,
but you don't say what the bug is. Remember that the summary is the only
information that will go into the release notes.
> Es index mapping set dynamic_templates without properties causing bug
> -
>
> Key: CALCITE-5974
> URL: https://issues.apache.org/jira/browse/CALCITE-5974
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.35.0
>Reporter: zhaowang
>Assignee: zhaowang
>Priority: Major
>
> When use es-adapter, we config a es cluster, it will fetchMapping during
> initialization.
> If a index set dynamic_templates bug no mappings like this:
>
> {code:java}
> {
> "test_index" : {
> "mappings" : {
> "dynamic_templates" : [
> {
> "integers" : {
> "match_mapping_type" : "long",
> "mapping" : {
> "type" : "integer"
> }
> }
> }
> ]
> }
> }
> } {code}
>
>
> org.apache.calcite.adapter.elasticsearch.ElasticsearchJson#visitMappingProperties
> throw exception:
>
> {code:java}
> java.lang.ClassCastException:
> com.fasterxml.jackson.databind.node.ArrayNode cannot be cast to
> com.fasterxml.jackson.databind.node.ObjectNode
> at
>
> org.apache.calcite.adapter.elasticsearch.ElasticsearchJson.visitMappingProperties(ElasticsearchJson.java:133)
> {code}
> So ElasticsearchTransport initialize failed, all index of this es cluster
> access failed.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{{}regexp{}}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
Example:
{{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
|result|
|4|
was:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{{}regexp{}}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
Example:
{{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
| result |
| 4 |
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}*
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{{}regexp{}}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
> Example:
> {{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
> |result|
> |4|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{{}regexp{}}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
Example:
{{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
| result |
| 4 |
was:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{{}regexp{}}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
Example:
{{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
++
| result |
++
| 4 |
++
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}*
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{{}regexp{}}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
> Example:
> {{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
> | result |
> | 4 |
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{{}regexp{}}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
Example:
{{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
++
| result |
++
| 4 |
++
was:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}*
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{{}regexp{}}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{{}string{}}}, otherwise it starts at the beginning of {{{}string{}}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
> Example:
> {{SELECT REGEXP_INSTR("abcadcabcaecghi", "adc") as result;}}
> ++
> | result |
> ++
> | 4 |
> ++
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5860) Decimal type conversion missing scale
[
https://issues.apache.org/jira/browse/CALCITE-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762484#comment-17762484
]
pengfei.zhan commented on CALCITE-5860:
---
Hi, [~zabetak], get your message. As you mentioned in the pr,
https://issues.apache.org/jira/browse/CALCITE-5923 seems to be done in this
issue much better.
I'm thinking how to resolve this more elegant.
> Decimal type conversion missing scale
> -
>
> Key: CALCITE-5860
> URL: https://issues.apache.org/jira/browse/CALCITE-5860
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.34.0
>Reporter: Guoliang Sun
>Assignee: pengfei.zhan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> Take the following SQL as an example
> {code:sql}
> SELECT CAST(((2.0) / SQRT(3.0)) AS DECIMAL(18, 0)) * SQRT(3.0) {code}
> The result of the SQL calculation should be {*}SQRT(3.0){*}.However, the
> actual result is {*}2.0{*}, which is not meet expectations.
>
> The following is the code generated by Janino
> {code:java}
> public Object[] apply(Object root0) {
> final java.math.BigDecimal literal_value = new java.math.BigDecimal(
> "2.0");
> final java.math.BigDecimal literal_value0 = new java.math.BigDecimal(
> "3.0");
> final java.math.BigDecimal literal_value1 = new java.math.BigDecimal(
> "0.5");
> final double method_name_call_value =
> org.apache.calcite.runtime.SqlFunctions.power(literal_value0, literal_value1);
> final java.math.BigDecimal cast_value = new java.math.BigDecimal(
> literal_value.doubleValue() / method_name_call_value);
> return new Object[] {
> cast_value == null ? 0.0D : cast_value.doubleValue() *
> method_name_call_value};
> } {code}
> We can see *((2.0) / SQRT(3.0)) AS DECIMAL(18, 0)* lost the scale.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}*
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
was:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}
*Returns the lowest 1-based position of the substring in {{string}} that
matches the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}*
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{regexp}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{string}}, otherwise it starts at the beginning of {{string}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
*{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}
*Returns the lowest 1-based position of the substring in {{string}} that
matches the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
was:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> *{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}
> *Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{regexp}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{string}}, otherwise it starts at the beginning of {{string}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
*{{REGEXP_REPLACE(value, regexp, replacement)}}*
Returns a STRING where all substrings of {{value}} that match regular
expression {{regexp}} are replaced with {{replacement}}.
backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
argument to insert text matching the corresponding parenthesized group in the
{{regexp}} pattern
was:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
{{REGEXP_REPLACE(value, regexp, replacement)}}
Returns a STRING where all substrings of value that match regular expression
regexp are replaced with replacement.
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> *{{REGEXP_REPLACE(value, regexp, replacement)}}*
> Returns a STRING where all substrings of {{value}} that match regular
> expression {{regexp}} are replaced with {{replacement}}.
> backslashed-escaped digits (\1 to \9) can be used within the {{replacement}}
> argument to insert text matching the corresponding parenthesized group in the
> {{regexp}} pattern
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5979) Add REGEXP_REPLACE function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5979:
Description:
Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
{{REGEXP_REPLACE(value, regexp, replacement)}}
Returns a STRING where all substrings of value that match regular expression
regexp are replaced with replacement.
was:Add support for [REGEXP_REPLACE
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
function from BigQuery.
> Add REGEXP_REPLACE function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5979
> URL: https://issues.apache.org/jira/browse/CALCITE-5979
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_REPLACE
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace]
> function from BigQuery.
> {{REGEXP_REPLACE(value, regexp, replacement)}}
> Returns a STRING where all substrings of value that match regular expression
> regexp are replaced with replacement.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5978:
Description:
Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
{{REGEXP_INSTR(string, regexp [, position [, occurrence [,
occurrence_position]]])}}
Returns the lowest 1-based position of the substring in {{string}} that matches
the {{regexp}}, Returns 0 if there is no match
* If {{position}} is specified, the search starts at this position in
{{string}}, otherwise it starts at the beginning of {{string}}.
* If {{occurrence}} is specified, the search returns index for the specific
occurrence of the {{regexp}} in value, otherwise returns the first match.
* If {{occurrence_position}} is specified as 1, returns the end index of
substring + 1 (default 0 returns start index of match)
was:Add support for [REGEXP_INSTR
|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
function from BigQuery.
> Add REGEXP_INSTR function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-5978
> URL: https://issues.apache.org/jira/browse/CALCITE-5978
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for [REGEXP_INSTR
> |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr]
> function from BigQuery.
> {{REGEXP_INSTR(string, regexp [, position [, occurrence [,
> occurrence_position]]])}}
> Returns the lowest 1-based position of the substring in {{string}} that
> matches the {{regexp}}, Returns 0 if there is no match
> * If {{position}} is specified, the search starts at this position in
> {{string}}, otherwise it starts at the beginning of {{string}}.
> * If {{occurrence}} is specified, the search returns index for the specific
> occurrence of the {{regexp}} in value, otherwise returns the first match.
> * If {{occurrence_position}} is specified as 1, returns the end index of
> substring + 1 (default 0 returns start index of match)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[ https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762482#comment-17762482 ] Jerin John commented on CALCITE-5978: - Sure Julian will update it. > Add REGEXP_INSTR function (enabled in BigQuery library) > --- > > Key: CALCITE-5978 > URL: https://issues.apache.org/jira/browse/CALCITE-5978 > Project: Calcite > Issue Type: Task >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > Labels: pull-request-available > > Add support for [REGEXP_INSTR > |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr] > function from BigQuery. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5978) Add REGEXP_INSTR function (enabled in BigQuery library)
[ https://issues.apache.org/jira/browse/CALCITE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762481#comment-17762481 ] Julian Hyde commented on CALCITE-5978: -- Please add a brief summary and example to the description. People may not be familiar with the REGEXP_INSTR function. Hyperlinks break over time. > Add REGEXP_INSTR function (enabled in BigQuery library) > --- > > Key: CALCITE-5978 > URL: https://issues.apache.org/jira/browse/CALCITE-5978 > Project: Calcite > Issue Type: Task >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > Labels: pull-request-available > > Add support for [REGEXP_INSTR > |https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_instr] > function from BigQuery. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5980) QuidemTests are not effectively executed on Windows
[
https://issues.apache.org/jira/browse/CALCITE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762480#comment-17762480
]
Julian Hyde edited comment on CALCITE-5980 at 9/6/23 6:33 PM:
--
Oh boy, good catch. I should have known that something was up when it 'just
worked' on Windows.
Are there any sanity checks that would have caught this? E.g. checking that
CoreQuidemTest executes at least one file? Or executes at least one query?
was (Author: julianhyde):
Oh boy, good catch. I should have known that something was up when it 'just
worked' on Windows.
> QuidemTests are not effectively executed on Windows
> ---
>
> Key: CALCITE-5980
> URL: https://issues.apache.org/jira/browse/CALCITE-5980
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.36.0
>
>
> Discovered by accident on my Windows+IntelliJ environment while I was trying
> to add new tests on a *.iq file. My new tests did not seem to be executed. I
> even tried adding syntax errors on purpose into different iq files to force
> them to fail, but the tests were still successful. The reason seems to be
> that, at least on my environment (Windows), the test files do not execute any
> of their statements. This seems caused by the changes introduced in
> CALCITE-5786.
> While debugging, I found this line in QuidemTest.java (that aims to create
> the inFile and outFile):
> {code:java}
> protected void checkRun(String path) throws Exception {
> ...
> // e.g. path = "sql/agg.iq"
> // inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // outDir = "/home/fred/calcite/core/build/quidem/test/sql"
> // outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
> inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
> outFile = replaceDir(inFile, "resources", "quidem");
> ...
> }
> {code}
> But in my case it results on {*}both files being the same{*}, thus when the
> outFile is created, it actually erases all the tests that were contained
> inside the inFile, so the test runs nothing.
> The reason for that is that the auxiliary method {{{}replaceDir{}}}:
> {code:java}
> private static File replaceDir(File file, String target, String
> replacement) {
> return new File(
> file.getAbsolutePath().replace(n2u('/' + target + '/'),
> n2u('/' + replacement + '/')));
> }
> {code}
> is trying to replace "/resources/" with "/quidem/" from the inFile absolute
> path, but in my case this path does not contain the pattern to be replaced,
> since it contains backslashes:
> "C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
> operation does nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5980) QuidemTests are not effectively executed on Windows
[
https://issues.apache.org/jira/browse/CALCITE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762480#comment-17762480
]
Julian Hyde commented on CALCITE-5980:
--
Oh boy, good catch. I should have known that something was up when it 'just
worked' on Windows.
> QuidemTests are not effectively executed on Windows
> ---
>
> Key: CALCITE-5980
> URL: https://issues.apache.org/jira/browse/CALCITE-5980
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.36.0
>
>
> Discovered by accident on my Windows+IntelliJ environment while I was trying
> to add new tests on a *.iq file. My new tests did not seem to be executed. I
> even tried adding syntax errors on purpose into different iq files to force
> them to fail, but the tests were still successful. The reason seems to be
> that, at least on my environment (Windows), the test files do not execute any
> of their statements. This seems caused by the changes introduced in
> CALCITE-5786.
> While debugging, I found this line in QuidemTest.java (that aims to create
> the inFile and outFile):
> {code:java}
> protected void checkRun(String path) throws Exception {
> ...
> // e.g. path = "sql/agg.iq"
> // inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // outDir = "/home/fred/calcite/core/build/quidem/test/sql"
> // outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
> inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
> outFile = replaceDir(inFile, "resources", "quidem");
> ...
> }
> {code}
> But in my case it results on {*}both files being the same{*}, thus when the
> outFile is created, it actually erases all the tests that were contained
> inside the inFile, so the test runs nothing.
> The reason for that is that the auxiliary method {{{}replaceDir{}}}:
> {code:java}
> private static File replaceDir(File file, String target, String
> replacement) {
> return new File(
> file.getAbsolutePath().replace(n2u('/' + target + '/'),
> n2u('/' + replacement + '/')));
> }
> {code}
> is trying to replace "/resources/" with "/quidem/" from the inFile absolute
> path, but in my case this path does not contain the pattern to be replaced,
> since it contains backslashes:
> "C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
> operation does nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5888) Assertion error in aggregate
[
https://issues.apache.org/jira/browse/CALCITE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762451#comment-17762451
]
Julian Hyde commented on CALCITE-5888:
--
I can see that[~pmsgd] has put considerable effort into creating a test case.
Can one of the committers in this thread volunteer to take a look? I think you
will find it straightforward to convert that test case into a simple test case
in RelBuilderTest.
> Assertion error in aggregate
>
>
> Key: CALCITE-5888
> URL: https://issues.apache.org/jira/browse/CALCITE-5888
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Petr Masopust
>Priority: Major
>
> We have {{relBuilder.aggregate(groupKey, aggregateCalls)}} in our code with
> values {{[AS($31, 'a_label_d_opp_stage_id_foodmart_bd4c9c91212f9f'),
> AS(CAST(FLOOR($34, FLAG(QUARTER))):TIMESTAMP(0),
> 'a_label_snapshot_timestamp_quarter_foodmart_89fe710c8628d9')]}} and
> {{[COUNT(CASE(AND(COALESCE($37, false), =($36, $13)), $15, null:NULL)),
> SUM(CASE(AND(COALESCE($37, false), =($36, $13)), $2, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL))].}}
> It works perfectly in version 1.34.0 but in 1.35.0 we got this exception:
> {{java.lang.AssertionError}}
> {{ at org.apache.calcite.rel.core.Aggregate.(Aggregate.java:175)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.(LogicalAggregate.java:72)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create_(LogicalAggregate.java:144)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create(LogicalAggregate.java:116)}}
> {{ at
> org.apache.calcite.rel.core.RelFactories$AggregateFactoryImpl.createAggregate(RelFactories.java:328)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2564)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2511)}}
> {{ at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:2348)}}
>
> I think it is either missing {{permute}} or assert should compare
> {{cardinality}} instead of {{length.}} Because it compares field index? to
> number of fields which looks like nonsense to me.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5977) RexLiteral returns the same value for TIMESTAMP and TIMESTAMP_WITH_LOCAL_TIME_ZONE
[
https://issues.apache.org/jira/browse/CALCITE-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762424#comment-17762424
]
Ruben Q L commented on CALCITE-5977:
My bad, everything is fine. I just realized that, for my case to work as
expected, I needed to use the explicit "CAST" operation with the appropriate
timezone code in the literal:
{code:sql}
CAST('2023-09-04 17:44:00 Europe/Paris' AS TIMESTAMP WITH LOCAL TIME ZONE)
{code}
Closing this ticket...
> RexLiteral returns the same value for TIMESTAMP and
> TIMESTAMP_WITH_LOCAL_TIME_ZONE
> --
>
> Key: CALCITE-5977
> URL: https://issues.apache.org/jira/browse/CALCITE-5977
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Priority: Major
>
> Perhaps I'm missing something, but this seems odd to me.
> If we want to get the value in Long format (i.e. milliseconds since
> 1970-01-01 00:00:00) of a TIMESTAMP / TIMESTAMP_WITH_LOCAL_TIME_ZONE
> RexLiteral, their code is exactly the same (even if they are in different
> "case" blocks):
> {code}
> public @Nullable Object getValue2() {
> ...
> switch (typeName) {
> ...
> case TIMESTAMP:
> case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
> return getValueAs(Long.class);
> ...
> }
> public @Nullable T getValueAs(Class clazz) {
> ...
> switch (typeName) {
> ...
> case TIMESTAMP:
> if (clazz == Long.class) {
> // Milliseconds since 1970-01-01 00:00:00
> return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
> }
> ...
> break;
> case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
> if (clazz == Long.class) {
> // Milliseconds since 1970-01-01 00:00:00
> return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
> }
> ...
> {code}
> In case of a TIMESTAMP_WITH_LOCAL_TIME_ZONE, shouldn't this code include some
> extra processing to "shift" the value with the proper offset / daylight
> savings?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5977) RexLiteral returns the same value for TIMESTAMP and TIMESTAMP_WITH_LOCAL_TIME_ZONE
[
https://issues.apache.org/jira/browse/CALCITE-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L closed CALCITE-5977.
--
Resolution: Not A Problem
> RexLiteral returns the same value for TIMESTAMP and
> TIMESTAMP_WITH_LOCAL_TIME_ZONE
> --
>
> Key: CALCITE-5977
> URL: https://issues.apache.org/jira/browse/CALCITE-5977
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Priority: Major
>
> Perhaps I'm missing something, but this seems odd to me.
> If we want to get the value in Long format (i.e. milliseconds since
> 1970-01-01 00:00:00) of a TIMESTAMP / TIMESTAMP_WITH_LOCAL_TIME_ZONE
> RexLiteral, their code is exactly the same (even if they are in different
> "case" blocks):
> {code}
> public @Nullable Object getValue2() {
> ...
> switch (typeName) {
> ...
> case TIMESTAMP:
> case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
> return getValueAs(Long.class);
> ...
> }
> public @Nullable T getValueAs(Class clazz) {
> ...
> switch (typeName) {
> ...
> case TIMESTAMP:
> if (clazz == Long.class) {
> // Milliseconds since 1970-01-01 00:00:00
> return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
> }
> ...
> break;
> case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
> if (clazz == Long.class) {
> // Milliseconds since 1970-01-01 00:00:00
> return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
> }
> ...
> {code}
> In case of a TIMESTAMP_WITH_LOCAL_TIME_ZONE, shouldn't this code include some
> extra processing to "shift" the value with the proper offset / daylight
> savings?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (CALCITE-5971) Add the RelRule to rewrite the Sample as Filter
[
https://issues.apache.org/jira/browse/CALCITE-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen reassigned CALCITE-5971:
-
Assignee: LakeShen
> Add the RelRule to rewrite the Sample as Filter
> ---
>
> Key: CALCITE-5971
> URL: https://issues.apache.org/jira/browse/CALCITE-5971
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>
> For the following SQL:
> {code:java}
> select deptno from "scott".dept tablesample bernoulli(50); {code}
> We could rewrite it to:
> {code:java}
> select deptno from "scott".dept where rand() < 0.5; {code}
> The sql :
> {code:java}
> select deptno from "scott".dept tablesample bernoulli(50) REPEATABLE(10);
> {code}
> We could rewrite it to:
> {code:java}
> select deptno from "scott".dept where rand(10) < 0.5; {code}
> This rule only rewrite the tablesample bernoulli.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5888) Assertion error in aggregate
[
https://issues.apache.org/jira/browse/CALCITE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762339#comment-17762339
]
Petr Masopust edited comment on CALCITE-5888 at 9/6/23 11:42 AM:
-
Minimal reproducer is here: [https://github.com/pmsgd/calcite5888] Just run the
only one test. It ends with the same assertion error like in description.
What I think is important are duplicated aggregate calls like:
{{relBuilder.aggregate(groupKey,}}
{{ aggregateCall1, aggregateCall2,}}
{{ aggregateCall1, aggregateCall2,}}
{{)}}
The same problem is when aggregate calls have different aliases what is our
case.
was (Author: pmsgd):
Minimal reproducer is here: [https://github.com/pmsgd/calcite5888] Just run the
only one test. It ends with the same assertion error like in description.
What I think is important are duplicated aggregate calls like:
{{relBuilder.aggregate(groupKey,}}
{{ aggregateCall1, aggregateCall2,}}
{{ aggregateCall1, aggregateCall2,}}
{{)}}
> Assertion error in aggregate
>
>
> Key: CALCITE-5888
> URL: https://issues.apache.org/jira/browse/CALCITE-5888
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Petr Masopust
>Priority: Major
>
> We have {{relBuilder.aggregate(groupKey, aggregateCalls)}} in our code with
> values {{[AS($31, 'a_label_d_opp_stage_id_foodmart_bd4c9c91212f9f'),
> AS(CAST(FLOOR($34, FLAG(QUARTER))):TIMESTAMP(0),
> 'a_label_snapshot_timestamp_quarter_foodmart_89fe710c8628d9')]}} and
> {{[COUNT(CASE(AND(COALESCE($37, false), =($36, $13)), $15, null:NULL)),
> SUM(CASE(AND(COALESCE($37, false), =($36, $13)), $2, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL))].}}
> It works perfectly in version 1.34.0 but in 1.35.0 we got this exception:
> {{java.lang.AssertionError}}
> {{ at org.apache.calcite.rel.core.Aggregate.(Aggregate.java:175)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.(LogicalAggregate.java:72)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create_(LogicalAggregate.java:144)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create(LogicalAggregate.java:116)}}
> {{ at
> org.apache.calcite.rel.core.RelFactories$AggregateFactoryImpl.createAggregate(RelFactories.java:328)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2564)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2511)}}
> {{ at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:2348)}}
>
> I think it is either missing {{permute}} or assert should compare
> {{cardinality}} instead of {{length.}} Because it compares field index? to
> number of fields which looks like nonsense to me.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5980) QuidemTests are not effectively executed on Windows
[
https://issues.apache.org/jira/browse/CALCITE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-5980:
---
Description:
Discovered by accident on my Windows+IntelliJ environment while I was trying to
add new tests on a *.iq file. My new tests did not seem to be executed. I even
tried adding syntax errors on purpose into different iq files to force them to
fail, but the tests were still successful. The reason seems to be that, at
least on my environment (Windows), the test files do not execute any of their
statements. This seems caused by the changes introduced in CALCITE-5786.
While debugging, I found this line in QuidemTest.java (that aims to create the
inFile and outFile):
{code:java}
protected void checkRun(String path) throws Exception {
...
// e.g. path = "sql/agg.iq"
// inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// outDir = "/home/fred/calcite/core/build/quidem/test/sql"
// outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
outFile = replaceDir(inFile, "resources", "quidem");
...
}
{code}
But in my case it results on {*}both files being the same{*}, thus when the
outFile is created, it actually erases all the tests that were contained inside
the inFile, so the test runs nothing.
The reason for that is that the auxiliary method {{{}replaceDir{}}}:
{code:java}
private static File replaceDir(File file, String target, String replacement) {
return new File(
file.getAbsolutePath().replace(n2u('/' + target + '/'),
n2u('/' + replacement + '/')));
}
{code}
is trying to replace "/resources/" with "/quidem/" from the inFile absolute
path, but in my case this path does not contain the pattern to be replaced,
since it contains backslashes:
"C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
operation does nothing.
was:
Discovered by accident on my Windows+IntelliJ environment while I was trying to
add new tests on a *.iq file. My new tests did not seem to be executed. I even
tried adding syntax errors on purpose into different iq files to force them to
fail, but the tests were still successful. The reason seems to be that, at
least on my environment (Windows), the test files do not execute any of their
statements. This seems a consequence of CALCITE-5786.
While debugging, I found this line in QuidemTest.java (that aims to create the
inFile and outFile):
{code:java}
protected void checkRun(String path) throws Exception {
...
// e.g. path = "sql/agg.iq"
// inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// outDir = "/home/fred/calcite/core/build/quidem/test/sql"
// outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
outFile = replaceDir(inFile, "resources", "quidem");
...
}
{code}
But it results on {*}both files being the same{*}, thus when the outFile is
created, it actually erases all the tests that were contained inside the
inFile, so the test runs nothing.
The reason for that is that the auxiliary method {{{}replaceDir{}}}:
{code:java}
private static File replaceDir(File file, String target, String replacement) {
return new File(
file.getAbsolutePath().replace(n2u('/' + target + '/'),
n2u('/' + replacement + '/')));
}
{code}
is trying to replace "/resources/" with "/quidem/" from the inFile absolute
path, but in my case this path does not contain the pattern to be replaced,
since it contains backslashes:
"C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
operation does nothing.
> QuidemTests are not effectively executed on Windows
> ---
>
> Key: CALCITE-5980
> URL: https://issues.apache.org/jira/browse/CALCITE-5980
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.36.0
>
>
> Discovered by accident on my Windows+IntelliJ environment while I was trying
> to add new tests on a *.iq file. My new tests did not seem to be executed. I
> even tried adding syntax errors on purpose into different iq files to force
> them to fail, but the tests were still successful. The reason seems to be
> that, at least on my environment (Windows), the test files do not execute any
> of their statements. This seems caused by the changes introduced in
> CALCITE-5786.
> While debugging, I found this line in QuidemTest.java (that aims to create
> the inFile and outFile):
>
[jira] [Updated] (CALCITE-5980) QuidemTests are not effectively executed on Windows
[
https://issues.apache.org/jira/browse/CALCITE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-5980:
---
Summary: QuidemTests are not effectively executed on Windows (was:
QuidemTest are not effectively executed on Windows)
> QuidemTests are not effectively executed on Windows
> ---
>
> Key: CALCITE-5980
> URL: https://issues.apache.org/jira/browse/CALCITE-5980
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.36.0
>
>
> Discovered by accident on my Windows+IntelliJ environment while I was trying
> to add new tests on a *.iq file. My new tests did not seem to be executed. I
> even tried adding syntax errors on purpose into different iq files to force
> them to fail, but the tests were still successful. The reason seems to be
> that, at least on my environment (Windows), the test files do not execute any
> of their statements. This seems a consequence of CALCITE-5786.
> While debugging, I found this line in QuidemTest.java (that aims to create
> the inFile and outFile):
> {code:java}
> protected void checkRun(String path) throws Exception {
> ...
> // e.g. path = "sql/agg.iq"
> // inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // outDir = "/home/fred/calcite/core/build/quidem/test/sql"
> // outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
> inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
> outFile = replaceDir(inFile, "resources", "quidem");
> ...
> }
> {code}
> But it results on {*}both files being the same{*}, thus when the outFile is
> created, it actually erases all the tests that were contained inside the
> inFile, so the test runs nothing.
> The reason for that is that the auxiliary method {{{}replaceDir{}}}:
> {code:java}
> private static File replaceDir(File file, String target, String
> replacement) {
> return new File(
> file.getAbsolutePath().replace(n2u('/' + target + '/'),
> n2u('/' + replacement + '/')));
> }
> {code}
> is trying to replace "/resources/" with "/quidem/" from the inFile absolute
> path, but in my case this path does not contain the pattern to be replaced,
> since it contains backslashes:
> "C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
> operation does nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (CALCITE-5980) QuidemTest are not effectively executed on Windows
[
https://issues.apache.org/jira/browse/CALCITE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L reassigned CALCITE-5980:
--
Assignee: Ruben Q L
> QuidemTest are not effectively executed on Windows
> --
>
> Key: CALCITE-5980
> URL: https://issues.apache.org/jira/browse/CALCITE-5980
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.36.0
>
>
> Discovered by accident on my Windows+IntelliJ environment while I was trying
> to add new tests on a *.iq file. My new tests did not seem to be executed. I
> even tried adding syntax errors on purpose into different iq files to force
> them to fail, but the tests were still successful. The reason seems to be
> that, at least on my environment (Windows), the test files do not execute any
> of their statements. This seems a consequence of CALCITE-5786.
> While debugging, I found this line in QuidemTest.java (that aims to create
> the inFile and outFile):
> {code:java}
> protected void checkRun(String path) throws Exception {
> ...
> // e.g. path = "sql/agg.iq"
> // inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
> // outDir = "/home/fred/calcite/core/build/quidem/test/sql"
> // outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
> inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
> outFile = replaceDir(inFile, "resources", "quidem");
> ...
> }
> {code}
> But it results on {*}both files being the same{*}, thus when the outFile is
> created, it actually erases all the tests that were contained inside the
> inFile, so the test runs nothing.
> The reason for that is that the auxiliary method {{{}replaceDir{}}}:
> {code:java}
> private static File replaceDir(File file, String target, String
> replacement) {
> return new File(
> file.getAbsolutePath().replace(n2u('/' + target + '/'),
> n2u('/' + replacement + '/')));
> }
> {code}
> is trying to replace "/resources/" with "/quidem/" from the inFile absolute
> path, but in my case this path does not contain the pattern to be replaced,
> since it contains backslashes:
> "C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
> operation does nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5888) Assertion error in aggregate
[
https://issues.apache.org/jira/browse/CALCITE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762339#comment-17762339
]
Petr Masopust commented on CALCITE-5888:
Minimal reproducer is here: [https://github.com/pmsgd/calcite5888] Just run the
only one test. It ends with the same assertion error like in description.
What I think is important are duplicated aggregate calls like:
{{relBuilder.aggregate(groupKey,}}
{{ aggregateCall1, aggregateCall2,}}
{{ aggregateCall1, aggregateCall2,}}
{{)}}
> Assertion error in aggregate
>
>
> Key: CALCITE-5888
> URL: https://issues.apache.org/jira/browse/CALCITE-5888
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Petr Masopust
>Priority: Major
>
> We have {{relBuilder.aggregate(groupKey, aggregateCalls)}} in our code with
> values {{[AS($31, 'a_label_d_opp_stage_id_foodmart_bd4c9c91212f9f'),
> AS(CAST(FLOOR($34, FLAG(QUARTER))):TIMESTAMP(0),
> 'a_label_snapshot_timestamp_quarter_foodmart_89fe710c8628d9')]}} and
> {{[COUNT(CASE(AND(COALESCE($37, false), =($36, $13)), $15, null:NULL)),
> SUM(CASE(AND(COALESCE($37, false), =($36, $13)), $2, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL)),
> MAX(CASE(AND(COALESCE($37, false), =($36, $13)), 0, null:NULL))].}}
> It works perfectly in version 1.34.0 but in 1.35.0 we got this exception:
> {{java.lang.AssertionError}}
> {{ at org.apache.calcite.rel.core.Aggregate.(Aggregate.java:175)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.(LogicalAggregate.java:72)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create_(LogicalAggregate.java:144)}}
> {{ at
> org.apache.calcite.rel.logical.LogicalAggregate.create(LogicalAggregate.java:116)}}
> {{ at
> org.apache.calcite.rel.core.RelFactories$AggregateFactoryImpl.createAggregate(RelFactories.java:328)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2564)}}
> {{ at
> org.apache.calcite.tools.RelBuilder.aggregate_(RelBuilder.java:2511)}}
> {{ at org.apache.calcite.tools.RelBuilder.aggregate(RelBuilder.java:2348)}}
>
> I think it is either missing {{permute}} or assert should compare
> {{cardinality}} instead of {{length.}} Because it compares field index? to
> number of fields which looks like nonsense to me.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5980) QuidemTest are not effectively executed on Windows
Ruben Q L created CALCITE-5980:
--
Summary: QuidemTest are not effectively executed on Windows
Key: CALCITE-5980
URL: https://issues.apache.org/jira/browse/CALCITE-5980
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Ruben Q L
Fix For: 1.36.0
Discovered by accident on my Windows+IntelliJ environment while I was trying to
add new tests on a *.iq file. My new tests did not seem to be executed. I even
tried adding syntax errors on purpose into different iq files to force them to
fail, but the tests were still successful. The reason seems to be that, at
least on my environment (Windows), the test files do not execute any of their
statements. This seems a consequence of CALCITE-5786.
While debugging, I found this line in QuidemTest.java (that aims to create the
inFile and outFile):
{code:java}
protected void checkRun(String path) throws Exception {
...
// e.g. path = "sql/agg.iq"
// inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
// outDir = "/home/fred/calcite/core/build/quidem/test/sql"
// outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
outFile = replaceDir(inFile, "resources", "quidem");
...
}
{code}
But it results on {*}both files being the same{*}, thus when the outFile is
created, it actually erases all the tests that were contained inside the
inFile, so the test runs nothing.
The reason for that is that the auxiliary method {{{}replaceDir{}}}:
{code:java}
private static File replaceDir(File file, String target, String replacement) {
return new File(
file.getAbsolutePath().replace(n2u('/' + target + '/'),
n2u('/' + replacement + '/')));
}
{code}
is trying to replace "/resources/" with "/quidem/" from the inFile absolute
path, but in my case this path does not contain the pattern to be replaced,
since it contains backslashes:
"C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement
operation does nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5967) UnsupportedOperationException while implementing a call that requires a special collator
[
https://issues.apache.org/jira/browse/CALCITE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-5967.
Resolution: Fixed
Fixed via
[{{164ff0a}}|https://github.com/apache/calcite/commit/164ff0a27e243850d294908dc5cff90760d0a35a]
> UnsupportedOperationException while implementing a call that requires a
> special collator
>
>
> Key: CALCITE-5967
> URL: https://issues.apache.org/jira/browse/CALCITE-5967
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> Regression introduced by a minor change within CALCITE-5914, detected while
> testing a downstream project with the latest Calcite main.
> CALCITE-5914 (see
> [2a96512c|https://github.com/apache/calcite/commit/2a96512c352bda4a5d9c0c80730f5c115ac363d6])
> introduced this apparently innocuous change in
> {{RexImpTable#AbstractRexCallImplementor#unboxIfNecessary}}:
> {code}
> // old
> argValueList.stream()
> .map(AbstractRexCallImplementor::unboxExpression)
> .collect(Collectors.toList());
> =>
> // new
> Util.transform(argValueList,
> AbstractRexCallImplementor::unboxExpression);
> {code}
> Both expressions seem equivalent, however there is a subtle difference: the
> old one returns an {{ArrayList}} (where we can add new elements); whereas the
> new one returns a {{TransformingList}} that extends {{AbstractList}} and that
> does not support {{List#add}}.
> After calling {{unboxIfNecessary}}, we might need to modify the argument list
> if we need a special collator to perform the operation:
> {code}
> private ParameterExpression genValueStatement(...) {
> ...
> optimizedArgValueList = unboxIfNecessary(optimizedArgValueList);
> final Expression callValue =
> implementSafe(translator, call, optimizedArgValueList);
> ...
> }
> @Override Expression implementSafe(...) {
> ...
> final Expression fieldComparator =
> generateCollatorExpression(relDataType0.getCollation());
> if (fieldComparator != null) {
> argValueList.add(fieldComparator); // <---
> UnsupportedOperationException!
> }
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5952) SemiJoinJoinTransposeRule should check if JoinType supports pushing predicates into its inputs
[
https://issues.apache.org/jira/browse/CALCITE-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-5952.
Resolution: Fixed
Fixed via
[{{d667123}}|https://github.com/apache/calcite/commit/d667123585bf518edd6a9bf93e23c1785fe03376]
Thanks [~lchistov1987] for the patch!{{{}{}}}
> SemiJoinJoinTransposeRule should check if JoinType supports pushing
> predicates into its inputs
> --
>
> Key: CALCITE-5952
> URL: https://issues.apache.org/jira/browse/CALCITE-5952
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.35.0
>Reporter: Leonid Chistov
>Assignee: Leonid Chistov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> The following test will fail if added to RelOptRulesTest.java
> {code:java}
> @Test void testCanNotPushSemiJoinToRightJoinBranch() {
> final Function relFn = b -> b
> .scan("EMP")
> .scan("DEPT")
> .join(JoinRelType.LEFT,
> b.equals(
> b.field(2, 0, "DEPTNO"),
> b.field(2, 1, "DEPTNO"))
> )
> .scan("BONUS")
> // semi join only relates to RHS fields of left join
> .semiJoin(b.equals(
> b.field(2, 0, "DNAME"),
> b.field(2, 1, "JOB")))
> .build();
> relFn(relFn).withRule(CoreRules.SEMI_JOIN_JOIN_TRANSPOSE).checkUnchanged();
> } {code}
> Produced plan will look like:
> {code:java}
> LogicalJoin(condition=[=($7, $8)], joinType=[left])
> LogicalTableScan(table=[[scott, EMP]])
> LogicalJoin(condition=[=($1, $4)], joinType=[semi])
> LogicalTableScan(table=[[scott, DEPT]])
> LogicalTableScan(table=[[scott, BONUS]]) {code}
> Which is different from the original plan:
> {code:java}
> LogicalJoin(condition=[=($9, $12)], joinType=[semi])
> LogicalJoin(condition=[=($7, $8)], joinType=[left])
> LogicalTableScan(table=[[scott, EMP]])
> LogicalTableScan(table=[[scott, DEPT]])
> LogicalTableScan(table=[[scott, BONUS]]) {code}
> This is not correct - in general case it is not correct to push semi-join to
> right side of left-join.
> The reason is the following:
> Consider rows from *EMP* that have no matching rows in {*}DEPT{*}. These rows
> will have *nulls* for *DEPT* columns in the result of left-join and they will
> be rejected by the top semi-join.
> But if we push semi-join to RHS of left-join, we are going to see rows from
> *EMP* with *nulls* on the *DEPT* side in the final result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5952) SemiJoinJoinTransposeRule should check if JoinType supports pushing predicates into its inputs
[
https://issues.apache.org/jira/browse/CALCITE-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-5952:
---
Fix Version/s: 1.36.0
> SemiJoinJoinTransposeRule should check if JoinType supports pushing
> predicates into its inputs
> --
>
> Key: CALCITE-5952
> URL: https://issues.apache.org/jira/browse/CALCITE-5952
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.35.0
>Reporter: Leonid Chistov
>Assignee: Leonid Chistov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> The following test will fail if added to RelOptRulesTest.java
> {code:java}
> @Test void testCanNotPushSemiJoinToRightJoinBranch() {
> final Function relFn = b -> b
> .scan("EMP")
> .scan("DEPT")
> .join(JoinRelType.LEFT,
> b.equals(
> b.field(2, 0, "DEPTNO"),
> b.field(2, 1, "DEPTNO"))
> )
> .scan("BONUS")
> // semi join only relates to RHS fields of left join
> .semiJoin(b.equals(
> b.field(2, 0, "DNAME"),
> b.field(2, 1, "JOB")))
> .build();
> relFn(relFn).withRule(CoreRules.SEMI_JOIN_JOIN_TRANSPOSE).checkUnchanged();
> } {code}
> Produced plan will look like:
> {code:java}
> LogicalJoin(condition=[=($7, $8)], joinType=[left])
> LogicalTableScan(table=[[scott, EMP]])
> LogicalJoin(condition=[=($1, $4)], joinType=[semi])
> LogicalTableScan(table=[[scott, DEPT]])
> LogicalTableScan(table=[[scott, BONUS]]) {code}
> Which is different from the original plan:
> {code:java}
> LogicalJoin(condition=[=($9, $12)], joinType=[semi])
> LogicalJoin(condition=[=($7, $8)], joinType=[left])
> LogicalTableScan(table=[[scott, EMP]])
> LogicalTableScan(table=[[scott, DEPT]])
> LogicalTableScan(table=[[scott, BONUS]]) {code}
> This is not correct - in general case it is not correct to push semi-join to
> right side of left-join.
> The reason is the following:
> Consider rows from *EMP* that have no matching rows in {*}DEPT{*}. These rows
> will have *nulls* for *DEPT* columns in the result of left-join and they will
> be rejected by the top semi-join.
> But if we push semi-join to RHS of left-join, we are going to see rows from
> *EMP* with *nulls* on the *DEPT* side in the final result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5967) UnsupportedOperationException while implementing a call that requires a special collator
[
https://issues.apache.org/jira/browse/CALCITE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762296#comment-17762296
]
Ruben Q L commented on CALCITE-5967:
Sure, I'll apply those suggestions before merging the patch.
> UnsupportedOperationException while implementing a call that requires a
> special collator
>
>
> Key: CALCITE-5967
> URL: https://issues.apache.org/jira/browse/CALCITE-5967
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> Regression introduced by a minor change within CALCITE-5914, detected while
> testing a downstream project with the latest Calcite main.
> CALCITE-5914 (see
> [2a96512c|https://github.com/apache/calcite/commit/2a96512c352bda4a5d9c0c80730f5c115ac363d6])
> introduced this apparently innocuous change in
> {{RexImpTable#AbstractRexCallImplementor#unboxIfNecessary}}:
> {code}
> // old
> argValueList.stream()
> .map(AbstractRexCallImplementor::unboxExpression)
> .collect(Collectors.toList());
> =>
> // new
> Util.transform(argValueList,
> AbstractRexCallImplementor::unboxExpression);
> {code}
> Both expressions seem equivalent, however there is a subtle difference: the
> old one returns an {{ArrayList}} (where we can add new elements); whereas the
> new one returns a {{TransformingList}} that extends {{AbstractList}} and that
> does not support {{List#add}}.
> After calling {{unboxIfNecessary}}, we might need to modify the argument list
> if we need a special collator to perform the operation:
> {code}
> private ParameterExpression genValueStatement(...) {
> ...
> optimizedArgValueList = unboxIfNecessary(optimizedArgValueList);
> final Expression callValue =
> implementSafe(translator, call, optimizedArgValueList);
> ...
> }
> @Override Expression implementSafe(...) {
> ...
> final Expression fieldComparator =
> generateCollatorExpression(relDataType0.getCollation());
> if (fieldComparator != null) {
> argValueList.add(fieldComparator); // <---
> UnsupportedOperationException!
> }
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5905) Documentation for CREATE TYPE is incorrect
[
https://issues.apache.org/jira/browse/CALCITE-5905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benchao Li resolved CALCITE-5905.
-
Assignee: Mihai Budiu
Resolution: Fixed
Merged via
[https://github.com/apache/calcite/commit/b57f2d2cb0fa1227ce17e4be872982c05f260249]
[~mbudiu] Thanks for the PR!
> Documentation for CREATE TYPE is incorrect
> --
>
> Key: CALCITE-5905
> URL: https://issues.apache.org/jira/browse/CALCITE-5905
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Assignee: Mihai Budiu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> I have tried to run the example CREATE TYPE statements from this page
> https://calcite.apache.org/docs/reference.html#declaring-objects-for-user-defined-types
> through the compiler:
> {code:sql}
> CREATE TYPE address_typ AS OBJECT (
>street VARCHAR2(30),
>cityVARCHAR2(20),
>state CHAR(2),
>postal_code VARCHAR2(6));
> {code}
> Calcite complains in two places: OBJECT and VARCHAR2.
> The following compiles fine:
> {code:sql}
> CREATE TYPE address_typ AS (
>street VARCHAR(30),
>cityVARCHAR(20),
>state CHAR(2),
>postal_code VARCHAR(6));
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
