[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if they are functionally dependent by other sort fields
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752288#comment-17752288
]
JingDas commented on CALCITE-5894:
--
It's good to do this work together. Maybe I can research and work out the
functional dependency architecture and some implementation firstly.
Contribute the init code to satisfy the minimize need of
`SortRemoveRedundantRule`, then split it to sub tasks for completing functional
dependency together in the future.
> Add SortRemoveRedundantRule to remove redundant sort fields if they are
> functionally dependent by other sort fields
> ---
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5904) The result should be empty list when table sample rate is 0
[
https://issues.apache.org/jira/browse/CALCITE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752268#comment-17752268
]
LakeShen commented on CALCITE-5904:
---
Hi [~jiajunbernoulli] ,yes, I will create new PR later,thanks for your review.
> The result should be empty list when table sample rate is 0
> ---
>
> Key: CALCITE-5904
> URL: https://issues.apache.org/jira/browse/CALCITE-5904
> Project: Calcite
> Issue Type: Sub-task
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> For The sql:
> {code:java}
> select * from emp tablesample bernoulli(0) where empno > 5
> {code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[>($0, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I think the correct result of above SQL is empty.
> But its result is the following SQL result:
> {code:java}
> select * from emp where empno > 5
> {code}
> It's an incorrect result
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5896) Document TLS client certificate JDBC URL options
[
https://issues.apache.org/jira/browse/CALCITE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5896:
-
Summary: Document TLS client certificate JDBC URL options (was: Document
client certificate JDBC URL options)
> Document TLS client certificate JDBC URL options
>
>
> Key: CALCITE-5896
> URL: https://issues.apache.org/jira/browse/CALCITE-5896
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Priority: Minor
> Labels: doc
>
> The client reference does not include the {_}keystore{_}, _keystore_password_
> and _key_password_ options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5904) The result should be empty list when table sample rate is 0
[
https://issues.apache.org/jira/browse/CALCITE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752264#comment-17752264
]
Jiajun Xie commented on CALCITE-5904:
-
You closed it, do you plan to create new PR?
> The result should be empty list when table sample rate is 0
> ---
>
> Key: CALCITE-5904
> URL: https://issues.apache.org/jira/browse/CALCITE-5904
> Project: Calcite
> Issue Type: Sub-task
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> For The sql:
> {code:java}
> select * from emp tablesample bernoulli(0) where empno > 5
> {code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[>($0, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I think the correct result of above SQL is empty.
> But its result is the following SQL result:
> {code:java}
> select * from emp where empno > 5
> {code}
> It's an incorrect result
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752261#comment-17752261
]
LakeShen commented on CALCITE-5913:
---
Maybe Functional dependencies is challenging to implement,I think we could
create a umbrella jira to track FunctionalDependency feature,at the same time,
under this umbrella jira, create multiple small task jiras,step by step into
calcite.It also makes it easier to review.
WDYT?
> Support to get functional dependency metadata in RelMetadataQuery
> -
>
> Key: CALCITE-5913
> URL: https://issues.apache.org/jira/browse/CALCITE-5913
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>
> Functional dependency analysis can be applied to various problems in query
> optimization:
> selectivity estimation, estimation of (intermediate) result sizes, *order
> optimization*
> *(in particular sort avoidance),* cost estimation, and various problems in
> the area of semantic query optimization, as said in the book《[Exploiting
> Functional Dependence in Query
> Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
>
> In calcite, it may be metadata that something like 'FunctionalDependency'
> BuiltInMetadata as following:
> {code:java}
> public abstract class BuiltInMetadata {
>// ...
>public interface FunctionalDependency extends Metadata {
>/** Returns whether column is functionally dependent on columns. */
>Boolean functionallyDetermine(ImmutableBitSet columns, int column);
>}
> } {code}
>
> As the above book said, functional dependency analysis is a valuable and
> challenging work. I think support order optimization
> (in particular sort avoidance) by the relevant functional dependency metadata
> firstly, and then get complete functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if they are functionally dependent by other sort fields
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752258#comment-17752258
]
LakeShen commented on CALCITE-5894:
---
FunctionalDependency is a very important optimization for an optimizer, and I
think it would be an exciting feature if calcite could support functional
dependencies.
Functional dependency can be used in many aspects of optimization. Including
order by, aggregate,selectivity estimation and so on.
We could create a umbrella jira to track FunctionalDependency feature,at the
same time, under this umbrella jira, create multiple small task jiras, so that
others can also contribute to the development.I would like to participate in
the development of this Feature.
[~julianhyde] [~libenchao] [~jingda] WDYT?
> Add SortRemoveRedundantRule to remove redundant sort fields if they are
> functionally dependent by other sort fields
> ---
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5844) Make RelMdColumnOrigins to be extendable for getting columnOrigins from viewTable
[
https://issues.apache.org/jira/browse/CALCITE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas closed CALCITE-5844.
Resolution: Not A Problem
> Make RelMdColumnOrigins to be extendable for getting columnOrigins from
> viewTable
> -
>
> Key: CALCITE-5844
> URL: https://issues.apache.org/jira/browse/CALCITE-5844
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: JingDas
>Priority: Minor
>
> In our scene, Getting field from view is expected when call
> RelMdColumnOrigins#getColumnOrigins.
> So I want to define customized RelMdColumnOrigins extend *RelMdColumnOrigins*
> which can *getColumnOrigins* from viewTable.
> Then I add customized RelMdColumnOrigins to {*}ChainedRelMetadataProvider{*},
> keep the sequence before DefaultRelMetadataProvider.INSTANCE.
> After doing all above, I can get columnOrigins from viewTable.
>
> But I found the private modifier constructor in class RelMdColumnOrigins(also
> such as
> RelMdCollation, RelMdColumnUniqueness, RelMdDistribution,
> RelMdExplainVisibility,
> RelMdPercentageOriginalRows, RelMdPopulationSize, RelMdUniqueKeys
> ),
> and protected modifier constructors(such as RelMdDistinctRowCount,
> RelMdExpressionLineage, RelMdLowerBoundCost,
> RelMdMemory, RelMdParallelism, RelMdSelectivity, RelMdSize,
> RelMdTableReferences
> )
> and public modifier constructors(such as RelMdAllPredicates,
> RelMdMaxRowCount, RelMdMinRowCount, RelMdNodeTypes,
> RelMdPredicates, RelMdRowCount,
> )
> I haven't had a chance to extend this class to add my custom logic because
> the constructor modifier is private.
> These "RelMd"prefixed class are default implementation, They should can be
> extended for user to add
> customed logic.
> For extensibility, I think this modifier maybe protected for these
> "RelMd"prefixed class whose modifier is private.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5844) Make RelMdColumnOrigins to be extendable for getting columnOrigins from viewTable
[
https://issues.apache.org/jira/browse/CALCITE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752242#comment-17752242
]
JingDas commented on CALCITE-5844:
--
Current capacity already support to get metadata from custom rel node by
reflection. Thanks for your reply. [~julianhyde] [~jiajunbernoulli]
> Make RelMdColumnOrigins to be extendable for getting columnOrigins from
> viewTable
> -
>
> Key: CALCITE-5844
> URL: https://issues.apache.org/jira/browse/CALCITE-5844
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: JingDas
>Priority: Minor
>
> In our scene, Getting field from view is expected when call
> RelMdColumnOrigins#getColumnOrigins.
> So I want to define customized RelMdColumnOrigins extend *RelMdColumnOrigins*
> which can *getColumnOrigins* from viewTable.
> Then I add customized RelMdColumnOrigins to {*}ChainedRelMetadataProvider{*},
> keep the sequence before DefaultRelMetadataProvider.INSTANCE.
> After doing all above, I can get columnOrigins from viewTable.
>
> But I found the private modifier constructor in class RelMdColumnOrigins(also
> such as
> RelMdCollation, RelMdColumnUniqueness, RelMdDistribution,
> RelMdExplainVisibility,
> RelMdPercentageOriginalRows, RelMdPopulationSize, RelMdUniqueKeys
> ),
> and protected modifier constructors(such as RelMdDistinctRowCount,
> RelMdExpressionLineage, RelMdLowerBoundCost,
> RelMdMemory, RelMdParallelism, RelMdSelectivity, RelMdSize,
> RelMdTableReferences
> )
> and public modifier constructors(such as RelMdAllPredicates,
> RelMdMaxRowCount, RelMdMinRowCount, RelMdNodeTypes,
> RelMdPredicates, RelMdRowCount,
> )
> I haven't had a chance to extend this class to add my custom logic because
> the constructor modifier is private.
> These "RelMd"prefixed class are default implementation, They should can be
> extended for user to add
> customed logic.
> For extensibility, I think this modifier maybe protected for these
> "RelMd"prefixed class whose modifier is private.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas updated CALCITE-5913:
-
Description:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the book《[Exploiting Functional
Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be metadata that something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the above book said, functional dependency analysis is a valuable and
challenging work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then get complete functional dependency function step by step.
was:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the book《[Exploiting Functional
Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then get complete functional dependency function step by step.
> Support to get functional dependency metadata in RelMetadataQuery
> -
>
> Key: CALCITE-5913
> URL: https://issues.apache.org/jira/browse/CALCITE-5913
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>
> Functional dependency analysis can be applied to various problems in query
> optimization:
> selectivity estimation, estimation of (intermediate) result sizes, *order
> optimization*
> *(in particular sort avoidance),* cost estimation, and various problems in
> the area of semantic query optimization, as said in the book《[Exploiting
> Functional Dependence in Query
> Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
>
> In calcite, it may be metadata that something like 'FunctionalDependency'
> BuiltInMetadata as following:
> {code:java}
> public abstract class BuiltInMetadata {
>// ...
>public interface FunctionalDependency extends Metadata {
>/** Returns whether column is functionally dependent on columns. */
>Boolean functionallyDetermine(ImmutableBitSet columns, int column);
>}
> } {code}
>
> As the above book said, functional dependency analysis is a valuable and
> challenging work. I think support order optimization
> (in particular sort avoidance) by the relevant functional dependency metadata
> firstly, and then get complete functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas updated CALCITE-5913:
-
Description:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the book《[Exploiting Functional
Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then get complete functional dependency function step by step.
was:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the book《[Exploiting Functional
Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
> Support to get functional dependency metadata in RelMetadataQuery
> -
>
> Key: CALCITE-5913
> URL: https://issues.apache.org/jira/browse/CALCITE-5913
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>
> Functional dependency analysis can be applied to various problems in query
> optimization:
> selectivity estimation, estimation of (intermediate) result sizes, *order
> optimization*
> *(in particular sort avoidance),* cost estimation, and various problems in
> the area of semantic query optimization, as said in the book《[Exploiting
> Functional Dependence in Query
> Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
>
> In calcite, it may be meta something like 'FunctionalDependency'
> BuiltInMetadata as following:
> {code:java}
> public abstract class BuiltInMetadata {
>// ...
>public interface FunctionalDependency extends Metadata {
>/** Returns whether column is functionally dependent on columns. */
>Boolean functionallyDetermine(ImmutableBitSet columns, int column);
>}
> } {code}
>
> As the book said, functional dependency analysis is a valuable and
> challenging work. I think support order optimization
> (in particular sort avoidance) by the relevant functional dependency metadata
> firstly, and then get complete functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if they are functionally dependent by other sort fields
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752235#comment-17752235
]
JingDas edited comment on CALCITE-5894 at 8/9/23 3:09 AM:
--
As said in the book [《Exploiting Functional Dependence in Query
Optimization》|[https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]]
Functional dependency analysis is a valuable and challenging work. I think
support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and the get complete functional dependency function step by step.
What do you think?
was (Author: JIRAUSER292370):
As said in the book [《Exploiting Functional Dependence in Query
Optimization》|[https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf],]
Functional dependency analysis is a valuable and challenging work. I think
support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and the get complete functional dependency function step by step.
What do you think?
> Add SortRemoveRedundantRule to remove redundant sort fields if they are
> functionally dependent by other sort fields
> ---
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if they are functionally dependent by other sort fields
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752235#comment-17752235
]
JingDas commented on CALCITE-5894:
--
As said in the book [《Exploiting Functional Dependence in Query
Optimization》|[https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf],]
Functional dependency analysis is a valuable and challenging work. I think
support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and the get complete functional dependency function step by step.
What do you think?
> Add SortRemoveRedundantRule to remove redundant sort fields if they are
> functionally dependent by other sort fields
> ---
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas updated CALCITE-5913:
-
Description:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the book《[Exploiting Functional
Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
was:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the paper《[Exploiting
Functional Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
> Support to get functional dependency metadata in RelMetadataQuery
> -
>
> Key: CALCITE-5913
> URL: https://issues.apache.org/jira/browse/CALCITE-5913
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>
> Functional dependency analysis can be applied to various problems in query
> optimization:
> selectivity estimation, estimation of (intermediate) result sizes, *order
> optimization*
> *(in particular sort avoidance),* cost estimation, and various problems in
> the area of semantic query optimization, as said in the book《[Exploiting
> Functional Dependence in Query
> Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
>
> In calcite, it may be meta something like 'FunctionalDependency'
> BuiltInMetadata as following:
> {code:java}
> public abstract class BuiltInMetadata {
>// ...
>public interface FunctionalDependency extends Metadata {
>/** Returns whether column is functionally dependent on columns. */
>Boolean functionallyDetermine(ImmutableBitSet columns, int column);
>}
> } {code}
>
> As the book said, functional dependency analysis is a valuable and
> challenging work. I think support order optimization
> (in particular sort avoidance) by the relevant functional dependency metadata
> firstly, and then complete to get functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas updated CALCITE-5913:
-
Description:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the paper《[Exploiting
Functional Dependence in Query
Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
was:
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the paper《Exploiting Functional
Dependence in Query Optimization》
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
> Support to get functional dependency metadata in RelMetadataQuery
> -
>
> Key: CALCITE-5913
> URL: https://issues.apache.org/jira/browse/CALCITE-5913
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>
> Functional dependency analysis can be applied to various problems in query
> optimization:
> selectivity estimation, estimation of (intermediate) result sizes, *order
> optimization*
> *(in particular sort avoidance),* cost estimation, and various problems in
> the area of semantic query optimization, as said in the paper《[Exploiting
> Functional Dependence in Query
> Optimization》|https://cs.uwaterloo.ca/research/tr/2000/11/CS-2000-11.thesis.pdf]
>
> In calcite, it may be meta something like 'FunctionalDependency'
> BuiltInMetadata as following:
> {code:java}
> public abstract class BuiltInMetadata {
>// ...
>public interface FunctionalDependency extends Metadata {
>/** Returns whether column is functionally dependent on columns. */
>Boolean functionallyDetermine(ImmutableBitSet columns, int column);
>}
> } {code}
>
> As the book said, functional dependency analysis is a valuable and
> challenging work. I think support order optimization
> (in particular sort avoidance) by the relevant functional dependency metadata
> firstly, and then complete to get functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5913) Support to get functional dependency metadata in RelMetadataQuery
JingDas created CALCITE-5913:
Summary: Support to get functional dependency metadata in
RelMetadataQuery
Key: CALCITE-5913
URL: https://issues.apache.org/jira/browse/CALCITE-5913
Project: Calcite
Issue Type: New Feature
Reporter: JingDas
Assignee: JingDas
Functional dependency analysis can be applied to various problems in query
optimization:
selectivity estimation, estimation of (intermediate) result sizes, *order
optimization*
*(in particular sort avoidance),* cost estimation, and various problems in the
area of semantic query optimization, as said in the paper《Exploiting Functional
Dependence in Query Optimization》
In calcite, it may be meta something like 'FunctionalDependency'
BuiltInMetadata as following:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
} {code}
As the book said, functional dependency analysis is a valuable and challenging
work. I think support order optimization
(in particular sort avoidance) by the relevant functional dependency metadata
firstly, and then complete to get functional dependency function step by step.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if they are functionally dependent by other sort fields
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JingDas updated CALCITE-5894:
-
Summary: Add SortRemoveRedundantRule to remove redundant sort fields if
they are functionally dependent by other sort fields (was: Add
SortRemoveRedundantRule to remove redundant sort fields if sort fields contains
unique key)
> Add SortRemoveRedundantRule to remove redundant sort fields if they are
> functionally dependent by other sort fields
> ---
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if sort fields contains unique key
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752234#comment-17752234
]
JingDas commented on CALCITE-5894:
--
[~julianhyde] I see your point and aggre with you,
`BuiltInMetadate.FunctionalDependency` is a wonderful, basic and general
ability for this. "Support to get functional dependency metadata in
RelMetadataQuery"
is another Jira case. I'll go back and finish this jira when anthor
BuiltInMetadate.FunctionalDependency Jira is ready. And I think it would be
better to change this Jira case title to "Add SortRemoveRedundantRule to remove
redundant sort fields if they are functionally dependent by other sort fields".
> Add SortRemoveRedundantRule to remove redundant sort fields if sort fields
> contains unique key
> --
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5912) Add an implementation of TABLESAMPLE in enumerable mode.
[
https://issues.apache.org/jira/browse/CALCITE-5912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5912:
--
Description:
In Calcite, the TableSample SQL statement will be converted to Sample RelNode
plan tree,for example:
{code:java}
select * from emp tablesample bernoulli(10){code}
The plan is :
{code:java}
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
Sample(mode=[bernoulli], rate=[10.0], repeatableSeed=[-])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
Currently in enumerable mode, we don't have EnumerableSample, So we can't add
TableSample quidem tests (such as in misc.iq).
After this jira,we could add the quidem test for TableSample(such as
tablesample.iq),although TableSample's results are inconclusive when the 0 <
sample rate < 100, but TampeSample(0) is certain result.
was:
In Calcite, the TableSample SQL statement will be converted to Sample RelNode
plan tree,for example:
{code:java}
select * from emp tablesample bernoulli(10){code}
The plan is :
{code:java}
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
Sample(mode=[bernoulli], rate=[10.0], repeatableSeed=[-])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
Currently in enumerable mode, we don't have EnumerableSample, So we can't add
TableSample quidem tests (such as in misc.iq).
After this jira,we could add the quidem test for TableSample(such as
tablesample.iq),although TableSample's results are inconclusive when the 0 <
sample rate < 100, but TampeSample(0) is certain result.
> Add an implementation of TABLESAMPLE in enumerable mode.
>
>
> Key: CALCITE-5912
> URL: https://issues.apache.org/jira/browse/CALCITE-5912
> Project: Calcite
> Issue Type: New Feature
>Reporter: LakeShen
>Priority: Major
>
> In Calcite, the TableSample SQL statement will be converted to Sample RelNode
> plan tree,for example:
> {code:java}
> select * from emp tablesample bernoulli(10){code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> Sample(mode=[bernoulli], rate=[10.0], repeatableSeed=[-])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
> Currently in enumerable mode, we don't have EnumerableSample, So we can't add
> TableSample quidem tests (such as in misc.iq).
> After this jira,we could add the quidem test for TableSample(such as
> tablesample.iq),although TableSample's results are inconclusive when the 0 <
> sample rate < 100, but TampeSample(0) is certain result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5912) Add an implementation of TABLESAMPLE in enumerable mode.
[
https://issues.apache.org/jira/browse/CALCITE-5912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5912:
--
Description:
In Calcite, the TableSample SQL statement will be converted to Sample RelNode
plan tree,for example:
{code:java}
select * from emp tablesample bernoulli(10){code}
The plan is :
{code:java}
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
Sample(mode=[bernoulli], rate=[10.0], repeatableSeed=[-])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
Currently in enumerable mode, we don't have EnumerableSample, So we can't add
TableSample quidem tests (such as in misc.iq).
After this jira,we could add the quidem test for TableSample(such as
tablesample.iq),although TableSample's results are inconclusive when the 0 <
sample rate < 100, but TampeSample(0) is certain result.
was:
In Calcite, the TableSample SQL statement will be converted to Sample RelNode
plan tree, currently in enumerable mode, we don't have EnumerableSample, So we
can't add TableSample quidem tests (such as in misc.iq).
After this jira,we could add the quidem test for TableSample(such as
tablesample.iq),although TableSample's results are inconclusive when the 0 <
sample rate < 100, but TampeSample(0) is certain result.
> Add an implementation of TABLESAMPLE in enumerable mode.
>
>
> Key: CALCITE-5912
> URL: https://issues.apache.org/jira/browse/CALCITE-5912
> Project: Calcite
> Issue Type: New Feature
>Reporter: LakeShen
>Priority: Major
>
> In Calcite, the TableSample SQL statement will be converted to Sample RelNode
> plan tree,for example:
> {code:java}
> select * from emp tablesample bernoulli(10){code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> Sample(mode=[bernoulli], rate=[10.0], repeatableSeed=[-])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
> Currently in enumerable mode, we don't have EnumerableSample, So we can't add
> TableSample quidem tests (such as in misc.iq).
>
> After this jira,we could add the quidem test for TableSample(such as
> tablesample.iq),although TableSample's results are inconclusive when the 0 <
> sample rate < 100, but TampeSample(0) is certain result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5912) Add an implementation of TABLESAMPLE in enumerable mode.
[ https://issues.apache.org/jira/browse/CALCITE-5912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LakeShen updated CALCITE-5912: -- Description: In Calcite, the TableSample SQL statement will be converted to Sample RelNode plan tree, currently in enumerable mode, we don't have EnumerableSample, So we can't add TableSample quidem tests (such as in misc.iq). After this jira,we could add the quidem test for TableSample(such as tablesample.iq),although TableSample's results are inconclusive when the 0 < sample rate < 100, but TampeSample(0) is certain result. > Add an implementation of TABLESAMPLE in enumerable mode. > > > Key: CALCITE-5912 > URL: https://issues.apache.org/jira/browse/CALCITE-5912 > Project: Calcite > Issue Type: New Feature >Reporter: LakeShen >Priority: Major > > In Calcite, the TableSample SQL statement will be converted to Sample RelNode > plan tree, currently in enumerable mode, we don't have EnumerableSample, So > we can't add TableSample quidem tests (such as in misc.iq). > After this jira,we could add the quidem test for TableSample(such as > tablesample.iq),although TableSample's results are inconclusive when the 0 < > sample rate < 100, but TampeSample(0) is certain result. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5912) Add an implementation of TABLESAMPLE in enumerable mode.
LakeShen created CALCITE-5912: - Summary: Add an implementation of TABLESAMPLE in enumerable mode. Key: CALCITE-5912 URL: https://issues.apache.org/jira/browse/CALCITE-5912 Project: Calcite Issue Type: New Feature Reporter: LakeShen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5895) The result of table sample rate 0 is incorrect result and should return the empty result
[
https://issues.apache.org/jira/browse/CALCITE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752224#comment-17752224
]
LakeShen commented on CALCITE-5895:
---
Hi,[~julianhyde] ,thank you very much for your patient reply,I have got your
point.You provided a lot of information for me.
I have the same thought with you for add implementation of {{TABLESAMPLE}} in
enumerable mode,because I was going to add the TABlESAMPLE(0) test case in
misc.iq, but it says that Sample doesn't have an implementation,we could track
the add implementation of TABLESAMPLE in enumerable mode by creating a JIRA.
> The result of table sample rate 0 is incorrect result and should return the
> empty result
> -
>
> Key: CALCITE-5895
> URL: https://issues.apache.org/jira/browse/CALCITE-5895
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-05-22-45-02-513.png
>
>
> For The sql:
> {code:java}
> select * from emp tablesample bernoulli(0) where empno > 5
> {code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[>($0, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I think the correct result of above SQL is empty.
> But its result is the following SQL result:
> {code:java}
> select * from emp where empno > 5
> {code}
> It's an incorrect result
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5909) Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE but passes when run from the command line
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752223#comment-17752223
]
Julian Hyde commented on CALCITE-5909:
--
I've reopened it, because a flaky test is a problem. It's not high priority,
but if someone can find a fix - as I and [~taoran] said above, we suspect
caching - it would be appreciated.
> Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE
> but passes when run from the command line
> --
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Minor
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE but passes when run from the command line
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-5909:
-
Summary: Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails
in the IDE but passes when run from the command line (was: SqlParserTest has a
failed unit test, but the Calcite pipeline passed all of them)
> Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE
> but passes when run from the command line
> --
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE but passes when run from the command line
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-5909:
-
Priority: Minor (was: Major)
> Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE
> but passes when run from the command line
> --
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Minor
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE but passes when run from the command line
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-5909:
-
Fix Version/s: (was: 1.36.0)
> Sometimes SqlParserTest.testNoUnintendedNewReservedKeywords fails in the IDE
> but passes when run from the command line
> --
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reopened CALCITE-5909:
--
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen closed CALCITE-5909.
-
Resolution: Not A Problem
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1775#comment-1775
]
LakeShen commented on CALCITE-5909:
---
Hi [~jhyde] [~taoran] ,thanks for your reply, I have got your point,and I will
close this jira.
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5895) The result of table sample rate 0 is incorrect result and should return the empty result
[
https://issues.apache.org/jira/browse/CALCITE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752205#comment-17752205
]
Julian Hyde commented on CALCITE-5895:
--
I agree this is a bug, and we should fix it. The parser correctly changes 'emp
TABLESAMPLE(100)' to 'emp'. But it should not convert 'emp TABLESAMPLE(0)' to
'emp', as it currently does.
I think implementation 2 would be better and easier. You would not need to
write a planner rule, and it seems clear to me where the logic should go:
* Move the {{if (fRate > 0.0f && fRate < 1.0f)}} logic from {{Parser.jj}} to
{{{}SqlToRelConverter.convertFrom{}}}.
* In {{{}SqlToRelConverter{}}}, you should use {{RelBuilder.empty()}} to
create an empty relation of the right type.
* For extra credit, move the {{if (rate.compareTo(BigDecimal.ZERO) < 0 ||
rate.compareTo(ONE_HUNDRED) > 0)}} logic from {{Parser.jj}} to
{{{}SqlValidatorImpl.validateQuery{}}}. The parser shouldn't be doing very much
validation. (And add a negative test to {{{}SqlValidatorTest{}}}.)
I really wish that we had an implementation of {{TABLESAMPLE}} in enumerable
mode. Then we could add a {{tablesample.iq}} test that actually executed
queries.
> The result of table sample rate 0 is incorrect result and should return the
> empty result
> -
>
> Key: CALCITE-5895
> URL: https://issues.apache.org/jira/browse/CALCITE-5895
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-05-22-45-02-513.png
>
>
> For The sql:
> {code:java}
> select * from emp tablesample bernoulli(0) where empno > 5
> {code}
> The plan is :
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[>($0, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I think the correct result of above SQL is empty.
> But its result is the following SQL result:
> {code:java}
> select * from emp where empno > 5
> {code}
> It's an incorrect result
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5869) LEAST_RESTRICTIVE does not use inner type of MEASURE for comparisons
[
https://issues.apache.org/jira/browse/CALCITE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanner Clary resolved CALCITE-5869.
---
Fix Version/s: 1.36.0
Resolution: Fixed
Merged via
[95c4ac|https://github.com/apache/calcite/commit/95c4acc3cccdc66a3fd39a41250cb0d7f86c56ee],
thanks for the reviews [~tjbanghart][~julianhyde]
> LEAST_RESTRICTIVE does not use inner type of MEASURE for comparisons
>
>
> Key: CALCITE-5869
> URL: https://issues.apache.org/jira/browse/CALCITE-5869
> Project: Calcite
> Issue Type: Bug
>Reporter: Tanner Clary
>Assignee: Tanner Clary
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> If you have a function with return type {{LEAST_RESTRICTIVE}}, that takes two
> arguments, and one of these arguments is an {{INTEGER}} and the other is a
> {{MEASURE}} you should expect that the latter is less restrictive.
> However, the logic in {{SqlTypeUtil#canCastFrom}} does not consider the case
> of measures where the type that should be used for comparisons is nested
> within. Because the {{SqlTypeFamily}} of a measure is {{ANY}}, that method
> will always return {{TRUE}}.
> I have a small fix I will open a PR for, any comments would be appreciated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Add REGEXP_EXTRACT and REGEXP_SUBSTR functions (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Description:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
and
[REGEXP_SUBSTR|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_substr]
functions from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
Synonym for REGEXP_EXTRACT
was:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
Synonym for REGEXP_EXTRACT
> Add REGEXP_EXTRACT and REGEXP_SUBSTR functions (enabled in BigQuery library)
>
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> and
> [REGEXP_SUBSTR|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_substr]
> functions from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
>
> *{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
> Synonym for REGEXP_EXTRACT
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Add REGEXP_EXTRACT function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Description:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
Synonym for REGEXP_EXTRACT
was:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
> Add REGEXP_EXTRACT function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
>
> *{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
> Synonym for REGEXP_EXTRACT
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Add REGEXP_EXTRACT and REGEXP_SUBSTR functions (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Summary: Add REGEXP_EXTRACT and REGEXP_SUBSTR functions (enabled in
BigQuery library) (was: Add REGEXP_EXTRACT function (enabled in BigQuery
library))
> Add REGEXP_EXTRACT and REGEXP_SUBSTR functions (enabled in BigQuery library)
>
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
>
> *{{REGEXP_SUBSTR(value, regexp[, position[, occurrence]])}}*
> Synonym for REGEXP_EXTRACT
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5910) Add REGEXP_EXTRACT function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752187#comment-17752187
]
Mihai Budiu edited comment on CALCITE-5910 at 8/8/23 9:58 PM:
--
First, let me make the disclaimer that I am still relatively new to Calcite, so
what I say below may be wrong. There may be some additional complications that
I do not even know about.
I have contemplated the problem of regular expressions myself and I have not
reached a happy conclusion. The difficulty is that there are *many* languages
for describing regular expressions. They tend to have a large common area, but
then they differ on corner cases, as the other comment points out. SQL has
LIKE, but also the more complex SIMILAR. On top of that databases add their own
regex dialects, for example Postgres has Posix regular expressions:
https://www.postgresql.org/docs/15/functions-matching.html
Calcite already has RLIKE, and a bunch of REGEX functions for various dialects.
RLIKE is *defined* to use the Java regex library, and that's very convenient
for Calcite.
As I see it, these functions can be evaluated in two places:
- at compilation, using the PROJECT_REDUCE_EXPRESSIONS optimizer rule, when the
plan is optimized
- at runtime, using whatever runtime you are using
These two evaluation methods better give the same results for constant
expressions, otherwise users may have very unpleasant surprises.
If you need 100% compatibility with a certain dialect of regexes, I think you
have two choices:
- use a Java library that faithfully implements the regex library
- prevent the compiler from ever evaluating such expressions at compile-time.
There are least two ways you can do that: (1) treat the function like a
user-defined function, or (2) claim it is non-deterministic
If you will always execute the Calcite-optimized code on your backend (e.g.,
BigQuery), maybe you can get away without implementing the functions in Java.
If you want to use Calcite as a runtime I believe that you have only one choice
left, and you better pray that there is a well-tested Java library implementing
your functions.
was (Author: JIRAUSER295926):
First, let me make the disclaimer that I am still relatively new to Calcite, so
what I say below may be wrong. There may be some additional complications that
I do not even know about.
I have contemplated the problem of regular expressions myself and I have not
reached a happy conclusion. The difficulty is that there are *many* languages
for describing regular expressions. They tend to have a large common area, but
then they differ on corner cases, as the other comments points out. SQL has
LIKE, but also the more complex SIMILAR. On top of that databases add their own
regex dialects, for example Postgres has Posix regular expressions:
https://www.postgresql.org/docs/15/functions-matching.html
Calcite already has RLIKE, and a bunch of REGEX functions for various dialects.
RLIKE is *defined* to use the Java regex library, and that's very convenient
for Calcite.
As I see it, these functions can be evaluated in two places:
- at compilation, using the PROJECT_REDUCE_EXPRESSIONS optimizer rule, when the
plan is optimized
- at runtime, using whatever runtime you are using
These two evaluation methods better give the same results for constant
expressions, otherwise users may have very unpleasant surprises.
If you need 100% compatibility with a certain dialect of regexes, I think you
have two choices:
- use a Java library that faithfully implements the regex library
- prevent the compiler from ever evaluating such expressions at compile-time.
There are least two ways you can do that: (1) treat the function like a
user-defined function, or (2) claim it is non-deterministic
If you will always execute the Calcite-optimized code on your backend (e.g.,
BigQuery), maybe you can get away without implementing the functions in Java.
If you want to use Calcite as a runtime I believe that you have only one choice
left, and you better pray that there is a well-tested Java library implementing
your functions.
> Add REGEXP_EXTRACT function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if
[jira] [Commented] (CALCITE-5910) Add REGEXP_EXTRACT function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752187#comment-17752187
]
Mihai Budiu commented on CALCITE-5910:
--
First, let me make the disclaimer that I am still relatively new to Calcite, so
what I say below may be wrong. There may be some additional complications that
I do not even know about.
I have contemplated the problem of regular expressions myself and I have not
reached a happy conclusion. The difficulty is that there are *many* languages
for describing regular expressions. They tend to have a large common area, but
then they differ on corner cases, as the other comments points out. SQL has
LIKE, but also the more complex SIMILAR. On top of that databases add their own
regex dialects, for example Postgres has Posix regular expressions:
https://www.postgresql.org/docs/15/functions-matching.html
Calcite already has RLIKE, and a bunch of REGEX functions for various dialects.
RLIKE is *defined* to use the Java regex library, and that's very convenient
for Calcite.
As I see it, these functions can be evaluated in two places:
- at compilation, using the PROJECT_REDUCE_EXPRESSIONS optimizer rule, when the
plan is optimized
- at runtime, using whatever runtime you are using
These two evaluation methods better give the same results for constant
expressions, otherwise users may have very unpleasant surprises.
If you need 100% compatibility with a certain dialect of regexes, I think you
have two choices:
- use a Java library that faithfully implements the regex library
- prevent the compiler from ever evaluating such expressions at compile-time.
There are least two ways you can do that: (1) treat the function like a
user-defined function, or (2) claim it is non-deterministic
If you will always execute the Calcite-optimized code on your backend (e.g.,
BigQuery), maybe you can get away without implementing the functions in Java.
If you want to use Calcite as a runtime I believe that you have only one choice
left, and you better pray that there is a well-tested Java library implementing
your functions.
> Add REGEXP_EXTRACT function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5910) Add REGEXP_EXTRACT function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752184#comment-17752184
]
Jerin John commented on CALCITE-5910:
-
Thanks for the pointer [~mbudiu], I tried implementing one of the other regex
functions with the java native library and we observed some variances in the
pattern processing between the two libraries (ref comment in CALCITE-5873).
Could you please review that comment and provide your suggestions if it would
be advisable to use the re2 java library instead.
> Add REGEXP_EXTRACT function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if sort fields contains unique key
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752182#comment-17752182
]
Julian Hyde commented on CALCITE-5894:
--
Here is a sketch for the new metadata:
{code:java}
public abstract class BuiltInMetadata {
// ...
public interface FunctionalDependency extends Metadata {
/** Returns whether column is functionally dependent on columns. */
Boolean functionallyDetermine(ImmutableBitSet columns, int column);
}
}
{code}
If {{columns}} contains a unique key (per
{{BuiltInMetadata.areColumnsUnique(columns, false)}}) then
{{functionallyDetermine(columns, column)}} will return true for all values of
{{column}}.
In the previous paragraph, {{ignoreNulls = false}} because null values do
matter in sorting.
It would be implemented in a new {{class RelMdFunctionalDependency}}.
For an expression e, the columns functionally determine e if they determine
every column in e.
> Add SortRemoveRedundantRule to remove redundant sort fields if sort fields
> contains unique key
> --
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if sort fields contains unique key
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752179#comment-17752179
]
Julian Hyde commented on CALCITE-5894:
--
We can generalize a little. If we have "ORDER BY w, x, y, z" then we can remove
"z" if "z" is functionally dependent (FD) on (w, x, y).
Clearly if w, x or y are individually unique then z is FD on (w, x, y).
But if (w, x, y) or (w, x) or (w, y) are unique (i.e. is a composite key) then
z can be removed also.
And if z may be FD for other reasons. Suppose we know that z == x + y. Then z
is FD on (x, y).
{{SortRemoveConstantKeysRule}} (added in CALCITE-873) is a case of this (a
constant is functionally dependent on anything).
{{Sort}} already ensures that the sort keys are distinct. "ORDER BY x, y, x"
would be a violation of the FD principle, because x is functionally dependent
on x.
If you can implement this in terms of FD, rather than unique keys per se, then
you will be able to handle cases such as "SELECT deptno FROM emp ORDER BY
deptno, deptno / 2". FD might require a new kind of metadata (similar to
{{RelMdUniqueKeys}} but going further) and therefore could be handled in a
separate Jira case.
> Add SortRemoveRedundantRule to remove redundant sort fields if sort fields
> contains unique key
> --
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5894) Add SortRemoveRedundantRule to remove redundant sort fields if sort fields contains unique key
[
https://issues.apache.org/jira/browse/CALCITE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752177#comment-17752177
]
Julian Hyde commented on CALCITE-5894:
--
[~libenchao], There is a significant benefit to removing redundant fields. If
you are doing an external merge sort, those redundant fields will be added to
the partition file. Even if those fields are never used, they incur an IO cost
when the file is written and read; also, assuming that sort memory is fixed,
the number of initial partitions in a merge-sort will be larger (because each
partition contains fewer rows), and therefore the tree depth (the 'log N' in 'N
log N') may be larger.
> Add SortRemoveRedundantRule to remove redundant sort fields if sort fields
> contains unique key
> --
>
> Key: CALCITE-5894
> URL: https://issues.apache.org/jira/browse/CALCITE-5894
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Minor
>
> In some scene, Sort fields can be reduct, if sort fields contain unique key
> For example
> {code:java}
> SELECT ename, salary FROM Emp
> order by empno, ename{code}
> where `empno` is a key, `ename` is redundant since `empno` alone is
> sufficient to determine the order of any two records.
> So the SQL can be optimized as following:
> {code:java}
> SELECT name, Emp.salary FROM Emp
> order by empno{code}
> For another example:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM
> (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename, e_agg.c {code}
> Although `e_agg.ename` is not a key but field `ename` is unique and not null,
> it can be optimized as following:
> {code:java}
> SELECT e_agg.c, e_agg.ename
> FROM (SELECT count(*) as c, ename, job FROM Emp GROUP BY ename, job) AS e_agg
> ORDER BY e_agg.ename{code}
> Sorting is an expensive operation, however. Therefore, it is imperative that
> sorting
> is optimized to avoid unnecessary sort field.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5889) Add the RelRule that converts Minus into UNION ALL..GROUP BY...WHERE
[ https://issues.apache.org/jira/browse/CALCITE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752168#comment-17752168 ] Julian Hyde edited comment on CALCITE-5889 at 8/8/23 8:13 PM: -- You're right. For regular EXCEPT the condition is "c0 > 0 AND c1 = 0 AND c2 = 0". (Where c0 is "count\(*) filter (where m = 0)", etc.) For EXCEPT ALL the number of copies to emit for each key is "c0 - c1 - c2". was (Author: julianhyde): You're right. For regular EXCEPT the condition is "c0 > 0 AND c1 = 0 AND c2 = 0". (Where c0 is "count(*) filter (where m = 0)", etc.) For EXCEPT ALL the number of copies to emit for each key is "c0 - c1 - c2". > Add the RelRule that converts Minus into UNION ALL..GROUP BY...WHERE > > > Key: CALCITE-5889 > URL: https://issues.apache.org/jira/browse/CALCITE-5889 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: LakeShen >Assignee: LakeShen >Priority: Major > Fix For: 1.36.0 > > > There are many projects that implement optimizers based on Calcite,our > optimizer is also based on Calcite. > Calcite has a lot of good rules in CoreRules.It has UnionToDistinctRule and > IntersectToDistinctRule RelRule ,UnionToDistinctRule is that converts > Union(all = false) to Union(all=true) + Aggregate,IntersectToDistinctRule is > that converts Intersect(all=false) to Union(all=true) + Aggregate + > Filter.None of these rules translate Minus to other RelNode combinations. > Normally, a computation engine does not have a Minus operator, so it is > common to convert Minus to some other relational algebra combination in the > optimizer.For example,in presto,it has the ImplementIntersectAndExceptAsUnion > PlanOptimizer that converts Minus to into UNION ALL..GROUP BY...WHERE. In > flink,it has the ReplaceMinusWithAntiJoinRule RelOptRule. In dremio-oss,it > has MinusToJoin RelOptRule.All of this rules,converts the Minus to other > composition of relational algebra. > Since there are no optimization rules for dealing with minus in calcite, > users of calcite generally need to write their own optimization rules for > dealing with Minus. > I think it makes sense to add the RelRule that converts Minus to other > composition of relational algebranto ,like UNION ALL.. GROUP BY... WHERE. So > for users of calcite,they don't have to go through the extra work of writing > rules for Minus to convert it to something else. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5889) Add the RelRule that converts Minus into UNION ALL..GROUP BY...WHERE
[ https://issues.apache.org/jira/browse/CALCITE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752168#comment-17752168 ] Julian Hyde commented on CALCITE-5889: -- You're right. For regular EXCEPT the condition is "c0 > 0 AND c1 = 0 AND c2 = 0". (Where c0 is "count(*) filter (where m = 0)", etc.) For EXCEPT ALL the number of copies to emit for each key is "c0 - c1 - c2". > Add the RelRule that converts Minus into UNION ALL..GROUP BY...WHERE > > > Key: CALCITE-5889 > URL: https://issues.apache.org/jira/browse/CALCITE-5889 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: LakeShen >Assignee: LakeShen >Priority: Major > Fix For: 1.36.0 > > > There are many projects that implement optimizers based on Calcite,our > optimizer is also based on Calcite. > Calcite has a lot of good rules in CoreRules.It has UnionToDistinctRule and > IntersectToDistinctRule RelRule ,UnionToDistinctRule is that converts > Union(all = false) to Union(all=true) + Aggregate,IntersectToDistinctRule is > that converts Intersect(all=false) to Union(all=true) + Aggregate + > Filter.None of these rules translate Minus to other RelNode combinations. > Normally, a computation engine does not have a Minus operator, so it is > common to convert Minus to some other relational algebra combination in the > optimizer.For example,in presto,it has the ImplementIntersectAndExceptAsUnion > PlanOptimizer that converts Minus to into UNION ALL..GROUP BY...WHERE. In > flink,it has the ReplaceMinusWithAntiJoinRule RelOptRule. In dremio-oss,it > has MinusToJoin RelOptRule.All of this rules,converts the Minus to other > composition of relational algebra. > Since there are no optimization rules for dealing with minus in calcite, > users of calcite generally need to write their own optimization rules for > dealing with Minus. > I think it makes sense to add the RelRule that converts Minus to other > composition of relational algebranto ,like UNION ALL.. GROUP BY... WHERE. So > for users of calcite,they don't have to go through the extra work of writing > rules for Minus to convert it to something else. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5905) Documentation for CREATE TYPE is incorrect
[
https://issues.apache.org/jira/browse/CALCITE-5905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-5905:
Labels: pull-request-available (was: )
> Documentation for CREATE TYPE is incorrect
> --
>
> Key: CALCITE-5905
> URL: https://issues.apache.org/jira/browse/CALCITE-5905
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Mihai Budiu
>Priority: Minor
> Labels: pull-request-available
>
> I have tried to run the example CREATE TYPE statements from this page
> https://calcite.apache.org/docs/reference.html#declaring-objects-for-user-defined-types
> through the compiler:
> {code:sql}
> CREATE TYPE address_typ AS OBJECT (
>street VARCHAR2(30),
>cityVARCHAR2(20),
>state CHAR(2),
>postal_code VARCHAR2(6));
> {code}
> Calcite complains in two places: OBJECT and VARCHAR2.
> The following compiles fine:
> {code:sql}
> CREATE TYPE address_typ AS (
>street VARCHAR(30),
>cityVARCHAR(20),
>state CHAR(2),
>postal_code VARCHAR(6));
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (CALCITE-5290) Add DATE_TRUNC function consistent with Postgres
[
https://issues.apache.org/jira/browse/CALCITE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TJ Banghart reassigned CALCITE-5290:
Assignee: Guillaume Massé (was: TJ Banghart)
> Add DATE_TRUNC function consistent with Postgres
>
>
> Key: CALCITE-5290
> URL: https://issues.apache.org/jira/browse/CALCITE-5290
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Guillaume Massé
>Priority: Major
>
> Add {{DATE_TRUNC}} function consistent with Postgres. Per the
> [specification|https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC],
> syntax is as follows:
> {code}
> DATE_TRUNC(, )
> {code}
> It is a different (and more complex) specification than BigQuery's DATE_TRUNC
> (see CALCITE-5180), which has the following syntax:
> {code}
> DATE_TRUNC(, )
> {code}
> Postgres DATE_TRUNC would be implemented after CALCITE-5155 and would allow
> user-defined time frames in the {{field}} argument.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5858) Add REGEXP_CONTAINS, REGEXP_EXTRACT, REGEXP_EXTRACT_ALL, REGEXP_INSTR, REGEXP_REPLACE and REGEXP_SUBSTR functions (enabled in BigQuery library)
[ https://issues.apache.org/jira/browse/CALCITE-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerin John updated CALCITE-5858: Summary: Add REGEXP_CONTAINS, REGEXP_EXTRACT, REGEXP_EXTRACT_ALL, REGEXP_INSTR, REGEXP_REPLACE and REGEXP_SUBSTR functions (enabled in BigQuery library) (was: Implement BigQuery functions REGEXP_CONTAINS, REGEXP_EXTRACT, REGEXP_EXTRACT_ALL, REGEXP_INSTR, REGEXP_REPLACE and REGEXP_SUBSTR) > Add REGEXP_CONTAINS, REGEXP_EXTRACT, REGEXP_EXTRACT_ALL, REGEXP_INSTR, > REGEXP_REPLACE and REGEXP_SUBSTR functions (enabled in BigQuery library) > --- > > Key: CALCITE-5858 > URL: https://issues.apache.org/jira/browse/CALCITE-5858 > Project: Calcite > Issue Type: New Feature >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > > Add support for REGEX_* string functions in BigQuery. > Function descriptions: > * REGEXP_CONTAINS: Returns TRUE if input value is a partial match for the > regular expression. > * REGEXP_EXTRACT: Returns the substring in input value that matches the > regular expression. Returns NULL if there is no match. > * REGEXP_EXTRACT_ALL: Returns an array of all substrings of input value that > match the regular expression. Returns an empty array if there is no match. > * REGEXP_INSTR: Returns the lowest 1-based position of a regular expression > in an input value. > * REGEXP_REPLACE: Returns a STRING where all substrings of input value that > match regular expression are replaced with the input replacement. > * REGEXP_SUBSTR: Synonym for REGEXP_EXTRACT, returns the substring in input > value that matches the regular expression. Returns NULL if there is no match. > More on these functions and their syntax may be found here: [BigQuery > Doc|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_contains] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Add REGEXP_EXTRACT function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Summary: Add REGEXP_EXTRACT function (enabled in BigQuery library) (was:
Implement BigQuery function REGEXP_EXTRACT)
> Add REGEXP_EXTRACT function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5873) Add REGEXP_CONTAINS function (enabled in BigQuery library)
[ https://issues.apache.org/jira/browse/CALCITE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerin John updated CALCITE-5873: Summary: Add REGEXP_CONTAINS function (enabled in BigQuery library) (was: Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library)) > Add REGEXP_CONTAINS function (enabled in BigQuery library) > -- > > Key: CALCITE-5873 > URL: https://issues.apache.org/jira/browse/CALCITE-5873 > Project: Calcite > Issue Type: Task >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > Labels: pull-request-available > > Add support for REGEXP_CONTAINS function from BigQuery. > Function returns TRUE if input value is a partial match for the regular > expression. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5873) Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library)
[ https://issues.apache.org/jira/browse/CALCITE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerin John updated CALCITE-5873: Summary: Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library) (was: Implement BigQuery functions REGEXP_CONTAINS) > Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library) > - > > Key: CALCITE-5873 > URL: https://issues.apache.org/jira/browse/CALCITE-5873 > Project: Calcite > Issue Type: Task >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > Labels: pull-request-available > > Add support for REGEXP_CONTAINS function from BigQuery. > Function returns TRUE if input value is a partial match for the regular > expression. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5911) Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5911:
Summary: Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library)
(was: Implement BigQuery function REGEXP_EXTRACT_ALL)
> Add REGEXP_EXTRACT_ALL function (enabled in BigQuery library)
> -
>
> Key: CALCITE-5911
> URL: https://issues.apache.org/jira/browse/CALCITE-5911
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> function from BigQuery.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
> * If the regular expression contains a capturing group ((...)), and there is
> a match for that capturing group, that match is added to the results. If
> there are multiple matches for a capturing group, the last match is added to
> the results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Implement BigQuery function REGEXP_EXTRACT
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Description:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
was:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
> Implement BigQuery function REGEXP_EXTRACT
> --
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Implement BigQuery function REGEXP_EXTRACT
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Description:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
function from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
was:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
and
[REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
functions from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
> Implement BigQuery function REGEXP_EXTRACT
> --
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> function from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence
> of the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5911) Implement BigQuery function REGEXP_EXTRACT_ALL
[
https://issues.apache.org/jira/browse/CALCITE-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5911:
Parent: (was: CALCITE-5858)
Issue Type: Task (was: Sub-task)
> Implement BigQuery function REGEXP_EXTRACT_ALL
> --
>
> Key: CALCITE-5911
> URL: https://issues.apache.org/jira/browse/CALCITE-5911
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> function from BigQuery.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
> * If the regular expression contains a capturing group ((...)), and there is
> a match for that capturing group, that match is added to the results. If
> there are multiple matches for a capturing group, the last match is added to
> the results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5911) Implement BigQuery function REGEXP_EXTRACT_ALL
[
https://issues.apache.org/jira/browse/CALCITE-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5911:
Parent: CALCITE-5858
Issue Type: Sub-task (was: Task)
> Implement BigQuery function REGEXP_EXTRACT_ALL
> --
>
> Key: CALCITE-5911
> URL: https://issues.apache.org/jira/browse/CALCITE-5911
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> function from BigQuery.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
> * If the regular expression contains a capturing group ((...)), and there is
> a match for that capturing group, that match is added to the results. If
> there are multiple matches for a capturing group, the last match is added to
> the results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5911) Implement BigQuery function REGEXP_EXTRACT_ALL
[
https://issues.apache.org/jira/browse/CALCITE-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5911:
Description:
Add support for
[REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
function from BigQuery.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
* If the regular expression contains a capturing group ((...)), and there is a
match for that capturing group, that match is added to the results. If there
are multiple matches for a capturing group, the last match is added to the
results.
was:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
and
[REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
functions from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
> Implement BigQuery function REGEXP_EXTRACT_ALL
> --
>
> Key: CALCITE-5911
> URL: https://issues.apache.org/jira/browse/CALCITE-5911
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> function from BigQuery.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
> * If the regular expression contains a capturing group ((...)), and there is
> a match for that capturing group, that match is added to the results. If
> there are multiple matches for a capturing group, the last match is added to
> the results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Description:
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
and
[REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
functions from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
was:
Add support for REGEXP_CONTAINS function from BigQuery.
Function returns TRUE if input value is a partial match for the regular
expression.
> Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL
> --
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> and
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> functions from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence of
> the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5911) Implement BigQuery function REGEXP_EXTRACT_ALL
Jerin John created CALCITE-5911:
---
Summary: Implement BigQuery function REGEXP_EXTRACT_ALL
Key: CALCITE-5911
URL: https://issues.apache.org/jira/browse/CALCITE-5911
Project: Calcite
Issue Type: Task
Reporter: Jerin John
Assignee: Jerin John
Add support for
[REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
and
[REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
functions from BigQuery.
*{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
Returns the substring in {{value}} that matches the regular expression
{{{}regexp{}}}. Returns {{NULL}} if there is no match.
* If the regular expression contains a capturing group ({{{}(...){}}}), and
there is a match for that capturing group, that match is returned. If there are
multiple matches for a capturing group, the last match is returned.
* If {{position}} is specified, the search starts at this position in
{{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
* If {{occurrence}} is specified, the search returns a specific occurrence of
the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
*{{REGEXP_EXTRACT_ALL(value, regexp)}}*
Returns an array of all substrings of value that match the re2 regular
expression, regexp. Returns an empty array if there is no match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5910) Implement BigQuery function REGEXP_EXTRACT
[
https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerin John updated CALCITE-5910:
Summary: Implement BigQuery function REGEXP_EXTRACT (was: Implement
BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL)
> Implement BigQuery function REGEXP_EXTRACT
> --
>
> Key: CALCITE-5910
> URL: https://issues.apache.org/jira/browse/CALCITE-5910
> Project: Calcite
> Issue Type: Task
>Reporter: Jerin John
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> Add support for
> [REGEXP_EXTRACT|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract]
> and
> [REGEXP_EXTRACT_ALL|https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract_all]
> functions from BigQuery.
> *{{REGEXP_EXTRACT(value, regexp[, position[, occurrence]])}}*
> Returns the substring in {{value}} that matches the regular expression
> {{{}regexp{}}}. Returns {{NULL}} if there is no match.
> * If the regular expression contains a capturing group ({{{}(...){}}}), and
> there is a match for that capturing group, that match is returned. If there
> are multiple matches for a capturing group, the last match is returned.
> * If {{position}} is specified, the search starts at this position in
> {{{}value{}}}, otherwise it starts at the beginning of {{{}value{}}}.
> * If {{occurrence}} is specified, the search returns a specific occurrence of
> the {{regexp}} in {{{}value{}}}, otherwise returns the first match.
> *{{REGEXP_EXTRACT_ALL(value, regexp)}}*
> Returns an array of all substrings of value that match the re2 regular
> expression, regexp. Returns an empty array if there is no match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5910) Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL
[ https://issues.apache.org/jira/browse/CALCITE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752146#comment-17752146 ] Mihai Budiu commented on CALCITE-5910: -- There are many languages that describe regular expressions, that differ subtly from one another. BigQuery uses re2: https://github.com/google/re2/wiki/Syntax which is similar to Java's regular expressions, but not identical. > Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL > -- > > Key: CALCITE-5910 > URL: https://issues.apache.org/jira/browse/CALCITE-5910 > Project: Calcite > Issue Type: Task >Reporter: Jerin John >Assignee: Jerin John >Priority: Major > Labels: pull-request-available > > Add support for REGEXP_CONTAINS function from BigQuery. > Function returns TRUE if input value is a partial match for the regular > expression. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5910) Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL
Jerin John created CALCITE-5910: --- Summary: Implement BigQuery functions REGEXP_EXTRACT and REGEXP_EXTRACT_ALL Key: CALCITE-5910 URL: https://issues.apache.org/jira/browse/CALCITE-5910 Project: Calcite Issue Type: Task Reporter: Jerin John Assignee: Jerin John Add support for REGEXP_CONTAINS function from BigQuery. Function returns TRUE if input value is a partial match for the regular expression. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752141#comment-17752141
]
Ran Tao edited comment on CALCITE-5908 at 8/8/23 5:32 PM:
--
Sorry for this. There is a concurrent modification problem, null check again
works in some particular scenarios. It's right. close the ticket already.
was (Author: lemonjing):
I didn't go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao reopened CALCITE-5908:
--
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao closed CALCITE-5908.
Resolution: Not A Problem
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[ https://issues.apache.org/jira/browse/CALCITE-5908 ]
Ran Tao deleted comment on CALCITE-5908:
--
was (Author: lemonjing):
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
[https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2]
then a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null.
WDYT?
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao closed CALCITE-5908.
Resolution: Not A Problem
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752141#comment-17752141
]
Ran Tao edited comment on CALCITE-5908 at 8/8/23 5:28 PM:
--
I didn't go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
was (Author: lemonjing):
I I didn't go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752141#comment-17752141
]
Ran Tao edited comment on CALCITE-5908 at 8/8/23 5:28 PM:
--
I I didn't go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
was (Author: lemonjing):
I'm not go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
Julian Hyde (Jira) 于2023年8月8日周二 23:34写道：
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao reopened CALCITE-5908:
--
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752141#comment-17752141
]
Ran Tao commented on CALCITE-5908:
--
I'm not go deep into the commit to understand why, Sorry for this.
It's right. close the ticket already.
Julian Hyde (Jira) 于2023年8月8日周二 23:34写道：
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao closed CALCITE-5908.
Resolution: Not A Problem
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guillaume Massé updated CALCITE-5909:
-
Description:
When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
failed,the exception like this:
{code:java}
java.lang.AssertionError: The parser has at least one new reserved keyword. Are
you sure it should be reserved? Difference: {code}
The picture like this:
!image-2023-08-08-23-32-55-466.png|width=1543,height=496!
I could fix this problem.More importantly, why is this method failing, but the
Calcite pipeline is passing? I think we should look at something we missed
was:
When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
failed,the exception like this:
{code:java}
java.lang.AssertionError: The parser has at least one new reserved keyword. Are
you sure it should be reserved? Difference: {code}
The picture like this:
!image-2023-08-08-23-32-55-466.png|width=709,height=228!
I could fix this problem.More importantly, why is this method failing, but the
Calcite pipeline is passing? I think we should look at something we missed
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1543,height=496!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5290) Add DATE_TRUNC function consistent with Postgres
[
https://issues.apache.org/jira/browse/CALCITE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752128#comment-17752128
]
Guillaume Massé commented on CALCITE-5290:
--
[~tjbanghart] yup you can re-assign this ticket to me.
> Add DATE_TRUNC function consistent with Postgres
>
>
> Key: CALCITE-5290
> URL: https://issues.apache.org/jira/browse/CALCITE-5290
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: TJ Banghart
>Priority: Major
>
> Add {{DATE_TRUNC}} function consistent with Postgres. Per the
> [specification|https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC],
> syntax is as follows:
> {code}
> DATE_TRUNC(, )
> {code}
> It is a different (and more complex) specification than BigQuery's DATE_TRUNC
> (see CALCITE-5180), which has the following syntax:
> {code}
> DATE_TRUNC(, )
> {code}
> Postgres DATE_TRUNC would be implemented after CALCITE-5155 and would allow
> user-defined time frames in the {{field}} argument.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5290) Add DATE_TRUNC function consistent with Postgres
[
https://issues.apache.org/jira/browse/CALCITE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752127#comment-17752127
]
TJ Banghart commented on CALCITE-5290:
--
[~MasseGuillaume] that would be great! I have not had time to take this on. May
I assign to you?
> Add DATE_TRUNC function consistent with Postgres
>
>
> Key: CALCITE-5290
> URL: https://issues.apache.org/jira/browse/CALCITE-5290
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: TJ Banghart
>Priority: Major
>
> Add {{DATE_TRUNC}} function consistent with Postgres. Per the
> [specification|https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC],
> syntax is as follows:
> {code}
> DATE_TRUNC(, )
> {code}
> It is a different (and more complex) specification than BigQuery's DATE_TRUNC
> (see CALCITE-5180), which has the following syntax:
> {code}
> DATE_TRUNC(, )
> {code}
> Postgres DATE_TRUNC would be implemented after CALCITE-5155 and would allow
> user-defined time frames in the {{field}} argument.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752121#comment-17752121
]
Ran Tao edited comment on CALCITE-5909 at 8/8/23 4:15 PM:
--
[~shenlang] +1
The IDE sometimes fail to execute the cases due to some cache or index problem.
At this time, you can switch to the shell to execute it again.
was (Author: lemonjing):
[~shenlang] +1
The IDE sometimes fail to execute the cases due to some cache or index. At this
time, you can switch to the shell to execute it again.
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=709,height=228!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752121#comment-17752121
]
Ran Tao commented on CALCITE-5909:
--
[~shenlang] +1
The IDE sometimes fail to execute the cases due to some cache or index. At this
time, you can switch to the shell to execute it again.
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=709,height=228!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752118#comment-17752118
]
Ran Tao edited comment on CALCITE-5908 at 8/8/23 4:11 PM:
--
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
[https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2]
then a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null.
WDYT?
was (Author: lemonjing):
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
[https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2]
then a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null in
else branch. WDYT?
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752118#comment-17752118
]
Ran Tao edited comment on CALCITE-5908 at 8/8/23 4:07 PM:
--
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
[https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2]
then a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null in
else branch. WDYT?
was (Author: lemonjing):
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2
And after a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null in
else branch. WDYT?
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752118#comment-17752118
]
Ran Tao commented on CALCITE-5908:
--
[~jhyde] I have checked it. it's a very old version in 1.13 make this change.
https://github.com/apache/calcite/pull/421/files#diff-1db549e5bc5ea88fef304211584d9fef15761178fc4d7f7bb22254db38695cf2
And after a commit change it to Guava Preconditions.checkNotNull.
And in another commit 'Change Guava Preconditions.checkNotNull to Java
Objects.requireNonNull. (Kevin Risden)' change to requireNonNull.
The latest commit change requireNonNull to requireNonNull(a, 'a') with message.
While follow the initial commit above, there is no need to check this null in
else branch. WDYT?
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752113#comment-17752113
]
Julian Hyde commented on CALCITE-5909:
--
I’ve seen this happen, too. I think the IDE may be getting a slightly different
file due to caching.
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=709,height=228!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-5908:
Labels: pull-request-available (was: )
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
> Labels: pull-request-available
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5909:
--
Description:
When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
failed,the exception like this:
{code:java}
java.lang.AssertionError: The parser has at least one new reserved keyword. Are
you sure it should be reserved? Difference: {code}
The picture like this:
!image-2023-08-08-23-32-55-466.png|width=709,height=228!
I could fix this problem.More importantly, why is this method failing, but the
Calcite pipeline is passing? I think we should look at something we missed
was:
When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
failed,the exception like this:
{code:java}
java.lang.AssertionError: The parser has at least one new reserved keyword. Are
you sure it should be reserved? Difference: {code}
The picture like this:
!image-2023-08-08-23-32-55-466.png|width=1271,height=409!
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=709,height=228!
> I could fix this problem.More importantly, why is this method failing, but
> the Calcite pipeline is passing? I think we should look at something we missed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5908) Remove unnecessary NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Tao updated CALCITE-5908:
-
Summary: Remove unnecessary NULL checks in CalciteSchema (was: Remove
misleading NULL checks in CalciteSchema)
> Remove unnecessary NULL checks in CalciteSchema
> ---
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5908) Remove misleading NULL checks in CalciteSchema
[
https://issues.apache.org/jira/browse/CALCITE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752102#comment-17752102
]
Julian Hyde commented on CALCITE-5908:
--
Do you understand why those checks were added? Read the file history.
> Remove misleading NULL checks in CalciteSchema
> --
>
> Key: CALCITE-5908
> URL: https://issues.apache.org/jira/browse/CALCITE-5908
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Minor
>
> When i read the source code of `CalciteSchema` found so many null checks
> below:
>
> {code:java}
> if (tableMap == null) {
> this.tableMap = new NameMap<>();
> } else {
> this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
> }
> if (latticeMap == null) {
> this.latticeMap = new NameMap<>();
> } else {
> this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
> } {code}
> i think we can remove it. it was misleading.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
[
https://issues.apache.org/jira/browse/CALCITE-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5909:
--
Attachment: image-2023-08-08-23-32-55-466.png
Component/s: tests
Fix Version/s: 1.36.0
Description:
When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
failed,the exception like this:
{code:java}
java.lang.AssertionError: The parser has at least one new reserved keyword. Are
you sure it should be reserved? Difference: {code}
The picture like this:
!image-2023-08-08-23-32-55-466.png|width=1271,height=409!
was:When I run the Sql
> SqlParserTest has a failed unit test, but the Calcite pipeline passed all of
> them
> -
>
> Key: CALCITE-5909
> URL: https://issues.apache.org/jira/browse/CALCITE-5909
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-23-32-55-466.png
>
>
> When I run the SqlParserTest,the testNoUnintendedNewReservedKeywords method
> failed,the exception like this:
> {code:java}
> java.lang.AssertionError: The parser has at least one new reserved keyword.
> Are you sure it should be reserved? Difference: {code}
> The picture like this:
> !image-2023-08-08-23-32-55-466.png|width=1271,height=409!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5909) SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them
LakeShen created CALCITE-5909: - Summary: SqlParserTest has a failed unit test, but the Calcite pipeline passed all of them Key: CALCITE-5909 URL: https://issues.apache.org/jira/browse/CALCITE-5909 Project: Calcite Issue Type: Bug Reporter: LakeShen When I run the Sql -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5908) Remove misleading NULL checks in CalciteSchema
Ran Tao created CALCITE-5908:
Summary: Remove misleading NULL checks in CalciteSchema
Key: CALCITE-5908
URL: https://issues.apache.org/jira/browse/CALCITE-5908
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.35.0
Reporter: Ran Tao
Assignee: Ran Tao
When i read the source code of `CalciteSchema` found so many null checks below:
{code:java}
if (tableMap == null) {
this.tableMap = new NameMap<>();
} else {
this.tableMap = Objects.requireNonNull(tableMap, "tableMap");
}
if (latticeMap == null) {
this.latticeMap = new NameMap<>();
} else {
this.latticeMap = Objects.requireNonNull(latticeMap, "latticeMap");
} {code}
i think we can remove it. it was misleading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5907) Unexpected boolean expression simplification for And expression
[
https://issues.apache.org/jira/browse/CALCITE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752095#comment-17752095
]
Julian Hyde commented on CALCITE-5907:
--
I agree with[~kgyrtkirk]. The RexCall to EQUALS is not well-formed unless there
is a CAST. It is not the responsibility of RexSimplify to check for that CAST.
Try to reproduce this in vanilla Calcite. I don’t think you will succeed.
> Unexpected boolean expression simplification for And expression
> ---
>
> Key: CALCITE-5907
> URL: https://issues.apache.org/jira/browse/CALCITE-5907
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Yunhong Zheng
>Priority: Major
> Fix For: 1.36.0
>
>
> As FLINK-27402 shown. If we have a table MyTable(a Int, b Boolean, c
> String). Calcite will not simplify this case ( c is Varchar type while
> SqlLiteral is boolean):
> {code:java}
> SELECT * FROM MyTable WHERE c = true;{code}
> As the logical plan is :
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[=($2, true)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> However, Calcite will simplify this case while simplifyAnd :
> {code:java}
> SELECT * FROM MyTable WHERE b = true and c = true;{code}
> As the logical plan is shown below: 'b = true' and 'c = true' both were
> simplified to 'b' and 'c':
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[AND($1, $2)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> This may cause error because of filter condition is a Varchar type literal.
>
> After reading Calcite's code, I found that. The logic of
> RexSimplify.implify() and RexSimplify.implifyAnd() is different, where the
> logic of RexSimplify.implifyAnd() is problematic:
> {code:java}
> // Simplify BOOLEAN expressions if possible
> while (term.getKind() == SqlKind.EQUALS) {
> RexCall call = (RexCall) term;
> if (call.getOperands().get(0).isAlwaysTrue()) {
> term = call.getOperands().get(1);
> terms.set(i, term);
> continue;
> } else if (call.getOperands().get(1).isAlwaysTrue()) {
> term = call.getOperands().get(0);
> terms.set(i, term);
> continue;
> }
> break;
> } {code}
> The above code cannot make such a simple judgment, as there may not be an
> implicit conversion to ensure that the types on both sides of the condition
> are consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5907) Unexpected boolean expression simplification for And expression
[
https://issues.apache.org/jira/browse/CALCITE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752094#comment-17752094
]
Zoltan Haindrich edited comment on CALCITE-5907 at 8/8/23 3:17 PM:
---
I believe both operands should be of the same type for an `EQUALS` ; I wonder
how it worked in your case ; I believe `RexSimplify` will not remove such casts
unless it can prove they are not needed.
Can you provide a test case on the `RexSimplify` api level?
You could try adding something like these to `RexProgramTest`:
{code}
@Test void testSimplify1() {
checkSimplify(
eq(vVarchar(), trueLiteral),
"something");
}
@Test void testSimplify2() {
checkSimplify(
eq(literal(3), trueLiteral),
"something");
}
{code}
was (Author: kgyrtkirk):
I believe both operands should be of the same type for an `EQUALS` ; I wonder
how it worked in your case ; I believe `RexSimplify` will not remove such casts
unless it can prove they are not needed.
Can you provide a test case on the `RexSimplify` api level?
You could try adding something like these to `RexProgramTest`:
```
@Test void testSimplify1() {
checkSimplify(
eq(vVarchar(), trueLiteral),
"something");
}
@Test void testSimplify2() {
checkSimplify(
eq(literal(3), trueLiteral),
"something");
}
```
> Unexpected boolean expression simplification for And expression
> ---
>
> Key: CALCITE-5907
> URL: https://issues.apache.org/jira/browse/CALCITE-5907
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Yunhong Zheng
>Priority: Major
> Fix For: 1.36.0
>
>
> As FLINK-27402 shown. If we have a table MyTable(a Int, b Boolean, c
> String). Calcite will not simplify this case ( c is Varchar type while
> SqlLiteral is boolean):
> {code:java}
> SELECT * FROM MyTable WHERE c = true;{code}
> As the logical plan is :
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[=($2, true)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> However, Calcite will simplify this case while simplifyAnd :
> {code:java}
> SELECT * FROM MyTable WHERE b = true and c = true;{code}
> As the logical plan is shown below: 'b = true' and 'c = true' both were
> simplified to 'b' and 'c':
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[AND($1, $2)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> This may cause error because of filter condition is a Varchar type literal.
>
> After reading Calcite's code, I found that. The logic of
> RexSimplify.implify() and RexSimplify.implifyAnd() is different, where the
> logic of RexSimplify.implifyAnd() is problematic:
> {code:java}
> // Simplify BOOLEAN expressions if possible
> while (term.getKind() == SqlKind.EQUALS) {
> RexCall call = (RexCall) term;
> if (call.getOperands().get(0).isAlwaysTrue()) {
> term = call.getOperands().get(1);
> terms.set(i, term);
> continue;
> } else if (call.getOperands().get(1).isAlwaysTrue()) {
> term = call.getOperands().get(0);
> terms.set(i, term);
> continue;
> }
> break;
> } {code}
> The above code cannot make such a simple judgment, as there may not be an
> implicit conversion to ensure that the types on both sides of the condition
> are consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5907) Unexpected boolean expression simplification for And expression
[
https://issues.apache.org/jira/browse/CALCITE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752094#comment-17752094
]
Zoltan Haindrich commented on CALCITE-5907:
---
I believe both operands should be of the same type for an `EQUALS` ; I wonder
how it worked in your case ; I believe `RexSimplify` will not remove such casts
unless it can prove they are not needed.
Can you provide a test case on the `RexSimplify` api level?
You could try adding something like these to `RexProgramTest`:
```
@Test void testSimplify1() {
checkSimplify(
eq(vVarchar(), trueLiteral),
"something");
}
@Test void testSimplify2() {
checkSimplify(
eq(literal(3), trueLiteral),
"something");
}
```
> Unexpected boolean expression simplification for And expression
> ---
>
> Key: CALCITE-5907
> URL: https://issues.apache.org/jira/browse/CALCITE-5907
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Yunhong Zheng
>Priority: Major
> Fix For: 1.36.0
>
>
> As FLINK-27402 shown. If we have a table MyTable(a Int, b Boolean, c
> String). Calcite will not simplify this case ( c is Varchar type while
> SqlLiteral is boolean):
> {code:java}
> SELECT * FROM MyTable WHERE c = true;{code}
> As the logical plan is :
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[=($2, true)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> However, Calcite will simplify this case while simplifyAnd :
> {code:java}
> SELECT * FROM MyTable WHERE b = true and c = true;{code}
> As the logical plan is shown below: 'b = true' and 'c = true' both were
> simplified to 'b' and 'c':
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[AND($1, $2)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> This may cause error because of filter condition is a Varchar type literal.
>
> After reading Calcite's code, I found that. The logic of
> RexSimplify.implify() and RexSimplify.implifyAnd() is different, where the
> logic of RexSimplify.implifyAnd() is problematic:
> {code:java}
> // Simplify BOOLEAN expressions if possible
> while (term.getKind() == SqlKind.EQUALS) {
> RexCall call = (RexCall) term;
> if (call.getOperands().get(0).isAlwaysTrue()) {
> term = call.getOperands().get(1);
> terms.set(i, term);
> continue;
> } else if (call.getOperands().get(1).isAlwaysTrue()) {
> term = call.getOperands().get(0);
> terms.set(i, term);
> continue;
> }
> break;
> } {code}
> The above code cannot make such a simple judgment, as there may not be an
> implicit conversion to ensure that the types on both sides of the condition
> are consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[
https://issues.apache.org/jira/browse/CALCITE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen reassigned CALCITE-5906:
-
Assignee: LakeShen
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png,
> image-2023-08-08-20-19-01-269.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[
https://issues.apache.org/jira/browse/CALCITE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-5906:
Labels: pull-request-available (was: )
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png,
> image-2023-08-08-20-19-01-269.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5827) Add IS_INF and IS_NAN functions (enabled in BigQuery library)
[
https://issues.apache.org/jira/browse/CALCITE-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanner Clary resolved CALCITE-5827.
---
Fix Version/s: 1.36.0
Resolution: Fixed
Merged via
[25f326c|https://github.com/apache/calcite/commit/25f326cf0a94d5e65bd82abc95b676cec6b1fd43],
thanks for the reviews [~Runking] [~julianhyde]!
> Add IS_INF and IS_NAN functions (enabled in BigQuery library)
> -
>
> Key: CALCITE-5827
> URL: https://issues.apache.org/jira/browse/CALCITE-5827
> Project: Calcite
> Issue Type: New Feature
>Reporter: Tanner Clary
>Assignee: Tanner Clary
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> BigQuery offers the
> [IS_INF|https://cloud.google.com/bigquery/docs/reference/standard-sql/mathematical_functions#is_inf]
> and
> [IS_NAN|https://cloud.google.com/bigquery/docs/reference/standard-sql/mathematical_functions#is_nan]
> functions. The first returns whether the specified value is infinite, the
> second returns whether the specified value is {{NaN}}.
> Example 1: {{SELECT IS_INF(3)}} would return {{FALSE}}.
> Example 2: {{SELECT IS_INF(CAST('Infinity` AS DOUBLE))}} would return
> {{TRUE}}.
> Example 3: {{SELECT IS_NAN(3)}} would return {{FALSE}}.
> Example 4: {{SELECT IS_INF(CAST('NaN' AS DOUBLE))}} would return {{TRUE}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5903) RelMdCollation does not define collations for EnumerableLimit
[
https://issues.apache.org/jira/browse/CALCITE-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-5903.
Resolution: Fixed
Fixed via
[{{0cea91b}}|https://github.com/apache/calcite/commit/0cea91b8426d46e72b572d99f6007cf12535f575]
> RelMdCollation does not define collations for EnumerableLimit
> -
>
> Key: CALCITE-5903
> URL: https://issues.apache.org/jira/browse/CALCITE-5903
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> RelMdCollation does not define collations for EnumerableLimit; it should
> define a method that returns the EnumerableLimit's input collations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[ https://issues.apache.org/jira/browse/CALCITE-5906 ]
LakeShen deleted comment on CALCITE-5906:
---
was (Author: shenlang):
Because of the TableSample percentage is float datatype, like this :
!image-2023-08-08-20-16-27-188.png|width=322,height=133!
When I tried to transform Sample to TABLESAMPLE SqlCall,it shows imprecise
precision,like this :
!image-2023-08-08-20-19-01-269.png|width=516,height=114!
I want to change the TableSample percentage to the type of precise
precision,such as bigdecimal,could anyone else give me some advice?
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png,
> image-2023-08-08-20-19-01-269.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5907) Unexpected boolean expression simplification for And expression
[
https://issues.apache.org/jira/browse/CALCITE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yunhong Zheng updated CALCITE-5907:
---
Description:
As FLINK-27402 shown. If we have a table MyTable(a Int, b Boolean, c String).
Calcite will not simplify this case ( c is Varchar type while SqlLiteral is
boolean):
{code:java}
SELECT * FROM MyTable WHERE c = true;{code}
As the logical plan is :
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[=($2, true)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
However, Calcite will simplify this case while simplifyAnd :
{code:java}
SELECT * FROM MyTable WHERE b = true and c = true;{code}
As the logical plan is shown below: 'b = true' and 'c = true' both were
simplified to 'b' and 'c':
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[AND($1, $2)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
This may cause error because of filter condition is a Varchar type literal.
After reading Calcite's code, I found that. The logic of RexSimplify.implify()
and RexSimplify.implifyAnd() is different, where the logic of
RexSimplify.implifyAnd() is problematic:
{code:java}
// Simplify BOOLEAN expressions if possible
while (term.getKind() == SqlKind.EQUALS) {
RexCall call = (RexCall) term;
if (call.getOperands().get(0).isAlwaysTrue()) {
term = call.getOperands().get(1);
terms.set(i, term);
continue;
} else if (call.getOperands().get(1).isAlwaysTrue()) {
term = call.getOperands().get(0);
terms.set(i, term);
continue;
}
break;
} {code}
The above code cannot make such a simple judgment, as there may not be an
implicit conversion to ensure that the types on both sides of the condition are
consistent.
was:
As [FLINK-27402|https://issues.apache.org/jira/browse/FLINK-27402] shown. If
we have a table MyTable(a Int, b Boolean, c String). Calcite will not
simplify this case ( c is Varchar type while SqlLiteral is boolean):
{code:java}
SELECT * FROM MyTable WHERE c = true;{code}
As the logical plan is :
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[=($2, true)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
However, Calcite will simplify this case while simplifyAnd :
{code:java}
SELECT * FROM MyTable WHERE b = true and c = true;{code}
As the logical plan is shown below: 'b = true' and 'c = true' both were
simplified to 'b' and 'c':
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[AND($1, $2)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
This may cause error because of filter condition is a Varchar type literal.
After reading Calcite's code, I found that. The logic of RexSimplify.implify()
and RexSimplify.implifyAnd() is different, where the logic of
RexSimplify.implifyAnd() is problematic:
{code:java}
// Simplify BOOLEAN expressions if possible
while (term.getKind() == SqlKind.EQUALS) {
RexCall call = (RexCall) term;
if (call.getOperands().get(0).isAlwaysTrue()) {
term = call.getOperands().get(1);
terms.set(i, term);
continue;
} else if (call.getOperands().get(1).isAlwaysTrue()) {
term = call.getOperands().get(0);
terms.set(i, term);
continue;
}
break;
} {code}
The above code cannot make such a simple judgment, as there may not be an
implicit conversion to ensure that the types on both sides of the condition are
consistent.
> Unexpected boolean expression simplification for And expression
> ---
>
> Key: CALCITE-5907
> URL: https://issues.apache.org/jira/browse/CALCITE-5907
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Yunhong Zheng
>Priority: Major
> Fix For: 1.36.0
>
>
> As FLINK-27402 shown. If we have a table MyTable(a Int, b Boolean, c
> String). Calcite will not simplify this case ( c is Varchar type while
> SqlLiteral is boolean):
> {code:java}
> SELECT * FROM MyTable WHERE c = true;{code}
> As the logical plan is :
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[=($2, true)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
> However, Calcite will simplify this case while simplifyAnd :
> {code:java}
> SELECT * FROM MyTable WHERE b = true an
[jira] [Commented] (CALCITE-5907) Unexpected boolean expression simplification for And expression
[
https://issues.apache.org/jira/browse/CALCITE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752022#comment-17752022
]
Yunhong Zheng commented on CALCITE-5907:
Hi, [~julianhyde] . This looks like the calcite bug. Can you take a look,
thanks!
> Unexpected boolean expression simplification for And expression
> ---
>
> Key: CALCITE-5907
> URL: https://issues.apache.org/jira/browse/CALCITE-5907
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Yunhong Zheng
>Priority: Major
> Fix For: 1.36.0
>
>
> As [FLINK-27402|https://issues.apache.org/jira/browse/FLINK-27402] shown. If
> we have a table MyTable(a Int, b Boolean, c String). Calcite will not
> simplify this case ( c is Varchar type while SqlLiteral is boolean):
>
> {code:java}
> SELECT * FROM MyTable WHERE c = true;{code}
>
> As the logical plan is :
>
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[=($2, true)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
>
> However, Calcite will simplify this case while simplifyAnd :
>
> {code:java}
> SELECT * FROM MyTable WHERE b = true and c = true;{code}
>
> As the logical plan is shown below: 'b = true' and 'c = true' both were
> simplified to 'b' and 'c':
>
> {code:java}
> LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
> +- LogicalProject(inputs=[0..2])
> +- LogicalFilter(condition=[AND($1, $2)])
> +- LogicalTableScan(table=[[default_catalog, default_database,
> MyTable]]){code}
>
> This may cause error because of filter condition is a Varchar type literal.
>
> After reading Calcite's code, I found that. The logic of
> RexSimplify.implify() and RexSimplify.implifyAnd() is different, where the
> logic of RexSimplify.implifyAnd() is problematic:
> {code:java}
> // Simplify BOOLEAN expressions if possible
> while (term.getKind() == SqlKind.EQUALS) {
> RexCall call = (RexCall) term;
> if (call.getOperands().get(0).isAlwaysTrue()) {
> term = call.getOperands().get(1);
> terms.set(i, term);
> continue;
> } else if (call.getOperands().get(1).isAlwaysTrue()) {
> term = call.getOperands().get(0);
> terms.set(i, term);
> continue;
> }
> break;
> } {code}
> The above code cannot make such a simple judgment, as there may not be an
> implicit conversion to ensure that the types on both sides of the condition
> are consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[
https://issues.apache.org/jira/browse/CALCITE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752021#comment-17752021
]
LakeShen commented on CALCITE-5906:
---
Because of the TableSample percentage is float datatype, like this :
!image-2023-08-08-20-16-27-188.png|width=322,height=133!
When I tried to transform Sample to TABLESAMPLE SqlCall,it shows imprecise
precision,like this :
!image-2023-08-08-20-19-01-269.png|width=516,height=114!
I want to change the TableSample percentage to the type of precise
precision,such as bigdecimal,could anyone else give me some advice?
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png,
> image-2023-08-08-20-19-01-269.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[
https://issues.apache.org/jira/browse/CALCITE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5906:
--
Attachment: image-2023-08-08-20-19-01-269.png
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png,
> image-2023-08-08-20-19-01-269.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5907) Unexpected boolean expression simplification for And expression
Yunhong Zheng created CALCITE-5907:
--
Summary: Unexpected boolean expression simplification for And
expression
Key: CALCITE-5907
URL: https://issues.apache.org/jira/browse/CALCITE-5907
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Yunhong Zheng
Fix For: 1.36.0
As [FLINK-27402|https://issues.apache.org/jira/browse/FLINK-27402] shown. If
we have a table MyTable(a Int, b Boolean, c String). Calcite will not
simplify this case ( c is Varchar type while SqlLiteral is boolean):
{code:java}
SELECT * FROM MyTable WHERE c = true;{code}
As the logical plan is :
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[=($2, true)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
However, Calcite will simplify this case while simplifyAnd :
{code:java}
SELECT * FROM MyTable WHERE b = true and c = true;{code}
As the logical plan is shown below: 'b = true' and 'c = true' both were
simplified to 'b' and 'c':
{code:java}
LogicalSink(table=[*anonymous_collect$1*], fields=[a, b, c])
+- LogicalProject(inputs=[0..2])
+- LogicalFilter(condition=[AND($1, $2)])
+- LogicalTableScan(table=[[default_catalog, default_database,
MyTable]]){code}
This may cause error because of filter condition is a Varchar type literal.
After reading Calcite's code, I found that. The logic of RexSimplify.implify()
and RexSimplify.implifyAnd() is different, where the logic of
RexSimplify.implifyAnd() is problematic:
{code:java}
// Simplify BOOLEAN expressions if possible
while (term.getKind() == SqlKind.EQUALS) {
RexCall call = (RexCall) term;
if (call.getOperands().get(0).isAlwaysTrue()) {
term = call.getOperands().get(1);
terms.set(i, term);
continue;
} else if (call.getOperands().get(1).isAlwaysTrue()) {
term = call.getOperands().get(0);
terms.set(i, term);
continue;
}
break;
} {code}
The above code cannot make such a simple judgment, as there may not be an
implicit conversion to ensure that the types on both sides of the condition are
consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5906) RelToSqlConverter throw AssertionError when RelNode tree has the Sample RelNode
[
https://issues.apache.org/jira/browse/CALCITE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LakeShen updated CALCITE-5906:
--
Attachment: image-2023-08-08-20-16-27-188.png
> RelToSqlConverter throw AssertionError when RelNode tree has the Sample
> RelNode
> ---
>
> Key: CALCITE-5906
> URL: https://issues.apache.org/jira/browse/CALCITE-5906
> Project: Calcite
> Issue Type: Bug
>Reporter: LakeShen
>Priority: Major
> Fix For: 1.36.0
>
> Attachments: image-2023-08-08-20-16-27-188.png
>
>
> Currently in the RelToSqlConverter class, there is no visit method to handle
> the Sample RelNode.If a SQL plan has Sample RelNode, RelToSqlConverter will
> throw a AssertionError like this:
> {code:java}
> java.lang.AssertionError: Need to implement
> org.apache.calcite.rel.core.Sample
> {code}
> This will cause the Sample RelNode plan to fail to translate to the
> corresponding dialect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5870) Allow literals like DECIMAL '12.3' (consistent with Postgres)
[
https://issues.apache.org/jira/browse/CALCITE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751941#comment-17751941
]
LakeShen commented on CALCITE-5870:
---
This PR is Ready, if others have time, please help me to review it, thank you
very much:)
> Allow literals like DECIMAL '12.3' (consistent with Postgres)
> -
>
> Key: CALCITE-5870
> URL: https://issues.apache.org/jira/browse/CALCITE-5870
> Project: Calcite
> Issue Type: New Feature
> Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
> Attachments: image-2023-07-24-19-23-30-316.png,
> image-2023-07-24-19-26-44-131.png
>
>
> In Postgresql,It supports the following sql grammar:
> {code:java}
> select DECIMAL '99.999' {code}
> Postgresql converts DECIMAL '99.999' to numeric constants：
> !image-2023-07-24-19-23-30-316.png|width=399,height=186!
> Now core sql parser converts DECIMAL '99.999' to AS SqlBasicCall,like this:
> !image-2023-07-24-19-26-44-131.png|width=529,height=224!
> this jira allow literals like DECIMAL '12.3' (consistent with Postgres)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
