[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the DECIMAL data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the DECIMAL data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the DECIMAL data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the DECIMAL data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the DECIMAL data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the Decimal data type. The implementation of Decimal that

[jira] [Updated] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40032: --- Description: Spark SQL today supports the TIMESTAMP data type. > Support Decimal128 type >

[jira] [Created] (SPARK-40032) Support Decimal128 type

2022-08-10 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-40032: -- Summary: Support Decimal128 type Key: SPARK-40032 URL: https://issues.apache.org/jira/browse/SPARK-40032 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-40028) Add binary examples for string expressions

2022-08-09 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-40028: -- Summary: Add binary examples for string expressions Key: SPARK-40028 URL: https://issues.apache.org/jira/browse/SPARK-40028 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-40013) DS V2 expressions should have the default implementation of toString

2022-08-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-40013. Resolution: Won't Fix > DS V2 expressions should have the default implementation of toString >

[jira] [Created] (SPARK-40013) DS V2 expressions should have the default implementation of toString

2022-08-08 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-40013: -- Summary: DS V2 expressions should have the default implementation of toString Key: SPARK-40013 URL: https://issues.apache.org/jira/browse/SPARK-40013 Project: Spark

[jira] [Updated] (SPARK-39964) DS V2 pushdown should unify the translate path

2022-08-02 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39964: --- Summary: DS V2 pushdown should unify the translate path (was: DS V2 pushdown should unify the

[jira] [Created] (SPARK-39964) DS V2 pushdown should unify the translate API

2022-08-02 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39964: -- Summary: DS V2 pushdown should unify the translate API Key: SPARK-39964 URL: https://issues.apache.org/jira/browse/SPARK-39964 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39963) Simplify the implementation of SimplifyCasts

2022-08-02 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39963: --- Summary: Simplify the implementation of SimplifyCasts (was: Improve the implementation of

[jira] [Created] (SPARK-39963) Improve the implementation of SimplifyCasts

2022-08-02 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39963: -- Summary: Improve the implementation of SimplifyCasts Key: SPARK-39963 URL: https://issues.apache.org/jira/browse/SPARK-39963 Project: Spark Issue Type:

[jira] [Created] (SPARK-39961) DS V2 push-down translate Cast if the cast is safe

2022-08-02 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39961: -- Summary: DS V2 push-down translate Cast if the cast is safe Key: SPARK-39961 URL: https://issues.apache.org/jira/browse/SPARK-39961 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

2022-07-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39909: --- Description: Currently, JDBCV2Suite have many test cases check the push-down information looks not

[jira] [Created] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

2022-07-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39909: -- Summary: Organize the check of push down information for JDBCV2Suite Key: SPARK-39909 URL: https://issues.apache.org/jira/browse/SPARK-39909 Project: Spark

[jira] [Created] (SPARK-39908) Translate common used aggregate functions for pushdown (non ANSI)

2022-07-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39908: -- Summary: Translate common used aggregate functions for pushdown (non ANSI) Key: SPARK-39908 URL: https://issues.apache.org/jira/browse/SPARK-39908 Project: Spark

[jira] [Updated] (SPARK-39819) DS V2 aggregate push down can work with Top N or Paging (Sort with group expressions)

2022-07-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39819: --- Summary: DS V2 aggregate push down can work with Top N or Paging (Sort with group expressions)

[jira] [Updated] (SPARK-39873) Remove OptimizeLimitZero and merge it into EliminateLimits

2022-07-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39873: --- Affects Version/s: 3.4.0 (was: 2.4.0) > Remove OptimizeLimitZero and

[jira] [Created] (SPARK-39894) Combine the similar binary comparison in boolean expression.

2022-07-27 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39894: -- Summary: Combine the similar binary comparison in boolean expression. Key: SPARK-39894 URL: https://issues.apache.org/jira/browse/SPARK-39894 Project: Spark

[jira] [Created] (SPARK-39873) Remove OptimizeLimitZero and merge it into EliminateLimits

2022-07-26 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39873: -- Summary: Remove OptimizeLimitZero and merge it into EliminateLimits Key: SPARK-39873 URL: https://issues.apache.org/jira/browse/SPARK-39873 Project: Spark Issue

[jira] [Updated] (SPARK-39858) Remove unnecessary AliasHelper or PredicateHelper for some rules

2022-07-24 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39858: --- Summary: Remove unnecessary AliasHelper or PredicateHelper for some rules (was: Remove unnecessary

[jira] [Created] (SPARK-39858) Remove unnecessary AliasHelper for some rules

2022-07-24 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39858: -- Summary: Remove unnecessary AliasHelper for some rules Key: SPARK-39858 URL: https://issues.apache.org/jira/browse/SPARK-39858 Project: Spark Issue Type:

[jira] [Created] (SPARK-39836) Simplify V2ExpressionBuilder by extract common method.

2022-07-21 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39836: -- Summary: Simplify V2ExpressionBuilder by extract common method. Key: SPARK-39836 URL: https://issues.apache.org/jira/browse/SPARK-39836 Project: Spark Issue

[jira] (SPARK-39819) DS V2 aggregate push down can work with Top N or Paging (Sort with group column)

2022-07-21 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819 ] jiaan.geng deleted comment on SPARK-39819: was (Author: beliefer): I'm working on. > DS V2 aggregate push down can work with Top N or Paging (Sort with group > column) >

[jira] [Updated] (SPARK-39819) DS V2 aggregate push down can work with Top N or Paging (Sort with group column)

2022-07-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39819: --- Summary: DS V2 aggregate push down can work with Top N or Paging (Sort with group column) (was: DS

[jira] [Updated] (SPARK-39819) DS V2 aggregate push down can work with Top N or Paging

2022-07-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39819: --- Summary: DS V2 aggregate push down can work with Top N or Paging (was: DS V2 aggregate push down

[jira] [Updated] (SPARK-39819) DS V2 aggregate push down can work with Top N

2022-07-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39819: --- Description: Currently, DS V2 aggregate push-down cannot work with Top N (order by ... limit ...)

[jira] [Commented] (SPARK-39819) DS V2 aggregate push down can work with Top N

2022-07-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568809#comment-17568809 ] jiaan.geng commented on SPARK-39819: I'm working on. > DS V2 aggregate push down can work with Top

[jira] [Created] (SPARK-39819) DS V2 aggregate push down can work with Top N

2022-07-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39819: -- Summary: DS V2 aggregate push down can work with Top N Key: SPARK-39819 URL: https://issues.apache.org/jira/browse/SPARK-39819 Project: Spark Issue Type:

[jira] [Created] (SPARK-39812) Simplify code to construct AggregateExpression with toAggregateExpression

2022-07-18 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39812: -- Summary: Simplify code to construct AggregateExpression with toAggregateExpression Key: SPARK-39812 URL: https://issues.apache.org/jira/browse/SPARK-39812 Project: Spark

[jira] [Created] (SPARK-39808) Support aggregate function MODE

2022-07-18 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39808: -- Summary: Support aggregate function MODE Key: SPARK-39808 URL: https://issues.apache.org/jira/browse/SPARK-39808 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39788: --- Description: Currently, processIndexProperties and getSupportedIndexTypeList in JdbcUtils have one

[jira] [Updated] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39788: --- Description: Currently, processIndexProperties and getSupportedIndexTypeList in JdbcUtils have one

[jira] [Updated] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39788: --- Description: Currently, > Rename catalogName to dialectName for JdbcUtils >

[jira] [Updated] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39788: --- Summary: Rename catalogName to dialectName for JdbcUtils (was: Implement createIndex and

[jira] [Commented] (SPARK-39788) Implement createIndex and IndexExists in JDBC (DB2 dialect)

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567083#comment-17567083 ] jiaan.geng commented on SPARK-39788: I'm working on. > Implement createIndex and IndexExists in

[jira] [Created] (SPARK-39788) Implement createIndex and IndexExists in JDBC (DB2 dialect)

2022-07-14 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39788: -- Summary: Implement createIndex and IndexExists in JDBC (DB2 dialect) Key: SPARK-39788 URL: https://issues.apache.org/jira/browse/SPARK-39788 Project: Spark

[jira] [Updated] (SPARK-39384) Compile build-in linear regression aggregate functions for JDBC dialect

2022-07-14 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39384: --- Summary: Compile build-in linear regression aggregate functions for JDBC dialect (was: Compile

[jira] [Created] (SPARK-39773) Update document of JDBC options for pushDownOffset

2022-07-13 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39773: -- Summary: Update document of JDBC options for pushDownOffset Key: SPARK-39773 URL: https://issues.apache.org/jira/browse/SPARK-39773 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39737) PERCENTILE_CONT and PERCENTILE_DISC should support aggregate filter

2022-07-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39737: --- Summary: PERCENTILE_CONT and PERCENTILE_DISC should support aggregate filter (was:

[jira] [Created] (SPARK-39737) percentile_cont and percentile_disc should support aggregate filter

2022-07-10 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39737: -- Summary: percentile_cont and percentile_disc should support aggregate filter Key: SPARK-39737 URL: https://issues.apache.org/jira/browse/SPARK-39737 Project: Spark

[jira] [Created] (SPARK-39707) Add SQL reference for aggregate functions

2022-07-07 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39707: -- Summary: Add SQL reference for aggregate functions Key: SPARK-39707 URL: https://issues.apache.org/jira/browse/SPARK-39707 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39651) Prune filter condition if compare with rand is deterministic

2022-07-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39651: --- Summary: Prune filter condition if compare with rand is deterministic (was: Prune filter condition

[jira] [Updated] (SPARK-39627) DS V2 pushdown should unify the compile API

2022-07-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39627: --- Summary: DS V2 pushdown should unify the compile API (was: Unify compileAggregate and

[jira] (SPARK-39627) Unify compileAggregate and compileExpression

2022-07-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39627 ] jiaan.geng deleted comment on SPARK-39627: was (Author: beliefer): I'm working on. > Unify compileAggregate and compileExpression > > >

[jira] [Updated] (SPARK-39651) Prune filter condition compare rand function with foldable expression

2022-06-30 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39651: --- Description: The SQL {code:java} SELECT * FROM tab WHERE rand(1) < 2 {code} evaluate rand(1) < 2

[jira] [Created] (SPARK-39651) Prune filter condition compare rand function with foldable expression

2022-06-30 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39651: -- Summary: Prune filter condition compare rand function with foldable expression Key: SPARK-39651 URL: https://issues.apache.org/jira/browse/SPARK-39651 Project: Spark

[jira] [Created] (SPARK-39627) Unify compileAggregate and compileExpression

2022-06-28 Thread jiaan.geng (Jira)
Title: Message Title jiaan.geng created

[jira] [Commented] (SPARK-39627) Unify compileAggregate and compileExpression

2022-06-28 Thread jiaan.geng (Jira)
Title: Message Title jiaan.geng commented

[jira] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148 ] jiaan.geng deleted comment on SPARK-39148: was (Author: beliefer): I'm working on. > DS V2 aggregate push down can work with OFFSET or LIMIT >

[jira] [Commented] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17558943#comment-17558943 ] jiaan.geng commented on SPARK-39148: I'm working on. > DS V2 aggregate push down can work with

[jira] [Updated] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39148: --- Description: Currently, DS V2 aggregate push-down cannot work with OFFSET and LIMIT. If it can work

[jira] [Updated] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39148: --- Description: Currently, DS V2 aggregate push-down cannot work with OFFSET and LIMIT. If we can

[jira] [Updated] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39148: --- Description: Currently, DS V2 aggregate push-down cannot work with OFFSET and LIMIT. If it can

[jira] [Updated] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39148: --- Summary: DS V2 aggregate push down can work with OFFSET or LIMIT (was: Support push down OFFSET

[jira] [Reopened] (SPARK-39148) Support push down OFFSET append LIMIT to JDBC data source V2

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng reopened SPARK-39148: > Support push down OFFSET append LIMIT to JDBC data source V2 >

[jira] (SPARK-39148) Support push down OFFSET append LIMIT to JDBC data source V2

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148 ] jiaan.geng deleted comment on SPARK-39148: was (Author: beliefer): I'm working on. > Support push down OFFSET append LIMIT to JDBC data source V2 >

[jira] [Resolved] (SPARK-39148) Support push down OFFSET append LIMIT to JDBC data source V2

2022-06-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-39148. Resolution: Duplicate > Support push down OFFSET append LIMIT to JDBC data source V2 >

[jira] [Updated] (SPARK-39577) Add SQL reference for built-in functions

2022-06-24 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39577: --- Summary: Add SQL reference for built-in functions (was: Add document into SQL reference for

[jira] [Created] (SPARK-39577) Add document into SQL reference for built-in functions

2022-06-24 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39577: -- Summary: Add document into SQL reference for built-in functions Key: SPARK-39577 URL: https://issues.apache.org/jira/browse/SPARK-39577 Project: Spark Issue

[jira] [Created] (SPARK-39537) Ensure that the expressions order returned by InferFiltersFromConstraints

2022-06-20 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39537: -- Summary: Ensure that the expressions order returned by InferFiltersFromConstraints Key: SPARK-39537 URL: https://issues.apache.org/jira/browse/SPARK-39537 Project: Spark

[jira] [Updated] (SPARK-39139) DS V2 supports push down DS V2 UDF

2022-06-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39139: --- Summary: DS V2 supports push down DS V2 UDF (was: DS V2 supports push-down DS V2 UDF) > DS V2

[jira] [Updated] (SPARK-39139) DS V2 supports push-down DS V2 UDF

2022-06-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39139: --- Summary: DS V2 supports push-down DS V2 UDF (was: DS V2 push-down supports DS V2 UDF) > DS V2

[jira] [Updated] (SPARK-38978) DS V2 supports push down OFFSET operator

2022-06-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38978: --- Summary: DS V2 supports push down OFFSET operator (was: Support push down OFFSET to DS V2) > DS

[jira] [Updated] (SPARK-38978) Support push down OFFSET to DS V2

2022-06-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38978: --- Summary: Support push down OFFSET to DS V2 (was: Support push down OFFSET to JDBC data source V2)

[jira] [Created] (SPARK-39479) DS V2 supports push down math functions(non ANSI)

2022-06-15 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39479: -- Summary: DS V2 supports push down math functions(non ANSI) Key: SPARK-39479 URL: https://issues.apache.org/jira/browse/SPARK-39479 Project: Spark Issue Type:

[jira] [Created] (SPARK-39453) DS V2 supports push down misc non-aggregate functions(non ANSI)

2022-06-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39453: -- Summary: DS V2 supports push down misc non-aggregate functions(non ANSI) Key: SPARK-39453 URL: https://issues.apache.org/jira/browse/SPARK-39453 Project: Spark

[jira] [Updated] (SPARK-38852) Better Data Source V2 operator pushdown framework

2022-06-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38852: --- Description: Currently, Spark supports push down Filters and Aggregates to data source. However,

[jira] [Updated] (SPARK-38852) Better Data Source V2 operator pushdown framework

2022-06-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38852: --- Description: Currently, Spark supports push down Filters and Aggregates to data source. However,

[jira] [Updated] (SPARK-38852) Better Data Source V2 operator pushdown framework

2022-06-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38852: --- Description: Currently, Spark supports push down Filters and Aggregates to data source. However,

[jira] [Created] (SPARK-39413) Capitalize sql keywords in JDBCV2Suite

2022-06-08 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39413: -- Summary: Capitalize sql keywords in JDBCV2Suite Key: SPARK-39413 URL: https://issues.apache.org/jira/browse/SPARK-39413 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-06-05 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39385: -- Summary: Translate linear regression aggregate functions for pushdown Key: SPARK-39385 URL: https://issues.apache.org/jira/browse/SPARK-39385 Project: Spark

[jira] [Created] (SPARK-39384) Compile linear regression aggregate functions of build-in JDBC dialect

2022-06-05 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39384: -- Summary: Compile linear regression aggregate functions of build-in JDBC dialect Key: SPARK-39384 URL: https://issues.apache.org/jira/browse/SPARK-39384 Project: Spark

[jira] [Resolved] (SPARK-39065) DS V2 Limit push-down should avoid out of memory

2022-06-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-39065. Resolution: Won't Fix We can't avoid OOM. > DS V2 Limit push-down should avoid out of memory >

[jira] [Commented] (SPARK-39320) Add the MEDIAN() function

2022-05-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543442#comment-17543442 ] jiaan.geng commented on SPARK-39320: [~maxgekk]Thank you for the ping. I will do the job. > Add the

[jira] [Updated] (SPARK-37623) Support ANSI Aggregate Function: regr_intercept

2022-05-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37623: --- Description: REGR_INTERCEPT is an ANSI aggregate functions. many database support it. (was:

[jira] [Updated] (SPARK-37623) Support ANSI Aggregate Function: regr_intercept

2022-05-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37623: --- Summary: Support ANSI Aggregate Function: regr_intercept (was: Support ANSI Aggregate Function:

[jira] [Created] (SPARK-39270) JDBC dialect supports registering dialect specific functions

2022-05-24 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39270: -- Summary: JDBC dialect supports registering dialect specific functions Key: SPARK-39270 URL: https://issues.apache.org/jira/browse/SPARK-39270 Project: Spark

[jira] [Updated] (SPARK-39230) Support ANSI Aggregate Function: regr_slope

2022-05-18 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39230: --- Description: REGR_SLOPE is an ANSI aggregate functions. many database support it. > Support ANSI

[jira] [Created] (SPARK-39230) Support ANSI Aggregate Function: regr_slope

2022-05-18 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39230: -- Summary: Support ANSI Aggregate Function: regr_slope Key: SPARK-39230 URL: https://issues.apache.org/jira/browse/SPARK-39230 Project: Spark Issue Type: Sub-task

[jira] (SPARK-39139) DS V2 push-down supports DS V2 UDF

2022-05-18 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139 ] jiaan.geng deleted comment on SPARK-39139: was (Author: beliefer): I'm working on. > DS V2 push-down supports DS V2 UDF > -- > > Key:

[jira] [Updated] (SPARK-39139) DS V2 push-down supports DS V2 UDF

2022-05-18 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39139: --- Summary: DS V2 push-down supports DS V2 UDF (was: DS V2 push-down supports UDF V2) > DS V2

[jira] [Updated] (SPARK-39171) Unify the Cast expression

2022-05-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39171: --- Issue Type: Improvement (was: New Feature) > Unify the Cast expression > -

[jira] [Created] (SPARK-39171) Unify the Cast expression

2022-05-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39171: -- Summary: Unify the Cast expression Key: SPARK-39171 URL: https://issues.apache.org/jira/browse/SPARK-39171 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-39162) Jdbc dialect should decide which function could be pushed down.

2022-05-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39162: --- Description: Regardless of whether the functions are ANSI or not, most databases are actually

[jira] [Created] (SPARK-39162) Jdbc dialect should decide which function could be pushed down.

2022-05-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39162: -- Summary: Jdbc dialect should decide which function could be pushed down. Key: SPARK-39162 URL: https://issues.apache.org/jira/browse/SPARK-39162 Project: Spark

[jira] [Created] (SPARK-39159) Add new Dataset API for Offset

2022-05-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39159: -- Summary: Add new Dataset API for Offset Key: SPARK-39159 URL: https://issues.apache.org/jira/browse/SPARK-39159 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-39157) H2Dialect should override getJDBCType so as make the data type is correct

2022-05-11 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39157: -- Summary: H2Dialect should override getJDBCType so as make the data type is correct Key: SPARK-39157 URL: https://issues.apache.org/jira/browse/SPARK-39157 Project: Spark

[jira] [Updated] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37463: --- Parent: SPARK-35662 Issue Type: Sub-task (was: Bug) > Read/Write Timestamp ntz from/to Orc

[jira] [Updated] (SPARK-38504) Can't read TimestampNTZ as TimestampLTZ

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38504: --- Parent: SPARK-35662 Issue Type: Sub-task (was: Improvement) > Can't read TimestampNTZ as

[jira] [Created] (SPARK-39148) Support push down OFFSET append LIMIT to JDBC data source V2

2022-05-11 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39148: -- Summary: Support push down OFFSET append LIMIT to JDBC data source V2 Key: SPARK-39148 URL: https://issues.apache.org/jira/browse/SPARK-39148 Project: Spark

[jira] [Updated] (SPARK-38978) Support push down OFFSET to JDBC data source V2

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38978: --- Summary: Support push down OFFSET to JDBC data source V2 (was: Support push down offset to JDBC

[jira] [Commented] (SPARK-39148) Support push down OFFSET append LIMIT to JDBC data source V2

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534737#comment-17534737 ] jiaan.geng commented on SPARK-39148: I'm working on. > Support push down OFFSET append LIMIT to

[jira] [Updated] (SPARK-38978) Support push down offset to JDBC data source V2

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38978: --- Description: Currently, DS V2 push-down supports LIMIT but OFFSET. If we can pushing down OFFSET to

[jira] [Updated] (SPARK-38978) Support push down offset to JDBC data source V2

2022-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38978: --- Summary: Support push down offset to JDBC data source V2 (was: Support push down limit and offset

[jira] [Commented] (SPARK-39139) DS V2 push-down supports UDF V2

2022-05-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534292#comment-17534292 ] jiaan.geng commented on SPARK-39139: I'm working on. > DS V2 push-down supports UDF V2 >

[jira] [Created] (SPARK-39139) DS V2 push-down supports UDF V2

2022-05-10 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39139: -- Summary: DS V2 push-down supports UDF V2 Key: SPARK-39139 URL: https://issues.apache.org/jira/browse/SPARK-39139 Project: Spark Issue Type: Sub-task

<    1   2   3   4   5   6   7   8   9   10   >