[jira] [Resolved] (SPARK-39776) Join‘ verbose string didn't contains JoinType

2022-07-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39776. - Fix Version/s: 3.4.0 Assignee: angerszhu Resolution: Fixed > Join‘ verbose

[jira] [Resolved] (SPARK-39836) Simplify V2ExpressionBuilder by extract common method.

2022-07-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39836. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37249

[jira] [Assigned] (SPARK-39836) Simplify V2ExpressionBuilder by extract common method.

2022-07-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39836: --- Assignee: jiaan.geng > Simplify V2ExpressionBuilder by extract common method. >

[jira] [Resolved] (SPARK-39469) Infer date type for CSV schema inference

2022-07-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39469. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36871

[jira] [Updated] (SPARK-38916) Tasks not killed caused by race conditions between killTask() and launchTask()

2022-07-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-38916: Fix Version/s: (was: 3.2.2) > Tasks not killed caused by race conditions between killTask()

[jira] [Assigned] (SPARK-39469) Infer date type for CSV schema inference

2022-07-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39469: --- Assignee: Jonathan Cui > Infer date type for CSV schema inference >

[jira] [Assigned] (SPARK-39810) tableExists can reuse getTable code

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39810: --- Assignee: Rui Wang > tableExists can reuse getTable code >

[jira] [Resolved] (SPARK-39810) tableExists can reuse getTable code

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39810. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37220

[jira] [Resolved] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39148. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37195

[jira] [Assigned] (SPARK-39148) DS V2 aggregate push down can work with OFFSET or LIMIT

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39148: --- Assignee: jiaan.geng > DS V2 aggregate push down can work with OFFSET or LIMIT >

[jira] [Assigned] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37287: --- Assignee: Allison Wang > Pull out dynamic partition and bucket sort from FileFormatWriter

[jira] [Resolved] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

2022-07-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37287. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37099

[jira] [Resolved] (SPARK-39806) Queries accessing METADATA struct crash on partitioned tables

2022-07-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39806. - Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-39806) Queries accessing METADATA struct crash on partitioned tables

2022-07-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39806: --- Assignee: Ala Luszczak > Queries accessing METADATA struct crash on partitioned tables >

[jira] [Assigned] (SPARK-39773) Update document of JDBC options for pushDownOffset

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39773: --- Assignee: jiaan.geng > Update document of JDBC options for pushDownOffset >

[jira] [Resolved] (SPARK-39773) Update document of JDBC options for pushDownOffset

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39773. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37186

[jira] [Resolved] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39788. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37201

[jira] [Assigned] (SPARK-39788) Rename catalogName to dialectName for JdbcUtils

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39788: --- Assignee: jiaan.geng > Rename catalogName to dialectName for JdbcUtils >

[jira] [Assigned] (SPARK-38904) Low cost DataFrame schema swap util

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38904: --- Assignee: Wenchen Fan > Low cost DataFrame schema swap util >

[jira] [Resolved] (SPARK-38904) Low cost DataFrame schema swap util

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38904. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37011

[jira] [Assigned] (SPARK-39625) add Dataset.as(StructType)

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39625: --- Assignee: Wenchen Fan > add Dataset.as(StructType) > -- > >

[jira] [Resolved] (SPARK-39625) add Dataset.as(StructType)

2022-07-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39625. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37011

[jira] [Resolved] (SPARK-39741) Support url encode/decode as built-in function

2022-07-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39741. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37113

[jira] [Assigned] (SPARK-39741) Support url encode/decode as built-in function

2022-07-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39741: --- Assignee: Yi kaifei > Support url encode/decode as built-in function >

[jira] [Resolved] (SPARK-39772) namespace should be null when database is null in the old constructors

2022-07-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39772. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37184

[jira] [Assigned] (SPARK-39772) namespace should be null when database is null in the old constructors

2022-07-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39772: --- Assignee: Ruifeng Zheng > namespace should be null when database is null in the old

[jira] [Assigned] (SPARK-39767) Remove UnresolvedDBObjectName and add UnresolvedIdentifier

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39767: --- Assignee: Wenchen Fan > Remove UnresolvedDBObjectName and add UnresolvedIdentifier >

[jira] [Resolved] (SPARK-39767) Remove UnresolvedDBObjectName and add UnresolvedIdentifier

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39767. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37178

[jira] [Resolved] (SPARK-39672) NotExists subquery failed with conflicting attributes

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39672. - Fix Version/s: 3.3.1 3.2.2 3.1.4 3.4.0

[jira] [Assigned] (SPARK-39672) NotExists subquery failed with conflicting attributes

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39672: --- Assignee: Manu Zhang > NotExists subquery failed with conflicting attributes >

[jira] [Created] (SPARK-39767) Remove UnresolvedDBObjectName and add UnresolvedIdentifier

2022-07-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39767: --- Summary: Remove UnresolvedDBObjectName and add UnresolvedIdentifier Key: SPARK-39767 URL: https://issues.apache.org/jira/browse/SPARK-39767 Project: Spark

[jira] [Resolved] (SPARK-39751) Better naming for hash aggregate key probing metric

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39751. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37164

[jira] [Assigned] (SPARK-39751) Better naming for hash aggregate key probing metric

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39751: --- Assignee: Cheng Su > Better naming for hash aggregate key probing metric >

[jira] [Created] (SPARK-39764) Make PhysicalOperation the same as ScanOperation

2022-07-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39764: --- Summary: Make PhysicalOperation the same as ScanOperation Key: SPARK-39764 URL: https://issues.apache.org/jira/browse/SPARK-39764 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-39651) Prune filter condition if compare with rand is deterministic

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39651. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37040

[jira] [Assigned] (SPARK-39651) Prune filter condition if compare with rand is deterministic

2022-07-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39651: --- Assignee: jiaan.geng > Prune filter condition if compare with rand is deterministic >

[jira] [Resolved] (SPARK-39699) Make CollapseProject smarter about collection creation expressions

2022-07-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39699. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37165

[jira] [Assigned] (SPARK-35208) Add docs for LATERAL subqueries

2022-07-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35208: --- Assignee: Huaxin Gao > Add docs for LATERAL subqueries > --- >

[jira] [Resolved] (SPARK-35208) Add docs for LATERAL subqueries

2022-07-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35208. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37080

[jira] [Resolved] (SPARK-39737) PERCENTILE_CONT and PERCENTILE_DISC should support aggregate filter

2022-07-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39737. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37150

[jira] [Assigned] (SPARK-39737) PERCENTILE_CONT and PERCENTILE_DISC should support aggregate filter

2022-07-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39737: --- Assignee: jiaan.geng > PERCENTILE_CONT and PERCENTILE_DISC should support aggregate filter

[jira] [Resolved] (SPARK-39627) DS V2 pushdown should unify the compile API

2022-07-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39627. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37047

[jira] [Assigned] (SPARK-39627) DS V2 pushdown should unify the compile API

2022-07-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39627: --- Assignee: jiaan.geng > DS V2 pushdown should unify the compile API >

[jira] [Resolved] (SPARK-38899) DS V2 supports push down datetime functions

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38899. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36663

[jira] [Assigned] (SPARK-38899) DS V2 supports push down datetime functions

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38899: --- Assignee: jiaan.geng > DS V2 supports push down datetime functions >

[jira] [Resolved] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39385. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36773

[jira] [Assigned] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39385: --- Assignee: jiaan.geng > Translate linear regression aggregate functions for pushdown >

[jira] [Updated] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-38018: Fix Version/s: 3.2.2 > Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly >

[jira] [Resolved] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39447. - Fix Version/s: 3.2.2 3.4.0 3.3.1 Resolution: Fixed

[jira] [Reopened] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-39447: - > Only non-broadcast query stage can propagate empty relation >

[jira] [Assigned] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39447: --- Assignee: XiDuo You > Only non-broadcast query stage can propagate empty relation >

[jira] [Resolved] (SPARK-33753) Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33753. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37043

[jira] [Assigned] (SPARK-33753) Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33753: --- Assignee: dzcxzl > Reduce the memory footprint and gc of the cache (hadoopJobMetadata) >

[jira] [Assigned] (SPARK-39679) TakeOrderedAndProjectExec should respect child output ordering

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39679: --- Assignee: XiDuo You > TakeOrderedAndProjectExec should respect child output ordering >

[jira] [Resolved] (SPARK-39679) TakeOrderedAndProjectExec should respect child output ordering

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39679. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37085

[jira] [Commented] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563530#comment-17563530 ] Wenchen Fan commented on SPARK-39622: - > ParquetIOSuite fails consistently with master branch. I'm

[jira] [Created] (SPARK-39699) Make CollapseProject smarter about collection creation expressions

2022-07-06 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39699: --- Summary: Make CollapseProject smarter about collection creation expressions Key: SPARK-39699 URL: https://issues.apache.org/jira/browse/SPARK-39699 Project: Spark

[jira] [Resolved] (SPARK-39584) Fix TPCDSQueryBenchmark Measuring Performance of Wrong Query Results

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39584. - Fix Version/s: 3.4.0 Assignee: Kazuyuki Tanimura Resolution: Fixed > Fix

[jira] [Assigned] (SPARK-39579) Make ListFunctions/getFunction/functionExists API compatible

2022-07-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39579: --- Assignee: Ruifeng Zheng > Make ListFunctions/getFunction/functionExists API compatible >

[jira] [Resolved] (SPARK-39579) Make ListFunctions/getFunction/functionExists API compatible

2022-07-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39579. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36977

[jira] [Resolved] (SPARK-39649) Make listDatabases / getDatabase / listColumns / refreshTable in PySpark support 3-layer-namespace

2022-07-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39649. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37039

[jira] [Assigned] (SPARK-39649) Make listDatabases / getDatabase / listColumns / refreshTable in PySpark support 3-layer-namespace

2022-07-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39649: --- Assignee: Ruifeng Zheng > Make listDatabases / getDatabase / listColumns / refreshTable in

[jira] [Assigned] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39503: --- Assignee: XiDuo You > Add session catalog name for v1 database table and function >

[jira] [Resolved] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39503. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37021

[jira] [Assigned] (SPARK-39645) Make getDatabase and listDatabases compatible with 3 layer namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39645: --- Assignee: Zach Schuermann > Make getDatabase and listDatabases compatible with 3 layer

[jira] [Resolved] (SPARK-39645) Make getDatabase and listDatabases compatible with 3 layer namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39645. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36968

[jira] [Resolved] (SPARK-39583) Make RefreshTable be compatible with 3 layer namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39583. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36983

[jira] [Assigned] (SPARK-39583) Make RefreshTable be compatible with 3 layer namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39583: --- Assignee: Rui Wang > Make RefreshTable be compatible with 3 layer namespace >

[jira] [Assigned] (SPARK-39139) DS V2 supports push down DS V2 UDF

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39139: --- Assignee: jiaan.geng > DS V2 supports push down DS V2 UDF >

[jira] [Resolved] (SPARK-39139) DS V2 supports push down DS V2 UDF

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39139. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36593

[jira] [Assigned] (SPARK-39597) Make GetTable, TableExists and DatabaseExists in the python side support 3-layer-namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39597: --- Assignee: Ruifeng Zheng > Make GetTable, TableExists and DatabaseExists in the python side

[jira] [Resolved] (SPARK-39597) Make GetTable, TableExists and DatabaseExists in the python side support 3-layer-namespace

2022-06-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39597. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36985

[jira] [Resolved] (SPARK-39570) inline table should allow expressions with alias

2022-06-28 Thread Wenchen Fan (Jira)
Title: Message Title Wenchen Fan resolved

[jira] [Assigned] (SPARK-39570) inline table should allow expressions with alias

2022-06-28 Thread Wenchen Fan (Jira)
Title: Message Title Wenchen Fan assigned

[jira] [Resolved] (SPARK-39479) DS V2 supports push down math functions(non ANSI)

2022-06-28 Thread Wenchen Fan (Jira)
Title: Message Title Wenchen Fan resolved

[jira] [Assigned] (SPARK-39479) DS V2 supports push down math functions(non ANSI)

2022-06-28 Thread Wenchen Fan (Jira)
Title: Message Title Wenchen Fan assigned

[jira] [Created] (SPARK-39625) add Dataset.as(StructType)

2022-06-27 Thread Wenchen Fan (Jira)
Title: Message Title Wenchen Fan created

[jira] [Resolved] (SPARK-39577) Add SQL reference for built-in functions

2022-06-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39577. - Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-39577) Add SQL reference for built-in functions

2022-06-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39577: --- Assignee: jiaan.geng > Add SQL reference for built-in functions >

[jira] [Assigned] (SPARK-39506) CacheTable, isCached, UncacheTable, setCurrentCatalog, currentCatalog, listCatalogs

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39506: --- Assignee: Rui Wang > CacheTable, isCached, UncacheTable, setCurrentCatalog,

[jira] [Resolved] (SPARK-39506) CacheTable, isCached, UncacheTable, setCurrentCatalog, currentCatalog, listCatalogs

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39506. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36904

[jira] [Assigned] (SPARK-39453) DS V2 supports push down misc non-aggregate functions(non ANSI)

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39453: --- Assignee: jiaan.geng > DS V2 supports push down misc non-aggregate functions(non ANSI) >

[jira] [Resolved] (SPARK-39453) DS V2 supports push down misc non-aggregate functions(non ANSI)

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39453. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36830

[jira] [Resolved] (SPARK-38978) DS V2 supports push down OFFSET operator

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38978. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36295

[jira] [Assigned] (SPARK-38978) DS V2 supports push down OFFSET operator

2022-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38978: --- Assignee: jiaan.geng > DS V2 supports push down OFFSET operator >

[jira] [Created] (SPARK-39570) inline table should allow expressions with alias

2022-06-23 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39570: --- Summary: inline table should allow expressions with alias Key: SPARK-39570 URL: https://issues.apache.org/jira/browse/SPARK-39570 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39548. - Fix Version/s: 3.3.1 3.2.2 3.4.0 Resolution: Fixed

[jira] [Assigned] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39548: --- Assignee: Rui Wang > CreateView Command with a window clause query hit a wrong window

[jira] [Assigned] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39547: --- Assignee: Prashant Singh > V2SessionCatalog should not throw NoSuchDatabaseException in >

[jira] [Resolved] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39547. - Fix Version/s: 3.4.0 3.3.1 Resolution: Fixed > V2SessionCatalog

[jira] [Assigned] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39543: --- Assignee: Yi kaifei > The option of DataFrameWriterV2 should be passed to storage

[jira] [Resolved] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39543. - Fix Version/s: 3.3.1 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-39340) DS v2 agg pushdown should allow dots in the name of top-level columns

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-39340: Fix Version/s: 3.4.0 3.3.1 > DS v2 agg pushdown should allow dots in the name

[jira] [Assigned] (SPARK-39551) Add AQE invalid plan check

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39551: --- Assignee: Wei Xue > Add AQE invalid plan check > -- > >

[jira] [Resolved] (SPARK-39551) Add AQE invalid plan check

2022-06-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39551. - Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-39195) Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

2022-06-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-39195: Summary: Spark OutputCommitCoordinator should abort stage when committed file not consistent with

[jira] [Assigned] (SPARK-39195) Spark should use two step update of outputCommitCoordinator

2022-06-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39195: --- Assignee: angerszhu > Spark should use two step update of outputCommitCoordinator >

[jira] [Resolved] (SPARK-39195) Spark should use two step update of outputCommitCoordinator

2022-06-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39195. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36564

[jira] [Resolved] (SPARK-39537) Ensure that the expressions order returned by InferFiltersFromConstraints

2022-06-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39537. - Resolution: Cannot Reproduce > Ensure that the expressions order returned by

[jira] [Resolved] (SPARK-39263) GetTable, TableExists and DatabaseExists

2022-06-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39263. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36641

<    10   11   12   13   14   15   16   17   18   19   >