[jira] [Resolved] (SPARK-48065) SPJ: allowJoinKeysSubsetOfPartitionKeys is too strict

2024-05-02 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-48065. -- Resolution: Fixed Issue resolved by pull request 46325 [https://github.com/apache/spark/pull/46325]

[jira] [Assigned] (SPARK-48065) SPJ: allowJoinKeysSubsetOfPartitionKeys is too strict

2024-05-02 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-48065: Assignee: Szehon Ho > SPJ: allowJoinKeysSubsetOfPartitionKeys is too strict >

[jira] [Resolved] (SPARK-48030) InternalRowComparableWrapper should cache rowOrdering to improve performace

2024-04-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-48030. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46265

[jira] [Assigned] (SPARK-48030) InternalRowComparableWrapper should cache rowOrdering to improve performace

2024-04-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-48030: Assignee: YE > InternalRowComparableWrapper should cache rowOrdering to improve performace >

[jira] [Resolved] (SPARK-47094) SPJ : Dynamically rebalance number of buckets when they are not equal

2024-04-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-47094. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45267

[jira] [Resolved] (SPARK-42040) SPJ: Introduce a new API for V2 input partition to report partition size

2024-03-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42040. -- Fix Version/s: 4.0.0 Resolution: Fixed > SPJ: Introduce a new API for V2 input partition to

[jira] [Assigned] (SPARK-42040) SPJ: Introduce a new API for V2 input partition to report partition size

2024-03-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42040: Assignee: Qi Zhu (was: zhuqi) > SPJ: Introduce a new API for V2 input partition to report

[jira] [Assigned] (SPARK-42040) SPJ: Introduce a new API for V2 input partition to report partition size

2024-03-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42040: Assignee: zhuqi > SPJ: Introduce a new API for V2 input partition to report partition size >

[jira] [Resolved] (SPARK-45731) Update partition statistics with ANALYZE TABLE command

2023-11-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45731. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43629

[jira] [Assigned] (SPARK-45731) Update partition statistics with ANALYZE TABLE command

2023-11-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45731: Assignee: Chao Sun > Update partition statistics with ANALYZE TABLE command >

[jira] [Created] (SPARK-45846) spark.sql.optimizeNullAwareAntiJoin should respect spark.sql.autoBroadcastJoinThreshold

2023-11-08 Thread Chao Sun (Jira)
Chao Sun created SPARK-45846: Summary: spark.sql.optimizeNullAwareAntiJoin should respect spark.sql.autoBroadcastJoinThreshold Key: SPARK-45846 URL: https://issues.apache.org/jira/browse/SPARK-45846

[jira] [Created] (SPARK-45731) Update partition statistics with ANALYZE TABLE command

2023-10-30 Thread Chao Sun (Jira)
Chao Sun created SPARK-45731: Summary: Update partition statistics with ANALYZE TABLE command Key: SPARK-45731 URL: https://issues.apache.org/jira/browse/SPARK-45731 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45652) SPJ: Handle empty input partitions after dynamic filtering

2023-10-30 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-45652: - Description: When the number of input partitions become 0 after dynamic filtering, in

[jira] [Assigned] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

2023-10-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45678: Assignee: L. C. Hsieh > Cover BufferReleasingInputStream.available under

[jira] [Resolved] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

2023-10-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45678. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43543

[jira] [Resolved] (SPARK-45652) SPJ: Handle empty input partitions after dynamic filtering

2023-10-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45652. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43531

[jira] [Assigned] (SPARK-45652) SPJ: Handle empty input partitions after dynamic filtering

2023-10-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45652: Assignee: Chao Sun > SPJ: Handle empty input partitions after dynamic filtering >

[jira] [Created] (SPARK-45652) SPJ: Handle empty input partitions after dynamic filtering

2023-10-24 Thread Chao Sun (Jira)
Chao Sun created SPARK-45652: Summary: SPJ: Handle empty input partitions after dynamic filtering Key: SPARK-45652 URL: https://issues.apache.org/jira/browse/SPARK-45652 Project: Spark Issue

[jira] [Assigned] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method

2023-09-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-44913: Assignee: Xianyang Liu > DS V2 supports push down V2 UDF that has magic method >

[jira] [Resolved] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method

2023-09-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-44913. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42612

[jira] [Assigned] (SPARK-45365) Allow the daily tests of branch-3.4 to use the new test group tags

2023-09-28 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45365: Assignee: Yang Jie > Allow the daily tests of branch-3.4 to use the new test group tags >

[jira] [Resolved] (SPARK-45365) Allow the daily tests of branch-3.4 to use the new test group tags

2023-09-28 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45365. -- Fix Version/s: 4.0.0 Resolution: Fixed > Allow the daily tests of branch-3.4 to use the new

[jira] [Resolved] (SPARK-36695) Allow passing V2 functions to data sources via V2 filters

2023-09-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-36695. -- Resolution: Duplicate > Allow passing V2 functions to data sources via V2 filters >

[jira] [Updated] (SPARK-44647) SPJ: Support SPJ when join key is subset of partition keys

2023-09-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-44647: - Summary: SPJ: Support SPJ when join key is subset of partition keys (was: Support SPJ when join key is

[jira] [Updated] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

2023-09-01 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-45054: - Affects Version/s: 3.4.1 3.3.2 3.2.4

[jira] [Updated] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

2023-09-01 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-45054: - Fix Version/s: 3.4.2 3.5.0 > HiveExternalCatalog.listPartitions should restore Spark

[jira] [Resolved] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

2023-09-01 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45054. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42777

[jira] [Assigned] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

2023-09-01 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45054: Assignee: Chao Sun > HiveExternalCatalog.listPartitions should restore Spark SQL stats >

[jira] [Created] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

2023-09-01 Thread Chao Sun (Jira)
Chao Sun created SPARK-45054: Summary: HiveExternalCatalog.listPartitions should restore Spark SQL stats Key: SPARK-45054 URL: https://issues.apache.org/jira/browse/SPARK-45054 Project: Spark

[jira] [Created] (SPARK-45036) SPJ: Refactor logic to handle partially clustered distribution

2023-08-31 Thread Chao Sun (Jira)
Chao Sun created SPARK-45036: Summary: SPJ: Refactor logic to handle partially clustered distribution Key: SPARK-45036 URL: https://issues.apache.org/jira/browse/SPARK-45036 Project: Spark

[jira] [Assigned] (SPARK-41471) SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

2023-08-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-41471: Assignee: Jia Fan > SPJ: Reduce Spark shuffle when only one side of a join is >

[jira] [Resolved] (SPARK-41471) SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

2023-08-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-41471. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42194

[jira] [Resolved] (SPARK-44641) SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

2023-08-07 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-44641. -- Fix Version/s: 3.4.2 3.5.0 Assignee: Chao Sun Resolution: Fixed >

[jira] [Commented] (SPARK-44660) Relax constraint for columnar shuffle check in AQE

2023-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750881#comment-17750881 ] Chao Sun commented on SPARK-44660: -- In fact the check is necessary, but it seems {code}

[jira] [Updated] (SPARK-44641) SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

2023-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-44641: - Priority: Blocker (was: Major) > SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled

[jira] [Created] (SPARK-44660) Relax constraint for columnar shuffle check in AQE

2023-08-03 Thread Chao Sun (Jira)
Chao Sun created SPARK-44660: Summary: Relax constraint for columnar shuffle check in AQE Key: SPARK-44660 URL: https://issues.apache.org/jira/browse/SPARK-44660 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44659) SPJ: Include keyGroupedPartitioning in StoragePartitionJoinParams equality check

2023-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-44659: - Summary: SPJ: Include keyGroupedPartitioning in StoragePartitionJoinParams equality check (was:

[jira] [Updated] (SPARK-44641) SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

2023-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-44641: - Summary: SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

[jira] [Created] (SPARK-44659) Include keyGroupedPartitioning in StoragePartitionJoinParams equality check

2023-08-03 Thread Chao Sun (Jira)
Chao Sun created SPARK-44659: Summary: Include keyGroupedPartitioning in StoragePartitionJoinParams equality check Key: SPARK-44659 URL: https://issues.apache.org/jira/browse/SPARK-44659 Project: Spark

[jira] [Updated] (SPARK-44641) Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

2023-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-44641: - Parent: SPARK-37375 Issue Type: Sub-task (was: Bug) > Results duplicated when SPJ

[jira] [Updated] (SPARK-42454) SPJ: encapsulate all SPJ related parameters in BatchScanExec

2023-07-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-42454: - Fix Version/s: 3.5.0 (was: 4.0.0) > SPJ: encapsulate all SPJ related parameters

[jira] [Resolved] (SPARK-42454) SPJ: encapsulate all SPJ related parameters in BatchScanExec

2023-07-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42454. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41990

[jira] [Assigned] (SPARK-42454) SPJ: encapsulate all SPJ related parameters in BatchScanExec

2023-07-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42454: Assignee: Szehon Ho > SPJ: encapsulate all SPJ related parameters in BatchScanExec >

[jira] [Resolved] (SPARK-36612) Support left outer join build left or right outer join build right in shuffled hash join

2023-06-02 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-36612. -- Fix Version/s: 3.5.0 Resolution: Fixed > Support left outer join build left or right outer

[jira] [Assigned] (SPARK-36612) Support left outer join build left or right outer join build right in shuffled hash join

2023-06-02 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-36612: Assignee: Szehon Ho > Support left outer join build left or right outer join build right in >

[jira] [Updated] (SPARK-43758) Upgrade snappy-java to 1.1.10.0

2023-05-23 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-43758: - Issue Type: Bug (was: Improvement) > Upgrade snappy-java to 1.1.10.0 > ---

[jira] [Updated] (SPARK-43758) Upgrade snappy-java to 1.1.10.0

2023-05-23 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-43758: - Affects Version/s: 3.4.0 (was: 3.5.0) > Upgrade snappy-java to 1.1.10.0 >

[jira] [Created] (SPARK-43758) Upgrade snappy-java to 1.1.10.0

2023-05-23 Thread Chao Sun (Jira)
Chao Sun created SPARK-43758: Summary: Upgrade snappy-java to 1.1.10.0 Key: SPARK-43758 URL: https://issues.apache.org/jira/browse/SPARK-43758 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-43494) Directly call `replicate()` instead of reflection in `SparkHadoopUtil#createFile`

2023-05-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43494: Assignee: Yang Jie > Directly call `replicate()` instead of reflection in >

[jira] [Resolved] (SPARK-43494) Directly call `replicate()` instead of reflection in `SparkHadoopUtil#createFile`

2023-05-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43494. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41164

[jira] [Assigned] (SPARK-43272) Replace reflection w/ direct calling for `SparkHadoopUtil#createFile`

2023-05-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43272: Assignee: Yang Jie > Replace reflection w/ direct calling for `SparkHadoopUtil#createFile` >

[jira] [Resolved] (SPARK-43272) Replace reflection w/ direct calling for `SparkHadoopUtil#createFile`

2023-05-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43272. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40945

[jira] [Resolved] (SPARK-43484) Kafka/Kinesis Assembly should not package hadoop-client-runtime

2023-05-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43484. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41152

[jira] [Assigned] (SPARK-43484) Kafka/Kinesis Assembly should not package hadoop-client-runtime

2023-05-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43484: Assignee: Cheng Pan > Kafka/Kinesis Assembly should not package hadoop-client-runtime >

[jira] [Assigned] (SPARK-43410) Improve vectorized loop for Packed skipValues

2023-05-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43410: Assignee: xiaochen zhou > Improve vectorized loop for Packed skipValues >

[jira] [Resolved] (SPARK-43410) Improve vectorized loop for Packed skipValues

2023-05-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43410. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41092

[jira] [Resolved] (SPARK-43248) Unnecessary serialize/deserialize of Path on parallel gather partition stats

2023-04-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43248. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40920

[jira] [Assigned] (SPARK-43248) Unnecessary serialize/deserialize of Path on parallel gather partition stats

2023-04-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43248: Assignee: Cheng Pan > Unnecessary serialize/deserialize of Path on parallel gather partition

[jira] [Assigned] (SPARK-43268) Use proper error classes when exceptions are constructed with a message

2023-04-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43268: Assignee: Anton Okolnychyi > Use proper error classes when exceptions are constructed with a

[jira] [Resolved] (SPARK-43268) Use proper error classes when exceptions are constructed with a message

2023-04-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43268. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40934

[jira] [Resolved] (SPARK-43211) Remove Hadoop2 support in IsolatedClientLoader

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43211. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40870

[jira] [Assigned] (SPARK-43211) Remove Hadoop2 support in IsolatedClientLoader

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43211: Assignee: Cheng Pan > Remove Hadoop2 support in IsolatedClientLoader >

[jira] [Resolved] (SPARK-43202) Replace reflection w/ direct calling for YARN Resource API

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43202. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40860

[jira] [Assigned] (SPARK-43202) Replace reflection w/ direct calling for YARN Resource API

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43202: Assignee: Cheng Pan > Replace reflection w/ direct calling for YARN Resource API >

[jira] [Assigned] (SPARK-43208) IsolatedClassLoader should close barrier class InputStream after reading

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43208: Assignee: Cheng Pan > IsolatedClassLoader should close barrier class InputStream after reading >

[jira] [Resolved] (SPARK-43208) IsolatedClassLoader should close barrier class InputStream after reading

2023-04-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43208. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40867

[jira] [Assigned] (SPARK-43196) Replace reflection w/ direct calling for `ContainerLaunchContext#setTokensConf`

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43196: Assignee: Yang Jie > Replace reflection w/ direct calling for >

[jira] [Resolved] (SPARK-43196) Replace reflection w/ direct calling for `ContainerLaunchContext#setTokensConf`

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43196. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40855

[jira] [Resolved] (SPARK-43191) Replace reflection w/ direct calling for Hadoop CallerContext

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43191. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40850

[jira] [Assigned] (SPARK-43191) Replace reflection w/ direct calling for Hadoop CallerContext

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43191: Assignee: Cheng Pan > Replace reflection w/ direct calling for Hadoop CallerContext >

[jira] [Resolved] (SPARK-43200) Remove Hadoop 2 reference in docs

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43200. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40857

[jira] [Assigned] (SPARK-43200) Remove Hadoop 2 reference in docs

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43200: Assignee: Cheng Pan > Remove Hadoop 2 reference in docs > - > >

[jira] [Resolved] (SPARK-43195) Remove unnecessary serializable wrapper in HadoopFSUtils

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43195. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40854

[jira] [Assigned] (SPARK-43195) Remove unnecessary serializable wrapper in HadoopFSUtils

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43195: Assignee: Cheng Pan > Remove unnecessary serializable wrapper in HadoopFSUtils >

[jira] [Resolved] (SPARK-43187) Remove workaround for MiniKdc's BindException

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43187. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40849

[jira] [Assigned] (SPARK-43187) Remove workaround for MiniKdc's BindException

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43187: Assignee: Cheng Pan > Remove workaround for MiniKdc's BindException >

[jira] [Resolved] (SPARK-43186) Remove workaround for FileSinkDesc

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43186. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40848

[jira] [Assigned] (SPARK-43186) Remove workaround for FileSinkDesc

2023-04-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43186: Assignee: Cheng Pan > Remove workaround for FileSinkDesc > -- >

[jira] [Resolved] (SPARK-42452) Remove hadoop-2 profile from Apache Spark

2023-04-18 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42452. -- Fix Version/s: 3.5.0 Assignee: Yang Jie Resolution: Fixed > Remove hadoop-2 profile

[jira] [Resolved] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters in vectorized reader

2023-04-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42388. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39950

[jira] [Assigned] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters in vectorized reader

2023-04-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42388: Assignee: Mars > Avoid unnecessary parquet footer reads when no filters in vectorized reader >

[jira] [Assigned] (SPARK-43150) Remove workaround for PARQUET-2160

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43150: Assignee: Cheng Pan > Remove workaround for PARQUET-2160 > -- >

[jira] [Resolved] (SPARK-43150) Remove workaround for PARQUET-2160

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43150. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40802

[jira] [Resolved] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43064. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40701

[jira] [Assigned] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43064: Assignee: angerszhu > Spark SQL CLI SQL tab should only show once statement once >

[jira] [Assigned] (SPARK-43104) Set `shadeTestJar` of protobuf module to false

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43104: Assignee: Yang Jie > Set `shadeTestJar` of protobuf module to false >

[jira] [Resolved] (SPARK-43104) Set `shadeTestJar` of protobuf module to false

2023-04-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43104. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40753

[jira] [Resolved] (SPARK-43126) mark two Hive UDF expressions as stateful

2023-04-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43126. -- Fix Version/s: 3.4.1 3.5.0 Resolution: Fixed > mark two Hive UDF

[jira] [Assigned] (SPARK-43126) mark two Hive UDF expressions as stateful

2023-04-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43126: Assignee: Wenchen Fan > mark two Hive UDF expressions as stateful >

[jira] [Assigned] (SPARK-43121) Use `BytesWritable.copyBytes` instead of manual copy in `HiveInspectors`

2023-04-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-43121: Assignee: Yang Jie > Use `BytesWritable.copyBytes` instead of manual copy in `HiveInspectors` >

[jira] [Resolved] (SPARK-43121) Use `BytesWritable.copyBytes` instead of manual copy in `HiveInspectors`

2023-04-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-43121. -- Fix Version/s: 3.5.0 Resolution: Fixed > Use `BytesWritable.copyBytes` instead of manual copy

[jira] [Updated] (SPARK-42480) Improve the performance of drop partitions

2023-03-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-42480: - Fix Version/s: 3.4.0 > Improve the performance of drop partitions >

[jira] [Updated] (SPARK-42480) Improve the performance of drop partitions

2023-03-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-42480: - Fix Version/s: (was: 3.5.0) > Improve the performance of drop partitions >

[jira] [Assigned] (SPARK-42480) Improve the performance of drop partitions

2023-03-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42480: Assignee: Wechar > Improve the performance of drop partitions >

[jira] [Resolved] (SPARK-42480) Improve the performance of drop partitions

2023-03-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42480. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40069

[jira] [Updated] (SPARK-42539) User-provided JARs can override Spark's Hive metadata client JARs when using "builtin"

2023-02-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-42539: - Fix Version/s: 3.4.0 > User-provided JARs can override Spark's Hive metadata client JARs when using >

[jira] [Resolved] (SPARK-42539) User-provided JARs can override Spark's Hive metadata client JARs when using "builtin"

2023-02-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-42539. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40144

[jira] [Assigned] (SPARK-42539) User-provided JARs can override Spark's Hive metadata client JARs when using "builtin"

2023-02-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-42539: Assignee: Erik Krogen > User-provided JARs can override Spark's Hive metadata client JARs when

[jira] [Resolved] (SPARK-41952) Upgrade Parquet to fix off-heap memory leaks in Zstd codec

2023-02-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-41952. -- Fix Version/s: 3.2.4 3.4.0 3.3.3 Resolution: Fixed >

[jira] [Assigned] (SPARK-41952) Upgrade Parquet to fix off-heap memory leaks in Zstd codec

2023-02-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-41952: Assignee: Cheng Pan > Upgrade Parquet to fix off-heap memory leaks in Zstd codec >

  1   2   3   4   5   >