[jira] [Updated] (SPARK-47299) Use the same `versions. json` in the dropdown of different versions of PySpark documents

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47299: --- Labels: pull-request-available (was: ) > Use the same `versions. json` in the dropdown of

[jira] [Created] (SPARK-47299) Use the same `versions. json` in the dropdown of different versions of PySpark documents

2024-03-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47299: --- Summary: Use the same `versions. json` in the dropdown of different versions of PySpark documents Key: SPARK-47299 URL: https://issues.apache.org/jira/browse/SPARK-47299

[jira] [Updated] (SPARK-47298) Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to `2.7.12`

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47298: --- Labels: pull-request-available (was: ) > Upgrade `mysql-connector-j` to `8.3.0` and

[jira] [Created] (SPARK-47298) Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to `2.7.12`

2024-03-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47298: --- Summary: Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to `2.7.12` Key: SPARK-47298 URL: https://issues.apache.org/jira/browse/SPARK-47298 Project:

[jira] [Created] (SPARK-47297) regular expressions

2024-03-05 Thread Jira
Uroš Bojanić created SPARK-47297: Summary: regular expressions Key: SPARK-47297 URL: https://issues.apache.org/jira/browse/SPARK-47297 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47293) Build batchSchema with total sparkSchema instead of append one by one

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47293. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45396

[jira] [Assigned] (SPARK-47293) Build batchSchema with total sparkSchema instead of append one by one

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47293: Assignee: Binjie Yang > Build batchSchema with total sparkSchema instead of append one by one >

[jira] [Created] (SPARK-47296) fail all unsupported functions

2024-03-05 Thread Jira
Uroš Bojanić created SPARK-47296: Summary: fail all unsupported functions Key: SPARK-47296 URL: https://issues.apache.org/jira/browse/SPARK-47296 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-47295) startswith, endswith (non-binary collations)

2024-03-05 Thread Jira
Uroš Bojanić created SPARK-47295: Summary: startswith, endswith (non-binary collations) Key: SPARK-47295 URL: https://issues.apache.org/jira/browse/SPARK-47295 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-46835) Join support for strings with collation

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46835. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45389

[jira] [Assigned] (SPARK-46835) Join support for strings with collation

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46835: --- Assignee: Aleksandar Tomic > Join support for strings with collation >

[jira] [Resolved] (SPARK-47294) OptimizeSkewInRebalanceRepartitions should support ProjectExec(_,ShuffleQueryStageExec)

2024-03-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-47294. --- Resolution: Not A Problem > OptimizeSkewInRebalanceRepartitions should support >

[jira] [Updated] (SPARK-47294) OptimizeSkewInRebalanceRepartitions should support ProjectExec(_,ShuffleQueryStageExec)

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47294: --- Labels: pull-request-available (was: ) > OptimizeSkewInRebalanceRepartitions should

[jira] [Updated] (SPARK-47294) OptimizeSkewInRebalanceRepartitions should support ProjectExec(_,ShuffleQueryStageExec)

2024-03-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-47294: -- Description: Current OptimizeSkewInRebalanceRepartitions only support match ShuffleQueryStageExec,

[jira] [Created] (SPARK-47294) OptimizeSkewInRebalanceRepartitions should support ProjectExec(_,ShuffleQueryStageExec)

2024-03-05 Thread angerszhu (Jira)
angerszhu created SPARK-47294: - Summary: OptimizeSkewInRebalanceRepartitions should support ProjectExec(_,ShuffleQueryStageExec) Key: SPARK-47294 URL: https://issues.apache.org/jira/browse/SPARK-47294

[jira] [Commented] (SPARK-19256) Hive bucketing write support

2024-03-05 Thread Shreyas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823851#comment-17823851 ] Shreyas commented on SPARK-19256: - Don't really like spamming in the comments - but this is a much

[jira] [Updated] (SPARK-47293) Build batchSchema with total sparkSchema instead of append one by one

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47293: --- Labels: pull-request-available (was: ) > Build batchSchema with total sparkSchema instead

[jira] [Created] (SPARK-47293) Build batchSchema with total sparkSchema instead of append one by one

2024-03-05 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-47293: --- Summary: Build batchSchema with total sparkSchema instead of append one by one Key: SPARK-47293 URL: https://issues.apache.org/jira/browse/SPARK-47293 Project: Spark

[jira] [Resolved] (SPARK-47280) Remove timezone limitation for ORACLE TIMESTAMP WITH TIMEZONE

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47280. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45384

[jira] [Resolved] (SPARK-47247) use smaller target size when coalescing partitions with exploding joins

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47247. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45357

[jira] [Assigned] (SPARK-47247) use smaller target size when coalescing partitions with exploding joins

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47247: --- Assignee: Wenchen Fan > use smaller target size when coalescing partitions with exploding

[jira] [Updated] (SPARK-40763) Should expose driver service name to config for user features

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-40763: --- Labels: pull-request-available (was: ) > Should expose driver service name to config for

[jira] [Updated] (SPARK-47146) Possible thread leak when doing sort merge join

2024-03-05 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-47146: Fix Version/s: 3.5.2 3.4.3 > Possible thread leak when doing

[jira] [Commented] (SPARK-47146) Possible thread leak when doing sort merge join

2024-03-05 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823832#comment-17823832 ] Mridul Muralidharan commented on SPARK-47146: - Backported to 3.5 and 3.4 in PR:

[jira] [Resolved] (SPARK-47285) AdaptiveSparkPlanExec should always use the context.session

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47285. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45388

[jira] [Assigned] (SPARK-47285) AdaptiveSparkPlanExec should always use the context.session

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47285: Assignee: XiDuo You > AdaptiveSparkPlanExec should always use the context.session >

[jira] [Created] (SPARK-47292) safeMapToJValue should consider when map is null

2024-03-05 Thread Wei Liu (Jira)
Wei Liu created SPARK-47292: --- Summary: safeMapToJValue should consider when map is null Key: SPARK-47292 URL: https://issues.apache.org/jira/browse/SPARK-47292 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-44746) Improve the documentation for TABLE input arguments for UDTFs

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44746. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45375

[jira] [Assigned] (SPARK-44746) Improve the documentation for TABLE input arguments for UDTFs

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44746: Assignee: Daniel > Improve the documentation for TABLE input arguments for UDTFs >

[jira] [Resolved] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47277. -- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in

[jira] [Created] (SPARK-47291) Add Parquet file reader metrics to scan

2024-03-05 Thread Parth Chandra (Jira)
Parth Chandra created SPARK-47291: - Summary: Add Parquet file reader metrics to scan Key: SPARK-47291 URL: https://issues.apache.org/jira/browse/SPARK-47291 Project: Spark Issue Type:

[jira] [Created] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-05 Thread Parth Chandra (Jira)
Parth Chandra created SPARK-47290: - Summary: Extend CustomTaskMetric to allow metric values from multiple sources Key: SPARK-47290 URL: https://issues.apache.org/jira/browse/SPARK-47290 Project:

[jira] [Comment Edited] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

2024-03-05 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823510#comment-17823510 ] Asif edited comment on SPARK-33152 at 3/5/24 6:43 PM: -- [~tedjenks] .. Unfortunately

[jira] [Created] (SPARK-47289) Allow extensions to log extended information in explain plan

2024-03-05 Thread Parth Chandra (Jira)
Parth Chandra created SPARK-47289: - Summary: Allow extensions to log extended information in explain plan Key: SPARK-47289 URL: https://issues.apache.org/jira/browse/SPARK-47289 Project: Spark

[jira] [Resolved] (SPARK-47033) EXECUTE IMMEDIATE USING does not recognize session variable names

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47033. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45293

[jira] [Resolved] (SPARK-43883) CTAS Command Nodes Prevent Some Optimizer Rules From Running

2024-03-05 Thread Ted Chester Jenks (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Chester Jenks resolved SPARK-43883. --- Resolution: Won't Fix > CTAS Command Nodes Prevent Some Optimizer Rules From

[jira] [Commented] (SPARK-47288) DataType __repr__ change breaks datatype checking (anit-)pattern

2024-03-05 Thread Ted Chester Jenks (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823717#comment-17823717 ] Ted Chester Jenks commented on SPARK-47288: --- [~gurwls223] I saw you on the original PR,

[jira] [Created] (SPARK-47288) DataType __repr__ change breaks datatype checking (anit-)pattern

2024-03-05 Thread Ted Chester Jenks (Jira)
Ted Chester Jenks created SPARK-47288: - Summary: DataType __repr__ change breaks datatype checking (anit-)pattern Key: SPARK-47288 URL: https://issues.apache.org/jira/browse/SPARK-47288 Project:

[jira] [Commented] (SPARK-46810) Clarify error class terminology

2024-03-05 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823713#comment-17823713 ] Nicholas Chammas commented on SPARK-46810: -- [~cloud_fan], [~LuciferYang], [~beliefer], and

[jira] [Updated] (SPARK-47281) Update the `versions. json` file for the already released spark version

2024-03-05 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-47281: Summary: Update the `versions. json` file for the already released spark version (was: Update

[jira] [Updated] (SPARK-47287) Aggregate in not causes

2024-03-05 Thread Ted Chester Jenks (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Chester Jenks updated SPARK-47287: -- Description:   The below snippet is confirmed working with Spark 3.2.1 and broken

[jira] [Created] (SPARK-47287) Aggregate in not causes

2024-03-05 Thread Ted Chester Jenks (Jira)
Ted Chester Jenks created SPARK-47287: - Summary: Aggregate in not causes Key: SPARK-47287 URL: https://issues.apache.org/jira/browse/SPARK-47287 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-47198) Is it possible to dynamically add backend service to ingress with Kubernetes?

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823616#comment-17823616 ] Kent Yao commented on SPARK-47198: -- Hi [~melin], it would be better to ask your questions via the

[jira] [Resolved] (SPARK-47198) Is it possible to dynamically add backend service to ingress with Kubernetes?

2024-03-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47198. -- Resolution: Information Provided > Is it possible to dynamically add backend service to ingress with

[jira] [Resolved] (SPARK-47168) Disable parquet filter pushdown for non default collated strings

2024-03-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47168. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45262

[jira] [Updated] (SPARK-47279) spark driver process hangs due to "unable to create new native thread"

2024-03-05 Thread TianyiMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TianyiMa updated SPARK-47279: - Description: we encounter that spark driver hangs for about 11 hours,  and finall killed by user. In

[jira] [Updated] (SPARK-47279) spark driver process hangs due to "unable to create new native thread"

2024-03-05 Thread TianyiMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TianyiMa updated SPARK-47279: - Attachment: oom-killer.png > spark driver process hangs due to "unable to create new native thread" >

[jira] [Assigned] (SPARK-47102) Add COLLATION_ENABLED config flag

2024-03-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-47102: Assignee: Mihailo Milosevic > Add COLLATION_ENABLED config flag >

[jira] [Resolved] (SPARK-47102) Add COLLATION_ENABLED config flag

2024-03-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47102. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45285

[jira] [Updated] (SPARK-46835) Join support for strings with collation

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46835: --- Labels: pull-request-available (was: ) > Join support for strings with collation >

[jira] [Updated] (SPARK-47239) Support distinct window function

2024-03-05 Thread Mingliang Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Zhu updated SPARK-47239: -- Issue Type: New Feature (was: Bug) > Support distinct window function >

[jira] [Created] (SPARK-47286) IN operator support

2024-03-05 Thread Aleksandar Tomic (Jira)
Aleksandar Tomic created SPARK-47286: Summary: IN operator support Key: SPARK-47286 URL: https://issues.apache.org/jira/browse/SPARK-47286 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-47285) AdaptiveSparkPlanExec should always use the context.session

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47285: --- Labels: pull-request-available (was: ) > AdaptiveSparkPlanExec should always use the

[jira] [Created] (SPARK-47285) AdaptiveSparkPlanExec should always use the context.session

2024-03-05 Thread XiDuo You (Jira)
XiDuo You created SPARK-47285: - Summary: AdaptiveSparkPlanExec should always use the context.session Key: SPARK-47285 URL: https://issues.apache.org/jira/browse/SPARK-47285 Project: Spark Issue

[jira] [Commented] (SPARK-45387) Partition key filter cannot be pushed down when using cast

2024-03-05 Thread TianyiMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823524#comment-17823524 ] TianyiMa commented on SPARK-45387: -- [~doki] the output execution plan is the final result, but the

[jira] [Updated] (SPARK-44259) Make `connect-jvm-client` module pass except arrow-related ones in Java 21

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44259: --- Labels: pull-request-available (was: ) > Make `connect-jvm-client` module pass except

[jira] [Commented] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

2024-03-05 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823512#comment-17823512 ] Asif commented on SPARK-33152: -- other than using my PR, the safe option would be to disable constraint

[jira] [Commented] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

2024-03-05 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823510#comment-17823510 ] Asif commented on SPARK-33152: -- [~tedjenks] .. Unfortunately I am not a committer. As part of workday , I

[jira] [Updated] (SPARK-47284) We should ensure enough parallelism when ShuffleExchangeLike join with specs without shuffle

2024-03-05 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated SPARK-47284: --- Description: The following case is introduced by https://issues.apache.org/jira/browse/SPARK-35703 //

[jira] [Created] (SPARK-47284) We should ensure enough parallelism when ShuffleExchangeLike join with specs without shuffle

2024-03-05 Thread Qi Zhu (Jira)
Qi Zhu created SPARK-47284: -- Summary: We should ensure enough parallelism when ShuffleExchangeLike join with specs without shuffle Key: SPARK-47284 URL: https://issues.apache.org/jira/browse/SPARK-47284

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: (was: Apache Spark) > PySpark util function assertDataFrameEqual

[jira] [Assigned] (SPARK-47283) Remove Spark version drop down to the PySpark doc site

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47283: -- Assignee: (was: Apache Spark) > Remove Spark version drop down to the PySpark

[jira] [Assigned] (SPARK-47283) Remove Spark version drop down to the PySpark doc site

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47283: -- Assignee: Apache Spark > Remove Spark version drop down to the PySpark doc site >

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: Apache Spark > PySpark util function assertDataFrameEqual should not

[jira] [Updated] (SPARK-47283) Remove Spark version drop down to the PySpark doc site

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47283: --- Labels: pull-request-available (was: ) > Remove Spark version drop down to the PySpark doc

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: (was: Apache Spark) > PySpark util function assertDataFrameEqual

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: Apache Spark > PySpark util function assertDataFrameEqual should not

[jira] [Commented] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

2024-03-05 Thread Ted Chester Jenks (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823506#comment-17823506 ] Ted Chester Jenks commented on SPARK-33152: --- [~ashahid7] I see. This is very painful for us

[jira] [Created] (SPARK-47283) Remove Spark version drop down to the PySpark doc site

2024-03-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47283: --- Summary: Remove Spark version drop down to the PySpark doc site Key: SPARK-47283 URL: https://issues.apache.org/jira/browse/SPARK-47283 Project: Spark Issue

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: (was: Apache Spark) > PySpark util function assertDataFrameEqual

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: Apache Spark > PySpark util function assertDataFrameEqual should not

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: (was: Apache Spark) > PySpark util function assertDataFrameEqual

[jira] [Assigned] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47281: -- Assignee: (was: Apache Spark) > Update the `versions. json` file for the already

[jira] [Assigned] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47277: -- Assignee: Apache Spark > PySpark util function assertDataFrameEqual should not

[jira] [Assigned] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47281: -- Assignee: Apache Spark > Update the `versions. json` file for the already released

[jira] [Assigned] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47281: -- Assignee: Apache Spark > Update the `versions. json` file for the already released

[jira] [Assigned] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47281: -- Assignee: (was: Apache Spark) > Update the `versions. json` file for the already

[jira] [Assigned] (SPARK-47148) Avoid to materialize AQE ExchangeQueryStageExec on the cancellation

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47148: -- Assignee: (was: Apache Spark) > Avoid to materialize AQE ExchangeQueryStageExec

[jira] [Created] (SPARK-47282) 'parseTableIdentifier' fails when a catalog name is provided

2024-03-05 Thread Denis Tarima (Jira)
Denis Tarima created SPARK-47282: Summary: 'parseTableIdentifier' fails when a catalog name is provided Key: SPARK-47282 URL: https://issues.apache.org/jira/browse/SPARK-47282 Project: Spark

[jira] [Updated] (SPARK-47177) Cached SQL plan do not display final AQE plan in explain string

2024-03-05 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-47177: -- Fix Version/s: 3.4.3 > Cached SQL plan do not display final AQE plan in explain string >

[jira] [Updated] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-47281: Affects Version/s: 3.5.1 > Update the `versions. json` file for the already released saprk

[jira] [Updated] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47281: --- Labels: pull-request-available (was: ) > Update the `versions. json` file for the already

[jira] [Updated] (SPARK-47279) spark driver process hangs due to "unable to create new native thread"

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47279: --- Labels: pull-request-available (was: ) > spark driver process hangs due to "unable to

[jira] [Created] (SPARK-47281) Update the `versions. json` file for the already released saprk version

2024-03-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47281: --- Summary: Update the `versions. json` file for the already released saprk version Key: SPARK-47281 URL: https://issues.apache.org/jira/browse/SPARK-47281 Project: Spark

[jira] [Updated] (SPARK-47280) Remove timezone limitation for ORACLE TIMESTAMP WITH TIMEZONE

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47280: --- Labels: pull-request-available (was: ) > Remove timezone limitation for ORACLE TIMESTAMP

[jira] [Updated] (SPARK-47210) Implicit casting on collated expressions

2024-03-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47210: --- Labels: pull-request-available (was: ) > Implicit casting on collated expressions >

[jira] [Created] (SPARK-47280) Remove timezone limitation for ORACLE TIMESTAMP WITH TIMEZONE

2024-03-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-47280: Summary: Remove timezone limitation for ORACLE TIMESTAMP WITH TIMEZONE Key: SPARK-47280 URL: https://issues.apache.org/jira/browse/SPARK-47280 Project: Spark Issue