[jira] [Assigned] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38077: Assignee: (was: Apache Spark) > Spark 3.2.1 breaks binary compatibility with Spark 3.

[jira] [Assigned] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38077: Assignee: Apache Spark > Spark 3.2.1 breaks binary compatibility with Spark 3.2.0 > -

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485072#comment-17485072 ] Apache Spark commented on SPARK-38077: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485047#comment-17485047 ] Apache Spark commented on SPARK-37958: -- User 'yoda-mon' has created a pull request

[jira] [Assigned] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37958: Assignee: (was: Apache Spark) > Pyspark SparkContext.AddFile() does not respect spark

[jira] [Assigned] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37958: Assignee: Apache Spark > Pyspark SparkContext.AddFile() does not respect spark.files.over

[jira] [Commented] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485046#comment-17485046 ] Apache Spark commented on SPARK-37958: -- User 'yoda-mon' has created a pull request

[jira] [Updated] (SPARK-38081) Support cloud-backend in K8s IT with SBT

[ https://issues.apache.org/jira/browse/SPARK-38081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38081: -- Parent: (was: SPARK-33772) Issue Type: Test (was: Sub-task) > Support cloud-backe

[jira] [Commented] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485039#comment-17485039 ] Kapil Singh commented on SPARK-37995: - Hey folks, any input on this issue? [~hyukjin

[jira] [Commented] (SPARK-38081) Support cloud-backend in K8s IT with SBT

[ https://issues.apache.org/jira/browse/SPARK-38081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485037#comment-17485037 ] Apache Spark commented on SPARK-38081: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-38081) Support cloud-backend in K8s IT with SBT

[ https://issues.apache.org/jira/browse/SPARK-38081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485036#comment-17485036 ] Apache Spark commented on SPARK-38081: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-38081) Support cloud-backend in K8s IT with SBT

[ https://issues.apache.org/jira/browse/SPARK-38081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38081: Assignee: (was: Apache Spark) > Support cloud-backend in K8s IT with SBT > --

[jira] [Assigned] (SPARK-38081) Support cloud-backend in K8s IT with SBT

[ https://issues.apache.org/jira/browse/SPARK-38081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38081: Assignee: Apache Spark > Support cloud-backend in K8s IT with SBT > -

[jira] [Created] (SPARK-38081) Support cloud-backend in K8s IT with SBT

Dongjoon Hyun created SPARK-38081: - Summary: Support cloud-backend in K8s IT with SBT Key: SPARK-38081 URL: https://issues.apache.org/jira/browse/SPARK-38081 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485032#comment-17485032 ] Leona Yoda commented on SPARK-37958: This is probably default behavior not only on k

[jira] [Updated] (SPARK-38075) Hive script transform with order by and limit will return fake rows

[ https://issues.apache.org/jira/browse/SPARK-38075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38075: -- Fix Version/s: 3.1.3 > Hive script transform with order by and limit will return fake rows > -

[jira] [Updated] (SPARK-38063) Support SQL split_part function

[ https://issues.apache.org/jira/browse/SPARK-38063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-38063: - Description: `split_part()` is a commonly supported function by other systems such as Postgres and some

[jira] [Updated] (SPARK-38063) Support SQL split_part function

[ https://issues.apache.org/jira/browse/SPARK-38063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-38063: - Description: `split_part()` is a commonly supported function by other systems such as Postgres and some

[jira] [Updated] (SPARK-38063) Support SQL split_part function

[ https://issues.apache.org/jira/browse/SPARK-38063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-38063: - Description: `split_part()` is a commonly supported function by other systems such as Postgres and some

[jira] [Commented] (SPARK-38075) Hive script transform with order by and limit will return fake rows

[ https://issues.apache.org/jira/browse/SPARK-38075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484997#comment-17484997 ] Apache Spark commented on SPARK-38075: -- User 'bersprockets' has created a pull requ

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484985#comment-17484985 ] Anton Okolnychyi commented on SPARK-34183: -- I am sorry it took me a while to co

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484982#comment-17484982 ] Apache Spark commented on SPARK-34183: -- User 'aokolnychyi' has created a pull reque

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484981#comment-17484981 ] Apache Spark commented on SPARK-34183: -- User 'aokolnychyi' has created a pull reque

[jira] [Comment Edited] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484941#comment-17484941 ] Nadav Samet edited comment on SPARK-38077 at 1/31/22, 9:14 PM: ---

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484941#comment-17484941 ] Nadav Samet commented on SPARK-38077: - I wasn't aware that the policy allows for it,

[jira] [Assigned] (SPARK-38047) Provide an option to only roll executors if they are outliers

[ https://issues.apache.org/jira/browse/SPARK-38047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38047: Assignee: Apache Spark (was: Alex Holmes) > Provide an option to only roll executors if

[jira] [Assigned] (SPARK-38047) Provide an option to only roll executors if they are outliers

[ https://issues.apache.org/jira/browse/SPARK-38047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38047: Assignee: Alex Holmes (was: Apache Spark) > Provide an option to only roll executors if

[jira] [Commented] (SPARK-38047) Provide an option to only roll executors if they are outliers

[ https://issues.apache.org/jira/browse/SPARK-38047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484933#comment-17484933 ] Apache Spark commented on SPARK-38047: -- User 'alexholmes' has created a pull reques

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484894#comment-17484894 ] Chao Sun commented on SPARK-38077: -- BTW [~thesamet] it seems Spark only guarantees API

[jira] [Commented] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484893#comment-17484893 ] Apache Spark commented on SPARK-38080: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38080: Assignee: (was: Apache Spark) > Flaky test: StreamingQueryManagerSuite: 'awaitAnyTerm

[jira] [Assigned] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38080: Assignee: Apache Spark > Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination wit

[jira] [Commented] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484891#comment-17484891 ] Apache Spark commented on SPARK-38080: -- User 'zsxwing' has created a pull request f

[jira] [Updated] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-38080: - Description: {code:java} [info] - awaitAnyTermination with timeout and resetTerminated *** FAILE

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484873#comment-17484873 ] Chao Sun commented on SPARK-38077: -- Sorry for breaking the binary compatibility. I wasn

[jira] [Updated] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38077: -- Affects Version/s: (was: 3.2.0) > Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[jira] [Commented] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484865#comment-17484865 ] Erik Krogen commented on SPARK-38062: - Thanks for taking a look btw! Even if we don'

[jira] [Commented] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484864#comment-17484864 ] Erik Krogen commented on SPARK-38062: - It still seems like a bug to me that the pres

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484863#comment-17484863 ] Dongjoon Hyun commented on SPARK-38077: --- cc [~sunchao] > Spark 3.2.1 breaks binar

[jira] [Updated] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

[ https://issues.apache.org/jira/browse/SPARK-38080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-38080: - Description: {code:java} [info] - awaitAnyTermination with timeout and resetTerminated *** FAILE

[jira] [Resolved] (SPARK-38075) Hive script transform with order by and limit will return fake rows

[ https://issues.apache.org/jira/browse/SPARK-38075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38075. --- Fix Version/s: 3.3.0 3.2.2 Assignee: Bruce Robbins Resolut

[jira] [Commented] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484855#comment-17484855 ] Dongjoon Hyun commented on SPARK-38062: --- I agree that you may think like that, but

[jira] [Updated] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38062: -- Affects Version/s: 3.3.0 (was: 3.1.1) > FallbackStorage shouldn't a

[jira] [Updated] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38062: -- Issue Type: Improvement (was: Bug) > FallbackStorage shouldn't attempt to resolve arbitrary "

[jira] [Created] (SPARK-38080) Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'

Shixiong Zhu created SPARK-38080: Summary: Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated' Key: SPARK-38080 URL: https://issues.apache.org/jira/browse/SPARK-38080

[jira] [Updated] (SPARK-38079) Not waiting for configmap before starting driver

[ https://issues.apache.org/jira/browse/SPARK-38079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben updated SPARK-38079: Description: *The problem* When you spark-submit to kubernetes in cluster-mode: # Kubernetes creates the driver

[jira] [Created] (SPARK-38079) Not waiting for configmap before starting driver

Ben created SPARK-38079: --- Summary: Not waiting for configmap before starting driver Key: SPARK-38079 URL: https://issues.apache.org/jira/browse/SPARK-38079 Project: Spark Issue Type: Bug Comp

[jira] [Comment Edited] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484806#comment-17484806 ] Erik Krogen edited comment on SPARK-38062 at 1/31/22, 5:33 PM: ---

[jira] [Commented] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484806#comment-17484806 ] Erik Krogen commented on SPARK-38062: - I think the issue I'm reporting is very simil

[jira] [Created] (SPARK-38078) Aggregation with Watermark in AppendMode is holding data beyong water mark boundary.

krishna created SPARK-38078: --- Summary: Aggregation with Watermark in AppendMode is holding data beyong water mark boundary. Key: SPARK-38078 URL: https://issues.apache.org/jira/browse/SPARK-38078 Project: S

[jira] [Created] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

Nadav Samet created SPARK-38077: --- Summary: Spark 3.2.1 breaks binary compatibility with Spark 3.2.0 Key: SPARK-38077 URL: https://issues.apache.org/jira/browse/SPARK-38077 Project: Spark Issue

[jira] [Updated] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nadav Samet updated SPARK-38077: Description: [PR 35243|https://github.com/apache/spark/pull/35243] introduced a new parameter to

[jira] [Commented] (SPARK-37946) Use error classes in the execution errors related to partitions

[ https://issues.apache.org/jira/browse/SPARK-37946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484592#comment-17484592 ] Apache Spark commented on SPARK-37946: -- User 'yutoacts' has created a pull request

[jira] [Assigned] (SPARK-37946) Use error classes in the execution errors related to partitions

[ https://issues.apache.org/jira/browse/SPARK-37946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37946: Assignee: Apache Spark > Use error classes in the execution errors related to partitions

[jira] [Assigned] (SPARK-37946) Use error classes in the execution errors related to partitions

[ https://issues.apache.org/jira/browse/SPARK-37946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37946: Assignee: Apache Spark > Use error classes in the execution errors related to partitions

[jira] [Assigned] (SPARK-37946) Use error classes in the execution errors related to partitions

[ https://issues.apache.org/jira/browse/SPARK-37946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37946: Assignee: (was: Apache Spark) > Use error classes in the execution errors related to