[jira] [Resolved] (SPARK-45513) Replace `scala.runtime.Tuple2Zipped` to `scala.collection.LazyZip2`

2023-10-13 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45513. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43351

[jira] [Updated] (SPARK-44662) SPIP: Improving performance of BroadcastHashJoin queries with stream side join key on non partition columns

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44662: --- Labels: pull-request-available (was: ) > SPIP: Improving performance of BroadcastHashJoin

[jira] [Updated] (SPARK-45534) Use `java.lang.ref.Cleaner` instead of `finalize` for `RemoteBlockPushResolver`

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45534: --- Labels: pull-request-available (was: ) > Use `java.lang.ref.Cleaner` instead of `finalize`

[jira] [Comment Edited] (SPARK-45539) Add assert and log error if watermark predicate is missing in append mode and streaming aggregation

2023-10-13 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775103#comment-17775103 ] Anish Shrigondekar edited comment on SPARK-45539 at 10/14/23 1:24 AM:

[jira] [Resolved] (SPARK-44735) Log a warning when inserting columns with the same name by row that don't match up

2023-10-13 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-44735. -- Fix Version/s: 4.0.0 Resolution: Fixed > Log a warning when inserting columns with the

[jira] [Assigned] (SPARK-44735) Log a warning when inserting columns with the same name by row that don't match up

2023-10-13 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reassigned SPARK-44735: Assignee: Jia Fan > Log a warning when inserting columns with the same name by row that

[jira] [Created] (SPARK-45539) Add assert and log error if watermark predicate is missing in append mode and streaming aggregation

2023-10-13 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-45539: -- Summary: Add assert and log error if watermark predicate is missing in append mode and streaming aggregation Key: SPARK-45539 URL:

[jira] [Updated] (SPARK-44210) Strengthen type checking and better comply with Connect specifications for `levenshtein` function

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44210: --- Labels: pull-request-available (was: ) > Strengthen type checking and better comply with

[jira] [Updated] (SPARK-43879) Decouple handle command and send response on server side

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43879: --- Labels: pull-request-available (was: ) > Decouple handle command and send response on

[jira] [Resolved] (SPARK-45487) Replace: _LEGACY_ERROR_TEMP_3007

2023-10-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-45487. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43342

[jira] [Assigned] (SPARK-45487) Replace: _LEGACY_ERROR_TEMP_3007

2023-10-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-45487: -- Assignee: Serge Rielau > Replace: _LEGACY_ERROR_TEMP_3007 >

[jira] [Updated] (SPARK-45517) Expand more exception constructors to support error framework parameters

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45517: --- Labels: pull-request-available (was: ) > Expand more exception constructors to support

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45508: -- Affects Version/s: 3.4.1 3.3.2 > Add

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45508: -- Fix Version/s: 3.4.2 3.5.1 3.3.4 > Add

[jira] [Assigned] (SPARK-45426) [CORE] Add support for a ReloadingTrustManager

2023-10-13 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-45426: --- Assignee: Hasnain Lakhani > [CORE] Add support for a ReloadingTrustManager

[jira] [Resolved] (SPARK-45426) [CORE] Add support for a ReloadingTrustManager

2023-10-13 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-45426. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Comment Edited] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng edited comment on SPARK-44679 at 10/13/23 4:12 PM: -- I think your

[jira] [Comment Edited] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng edited comment on SPARK-44679 at 10/13/23 4:11 PM: -- I think your

[jira] [Comment Edited] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng edited comment on SPARK-44679 at 10/13/23 4:07 PM: -- I think your

[jira] [Comment Edited] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng edited comment on SPARK-44679 at 10/13/23 4:00 PM: -- I think your

[jira] [Comment Edited] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng edited comment on SPARK-44679 at 10/13/23 3:57 PM: -- I think your

[jira] [Resolved] (SPARK-45495) Support stage level task resource profile for k8s cluster when dynamic allocation disabled

2023-10-13 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-45495. --- Resolution: Fixed > Support stage level task resource profile for k8s cluster when dynamic

[jira] [Commented] (SPARK-44679) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2023-10-13 Thread huangsheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774963#comment-17774963 ] huangsheng commented on SPARK-44679: I think your problem is the same as [#SPARK-40622] >

[jira] [Updated] (SPARK-45387) Partition key filter cannot be pushed down when using cast

2023-10-13 Thread TianyiMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TianyiMa updated SPARK-45387: - Attachment: PruneFileSourcePartitions.diff > Partition key filter cannot be pushed down when using cast

[jira] [Commented] (SPARK-45527) Task fraction resource request is not expected

2023-10-13 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774957#comment-17774957 ] Thomas Graves commented on SPARK-45527: --- thanks for filing and digging into this. I assume this is

[jira] [Assigned] (SPARK-45536) Lower the default -Xmx of build/mvn

2023-10-13 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-45536: Assignee: Yang Jie > Lower the default -Xmx of build/mvn > --- >

[jira] [Resolved] (SPARK-45536) Lower the default -Xmx of build/mvn

2023-10-13 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45536. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43364

[jira] [Updated] (SPARK-45538) pyspark connect overwrite_partitions bug

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45538: --- Labels: patch pull-request-available (was: patch) > pyspark connect overwrite_partitions

[jira] [Assigned] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45476: -- Assignee: (was: Apache Spark) > Raise exception directly instead of calling

[jira] [Assigned] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45476: -- Assignee: Apache Spark > Raise exception directly instead of calling

[jira] [Created] (SPARK-45538) pyspark connect overwrite_partitions bug

2023-10-13 Thread xie shuiahu (Jira)
xie shuiahu created SPARK-45538: --- Summary: pyspark connect overwrite_partitions bug Key: SPARK-45538 URL: https://issues.apache.org/jira/browse/SPARK-45538 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45535) Do not compile tests for snapshots and release build

2023-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45535. -- Resolution: Invalid > Do not compile tests for snapshots and release build >

[jira] [Updated] (SPARK-45537) The last Task may get stuck in multi resource-profile

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45537: --- Labels: pull-request-available (was: ) > The last Task may get stuck in multi

[jira] [Created] (SPARK-45537) The last Task may get stuck in multi resource-profile

2023-10-13 Thread liangyongyuan (Jira)
liangyongyuan created SPARK-45537: - Summary: The last Task may get stuck in multi resource-profile Key: SPARK-45537 URL: https://issues.apache.org/jira/browse/SPARK-45537 Project: Spark

[jira] [Updated] (SPARK-45536) Lower the default -Xmx of build/mvn

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45536: --- Labels: pull-request-available (was: ) > Lower the default -Xmx of build/mvn >

[jira] [Updated] (SPARK-45535) Do not compile tests for snapshots and release build

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45535: --- Labels: pull-request-available (was: ) > Do not compile tests for snapshots and release

[jira] [Created] (SPARK-45536) Lower the default -Xmx of build/mvn

2023-10-13 Thread Yang Jie (Jira)
Yang Jie created SPARK-45536: Summary: Lower the default -Xmx of build/mvn Key: SPARK-45536 URL: https://issues.apache.org/jira/browse/SPARK-45536 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45535) Do not compile tests for snapshots and release build

2023-10-13 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45535: Summary: Do not compile tests for snapshots and release build Key: SPARK-45535 URL: https://issues.apache.org/jira/browse/SPARK-45535 Project: Spark Issue

[jira] [Updated] (SPARK-45514) Replace `scala.runtime.Tuple3Zipped` to `scala.collection.LazyZip3`

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45514: --- Labels: pull-request-available (was: ) > Replace `scala.runtime.Tuple3Zipped` to

[jira] [Commented] (SPARK-45534) Use `java.lang.ref.Cleaner` instead of `finalize` for `RemoteBlockPushResolver`

2023-10-13 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774804#comment-17774804 ] Yang Jie commented on SPARK-45534: -- Please go ahead ~ thank you very much. [~zhaomin]            > Use

[jira] [Commented] (SPARK-45534) Use `java.lang.ref.Cleaner` instead of `finalize` for `RemoteBlockPushResolver`

2023-10-13 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774801#comment-17774801 ] Min Zhao commented on SPARK-45534: -- Hello，Could you assign it to me?   > Use `java.lang.ref.Cleaner`

[jira] [Updated] (SPARK-45529) Fix flaky KafkaSourceStressSuite

2023-10-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45529: --- Labels: pull-request-available (was: ) > Fix flaky KafkaSourceStressSuite >

[jira] [Resolved] (SPARK-45532) Restore codetabs for the Protobuf Data Source Guide

2023-10-13 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-45532. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-45532) Restore codetabs for the Protobuf Data Source Guide

2023-10-13 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-45532: Assignee: Kent Yao > Restore codetabs for the Protobuf Data Source Guide >