[jira] [Resolved] (SPARK-40522) Upgrade Apache Kafka from 3.2.1 to 3.2.3

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40522. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37958

[jira] [Assigned] (SPARK-40522) Upgrade Apache Kafka from 3.2.1 to 3.2.3

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40522: - Assignee: Bjørn Jørgensen > Upgrade Apache Kafka from 3.2.1 to 3.2.3 >

[jira] [Resolved] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40327. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37948

[jira] [Assigned] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40327: - Assignee: Ruifeng Zheng > Increase pandas API coverage for pandas API on Spark >

[jira] [Assigned] (SPARK-40434) Implement applyInPandasWithState in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40434: Assignee: Jungtaek Lim > Implement applyInPandasWithState in PySpark >

[jira] [Commented] (SPARK-40434) Implement applyInPandasWithState in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608067#comment-17608067 ] Apache Spark commented on SPARK-40434: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Resolved] (SPARK-40434) Implement applyInPandasWithState in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40434. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37893

[jira] [Resolved] (SPARK-40487) Make defaultJoin in BroadcastNestedLoopJoinExec running in parallel

[ https://issues.apache.org/jira/browse/SPARK-40487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40487. - Fix Version/s: 3.4.0 Assignee: Xingchao, Zhang Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-40490) `YarnShuffleIntegrationSuite` no longer verifies `registeredExecFile` reload after SPARK-17321

[ https://issues.apache.org/jira/browse/SPARK-40490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608042#comment-17608042 ] Apache Spark commented on SPARK-40490: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-40490) `YarnShuffleIntegrationSuite` no longer verifies `registeredExecFile` reload after SPARK-17321

[ https://issues.apache.org/jira/browse/SPARK-40490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608043#comment-17608043 ] Apache Spark commented on SPARK-40490: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-40526: Description: release notes: [https://github.com/scala/scala/releases/tag/v2.13.9]

[jira] [Assigned] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40526: Assignee: (was: Apache Spark) > Upgrade Scala to 2.13.9 > --- >

[jira] [Commented] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608041#comment-17608041 ] Apache Spark commented on SPARK-40526: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40526: Assignee: Apache Spark > Upgrade Scala to 2.13.9 > --- > >

[jira] [Updated] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-40526: Attachment: image-2022-09-22-10-53-10-579.png > Upgrade Scala to 2.13.9 > ---

[jira] [Commented] (SPARK-40526) Upgrade Scala to 2.13.9

[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608040#comment-17608040 ] Apache Spark commented on SPARK-40526: -- User 'panbingkun' has created a pull request for this

[jira] [Created] (SPARK-40526) Upgrade Scala to 2.13.9

BingKun Pan created SPARK-40526: --- Summary: Upgrade Scala to 2.13.9 Key: SPARK-40526 URL: https://issues.apache.org/jira/browse/SPARK-40526 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-40385) Classes with companion object constructor fails interpreted path

[ https://issues.apache.org/jira/browse/SPARK-40385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40385: - Fix Version/s: 3.3.2 (was: 3.3.1) > Classes with companion object

[jira] [Resolved] (SPARK-40385) Classes with companion object constructor fails interpreted path

[ https://issues.apache.org/jira/browse/SPARK-40385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40385. -- Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-40385) Classes with companion object constructor fails interpreted path

[ https://issues.apache.org/jira/browse/SPARK-40385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40385: Assignee: Emil Ejbyfeldt > Classes with companion object constructor fails interpreted

[jira] [Updated] (SPARK-40525) Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-40525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xsys updated SPARK-40525: - Description: h3. Describe the bug Storing an invalid INT value {{1.1}} using DataFrames via {{spark-shell}} 

[jira] [Updated] (SPARK-40525) Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-40525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xsys updated SPARK-40525: - Description: h3. Describe the bug Storing an invalid INT value {{1.1}} using DataFrames via {{spark-shell}} 

[jira] [Created] (SPARK-40525) Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL

xsys created SPARK-40525: Summary: Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL Key: SPARK-40525 URL:

[jira] [Comment Edited] (SPARK-38819) Run Pandas on Spark with Pandas 1.4.x

[ https://issues.apache.org/jira/browse/SPARK-38819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543450#comment-17543450 ] Yikun Jiang edited comment on SPARK-38819 at 9/22/22 1:37 AM: -- All UT /

[jira] [Commented] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

[ https://issues.apache.org/jira/browse/SPARK-39200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608021#comment-17608021 ] Apache Spark commented on SPARK-39200: -- User 'ukby1234' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

[ https://issues.apache.org/jira/browse/SPARK-39200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39200: Assignee: Apache Spark > Stream is corrupted Exception while fetching the blocks from

[jira] [Assigned] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

[ https://issues.apache.org/jira/browse/SPARK-39200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39200: Assignee: (was: Apache Spark) > Stream is corrupted Exception while fetching the

[jira] [Commented] (SPARK-40142) Make pyspark.sql.functions examples self-contained

[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608012#comment-17608012 ] Apache Spark commented on SPARK-40142: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-40142) Make pyspark.sql.functions examples self-contained

[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608011#comment-17608011 ] Apache Spark commented on SPARK-40142: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

[ https://issues.apache.org/jira/browse/SPARK-39200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608008#comment-17608008 ] Frank Yin commented on SPARK-39200: --- We've seen this exception as well. Is there a patch coming?  >

[jira] [Resolved] (SPARK-40303) The performance will be worse after codegen

[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40303. - Resolution: Won't Fix Issue fixed by [JDK-8159720|https://bugs.openjdk.org/browse/JDK-8159720].

[jira] [Updated] (SPARK-40524) local mode with resource scheduling can hang

[ https://issues.apache.org/jira/browse/SPARK-40524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-40524: -- Summary: local mode with resource scheduling can hang (was: local mode with resource

[jira] [Created] (SPARK-40524) local mode with resource scheduling should just fail

Thomas Graves created SPARK-40524: - Summary: local mode with resource scheduling should just fail Key: SPARK-40524 URL: https://issues.apache.org/jira/browse/SPARK-40524 Project: Spark Issue

[jira] [Created] (SPARK-40523) pyspark dataframe methods (i.e. show()) won't run in VSCode debug console

Eli created SPARK-40523: --- Summary: pyspark dataframe methods (i.e. show()) won't run in VSCode debug console Key: SPARK-40523 URL: https://issues.apache.org/jira/browse/SPARK-40523 Project: Spark

[jira] [Comment Edited] (SPARK-40457) upgrade jackson data mapper to latest

[ https://issues.apache.org/jira/browse/SPARK-40457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607949#comment-17607949 ] Bjørn Jørgensen edited comment on SPARK-40457 at 9/21/22 7:49 PM: --

[jira] [Commented] (SPARK-40457) upgrade jackson data mapper to latest

[ https://issues.apache.org/jira/browse/SPARK-40457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607949#comment-17607949 ] Bjørn Jørgensen commented on SPARK-40457: - [~bilna123] Yes, there are no version to upgrade to

[jira] [Updated] (SPARK-40522) Upgrade Apache Kafka from 3.2.1 to 3.2.3

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40522: Summary: Upgrade Apache Kafka from 3.2.1 to 3.2.3 (was: Upgrade kafka from 3.2.1 to

[jira] [Updated] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.3

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40522: Summary: Upgrade kafka from 3.2.1 to 3.2.3 (was: Upgrade kafka from 3.2.1 to 3.2.2) >

[jira] [Comment Edited] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607918#comment-17607918 ] Dongjoon Hyun edited comment on SPARK-40508 at 9/21/22 5:47 PM:

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607918#comment-17607918 ] Dongjoon Hyun commented on SPARK-40508: --- Previously, you are in `Contributor` and `Administrator`.

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607919#comment-17607919 ] Sun Chao commented on SPARK-40508: -- Great to know. Thanks! > Treat unknown partitioning as

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607917#comment-17607917 ] Dongjoon Hyun commented on SPARK-40508: --- Ya, the merge script sometimes hit the corner cases. BTW,

[jira] [Assigned] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40522: Assignee: Apache Spark > Upgrade kafka from 3.2.1 to 3.2.2 >

[jira] [Commented] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607910#comment-17607910 ] Apache Spark commented on SPARK-40522: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40522: Assignee: (was: Apache Spark) > Upgrade kafka from 3.2.1 to 3.2.2 >

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket https://issues.apache.org/jira/browse/SPARK-39469, we introduced

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket https://issues.apache.org/jira/browse/SPARK-39469, we introduced

[jira] (SPARK-40341) Implement `Rolling.median`.

[ https://issues.apache.org/jira/browse/SPARK-40341 ] Artsiom Yudovin deleted comment on SPARK-40341: - was (Author: ayudovin): I'm working on this > Implement `Rolling.median`. > --- > > Key:

[jira] [Commented] (SPARK-40341) Implement `Rolling.median`.

[ https://issues.apache.org/jira/browse/SPARK-40341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607908#comment-17607908 ] Artsiom Yudovin commented on SPARK-40341: - I'm working on this > Implement `Rolling.median`. >

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607902#comment-17607902 ] Sun Chao commented on SPARK-40508: -- Oh, thanks [~viirya] ! For some reason the merge script was

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607900#comment-17607900 ] L. C. Hsieh commented on SPARK-40508: - [~csun] Seems he is already in contributor list. I just

[jira] [Assigned] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40508: --- Assignee: Ted Yu > Treat unknown partitioning as UnknownPartitioning >

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Summary: Correct CSV schema inference and data parsing behavior on columns with mixed dates and

[jira] [Commented] (SPARK-40521) PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions instead of the conflicting partition

[ https://issues.apache.org/jira/browse/SPARK-40521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607897#comment-17607897 ] Serge Rielau commented on SPARK-40521: -- Hive does return the offending partition. We just need to

[jira] [Updated] (SPARK-40521) PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions instead of the conflicting partition

[ https://issues.apache.org/jira/browse/SPARK-40521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-40521: - Attachment: Screen Shot 2022-09-21 at 10.08.52 AM.png Screen Shot 2022-09-21 at

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607895#comment-17607895 ] Apache Spark commented on SPARK-40508: -- User 'tedyu' has created a pull request for this issue:

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607893#comment-17607893 ] Apache Spark commented on SPARK-40508: -- User 'tedyu' has created a pull request for this issue:

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607869#comment-17607869 ] Chao Sun commented on SPARK-40508: -- [~dongjoon][~viirya] could you add [~yuzhih...@gmail.com] to the

[jira] [Resolved] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-40508. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37952

[jira] [Updated] (SPARK-40521) PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions instead of the conflicting partition

[ https://issues.apache.org/jira/browse/SPARK-40521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-40521: - Description: PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions

[jira] [Comment Edited] (SPARK-40427) Add error classes for LIMIT/OFFSET CheckAnalysis failures

[ https://issues.apache.org/jira/browse/SPARK-40427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607824#comment-17607824 ] Franck Thang edited comment on SPARK-40427 at 9/21/22 3:33 PM: --- Hi

[jira] [Commented] (SPARK-40427) Add error classes for LIMIT/OFFSET CheckAnalysis failures

[ https://issues.apache.org/jira/browse/SPARK-40427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607824#comment-17607824 ] Franck Thang commented on SPARK-40427: -- Hi [~dtenedor] , I think this is a duplicated ticket with

[jira] [Updated] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40522: Description: [Memory Allocation with Excessive Size Value

[jira] [Updated] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40522: Description: [https://security.snyk.io/vuln/SNYK-JAVA-ORGAPACHEKAFKA-3027430 Memory

[jira] [Updated] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

[ https://issues.apache.org/jira/browse/SPARK-40522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40522: Description: [https://security.snyk.io/vuln/SNYK-JAVA-ORGAPACHEKAFKA-3027430|Memory

[jira] [Created] (SPARK-40522) Upgrade kafka from 3.2.1 to 3.2.2

Bjørn Jørgensen created SPARK-40522: --- Summary: Upgrade kafka from 3.2.1 to 3.2.2 Key: SPARK-40522 URL: https://issues.apache.org/jira/browse/SPARK-40522 Project: Spark Issue Type:

[jira] [Created] (SPARK-40521) PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions instead of the conflicting partition

Serge Rielau created SPARK-40521: Summary: PartitionsAlreadyExistException in Hive V1 Command V1 reports all partitions instead of the conflicting partition Key: SPARK-40521 URL:

[jira] [Resolved] (SPARK-40490) `YarnShuffleIntegrationSuite` no longer verifies `registeredExecFile` reload after SPARK-17321

[ https://issues.apache.org/jira/browse/SPARK-40490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-40490. --- Fix Version/s: 3.4.0 Assignee: Yang Jie Resolution: Fixed >

[jira] [Created] (SPARK-40520) Add a script to generate DOI mainifest

Yikun Jiang created SPARK-40520: --- Summary: Add a script to generate DOI mainifest Key: SPARK-40520 URL: https://issues.apache.org/jira/browse/SPARK-40520 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-40516) Add official image dockerfile for Spark v3.3.0

[ https://issues.apache.org/jira/browse/SPARK-40516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40516: Description: Example: [https://github.com/Yikun/spark-docker/tree/master/3.3.0] Test:

[jira] [Created] (SPARK-40519) Add "Publish workflow" to help release apache/spark image

Yikun Jiang created SPARK-40519: --- Summary: Add "Publish workflow" to help release apache/spark image Key: SPARK-40519 URL: https://issues.apache.org/jira/browse/SPARK-40519 Project: Spark

[jira] [Updated] (SPARK-40516) Add official image dockerfile for Spark v3.3.0

[ https://issues.apache.org/jira/browse/SPARK-40516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40516: Description: Example: [https://github.com/Yikun/spark-docker/tree/master/3.3.0]   > Add

[jira] [Created] (SPARK-40517) Add DOI manifest file for Spark Docker Official Image

Yikun Jiang created SPARK-40517: --- Summary: Add DOI manifest file for Spark Docker Official Image Key: SPARK-40517 URL: https://issues.apache.org/jira/browse/SPARK-40517 Project: Spark Issue

[jira] [Created] (SPARK-40518) Add Spark Docker Official Image doc

Yikun Jiang created SPARK-40518: --- Summary: Add Spark Docker Official Image doc Key: SPARK-40518 URL: https://issues.apache.org/jira/browse/SPARK-40518 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-40516) Add official image dockerfile for Spark v3.3.0

Yikun Jiang created SPARK-40516: --- Summary: Add official image dockerfile for Spark v3.3.0 Key: SPARK-40516 URL: https://issues.apache.org/jira/browse/SPARK-40516 Project: Spark Issue Type:

[jira] [Created] (SPARK-40515) Add apache/spark-docker repo

Yikun Jiang created SPARK-40515: --- Summary: Add apache/spark-docker repo Key: SPARK-40515 URL: https://issues.apache.org/jira/browse/SPARK-40515 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-40175) Converting Tuple2 to Scala Map via `.toMap` is slow

[ https://issues.apache.org/jira/browse/SPARK-40175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40175: - Priority: Minor (was: Major) > Converting Tuple2 to Scala Map via `.toMap` is slow >

[jira] [Resolved] (SPARK-40494) Optimize the performance of `keys.zipWithIndex.toMap` code pattern

[ https://issues.apache.org/jira/browse/SPARK-40494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40494. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37940

[jira] [Assigned] (SPARK-40494) Optimize the performance of `keys.zipWithIndex.toMap` code pattern

[ https://issues.apache.org/jira/browse/SPARK-40494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40494: --- Assignee: Yang Jie > Optimize the performance of `keys.zipWithIndex.toMap` code pattern >

[jira] [Commented] (SPARK-40514) Python related tests need to check the Python version

[ https://issues.apache.org/jira/browse/SPARK-40514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607606#comment-17607606 ] Apache Spark commented on SPARK-40514: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-40514) Python related tests need to check the Python version

[ https://issues.apache.org/jira/browse/SPARK-40514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40514: Assignee: Apache Spark > Python related tests need to check the Python version >

[jira] [Assigned] (SPARK-40514) Python related tests need to check the Python version

[ https://issues.apache.org/jira/browse/SPARK-40514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40514: Assignee: (was: Apache Spark) > Python related tests need to check the Python

[jira] [Commented] (SPARK-40514) Python related tests need to check the Python version

[ https://issues.apache.org/jira/browse/SPARK-40514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607605#comment-17607605 ] Apache Spark commented on SPARK-40514: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-40498) Implement `kendall` and `min_periods` in `Series.corr`

[ https://issues.apache.org/jira/browse/SPARK-40498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40498: - Assignee: Ruifeng Zheng > Implement `kendall` and `min_periods` in `Series.corr` >

[jira] [Resolved] (SPARK-40498) Implement `kendall` and `min_periods` in `Series.corr`

[ https://issues.apache.org/jira/browse/SPARK-40498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40498. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37945

[jira] [Created] (SPARK-40514) Python related tests need to check the Python version

Yang Jie created SPARK-40514: Summary: Python related tests need to check the Python version Key: SPARK-40514 URL: https://issues.apache.org/jira/browse/SPARK-40514 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40513) SPIP: Support Docker Official Image for Spark

[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40513: Issue Type: New Feature (was: Umbrella) > SPIP: Support Docker Official Image for Spark >

[jira] [Updated] (SPARK-40513) SPIP: Support Docker Official Image for Spark

[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40513: Issue Type: Umbrella (was: Bug) > SPIP: Support Docker Official Image for Spark >

[jira] [Updated] (SPARK-40513) SPIP: Support Docker Official Image for Spark

[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40513: Labels: SPIP (was: ) > SPIP: Support Docker Official Image for Spark >

[jira] [Updated] (SPARK-40513) SPIP: Support Docker Official Image for Spark

[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40513: Description: This SPIP is proposed to add [Docker Official

[jira] [Updated] (SPARK-40513) SPIP: Support Docker Official Image for Spark

[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40513: Description: This SPIP is proposed to add [Docker Official

[jira] [Created] (SPARK-40513) SPIP: Support Docker Official Image for Spark

Yikun Jiang created SPARK-40513: --- Summary: SPIP: Support Docker Official Image for Spark Key: SPARK-40513 URL: https://issues.apache.org/jira/browse/SPARK-40513 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-40512) Upgrade pandas to 1.5.0

[ https://issues.apache.org/jira/browse/SPARK-40512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607527#comment-17607527 ] Apache Spark commented on SPARK-40512: -- User 'itholic' has created a pull request for this issue:

[jira] (SPARK-34805) PySpark loses metadata in DataFrame fields when selecting nested columns

[ https://issues.apache.org/jira/browse/SPARK-34805 ] Joost Farla deleted comment on SPARK-34805: - was (Author: JIRAUSER295969): [~cloud_fan] I was running into the exact same issue using Spark v3.3.0. It looks like the fix was merged into the

[jira] [Assigned] (SPARK-40512) Upgrade pandas to 1.5.0

[ https://issues.apache.org/jira/browse/SPARK-40512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40512: Assignee: Apache Spark > Upgrade pandas to 1.5.0 > --- > >

[jira] [Assigned] (SPARK-40512) Upgrade pandas to 1.5.0

[ https://issues.apache.org/jira/browse/SPARK-40512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40512: Assignee: (was: Apache Spark) > Upgrade pandas to 1.5.0 > --- >

[jira] [Comment Edited] (SPARK-40502) Support dataframe API use jdbc data source in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607523#comment-17607523 ] CaoYu edited comment on SPARK-40502 at 9/21/22 6:07 AM: I am a teacher Recently

[jira] [Commented] (SPARK-40502) Support dataframe API use jdbc data source in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607524#comment-17607524 ] CaoYu commented on SPARK-40502: --- When I designed the Python Flink course It is found that PyFlink does not

[jira] [Created] (SPARK-40512) Upgrade pandas to 1.5.0

Haejoon Lee created SPARK-40512: --- Summary: Upgrade pandas to 1.5.0 Key: SPARK-40512 URL: https://issues.apache.org/jira/browse/SPARK-40512 Project: Spark Issue Type: Improvement