[jira] [Updated] (SPARK-45185) Ignore type check for preventing unexpected linter failure

[ https://issues.apache.org/jira/browse/SPARK-45185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45185: --- Labels: pull-request-available (was: ) > Ignore type check for preventing unexpected

[jira] [Created] (SPARK-45185) Ignore type check for preventing unexpected linter failure

Haejoon Lee created SPARK-45185: --- Summary: Ignore type check for preventing unexpected linter failure Key: SPARK-45185 URL: https://issues.apache.org/jira/browse/SPARK-45185 Project: Spark

[jira] [Updated] (SPARK-45184) Remove orphaned error class documents

[ https://issues.apache.org/jira/browse/SPARK-45184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45184: --- Labels: pull-request-available (was: ) > Remove orphaned error class documents >

[jira] [Created] (SPARK-45184) Remove orphaned error class documents

BingKun Pan created SPARK-45184: --- Summary: Remove orphaned error class documents Key: SPARK-45184 URL: https://issues.apache.org/jira/browse/SPARK-45184 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-45181) Upgrade buf to v1.26.1

[ https://issues.apache.org/jira/browse/SPARK-45181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45181: - Assignee: Ruifeng Zheng > Upgrade buf to v1.26.1 > -- > >

[jira] [Resolved] (SPARK-45181) Upgrade buf to v1.26.1

[ https://issues.apache.org/jira/browse/SPARK-45181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45181. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42947

[jira] [Resolved] (SPARK-40497) Upgrade Scala to 2.13.11

[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40497. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42918

[jira] [Resolved] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45164. --- Resolution: Fixed Issue resolved by pull request 42926

[jira] [Assigned] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45164: - Assignee: Haejoon Lee > Remove deprecated Index APIs > >

[jira] [Resolved] (SPARK-45173) Remove some unnecessary sourceMapping files in UI

[ https://issues.apache.org/jira/browse/SPARK-45173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45173. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42935

[jira] [Assigned] (SPARK-45173) Remove some unnecessary sourceMapping files in UI

[ https://issues.apache.org/jira/browse/SPARK-45173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45173: - Assignee: Kent Yao > Remove some unnecessary sourceMapping files in UI >

[jira] [Commented] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

[ https://issues.apache.org/jira/browse/SPARK-45183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765886#comment-17765886 ] hgs commented on SPARK-45183: - [~dongjoon] , will you have a look at this. > ExecutorPodsLifecycleManager

[jira] [Created] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

hgs created SPARK-45183: --- Summary: ExecutorPodsLifecycleManager delete a pod multi times. Key: SPARK-45183 URL: https://issues.apache.org/jira/browse/SPARK-45183 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45078) The ArrayInsert function should make explicit casting when element type not equals derived component type

[ https://issues.apache.org/jira/browse/SPARK-45078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45078: --- Labels: pull-request-available (was: ) > The ArrayInsert function should make explicit

[jira] [Updated] (SPARK-45182) Ignore task completion from old stage after retrying indeterminate stages

[ https://issues.apache.org/jira/browse/SPARK-45182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45182: --- Labels: pull-request-available (was: ) > Ignore task completion from old stage after

[jira] [Updated] (SPARK-45182) Ignore task completion from old stage after retrying indeterminate stages

[ https://issues.apache.org/jira/browse/SPARK-45182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayur Bhosale updated SPARK-45182: -- Description: SPARK-25342 Added a support for rolling back shuffle map stage so that all

[jira] [Created] (SPARK-45182) Ignore task completion from old stage after retrying indeterministic stages

Mayur Bhosale created SPARK-45182: - Summary: Ignore task completion from old stage after retrying indeterministic stages Key: SPARK-45182 URL: https://issues.apache.org/jira/browse/SPARK-45182

[jira] [Updated] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method

[ https://issues.apache.org/jira/browse/SPARK-44913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44913: --- Labels: pull-request-available (was: ) > DS V2 supports push down V2 UDF that has magic

[jira] [Resolved] (SPARK-36695) Allow passing V2 functions to data sources via V2 filters

[ https://issues.apache.org/jira/browse/SPARK-36695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-36695. -- Resolution: Duplicate > Allow passing V2 functions to data sources via V2 filters >

[jira] [Updated] (SPARK-43496) Have a separate config for Memory limits for kubernetes pods

[ https://issues.apache.org/jira/browse/SPARK-43496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43496: --- Labels: pull-request-available (was: ) > Have a separate config for Memory limits for

[jira] [Updated] (SPARK-45093) AddArtifacts should give proper error messages if it fails

[ https://issues.apache.org/jira/browse/SPARK-45093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45093: --- Labels: pull-request-available (was: ) > AddArtifacts should give proper error messages if

[jira] [Assigned] (SPARK-45179) Increase Numpy minimum version to 1.21

[ https://issues.apache.org/jira/browse/SPARK-45179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45179: - Assignee: Ruifeng Zheng > Increase Numpy minimum version to 1.21 >

[jira] [Resolved] (SPARK-45179) Increase Numpy minimum version to 1.21

[ https://issues.apache.org/jira/browse/SPARK-45179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45179. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42944

[jira] [Updated] (SPARK-45154) Pyspark DecisionTreeClassifier: results and tree structure in spark3 very different from that of the spark2 version on the same data and with the same hyperparameters.

[ https://issues.apache.org/jira/browse/SPARK-45154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oumar Nour updated SPARK-45154: --- Component/s: Spark Core > Pyspark DecisionTreeClassifier: results and tree structure in spark3 very

[jira] [Updated] (SPARK-45181) Upgrade buf to v1.26.1

[ https://issues.apache.org/jira/browse/SPARK-45181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45181: --- Labels: pull-request-available (was: ) > Upgrade buf to v1.26.1 > -- >

[jira] [Created] (SPARK-45181) Upgrade buf to v1.26.1

Ruifeng Zheng created SPARK-45181: - Summary: Upgrade buf to v1.26.1 Key: SPARK-45181 URL: https://issues.apache.org/jira/browse/SPARK-45181 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-45151) Direct Task ThreadDump

[ https://issues.apache.org/jira/browse/SPARK-45151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-45151. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42904

[jira] [Assigned] (SPARK-45151) Direct Task ThreadDump

[ https://issues.apache.org/jira/browse/SPARK-45151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-45151: Assignee: Kent Yao > Direct Task ThreadDump > -- > > Key:

[jira] [Assigned] (SPARK-43752) default column value should support v2 write commands

[ https://issues.apache.org/jira/browse/SPARK-43752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43752: -- Assignee: (was: Apache Spark) > default column value should support v2 write

[jira] [Assigned] (SPARK-43752) default column value should support v2 write commands

[ https://issues.apache.org/jira/browse/SPARK-43752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43752: -- Assignee: Apache Spark > default column value should support v2 write commands >

[jira] [Assigned] (SPARK-43752) default column value should support v2 write commands

[ https://issues.apache.org/jira/browse/SPARK-43752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43752: -- Assignee: (was: Apache Spark) > default column value should support v2 write

[jira] [Assigned] (SPARK-45175) download krb5.conf from remote storage in spark-submit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45175: -- Assignee: (was: Apache Spark) > download krb5.conf from remote storage in

[jira] [Assigned] (SPARK-45175) download krb5.conf from remote storage in spark-submit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45175: -- Assignee: Apache Spark > download krb5.conf from remote storage in spark-submit on

[jira] [Assigned] (SPARK-43752) default column value should support v2 write commands

[ https://issues.apache.org/jira/browse/SPARK-43752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43752: -- Assignee: Apache Spark > default column value should support v2 write commands >

[jira] [Assigned] (SPARK-45175) download krb5.conf from remote storage in spark-submit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45175: -- Assignee: Apache Spark > download krb5.conf from remote storage in spark-submit on

[jira] [Assigned] (SPARK-45175) download krb5.conf from remote storage in spark-submit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45175: -- Assignee: (was: Apache Spark) > download krb5.conf from remote storage in

[jira] [Assigned] (SPARK-45065) Support Pandas 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-45065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45065: -- Assignee: (was: Apache Spark) > Support Pandas 2.1.0 > > >

[jira] [Assigned] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45164: -- Assignee: (was: Apache Spark) > Remove deprecated Index APIs >

[jira] [Assigned] (SPARK-45180) Remove boolean inputs for inclusive from Series.between

[ https://issues.apache.org/jira/browse/SPARK-45180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45180: -- Assignee: (was: Apache Spark) > Remove boolean inputs for inclusive from

[jira] [Assigned] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45164: -- Assignee: Apache Spark > Remove deprecated Index APIs >

[jira] [Assigned] (SPARK-45065) Support Pandas 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-45065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45065: -- Assignee: Apache Spark > Support Pandas 2.1.0 > > >

[jira] [Assigned] (SPARK-45180) Remove boolean inputs for inclusive from Series.between

[ https://issues.apache.org/jira/browse/SPARK-45180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45180: -- Assignee: Apache Spark > Remove boolean inputs for inclusive from Series.between >

[jira] [Updated] (SPARK-45175) download krb5.conf from remote storage in spark-submit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Sun updated SPARK-45175: - Summary: download krb5.conf from remote storage in spark-submit on k8s (was: download krb5.conf from

[jira] [Assigned] (SPARK-45177) Remove `col_space` parameter from `DataFrame.to_latex`

[ https://issues.apache.org/jira/browse/SPARK-45177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45177: Assignee: Haejoon Lee > Remove `col_space` parameter from `DataFrame.to_latex` >

[jira] [Resolved] (SPARK-45177) Remove `col_space` parameter from `DataFrame.to_latex`

[ https://issues.apache.org/jira/browse/SPARK-45177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45177. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42937

[jira] [Updated] (SPARK-43752) default column value should support v2 write commands

[ https://issues.apache.org/jira/browse/SPARK-43752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43752: --- Labels: pull-request-available (was: ) > default column value should support v2 write

[jira] [Assigned] (SPARK-45166) Clean up unused code paths for pyarrow<4

[ https://issues.apache.org/jira/browse/SPARK-45166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45166: Assignee: Ruifeng Zheng > Clean up unused code paths for pyarrow<4 >

[jira] [Resolved] (SPARK-45166) Clean up unused code paths for pyarrow<4

[ https://issues.apache.org/jira/browse/SPARK-45166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45166. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42928

[jira] [Resolved] (SPARK-45128) Support CalendarIntervalType in Arrow

[ https://issues.apache.org/jira/browse/SPARK-45128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45128. -- Fix Version/s: 4.0.0 Assignee: Ruifeng Zheng Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-25689) Move token renewal logic to driver in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-25689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765477#comment-17765477 ] RickyMa commented on SPARK-25689: - I've added a follow up pull request for this issue:

[jira] [Commented] (SPARK-43561) Enable DataFrameConversionTests.test_to_latex for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765476#comment-17765476 ] Hudson commented on SPARK-43561: User 'itholic' has created a pull request for this issue:

[jira] [Updated] (SPARK-45180) Remove boolean inputs for inclusive from Series.between

[ https://issues.apache.org/jira/browse/SPARK-45180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45180: --- Labels: pull-request-available (was: ) > Remove boolean inputs for inclusive from

[jira] [Created] (SPARK-45180) Remove boolean inputs for inclusive from Series.between

Haejoon Lee created SPARK-45180: --- Summary: Remove boolean inputs for inclusive from Series.between Key: SPARK-45180 URL: https://issues.apache.org/jira/browse/SPARK-45180 Project: Spark Issue

[jira] [Commented] (SPARK-44113) Make Scala 2.13+ as default Scala version

[ https://issues.apache.org/jira/browse/SPARK-44113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765474#comment-17765474 ] Yang Jie commented on SPARK-44113: -- [~dongjoon] Spark 3.5.0 has been released. Will we start working on

[jira] [Updated] (SPARK-45179) Increase Numpy minimum version to 1.21

[ https://issues.apache.org/jira/browse/SPARK-45179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45179: --- Labels: pull-request-available (was: ) > Increase Numpy minimum version to 1.21 >

[jira] [Updated] (SPARK-45175) download krb5.conf from remote storage in spark-sumbit on k8s

[ https://issues.apache.org/jira/browse/SPARK-45175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45175: --- Labels: pull-request-available (was: ) > download krb5.conf from remote storage in

[jira] [Updated] (SPARK-44376) Build using maven is broken using 2.13 and Java 11 and Java 17

[ https://issues.apache.org/jira/browse/SPARK-44376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44376: --- Labels: pull-request-available (was: ) > Build using maven is broken using 2.13 and Java

[jira] [Created] (SPARK-45179) Increase Numpy minimum version to 1.21

Ruifeng Zheng created SPARK-45179: - Summary: Increase Numpy minimum version to 1.21 Key: SPARK-45179 URL: https://issues.apache.org/jira/browse/SPARK-45179 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-26365: --- Labels: pull-request-available (was: ) > spark-submit for k8s cluster doesn't propagate

[jira] [Updated] (SPARK-43874) Enable GroupByTests for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43874: --- Labels: pull-request-available (was: ) > Enable GroupByTests for pandas 2.0.0. >

[jira] [Resolved] (SPARK-43811) Enable DataFrameTests.test_reindex for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43811. - Resolution: Fixed > Enable DataFrameTests.test_reindex for pandas 2.0.0. >

[jira] [Resolved] (SPARK-44276) Match behavior with pandas for `SeriesStringTests.test_string_replace`

[ https://issues.apache.org/jira/browse/SPARK-44276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-44276. - Resolution: Fixed > Match behavior with pandas for `SeriesStringTests.test_string_replace` >

[jira] [Resolved] (SPARK-43644) Enable DatetimeIndexTests.test_indexer_between_time for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43644. - Resolution: Fixed > Enable DatetimeIndexTests.test_indexer_between_time for pandas 2.0.0. >

[jira] [Resolved] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43433. - Resolution: Fixed > Match `GroupBy.nth` behavior with new pandas behavior >

[jira] [Resolved] (SPARK-43291) Generate proper warning on different behavior with numeric_only

[ https://issues.apache.org/jira/browse/SPARK-43291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43291. - Resolution: Won't Fix We should match the behavior w/ Pandas instead of warning. > Generate

[jira] [Resolved] (SPARK-43282) Investigate DataFrame.sort_values with pandas behavior.

[ https://issues.apache.org/jira/browse/SPARK-43282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43282. - Resolution: Won't Fix > Investigate DataFrame.sort_values with pandas behavior. >