[jira] [Updated] (SPARK-50391) DataFrame conversion to table argument

[ https://issues.apache.org/jira/browse/SPARK-50391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-50391: - Description: {color:#172b4d}Table-Valued Functions (TVFs) and User-Defined Table Functions (UDTF

[jira] [Updated] (SPARK-50392) DataFrame conversion to table argument in Spark Classic

[ https://issues.apache.org/jira/browse/SPARK-50392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-50392: - Summary: DataFrame conversion to table argument in Spark Classic (was: Implement partitionBy, o

[jira] [Created] (SPARK-50488) `No enum constant java.sql.JDBCType.varchar` error writing array values

Javier Neira Sanchez created SPARK-50488: Summary: `No enum constant java.sql.JDBCType.varchar` error writing array values Key: SPARK-50488 URL: https://issues.apache.org/jira/browse/SPARK-50488

[jira] [Updated] (SPARK-16675) Avoid per-record type dispatch in JDBC when writing

[ https://issues.apache.org/jira/browse/SPARK-16675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-16675: --- Labels: pull-request-available (was: ) > Avoid per-record type dispatch in JDBC when writin

[jira] [Updated] (SPARK-50487) Update broken jira link

[ https://issues.apache.org/jira/browse/SPARK-50487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50487: --- Labels: pull-request-available (was: ) > Update broken jira link > ---

[jira] [Created] (SPARK-50487) Update broken jira link

xiaoping.huang created SPARK-50487: -- Summary: Update broken jira link Key: SPARK-50487 URL: https://issues.apache.org/jira/browse/SPARK-50487 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-50486) Refine the docstring for string functions - part 2

[ https://issues.apache.org/jira/browse/SPARK-50486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50486: Assignee: Ruifeng Zheng > Refine the docstring for string functions - part 2 > --

[jira] [Resolved] (SPARK-50486) Refine the docstring for string functions - part 2

[ https://issues.apache.org/jira/browse/SPARK-50486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50486. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49049 [https://gi

[jira] [Updated] (SPARK-50485) Unwrap SparkThrowable in UncheckedExecutionException thrown by tableRelationCache

[ https://issues.apache.org/jira/browse/SPARK-50485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50485: --- Labels: pull-request-available (was: ) > Unwrap SparkThrowable in UncheckedExecutionExcepti

[jira] [Updated] (SPARK-50486) Refine the docstring for string functions - part 2

[ https://issues.apache.org/jira/browse/SPARK-50486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50486: --- Labels: pull-request-available (was: ) > Refine the docstring for string functions - part 2

[jira] [Created] (SPARK-50486) Refine the docstring for string functions - part 2

Ruifeng Zheng created SPARK-50486: - Summary: Refine the docstring for string functions - part 2 Key: SPARK-50486 URL: https://issues.apache.org/jira/browse/SPARK-50486 Project: Spark Issue Ty

[jira] [Created] (SPARK-50485) Unwrap SparkThrowable in UncheckedExecutionException thrown by tableRelationCache

Kent Yao created SPARK-50485: Summary: Unwrap SparkThrowable in UncheckedExecutionException thrown by tableRelationCache Key: SPARK-50485 URL: https://issues.apache.org/jira/browse/SPARK-50485 Project: Sp

[jira] [Resolved] (SPARK-50481) Improve `SortShuffleManager.unregisterShuffle` to skip checksum file logic if checksum is disabled

[ https://issues.apache.org/jira/browse/SPARK-50481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50481. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49046 [https://gi

[jira] [Assigned] (SPARK-50481) Improve `SortShuffleManager.unregisterShuffle` to skip checksum file logic if checksum is disabled

[ https://issues.apache.org/jira/browse/SPARK-50481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50481: Assignee: Dongjoon Hyun > Improve `SortShuffleManager.unregisterShuffle` to skip checksum

[jira] [Resolved] (SPARK-50482) Deprecated no-op `spark.shuffle.spill` config

[ https://issues.apache.org/jira/browse/SPARK-50482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50482. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49047 [https://gi

[jira] [Assigned] (SPARK-50482) Deprecated no-op `spark.shuffle.spill` config

[ https://issues.apache.org/jira/browse/SPARK-50482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50482: Assignee: Dongjoon Hyun > Deprecated no-op `spark.shuffle.spill` config > ---

[jira] [Resolved] (SPARK-50484) Skip ClassificationTestsOnConnect tests properly if dependences are not found

[ https://issues.apache.org/jira/browse/SPARK-50484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50484. -- Resolution: Invalid > Skip ClassificationTestsOnConnect tests properly if dependences are not

[jira] [Updated] (SPARK-50484) Skip ClassificationTestsOnConnect tests properly if dependences are not found

[ https://issues.apache.org/jira/browse/SPARK-50484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50484: --- Labels: pull-request-available (was: ) > Skip ClassificationTestsOnConnect tests properly i

[jira] [Created] (SPARK-50484) Skip ClassificationTestsOnConnect tests properly if dependences are not found

Hyukjin Kwon created SPARK-50484: Summary: Skip ClassificationTestsOnConnect tests properly if dependences are not found Key: SPARK-50484 URL: https://issues.apache.org/jira/browse/SPARK-50484 Project

[jira] [Updated] (SPARK-50483) Still throwing BlockMissingException even with ignoreCorruptFiles enabled

[ https://issues.apache.org/jira/browse/SPARK-50483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-50483: Attachment: success-task.png > Still throwing BlockMissingException even with ignoreCorruptFiles e

[jira] [Created] (SPARK-50483) Still throwing BlockMissingException even with ignoreCorruptFiles enabled

Yuming Wang created SPARK-50483: --- Summary: Still throwing BlockMissingException even with ignoreCorruptFiles enabled Key: SPARK-50483 URL: https://issues.apache.org/jira/browse/SPARK-50483 Project: Spar

[jira] [Resolved] (SPARK-50470) Block usage of collations for maps' key

[ https://issues.apache.org/jira/browse/SPARK-50470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-50470. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49030 [https://github.com

[jira] [Updated] (SPARK-50481) Improve `SortShuffleManager.unregisterShuffle` to skip checksum file logic if checksum is disabled

[ https://issues.apache.org/jira/browse/SPARK-50481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50481: -- Summary: Improve `SortShuffleManager.unregisterShuffle` to skip checksum file logic if checksu

[jira] [Updated] (SPARK-50384) Update docs with YuniKorn 1.6.0 and first-time installation guide

[ https://issues.apache.org/jira/browse/SPARK-50384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50384: -- Summary: Update docs with YuniKorn 1.6.0 and first-time installation guide (was: Update docs

[jira] [Resolved] (SPARK-50384) Update docs for first-time YuniKorn installation

[ https://issues.apache.org/jira/browse/SPARK-50384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50384. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-50482) Deprecated no-op `spark.shuffle.spill` config

[ https://issues.apache.org/jira/browse/SPARK-50482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50482: -- Description: Previously, Spark shows a warning when `spark.shuffle.spill=false`.  After this,

[jira] [Updated] (SPARK-50482) Deprecated no-op `spark.shuffle.spill` config

[ https://issues.apache.org/jira/browse/SPARK-50482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50482: --- Labels: pull-request-available (was: ) > Deprecated no-op `spark.shuffle.spill` config > --

[jira] [Updated] (SPARK-50482) Deprecated no-op `spark.shuffle.spill` config

[ https://issues.apache.org/jira/browse/SPARK-50482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50482: -- Summary: Deprecated no-op `spark.shuffle.spill` config (was: Deprecated `spark.shuffle.spill`

[jira] [Created] (SPARK-50482) Deprecated `spark.shuffle.spill` config

Dongjoon Hyun created SPARK-50482: - Summary: Deprecated `spark.shuffle.spill` config Key: SPARK-50482 URL: https://issues.apache.org/jira/browse/SPARK-50482 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-50481) Improve `SortShuffleManager.unregisterShuffle` to skip checksum file logic if checksum is disabled

[ https://issues.apache.org/jira/browse/SPARK-50481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50481: --- Labels: pull-request-available (was: ) > Improve `SortShuffleManager.unregisterShuffle` to

[jira] [Created] (SPARK-50481) Improve `IndexShuffleBlockResolver.removeDataByMap` to skip checksum file logic if checksum is disabled

Dongjoon Hyun created SPARK-50481: - Summary: Improve `IndexShuffleBlockResolver.removeDataByMap` to skip checksum file logic if checksum is disabled Key: SPARK-50481 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Assigned] (SPARK-50463) Reading partition values from parquet over multiple batches fails

[ https://issues.apache.org/jira/browse/SPARK-50463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50463: - Assignee: Richard Chen > Reading partition values from parquet over multiple batches fa

[jira] [Resolved] (SPARK-50463) Reading partition values from parquet over multiple batches fails

[ https://issues.apache.org/jira/browse/SPARK-50463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50463. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49021 [https://

[jira] [Updated] (SPARK-50478) Fix collated string matching

[ https://issues.apache.org/jira/browse/SPARK-50478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jovan Markovic updated SPARK-50478: --- Summary: Fix collated string matching (was: UpCasting to collated string) > Fix collated st

[jira] [Updated] (SPARK-50433) Fix confguring log4j2 guide docs for Spark on YARN

[ https://issues.apache.org/jira/browse/SPARK-50433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50433: -- Fix Version/s: 3.5.4 > Fix confguring log4j2 guide docs for Spark on YARN > --

[jira] [Updated] (SPARK-50425) Bump Apache Parquet to 1.15.0

[ https://issues.apache.org/jira/browse/SPARK-50425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50425: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Bump Apache Parquet to

[jira] [Assigned] (SPARK-50425) Bump Apache Parquet to 1.15.0

[ https://issues.apache.org/jira/browse/SPARK-50425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50425: - Assignee: Fokko Driesprong > Bump Apache Parquet to 1.15.0 > --

[jira] [Resolved] (SPARK-50425) Bump Apache Parquet to 1.15.0

[ https://issues.apache.org/jira/browse/SPARK-50425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50425. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48970 [https://

[jira] [Commented] (SPARK-49570) Use agnostic encoders instead of expression encoders in user facing API

[ https://issues.apache.org/jira/browse/SPARK-49570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902692#comment-17902692 ] Herman van Hövell commented on SPARK-49570: --- This is done. > Use agnostic enc

[jira] [Assigned] (SPARK-49436) Add shared SQLContext interface

[ https://issues.apache.org/jira/browse/SPARK-49436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49436: - Assignee: Pengfei Xu > Add shared SQLContext interface > --

[jira] [Assigned] (SPARK-43415) Impl mapValues for KVGDS#mapValues

[ https://issues.apache.org/jira/browse/SPARK-43415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-43415: - Assignee: Pengfei Xu > Impl mapValues for KVGDS#mapValues > ---

[jira] [Resolved] (SPARK-49769) Replace ExpressionUtils.colum with ClassicConversions

[ https://issues.apache.org/jira/browse/SPARK-49769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-49769. --- Fix Version/s: 4.0.0 Assignee: Herman van Hövell Resolution: Fixed F

[jira] [Assigned] (SPARK-49570) Use agnostic encoders instead of expression encoders in user facing API

[ https://issues.apache.org/jira/browse/SPARK-49570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49570: - Assignee: Herman van Hövell > Use agnostic encoders instead of expression encod

[jira] [Resolved] (SPARK-49570) Use agnostic encoders instead of expression encoders in user facing API

[ https://issues.apache.org/jira/browse/SPARK-49570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-49570. --- Fix Version/s: 4.0.0 Resolution: Fixed > Use agnostic encoders instead of exp

[jira] [Resolved] (SPARK-50473) Add generic ColumnConversions helper

[ https://issues.apache.org/jira/browse/SPARK-50473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-50473. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49038 [

[jira] [Commented] (SPARK-50277) [k8s] Apply for executor pods in parallel

[ https://issues.apache.org/jira/browse/SPARK-50277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902688#comment-17902688 ] Dongjoon Hyun commented on SPARK-50277: --- This is a performance issue of K8s Master

[jira] [Resolved] (SPARK-50474) Upgrade checkstyle to 10.20.2

[ https://issues.apache.org/jira/browse/SPARK-50474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-50474. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49040 [https://github.com

[jira] [Updated] (SPARK-50480) Extend CharType and VarcharType from StringType

[ https://issues.apache.org/jira/browse/SPARK-50480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jovan Markovic updated SPARK-50480: --- Description: Change type hierarchy so that CharType and VarcharType extend StringType. This

[jira] [Created] (SPARK-50480) Extend CharType and VarcharType from StringType

Jovan Markovic created SPARK-50480: -- Summary: Extend CharType and VarcharType from StringType Key: SPARK-50480 URL: https://issues.apache.org/jira/browse/SPARK-50480 Project: Spark Issue Typ

[jira] [Created] (SPARK-50479) Improving char/varchar support

Jovan Markovic created SPARK-50479: -- Summary: Improving char/varchar support Key: SPARK-50479 URL: https://issues.apache.org/jira/browse/SPARK-50479 Project: Spark Issue Type: Epic

[jira] [Updated] (SPARK-50478) UpCasting to collated string

[ https://issues.apache.org/jira/browse/SPARK-50478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50478: --- Labels: pull-request-available (was: ) > UpCasting to collated string > ---

[jira] [Created] (SPARK-50478) UpCasting to collated string

Jovan Markovic created SPARK-50478: -- Summary: UpCasting to collated string Key: SPARK-50478 URL: https://issues.apache.org/jira/browse/SPARK-50478 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-50477) Add a separate docker file for python 3.9 daily build

[ https://issues.apache.org/jira/browse/SPARK-50477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50477: --- Labels: pull-request-available (was: ) > Add a separate docker file for python 3.9 daily bu

[jira] [Created] (SPARK-50477) Add a separate docker file for python 3.9 daily build

Ruifeng Zheng created SPARK-50477: - Summary: Add a separate docker file for python 3.9 daily build Key: SPARK-50477 URL: https://issues.apache.org/jira/browse/SPARK-50477 Project: Spark Issue

[jira] [Updated] (SPARK-50460) Generalize and simplify Connect exception handling

[ https://issues.apache.org/jira/browse/SPARK-50460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-50460: Summary: Generalize and simplify Connect exception handling (was: Generalize Spark Connect except

[jira] [Updated] (SPARK-50460) Generalize Spark Connect exceptions

[ https://issues.apache.org/jira/browse/SPARK-50460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50460: --- Labels: pull-request-available (was: ) > Generalize Spark Connect exceptions >

[jira] [Created] (SPARK-50475) NullPointerException when saving a df with 'noop' format after joining wide dfs

Cedric Cuypers created SPARK-50475: -- Summary: NullPointerException when saving a df with 'noop' format after joining wide dfs Key: SPARK-50475 URL: https://issues.apache.org/jira/browse/SPARK-50475 P

[jira] [Updated] (SPARK-50474) Upgrade checkstyle to 10.20.2

[ https://issues.apache.org/jira/browse/SPARK-50474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50474: --- Labels: pull-request-available (was: ) > Upgrade checkstyle to 10.20.2 > --

[jira] [Updated] (SPARK-50393) Support DataFrame conversion to table argument in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-50393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-50393: - Summary: Support DataFrame conversion to table argument in Spark Connect (was: Support df.argum

[jira] [Updated] (SPARK-50133) Support DataFrame conversion to table argument in Spark Classic

[ https://issues.apache.org/jira/browse/SPARK-50133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-50133: - Summary: Support DataFrame conversion to table argument in Spark Classic (was: Support df.argum