[jira] [Resolved] (SPARK-41935) Skip snapshot check and transfer progress log during publishing snapshots

[ https://issues.apache.org/jira/browse/SPARK-41935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41935. --- Resolution: Invalid > Skip snapshot check and transfer progress log during publishing

[jira] [Assigned] (SPARK-41935) Skip snapshot check and transfer progress log during publishing snapshots

[ https://issues.apache.org/jira/browse/SPARK-41935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41935: Assignee: (was: Apache Spark) > Skip snapshot check and transfer progress log during

[jira] [Assigned] (SPARK-41935) Skip snapshot check and transfer progress log during publishing snapshots

[ https://issues.apache.org/jira/browse/SPARK-41935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41935: Assignee: Apache Spark > Skip snapshot check and transfer progress log during publishing

[jira] [Commented] (SPARK-41935) Skip snapshot check and transfer progress log during publishing snapshots

[ https://issues.apache.org/jira/browse/SPARK-41935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655659#comment-17655659 ] Apache Spark commented on SPARK-41935: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-41935) Skip snapshot check and transfer progress log in release-build.sh

Dongjoon Hyun created SPARK-41935: - Summary: Skip snapshot check and transfer progress log in release-build.sh Key: SPARK-41935 URL: https://issues.apache.org/jira/browse/SPARK-41935 Project: Spark

[jira] [Updated] (SPARK-41935) Skip snapshot check and transfer progress log during publishing snapshots

[ https://issues.apache.org/jira/browse/SPARK-41935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41935: -- Summary: Skip snapshot check and transfer progress log during publishing snapshots (was:

[jira] [Assigned] (SPARK-41934) Add the unsupported function list for `session`

[ https://issues.apache.org/jira/browse/SPARK-41934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41934: Assignee: (was: Apache Spark) > Add the unsupported function list for `session` >

[jira] [Assigned] (SPARK-41934) Add the unsupported function list for `session`

[ https://issues.apache.org/jira/browse/SPARK-41934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41934: Assignee: Apache Spark > Add the unsupported function list for `session` >

[jira] [Assigned] (SPARK-41934) Add the unsupported function list for `session`

[ https://issues.apache.org/jira/browse/SPARK-41934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41934: Assignee: (was: Apache Spark) > Add the unsupported function list for `session` >

[jira] [Commented] (SPARK-41934) Add the unsupported function list for `session`

[ https://issues.apache.org/jira/browse/SPARK-41934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655658#comment-17655658 ] Apache Spark commented on SPARK-41934: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41933) Provide local mode that automatically starts the server

[ https://issues.apache.org/jira/browse/SPARK-41933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41933: Assignee: (was: Apache Spark) > Provide local mode that automatically starts the

[jira] [Assigned] (SPARK-41933) Provide local mode that automatically starts the server

[ https://issues.apache.org/jira/browse/SPARK-41933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41933: Assignee: Apache Spark > Provide local mode that automatically starts the server >

[jira] [Commented] (SPARK-41933) Provide local mode that automatically starts the server

[ https://issues.apache.org/jira/browse/SPARK-41933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655657#comment-17655657 ] Apache Spark commented on SPARK-41933: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-41934) Add the unsupported function list for `session`

Ruifeng Zheng created SPARK-41934: - Summary: Add the unsupported function list for `session` Key: SPARK-41934 URL: https://issues.apache.org/jira/browse/SPARK-41934 Project: Spark Issue

[jira] [Created] (SPARK-41933) Provide local mode that automatically starts the server

Hyukjin Kwon created SPARK-41933: Summary: Provide local mode that automatically starts the server Key: SPARK-41933 URL: https://issues.apache.org/jira/browse/SPARK-41933 Project: Spark

[jira] [Created] (SPARK-41932) Bootstrapping Spark Connect

Hyukjin Kwon created SPARK-41932: Summary: Bootstrapping Spark Connect Key: SPARK-41932 URL: https://issues.apache.org/jira/browse/SPARK-41932 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-40451) Type annotations for Spark Connect Python client

[ https://issues.apache.org/jira/browse/SPARK-40451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655647#comment-17655647 ] Hyukjin Kwon commented on SPARK-40451: -- I believe this is done. > Type annotations for Spark

[jira] [Resolved] (SPARK-40451) Type annotations for Spark Connect Python client

[ https://issues.apache.org/jira/browse/SPARK-40451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40451. -- Assignee: Hyukjin Kwon Resolution: Done > Type annotations for Spark Connect Python

[jira] [Resolved] (SPARK-41927) Add the unsupported list for `GroupedData`

[ https://issues.apache.org/jira/browse/SPARK-41927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41927. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39437

[jira] [Assigned] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41824: Assignee: jiaan.geng > Implement DataFrame.explain format to be similar to PySpark >

[jira] [Resolved] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41824. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39436

[jira] [Assigned] (SPARK-41927) Add the unsupported list for `GroupedData`

[ https://issues.apache.org/jira/browse/SPARK-41927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41927: Assignee: Ruifeng Zheng > Add the unsupported list for `GroupedData` >

[jira] [Resolved] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41928. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39438

[jira] [Assigned] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41928: Assignee: Ruifeng Zheng > Add the unsupported list for functions >

[jira] [Assigned] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41929: Assignee: Ruifeng Zheng > Add function array_compact > -- > >

[jira] [Assigned] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

[ https://issues.apache.org/jira/browse/SPARK-41930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41930: - Assignee: Dongjoon Hyun > Remove `branch-3.1` from publish_snapshot job >

[jira] [Resolved] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

[ https://issues.apache.org/jira/browse/SPARK-41930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41930. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39440

[jira] [Resolved] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41929. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39439

[jira] [Created] (SPARK-41931) Improve UNSUPPORTED_DATA_TYPE message for complex types

Serge Rielau created SPARK-41931: Summary: Improve UNSUPPORTED_DATA_TYPE message for complex types Key: SPARK-41931 URL: https://issues.apache.org/jira/browse/SPARK-41931 Project: Spark

[jira] [Assigned] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

[ https://issues.apache.org/jira/browse/SPARK-41930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41930: Assignee: Apache Spark > Remove `branch-3.1` from publish_snapshot job >

[jira] [Commented] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

[ https://issues.apache.org/jira/browse/SPARK-41930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655636#comment-17655636 ] Apache Spark commented on SPARK-41930: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

[ https://issues.apache.org/jira/browse/SPARK-41930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41930: Assignee: (was: Apache Spark) > Remove `branch-3.1` from publish_snapshot job >

[jira] [Created] (SPARK-41930) Remove `branch-3.1` from publish_snapshot job

Dongjoon Hyun created SPARK-41930: - Summary: Remove `branch-3.1` from publish_snapshot job Key: SPARK-41930 URL: https://issues.apache.org/jira/browse/SPARK-41930 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41904) Fix Function `nth_value` functions output

[ https://issues.apache.org/jira/browse/SPARK-41904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655629#comment-17655629 ] jiaan.geng commented on SPARK-41904: [~techaddict]Could you tell me how to reproduce this issue? I

[jira] [Commented] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655628#comment-17655628 ] Apache Spark commented on SPARK-41929: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655627#comment-17655627 ] Apache Spark commented on SPARK-41929: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41929: Assignee: (was: Apache Spark) > Add function array_compact >

[jira] [Assigned] (SPARK-41929) Add function array_compact

[ https://issues.apache.org/jira/browse/SPARK-41929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41929: Assignee: Apache Spark > Add function array_compact > -- > >

[jira] [Created] (SPARK-41929) Add function array_compact

Ruifeng Zheng created SPARK-41929: - Summary: Add function array_compact Key: SPARK-41929 URL: https://issues.apache.org/jira/browse/SPARK-41929 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655626#comment-17655626 ] Apache Spark commented on SPARK-41928: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41928: Assignee: (was: Apache Spark) > Add the unsupported list for functions >

[jira] [Assigned] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41928: Assignee: Apache Spark > Add the unsupported list for functions >

[jira] [Commented] (SPARK-41928) Add the unsupported list for functions

[ https://issues.apache.org/jira/browse/SPARK-41928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655625#comment-17655625 ] Apache Spark commented on SPARK-41928: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-41928) Add the unsupported list for functions

Ruifeng Zheng created SPARK-41928: - Summary: Add the unsupported list for functions Key: SPARK-41928 URL: https://issues.apache.org/jira/browse/SPARK-41928 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41927) Add the unsupported list for `GroupedData`

[ https://issues.apache.org/jira/browse/SPARK-41927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41927: Assignee: (was: Apache Spark) > Add the unsupported list for `GroupedData` >

[jira] [Commented] (SPARK-41927) Add the unsupported list for `GroupedData`

[ https://issues.apache.org/jira/browse/SPARK-41927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655624#comment-17655624 ] Apache Spark commented on SPARK-41927: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41927) Add the unsupported list for `GroupedData`

[ https://issues.apache.org/jira/browse/SPARK-41927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41927: Assignee: Apache Spark > Add the unsupported list for `GroupedData` >

[jira] [Assigned] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41824: Assignee: (was: Apache Spark) > Implement DataFrame.explain format to be similar to

[jira] [Commented] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655623#comment-17655623 ] Apache Spark commented on SPARK-41824: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41824: Assignee: Apache Spark > Implement DataFrame.explain format to be similar to PySpark >

[jira] [Created] (SPARK-41927) Add the unsupported list for `GroupedData`

Ruifeng Zheng created SPARK-41927: - Summary: Add the unsupported list for `GroupedData` Key: SPARK-41927 URL: https://issues.apache.org/jira/browse/SPARK-41927 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41926) Add Github action test job with RocksDB as UI backend

[ https://issues.apache.org/jira/browse/SPARK-41926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41926: Assignee: Gengliang Wang (was: Apache Spark) > Add Github action test job with RocksDB

[jira] [Assigned] (SPARK-41926) Add Github action test job with RocksDB as UI backend

[ https://issues.apache.org/jira/browse/SPARK-41926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41926: Assignee: Apache Spark (was: Gengliang Wang) > Add Github action test job with RocksDB

[jira] [Commented] (SPARK-41926) Add Github action test job with RocksDB as UI backend

[ https://issues.apache.org/jira/browse/SPARK-41926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655622#comment-17655622 ] Apache Spark commented on SPARK-41926: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-41926) Add Github action test job with RocksDB as UI backend

Gengliang Wang created SPARK-41926: -- Summary: Add Github action test job with RocksDB as UI backend Key: SPARK-41926 URL: https://issues.apache.org/jira/browse/SPARK-41926 Project: Spark

[jira] [Resolved] (SPARK-41875) Throw proper errors in Dataset.to()

[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41875. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39422

[jira] [Assigned] (SPARK-41875) Throw proper errors in Dataset.to()

[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41875: - Assignee: jiaan.geng > Throw proper errors in Dataset.to() >

[jira] [Resolved] (SPARK-41898) Window.rowsBetween should handle `float("-inf")` and `float("+inf")` as argument

[ https://issues.apache.org/jira/browse/SPARK-41898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41898. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39433

[jira] [Assigned] (SPARK-41898) Window.rowsBetween should handle `float("-inf")` and `float("+inf")` as argument

[ https://issues.apache.org/jira/browse/SPARK-41898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41898: - Assignee: Sandeep Singh > Window.rowsBetween should handle `float("-inf")` and

[jira] [Assigned] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

[ https://issues.apache.org/jira/browse/SPARK-41924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41924: - Assignee: Ruifeng Zheng > Make StructType support metadata and Implement

[jira] [Resolved] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

[ https://issues.apache.org/jira/browse/SPARK-41924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41924. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39432

[jira] [Commented] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655604#comment-17655604 ] Dongjoon Hyun commented on SPARK-35743: --- Thank you for updating! > Improve Parquet vectorized

[jira] [Updated] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35743: -- Labels: parquet releasenotes (was: parquet) > Improve Parquet vectorized reader >

[jira] [Updated] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

[ https://issues.apache.org/jira/browse/SPARK-36529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36529: - Parent: (was: SPARK-35743) Issue Type: Bug (was: Sub-task) > Decouple CPU with IO work in

[jira] [Updated] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

[ https://issues.apache.org/jira/browse/SPARK-36529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36529: - Issue Type: Improvement (was: Bug) > Decouple CPU with IO work in vectorized Parquet reader >

[jira] [Resolved] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41925. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39434

[jira] [Updated] (SPARK-36528) Implement lazy decoding for the vectorized Parquet reader

[ https://issues.apache.org/jira/browse/SPARK-36528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36528: - Parent: (was: SPARK-35743) Issue Type: Bug (was: Sub-task) > Implement lazy decoding for

[jira] [Updated] (SPARK-36528) Implement lazy decoding for the vectorized Parquet reader

[ https://issues.apache.org/jira/browse/SPARK-36528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36528: - Issue Type: New Feature (was: Bug) > Implement lazy decoding for the vectorized Parquet reader >

[jira] [Assigned] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41925: - Assignee: Dongjoon Hyun > Enable spark.sql.orc.enableNestedColumnVectorizedReader by

[jira] [Resolved] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-35743. -- Fix Version/s: 3.4.0 Resolution: Fixed > Improve Parquet vectorized reader >

[jira] [Updated] (SPARK-36527) Implement lazy materialization for the vectorized Parquet reader

[ https://issues.apache.org/jira/browse/SPARK-36527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36527: - Parent: (was: SPARK-35743) Issue Type: Improvement (was: Sub-task) > Implement lazy

[jira] [Commented] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655560#comment-17655560 ] Dongjoon Hyun commented on SPARK-35743: --- Hi, [~csun] . Please update the JIRA's target version and

[jira] [Updated] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35743: -- Target Version/s: (was: 3.3.0) > Improve Parquet vectorized reader >

[jira] [Resolved] (SPARK-41895) Add tests for streaming UI with RocksDB backend

[ https://issues.apache.org/jira/browse/SPARK-41895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41895. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39415

[jira] [Commented] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1767#comment-1767 ] Apache Spark commented on SPARK-41925: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1766#comment-1766 ] Apache Spark commented on SPARK-41925: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41925: Assignee: (was: Apache Spark) > Enable

[jira] [Assigned] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

[ https://issues.apache.org/jira/browse/SPARK-41925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41925: Assignee: Apache Spark > Enable spark.sql.orc.enableNestedColumnVectorizedReader by

[jira] [Created] (SPARK-41925) Enable spark.sql.orc.enableNestedColumnVectorizedReader by default

Dongjoon Hyun created SPARK-41925: - Summary: Enable spark.sql.orc.enableNestedColumnVectorizedReader by default Key: SPARK-41925 URL: https://issues.apache.org/jira/browse/SPARK-41925 Project: Spark

[jira] [Commented] (SPARK-41918) Refine the naming in proto messages

[ https://issues.apache.org/jira/browse/SPARK-41918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1764#comment-1764 ] Rui Wang commented on SPARK-41918: -- I did some tests locally and find something as the below: If I

[jira] [Updated] (SPARK-41820) DataFrame.createOrReplaceGlobalTempView - SparkConnectException: requirement failed

[ https://issues.apache.org/jira/browse/SPARK-41820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41820: -- Description: {code:java} >>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")],

[jira] [Comment Edited] (SPARK-41918) Refine the naming in proto messages

[ https://issues.apache.org/jira/browse/SPARK-41918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655545#comment-17655545 ] Rui Wang edited comment on SPARK-41918 at 1/6/23 6:35 PM: -- [~grundprinzip-db]

[jira] [Commented] (SPARK-41918) Refine the naming in proto messages

[ https://issues.apache.org/jira/browse/SPARK-41918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655545#comment-17655545 ] Rui Wang commented on SPARK-41918: -- [~grundprinzip-db] I am a bit confused on the renaming and what

[jira] [Assigned] (SPARK-41898) Window.rowsBetween should handle `float("-inf")` and `float("+inf")` as argument

[ https://issues.apache.org/jira/browse/SPARK-41898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41898: Assignee: (was: Apache Spark) > Window.rowsBetween should handle `float("-inf")` and

[jira] [Assigned] (SPARK-41898) Window.rowsBetween should handle `float("-inf")` and `float("+inf")` as argument

[ https://issues.apache.org/jira/browse/SPARK-41898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41898: Assignee: Apache Spark > Window.rowsBetween should handle `float("-inf")` and

[jira] [Commented] (SPARK-41898) Window.rowsBetween should handle `float("-inf")` and `float("+inf")` as argument

[ https://issues.apache.org/jira/browse/SPARK-41898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655539#comment-17655539 ] Apache Spark commented on SPARK-41898: -- User 'techaddict' has created a pull request for this

[jira] [Commented] (SPARK-41911) Add version fields to Connect proto

[ https://issues.apache.org/jira/browse/SPARK-41911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655537#comment-17655537 ] Rui Wang commented on SPARK-41911: -- We will know better where we need the versions during the process

[jira] [Commented] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655532#comment-17655532 ] L. C. Hsieh commented on SPARK-41049: - For a correctness bug, I think we should backport it, though

[jira] [Commented] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655516#comment-17655516 ] Erik Krogen commented on SPARK-41049: - Thanks! [~cloud_fan]  [~viirya]  shall we backport this to

[jira] [Resolved] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-41890. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39399

[jira] [Assigned] (SPARK-41800) Upgrade commons-compress to 1.22

[ https://issues.apache.org/jira/browse/SPARK-41800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-41800: Assignee: BingKun Pan > Upgrade commons-compress to 1.22 >

[jira] [Resolved] (SPARK-41800) Upgrade commons-compress to 1.22

[ https://issues.apache.org/jira/browse/SPARK-41800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-41800. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39326

[jira] [Assigned] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-41890: Assignee: Yang Jie > Reduce `toSeq` in >

[jira] [Commented] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655442#comment-17655442 ] Hyukjin Kwon commented on SPARK-41824: -- It's actually implementation details in PySpark. It would

[jira] [Assigned] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

[ https://issues.apache.org/jira/browse/SPARK-41924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41924: Assignee: Apache Spark > Make StructType support metadata and Implement

[jira] [Assigned] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

[ https://issues.apache.org/jira/browse/SPARK-41924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41924: Assignee: (was: Apache Spark) > Make StructType support metadata and Implement

[jira] [Commented] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

[ https://issues.apache.org/jira/browse/SPARK-41924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655436#comment-17655436 ] Apache Spark commented on SPARK-41924: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-41924) Make StructType support metadata and Implement `DataFrame.withMetadata`

Ruifeng Zheng created SPARK-41924: - Summary: Make StructType support metadata and Implement `DataFrame.withMetadata` Key: SPARK-41924 URL: https://issues.apache.org/jira/browse/SPARK-41924 Project:

[jira] [Comment Edited] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655434#comment-17655434 ] jiaan.geng edited comment on SPARK-41824 at 1/6/23 12:56 PM: - I found scala

[jira] [Comment Edited] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655434#comment-17655434 ] jiaan.geng edited comment on SPARK-41824 at 1/6/23 12:56 PM: - I found scala

  1   2   >