[jira] [Assigned] (SPARK-46651) Split `FrameTakeTests`

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46651: Assignee: Ruifeng Zheng > Split `FrameTakeTests` > -- > >

[jira] [Resolved] (SPARK-46651) Split `FrameTakeTests`

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46651. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44656

[jira] [Updated] (SPARK-46655) Skip query context catching in DataFrame methods

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46655: --- Labels: pull-request-available (was: ) > Skip query context catching in DataFrame methods

[jira] [Created] (SPARK-46655) Skip query context catching in DataFrame methods

2024-01-09 Thread Max Gekk (Jira)
Max Gekk created SPARK-46655: Summary: Skip query context catching in DataFrame methods Key: SPARK-46655 URL: https://issues.apache.org/jira/browse/SPARK-46655 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46590) Coalesce partiton assert error after skew join optimization

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46590: --- Labels: pull-request-available (was: ) > Coalesce partiton assert error after skew join

[jira] [Created] (SPARK-46654) df.show() of pyspark displayed different results between Regular Spark and Spark Connect

2024-01-09 Thread Yang Jie (Jira)
Yang Jie created SPARK-46654: Summary: df.show() of pyspark displayed different results between Regular Spark and Spark Connect Key: SPARK-46654 URL: https://issues.apache.org/jira/browse/SPARK-46654

[jira] [Resolved] (SPARK-46648) Use `zstd` as the default ORC compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46648. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44654

[jira] [Updated] (SPARK-46653) Code-gen for full outer sort merge join output line by line

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46653: --- Labels: pull-request-available (was: ) > Code-gen for full outer sort merge join output

[jira] [Created] (SPARK-46653) Code-gen for full outer sort merge join output line by line

2024-01-09 Thread Mingliang Zhu (Jira)
Mingliang Zhu created SPARK-46653: - Summary: Code-gen for full outer sort merge join output line by line Key: SPARK-46653 URL: https://issues.apache.org/jira/browse/SPARK-46653 Project: Spark

[jira] [Updated] (SPARK-46642) Add `getMessageTemplate` to PySpark error framework

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46642: --- Labels: pull-request-available (was: ) > Add `getMessageTemplate` to PySpark error

[jira] [Updated] (SPARK-46652) Remove `Snappy` from `TPCDSQueryBenchmark` benchmark case name

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46652: --- Labels: pull-request-available (was: ) > Remove `Snappy` from `TPCDSQueryBenchmark`

[jira] [Updated] (SPARK-46652) Remove `Snappy` from `TPCDSQueryBenchmark` benchmark case name

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46652: -- Summary: Remove `Snappy` from `TPCDSQueryBenchmark` benchmark case name (was: Remove

[jira] [Created] (SPARK-46652) Remove `Snappy` from the benchmark case name

2024-01-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46652: - Summary: Remove `Snappy` from the benchmark case name Key: SPARK-46652 URL: https://issues.apache.org/jira/browse/SPARK-46652 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46651) Split `FrameTakeTests`

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46651: --- Labels: pull-request-available (was: ) > Split `FrameTakeTests` > -- >

[jira] [Created] (SPARK-46651) Split `FrameTakeTests`

2024-01-09 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46651: - Summary: Split `FrameTakeTests` Key: SPARK-46651 URL: https://issues.apache.org/jira/browse/SPARK-46651 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46650) Replace AtomicBoolean with volatile boolean

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46650: --- Labels: pull-request-available (was: ) > Replace AtomicBoolean with volatile boolean >

[jira] [Created] (SPARK-46650) Replace AtomicBoolean with volatile boolean

2024-01-09 Thread Yang Jie (Jira)
Yang Jie created SPARK-46650: Summary: Replace AtomicBoolean with volatile boolean Key: SPARK-46650 URL: https://issues.apache.org/jira/browse/SPARK-46650 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-46649) Run PyPy 3 and Python 3.10 tests independently

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46649: Assignee: Hyukjin Kwon > Run PyPy 3 and Python 3.10 tests independently >

[jira] [Assigned] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46645: Assignee: Hyukjin Kwon > Exclude unittest-xml-reporting in Python 3.12 image >

[jira] [Resolved] (SPARK-46649) Run PyPy 3 and Python 3.10 tests independently

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46649. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44655

[jira] [Resolved] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46645. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44652

[jira] [Updated] (SPARK-46649) Run PyPy 3 and Python 3.10 tests independently

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46649: --- Labels: pull-request-available (was: ) > Run PyPy 3 and Python 3.10 tests independently >

[jira] [Updated] (SPARK-46649) Run PyPy 3 and Python 3.10 tests independently

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46649: - Description: https://github.com/apache/spark/actions/runs/7462843546/job/20306241275 Seems

[jira] [Created] (SPARK-46649) Run PyPy 3 and Python 3.10 tests independently

2024-01-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46649: Summary: Run PyPy 3 and Python 3.10 tests independently Key: SPARK-46649 URL: https://issues.apache.org/jira/browse/SPARK-46649 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-46536) Support GROUP BY calendar_interval_type

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46536. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44538

[jira] [Updated] (SPARK-46643) Fix ORC tests to be independent from default compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46643: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Test) > Fix ORC tests to be

[jira] [Updated] (SPARK-46648) Use `zstd` as the default ORC compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46648: -- Summary: Use `zstd` as the default ORC compression (was: Use `zstd` as the default value of

[jira] [Updated] (SPARK-46648) Use `zstd` as the default value of `spark.sql.orc.compression.codec`

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46648: --- Labels: pull-request-available (was: ) > Use `zstd` as the default value of

[jira] [Resolved] (SPARK-46646) Improve `TPCDSQueryBenchmark` to support other file formats

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46646. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44651

[jira] [Updated] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46645: --- Labels: pull-request-available (was: ) > Exclude unittest-xml-reporting in Python 3.12

[jira] [Updated] (SPARK-46646) Improve `TPCDSQueryBenchmark` to support other file formats

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46646: --- Labels: pull-request-available (was: ) > Improve `TPCDSQueryBenchmark` to support other

[jira] [Created] (SPARK-46647) Add unittest-xml-reporting into Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46647: Summary: Add unittest-xml-reporting into Python 3.12 image Key: SPARK-46647 URL: https://issues.apache.org/jira/browse/SPARK-46647 Project: Spark Issue

[jira] [Updated] (SPARK-46646) Improve `TPCDSQueryBenchmark` to support other file formats

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46646: -- Summary: Improve `TPCDSQueryBenchmark` to support other file formats (was: Improve

[jira] [Created] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46645: Summary: Exclude unittest-xml-reporting in Python 3.12 image Key: SPARK-46645 URL: https://issues.apache.org/jira/browse/SPARK-46645 Project: Spark Issue

[jira] [Updated] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46645: - Issue Type: Test (was: Improvement) > Exclude unittest-xml-reporting in Python 3.12 image >

[jira] [Updated] (SPARK-46645) Exclude unittest-xml-reporting in Python 3.12 image

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46645: - Issue Type: Improvement (was: Bug) > Exclude unittest-xml-reporting in Python 3.12 image >

[jira] [Resolved] (SPARK-46643) Fix ORC tests to be independent from default compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46643. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44648

[jira] [Assigned] (SPARK-46643) Fix ORC tests to be independent from default compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46643: - Assignee: Dongjoon Hyun > Fix ORC tests to be independent from default compression >

[jira] [Updated] (SPARK-46644) Fix add in SQLMetric

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46644: --- Labels: pull-request-available (was: ) > Fix add in SQLMetric > > >

[jira] [Created] (SPARK-46644) Fix add in SQLMetric

2024-01-09 Thread Davin Tjong (Jira)
Davin Tjong created SPARK-46644: --- Summary: Fix add in SQLMetric Key: SPARK-46644 URL: https://issues.apache.org/jira/browse/SPARK-46644 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-46643) Fix ORC tests to be independent from default compression

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46643: --- Labels: pull-request-available (was: ) > Fix ORC tests to be independent from default

[jira] [Updated] (SPARK-46643) Fix ORC tests to be independent from default compression

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46643: -- Summary: Fix ORC tests to be independent from default compression (was: Fix OrcQuerySuite to

[jira] [Created] (SPARK-46643) Fix OrcQuerySuite to be independent from default compression

2024-01-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46643: - Summary: Fix OrcQuerySuite to be independent from default compression Key: SPARK-46643 URL: https://issues.apache.org/jira/browse/SPARK-46643 Project: Spark

[jira] [Created] (SPARK-46642) Add `getMessageTemplate` to PySpark error framework

2024-01-09 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46642: --- Summary: Add `getMessageTemplate` to PySpark error framework Key: SPARK-46642 URL: https://issues.apache.org/jira/browse/SPARK-46642 Project: Spark Issue

[jira] [Assigned] (SPARK-37039) np.nan series.astype(bool) should be True

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37039: Assignee: Haejoon Lee > np.nan series.astype(bool) should be True >

[jira] [Resolved] (SPARK-37039) np.nan series.astype(bool) should be True

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37039. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44570

[jira] [Updated] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46641: --- Labels: pull-request-available (was: ) > Add maxBytesPerTrigger threshold option >

[jira] [Created] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-09 Thread Maksim Konstantinov (Jira)
Maksim Konstantinov created SPARK-46641: --- Summary: Add maxBytesPerTrigger threshold option Key: SPARK-46641 URL: https://issues.apache.org/jira/browse/SPARK-46641 Project: Spark Issue

[jira] [Resolved] (SPARK-46633) Reading a non-empty Avro file with empty blocks returns 0 records

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46633. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44635

[jira] [Assigned] (SPARK-46633) Reading a non-empty Avro file with empty blocks returns 0 records

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46633: Assignee: Ivan Sadikov > Reading a non-empty Avro file with empty blocks returns 0

[jira] [Assigned] (SPARK-46437) Enable conditional includes in Jekyll documentation

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46437: Assignee: Nicholas Chammas > Enable conditional includes in Jekyll documentation >

[jira] [Resolved] (SPARK-46437) Enable conditional includes in Jekyll documentation

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46437. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44630

[jira] [Resolved] (SPARK-46593) Refactor `data_type_ops` tests

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46593. -- Resolution: Fixed Issue resolved by pull request 44637

[jira] [Resolved] (SPARK-46637) Enhancing the Visual Appeal of Spark doc website

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46637. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44642

[jira] [Resolved] (SPARK-46630) XML: Validate XML element name on write

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46630. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44634

[jira] [Assigned] (SPARK-46630) XML: Validate XML element name on write

2024-01-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46630: Assignee: Sandip Agarwala > XML: Validate XML element name on write >

[jira] [Updated] (SPARK-46639) Add WindowExec SQLMetrics

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46639: --- Labels: pull-request-available (was: ) > Add WindowExec SQLMetrics >

[jira] [Updated] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46640: --- Labels: pull-request-available (was: ) > RemoveRedundantAliases does not account for

[jira] [Updated] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-09 Thread Nikhil Sheoran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Sheoran updated SPARK-46640: --- Summary: RemoveRedundantAliases does not account for SubqueryExpression when removing

[jira] [Created] (SPARK-46640) RemoveRedundantAliases does not account for references in SubqueryExpression when removing aliases

2024-01-09 Thread Nikhil Sheoran (Jira)
Nikhil Sheoran created SPARK-46640: -- Summary: RemoveRedundantAliases does not account for references in SubqueryExpression when removing aliases Key: SPARK-46640 URL:

[jira] [Created] (SPARK-46639) Add WindowExec SQLMetrics

2024-01-09 Thread Eren Avsarogullari (Jira)
Eren Avsarogullari created SPARK-46639: -- Summary: Add WindowExec SQLMetrics Key: SPARK-46639 URL: https://issues.apache.org/jira/browse/SPARK-46639 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-45959) SPIP: Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2024-01-09 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45959: - Description: Though documentation clearly recommends to add all columns in a single shot, but in reality is

[jira] [Updated] (SPARK-45959) SPIP: Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2024-01-09 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45959: - Summary: SPIP: Abusing DataSet.withColumn can cause huge tree with severe perf degradation (was: Abusing

[jira] [Updated] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-01-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46638: --- Summary: Create API to acquire execution memory for 'eval' and 'terminate' methods (was: Create API to

[jira] [Created] (SPARK-46638) Create API to acquire execution memory for eval and terminate methods

2024-01-09 Thread Daniel (Jira)
Daniel created SPARK-46638: -- Summary: Create API to acquire execution memory for eval and terminate methods Key: SPARK-46638 URL: https://issues.apache.org/jira/browse/SPARK-46638 Project: Spark

[jira] [Updated] (SPARK-46637) Enhancing the Visual Appeal of Spark doc website

2024-01-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-46637: - Priority: Minor (was: Major) > Enhancing the Visual Appeal of Spark doc website >

[jira] [Updated] (SPARK-46637) Enhancing the Visual Appeal of Spark doc website

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46637: --- Labels: pull-request-available (was: ) > Enhancing the Visual Appeal of Spark doc website

[jira] [Updated] (SPARK-46637) Enhancing the Visual Appeal of Spark doc website

2024-01-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-46637: --- Affects Version/s: 3.5.1 (was: 3.4.3) > Enhancing the Visual

[jira] [Created] (SPARK-46637) Enhancing the Visual Appeal of Spark doc website

2024-01-09 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-46637: -- Summary: Enhancing the Visual Appeal of Spark doc website Key: SPARK-46637 URL: https://issues.apache.org/jira/browse/SPARK-46637 Project: Spark Issue

[jira] [Resolved] (SPARK-46634) literal validation should not drill down to null fields

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46634. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44640

[jira] [Resolved] (SPARK-46622) Should override `toString` method for `shuffledb.StoreVersion`

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46622. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44624

[jira] [Assigned] (SPARK-46622) Should override `toString` method for `shuffledb.StoreVersion`

2024-01-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46622: - Assignee: Yang Jie > Should override `toString` method for `shuffledb.StoreVersion` >

[jira] [Updated] (SPARK-46636) Pyspark throwing TypeError while collecting a RDD

2024-01-09 Thread Shubham Patil (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Patil updated SPARK-46636: -- Affects Version/s: 3.4.1 (was: 3.3.4) > Pyspark throwing TypeError

[jira] [Created] (SPARK-46636) Pyspark throwing TypeError while collecting a RDD

2024-01-09 Thread Shubham Patil (Jira)
Shubham Patil created SPARK-46636: - Summary: Pyspark throwing TypeError while collecting a RDD Key: SPARK-46636 URL: https://issues.apache.org/jira/browse/SPARK-46636 Project: Spark Issue

[jira] [Updated] (SPARK-46634) literal validation should not drill down to null fields

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46634: --- Labels: pull-request-available (was: ) > literal validation should not drill down to null

[jira] [Updated] (SPARK-46635) Refine docstring of `from_csv/schema_of_csv/to_csv`

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46635: --- Labels: pull-request-available (was: ) > Refine docstring of

[jira] [Created] (SPARK-46635) Refine docstring of `from_csv/schema_of_csv/to_csv`

2024-01-09 Thread Yang Jie (Jira)
Yang Jie created SPARK-46635: Summary: Refine docstring of `from_csv/schema_of_csv/to_csv` Key: SPARK-46635 URL: https://issues.apache.org/jira/browse/SPARK-46635 Project: Spark Issue Type:

[jira] [Created] (SPARK-46634) literal validation should not drill down to null fields

2024-01-09 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46634: --- Summary: literal validation should not drill down to null fields Key: SPARK-46634 URL: https://issues.apache.org/jira/browse/SPARK-46634 Project: Spark Issue

[jira] [Assigned] (SPARK-46331) Removing CodeGenFallback trait from subset of datetime and spark version functions

2024-01-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46331: --- Assignee: Aleksandar Tomic > Removing CodeGenFallback trait from subset of datetime and

[jira] [Resolved] (SPARK-46331) Removing CodeGenFallback trait from subset of datetime and spark version functions

2024-01-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46331. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44261

[jira] [Commented] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2024-01-09 Thread Steven Ottens (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804683#comment-17804683 ] Steven Ottens commented on SPARK-46032: --- I've reported a similar issue and gave a way to test it

[jira] [Updated] (SPARK-46627) Streaming UI hover-over shows incorrect value

2024-01-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-46627: --- Issue Type: Bug (was: Task) > Streaming UI hover-over shows incorrect value >

[jira] [Resolved] (SPARK-46627) Streaming UI hover-over shows incorrect value

2024-01-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-46627. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved in

[jira] [Assigned] (SPARK-46627) Streaming UI hover-over shows incorrect value

2024-01-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-46627: -- Assignee: Kent Yao > Streaming UI hover-over shows incorrect value >

[jira] [Assigned] (SPARK-46547) Fix deadlock issue between maintenance thread and streaming agg physical operators

2024-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46547: -- Assignee: (was: Apache Spark) > Fix deadlock issue between maintenance thread