[jira] [Resolved] (SPARK-46427) Change Python Data Source's description to be pretty in explain

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46427. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44379 [https://gi

[jira] [Assigned] (SPARK-46427) Change Python Data Source's description to be pretty in explain

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46427: Assignee: Hyukjin Kwon > Change Python Data Source's description to be pretty in explain

[jira] [Assigned] (SPARK-46403) Decode parquet binary with getBytesUnsafe method

2023-12-15 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-46403: -- Assignee: Wan Kun > Decode parquet binary with getBytesUnsafe method > --

[jira] [Resolved] (SPARK-46403) Decode parquet binary with getBytesUnsafe method

2023-12-15 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46403. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44351 [https://github

[jira] [Updated] (SPARK-46427) Change Python Data Source's description to be pretty in explain

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46427: --- Labels: pull-request-available (was: ) > Change Python Data Source's description to be pret

[jira] [Created] (SPARK-46427) Change Python Data Source's description to be pretty in explain

2023-12-15 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46427: Summary: Change Python Data Source's description to be pretty in explain Key: SPARK-46427 URL: https://issues.apache.org/jira/browse/SPARK-46427 Project: Spark

[jira] [Resolved] (SPARK-46425) Pin the bundler version in CI

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46425. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44376 [https://gi

[jira] [Assigned] (SPARK-46425) Pin the bundler version in CI

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46425: Assignee: Hyukjin Kwon > Pin the bundler version in CI > - >

[jira] [Resolved] (SPARK-46414) Use prependBaseUri to render javascript imports

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46414. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44363 [https://gi

[jira] [Resolved] (SPARK-46422) Move `test_window` to `pyspark.pandas.tests.window.*`

2023-12-15 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46422. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44371 [https://

[jira] [Updated] (SPARK-42829) Add Identifier to the cached RDD node on the Stages page

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42829: --- Labels: pull-request-available (was: ) > Add Identifier to the cached RDD node on the Stage

[jira] [Deleted] (SPARK-46426) Uses sum metrics for number of output length

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46426: - > Uses sum metrics for number of output length > > >

[jira] [Updated] (SPARK-46426) Uses sum metrics for number of output length

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46426: - Attachment: Screenshot 2023-12-15 at 3.23.09 PM.png > Uses sum metrics for number of output leng

[jira] [Updated] (SPARK-46426) Uses sum metrics for number of output length

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46426: - Description: screenhot attached. it shouldn't look like bytes > Uses sum metrics for number of o

[jira] [Created] (SPARK-46426) Uses sum metrics for number of output length

2023-12-15 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46426: Summary: Uses sum metrics for number of output length Key: SPARK-46426 URL: https://issues.apache.org/jira/browse/SPARK-46426 Project: Spark Issue Type: Impr

[jira] [Assigned] (SPARK-46423) Refactor Python Data Source instance loading

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46423: Assignee: Hyukjin Kwon > Refactor Python Data Source instance loading > -

[jira] [Resolved] (SPARK-46423) Refactor Python Data Source instance loading

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46423. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44374 [https://gi

[jira] [Updated] (SPARK-46425) Pin the bundler version in CI

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46425: --- Labels: pull-request-available (was: ) > Pin the bundler version in CI > --

[jira] [Created] (SPARK-46425) Pin the bundler version in CI

2023-12-15 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46425: Summary: Pin the bundler version in CI Key: SPARK-46425 URL: https://issues.apache.org/jira/browse/SPARK-46425 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-46424) Support PythonSQLMetrics.pythonMetrics

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46424: --- Labels: pull-request-available (was: ) > Support PythonSQLMetrics.pythonMetrics > -

[jira] [Resolved] (SPARK-45807) DataSourceV2: Improve ViewCatalog API

2023-12-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45807. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44330 [https://gith

[jira] [Updated] (SPARK-46424) Support PythonSQLMetrics.pythonMetrics

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46424: - Summary: Support PythonSQLMetrics.pythonMetrics (was: Support PythonSQLMetrics.pythonMetrics vi

[jira] [Updated] (SPARK-46424) Support PythonSQLMetrics.pythonMetrics via custom metrics API in DSv2

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46424: - Priority: Minor (was: Major) > Support PythonSQLMetrics.pythonMetrics via custom metrics API in

[jira] [Created] (SPARK-46424) Support PythonSQLMetrics.pythonMetrics via custom metrics API in DSv2

2023-12-15 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46424: Summary: Support PythonSQLMetrics.pythonMetrics via custom metrics API in DSv2 Key: SPARK-46424 URL: https://issues.apache.org/jira/browse/SPARK-46424 Project: Spark

[jira] [Updated] (SPARK-46423) Refactor Python Data Source instance loading

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46423: --- Labels: pull-request-available (was: ) > Refactor Python Data Source instance loading > ---

[jira] [Assigned] (SPARK-46419) Reorganize `DatetimeIndexTests`: Factor out 3 slow tests

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46419: Assignee: Ruifeng Zheng > Reorganize `DatetimeIndexTests`: Factor out 3 slow tests >

[jira] [Resolved] (SPARK-46419) Reorganize `DatetimeIndexTests`: Factor out 3 slow tests

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46419. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44369 [https://gi

[jira] [Created] (SPARK-46423) Refactor Python Data Source instance loading

2023-12-15 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46423: Summary: Refactor Python Data Source instance loading Key: SPARK-46423 URL: https://issues.apache.org/jira/browse/SPARK-46423 Project: Spark Issue Type: Sub-

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45597: Assignee: Hyukjin Kwon (was: Allison Wang) > Support creating table using a Python data

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45597: Assignee: Allison Wang > Support creating table using a Python data source in SQL > -

[jira] [Resolved] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45597. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44305 [https://gi

[jira] [Commented] (SPARK-32710) Add Hive Murmur3Hash expression

2023-12-15 Thread Eric Xiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17797261#comment-17797261 ] Eric Xiao commented on SPARK-32710: --- Hi [~chengsu], I am interested in working on enab

[jira] [Resolved] (SPARK-46409) Spark Connect Repl does not work with ClosureCleaner

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46409. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44360 [https://gi

[jira] [Assigned] (SPARK-46409) Spark Connect Repl does not work with ClosureCleaner

2023-12-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46409: Assignee: Vsevolod Stepanov > Spark Connect Repl does not work with ClosureCleaner >

[jira] [Resolved] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2023-12-15 Thread Andrew Otto (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Otto resolved SPARK-23890. - Fix Version/s: 3.0.0 Resolution: Fixed Ah! This is supported in DataSource v2 after all,

[jira] [Updated] (SPARK-46422) Move `test_window` to `pyspark.pandas.tests.window.*`

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46422: --- Labels: pull-request-available (was: ) > Move `test_window` to `pyspark.pandas.tests.window

[jira] [Created] (SPARK-46422) Move `test_window` to `pyspark.pandas.tests.window.*`

2023-12-15 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46422: - Summary: Move `test_window` to `pyspark.pandas.tests.window.*` Key: SPARK-46422 URL: https://issues.apache.org/jira/browse/SPARK-46422 Project: Spark Issue

[jira] [Updated] (SPARK-46421) Broken support for explode on a Map in typed API

2023-12-15 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-46421: --- Description:   {code:java} scala> spark.createDataset(Seq(Tuple1(Map(1 -> 2.select(expl

[jira] [Created] (SPARK-46421) Broken support for explode on a Map in typed API

2023-12-15 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-46421: -- Summary: Broken support for explode on a Map in typed API Key: SPARK-46421 URL: https://issues.apache.org/jira/browse/SPARK-46421 Project: Spark Issue Ty

[jira] [Updated] (SPARK-46420) Remove unused transport from SparkSQLCLIDriver

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46420: --- Labels: pull-request-available (was: ) > Remove unused transport from SparkSQLCLIDriver > -

[jira] [Created] (SPARK-46420) Remove unused transport from SparkSQLCLIDriver

2023-12-15 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-46420: - Summary: Remove unused transport from SparkSQLCLIDriver Key: SPARK-46420 URL: https://issues.apache.org/jira/browse/SPARK-46420 Project: Spark Issue Type: Improvem

[jira] [Updated] (SPARK-46419) Reorganize `DatetimeIndexTests`: Factor out 3 slow tests

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46419: --- Labels: pull-request-available (was: ) > Reorganize `DatetimeIndexTests`: Factor out 3 slow

[jira] [Created] (SPARK-46419) Reorganize `DatetimeIndexTests`: Factor out 3 slow tests

2023-12-15 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46419: - Summary: Reorganize `DatetimeIndexTests`: Factor out 3 slow tests Key: SPARK-46419 URL: https://issues.apache.org/jira/browse/SPARK-46419 Project: Spark Is

[jira] [Updated] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-40876: --- Labels: pull-request-available (was: ) > Spark's Vectorized ParquetReader should support ty

[jira] [Resolved] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46417. -- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-46417: Assignee: Wenchen Fan > do not fail when calling getTable and throwException is false > -

[jira] [Resolved] (SPARK-46418) Reorganize `ReshapeTests`

2023-12-15 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46418. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44365 [https://

[jira] [Assigned] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46417: -- Assignee: (was: Apache Spark) > do not fail when calling getTable and throwExcept

[jira] [Assigned] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46417: -- Assignee: Apache Spark > do not fail when calling getTable and throwException is fals

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45597: -- Assignee: (was: Apache Spark) > Support creating table using a Python data source

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45597: -- Assignee: Apache Spark > Support creating table using a Python data source in SQL > -

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45597: -- Assignee: Apache Spark > Support creating table using a Python data source in SQL > -

[jira] [Assigned] (SPARK-45597) Support creating table using a Python data source in SQL

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45597: -- Assignee: (was: Apache Spark) > Support creating table using a Python data source

[jira] [Assigned] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46417: -- Assignee: (was: Apache Spark) > do not fail when calling getTable and throwExcept

[jira] [Assigned] (SPARK-46417) do not fail when calling getTable and throwException is false

2023-12-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46417: -- Assignee: Apache Spark > do not fail when calling getTable and throwException is fals