[jira] [Updated] (SPARK-46153) XML: Add TimestampNTZType support

[ https://issues.apache.org/jira/browse/SPARK-46153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46153: --- Labels: pull-request-available (was: ) > XML: Add TimestampNTZType support >

[jira] [Updated] (SPARK-46153) XML: Add TimestampNTZType support

[ https://issues.apache.org/jira/browse/SPARK-46153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandip Agarwala updated SPARK-46153: Summary: XML: Add TimestampNTZType support (was: XML: Add TimestampNTZType support in

[jira] [Resolved] (SPARK-46379) Reorganize `FrameInterpolateTests`

[ https://issues.apache.org/jira/browse/SPARK-46379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46379. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44313

[jira] [Assigned] (SPARK-46379) Reorganize `FrameInterpolateTests`

[ https://issues.apache.org/jira/browse/SPARK-46379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46379: Assignee: Ruifeng Zheng > Reorganize `FrameInterpolateTests` >

[jira] [Updated] (SPARK-46389) Should manually close the RocksDB/LevelDB when checkVersion fails

[ https://issues.apache.org/jira/browse/SPARK-46389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46389: --- Labels: pull-request-available (was: ) > Should manually close the RocksDB/LevelDB when

[jira] [Created] (SPARK-46389) Should manually close the RocksDB/LevelDB when checkVersion fails

Yang Jie created SPARK-46389: Summary: Should manually close the RocksDB/LevelDB when checkVersion fails Key: SPARK-46389 URL: https://issues.apache.org/jira/browse/SPARK-46389 Project: Spark

[jira] [Resolved] (SPARK-46132) [CORE] Support key password for JKS keys for RPC SSL

[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46132. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-46132) [CORE] Support key password for JKS keys for RPC SSL

[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46132: --- Assignee: Hasnain Lakhani > [CORE] Support key password for JKS keys for

[jira] [Updated] (SPARK-46388) HiveAnalysis misses pattern guard `query.resolved`

[ https://issues.apache.org/jira/browse/SPARK-46388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46388: --- Labels: pull-request-available (was: ) > HiveAnalysis misses pattern guard

[jira] [Created] (SPARK-46388) HiveAnalysis misses pattern guard `query.resolved`

Kent Yao created SPARK-46388: Summary: HiveAnalysis misses pattern guard `query.resolved` Key: SPARK-46388 URL: https://issues.apache.org/jira/browse/SPARK-46388 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-46386) Improve and test assertions of observation (pyspark.sql.observation)

[ https://issues.apache.org/jira/browse/SPARK-46386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46386: --- Labels: pull-request-available (was: ) > Improve and test assertions of observation

[jira] [Created] (SPARK-46386) Improve and test assertions of observation (pyspark.sql.observation)

Xinrong Meng created SPARK-46386: Summary: Improve and test assertions of observation (pyspark.sql.observation) Key: SPARK-46386 URL: https://issues.apache.org/jira/browse/SPARK-46386 Project: Spark

[jira] [Updated] (SPARK-46385) Test aggregate functions for groups (pyspark.sql.group)

[ https://issues.apache.org/jira/browse/SPARK-46385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46385: --- Labels: pull-request-available (was: ) > Test aggregate functions for groups

[jira] [Created] (SPARK-46385) Test aggregate functions for groups (pyspark.sql.group)

Xinrong Meng created SPARK-46385: Summary: Test aggregate functions for groups (pyspark.sql.group) Key: SPARK-46385 URL: https://issues.apache.org/jira/browse/SPARK-46385 Project: Spark

[jira] [Updated] (SPARK-46384) Streaming UI doesn't display graph correctly

[ https://issues.apache.org/jira/browse/SPARK-46384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-46384: Summary: Streaming UI doesn't display graph correctly (was: Streaming UI doesn't show graph) >

[jira] [Updated] (SPARK-46384) Structured Streaming UI doesn't display graph correctly

[ https://issues.apache.org/jira/browse/SPARK-46384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-46384: Summary: Structured Streaming UI doesn't display graph correctly (was: Streaming UI doesn't display

[jira] [Created] (SPARK-46384) Streaming UI doesn't show graph

Wei Liu created SPARK-46384: --- Summary: Streaming UI doesn't show graph Key: SPARK-46384 URL: https://issues.apache.org/jira/browse/SPARK-46384 Project: Spark Issue Type: Task Components:

[jira] [Resolved] (SPARK-46370) Fix bug when querying from table after changing defaults

[ https://issues.apache.org/jira/browse/SPARK-46370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46370. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44302

[jira] [Assigned] (SPARK-46253) Plan Python data source read using mapInArrow

[ https://issues.apache.org/jira/browse/SPARK-46253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46253: Assignee: Allison Wang > Plan Python data source read using mapInArrow >

[jira] [Resolved] (SPARK-46253) Plan Python data source read using mapInArrow

[ https://issues.apache.org/jira/browse/SPARK-46253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46253. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44170

[jira] [Updated] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46383: --- Labels: pull-request-available (was: ) > Reduce Driver Heap Usage by Reducing the Lifespan

[jira] [Commented] (SPARK-40894) Introduce error sub-classes of the operation not allowed parse error

[ https://issues.apache.org/jira/browse/SPARK-40894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795946#comment-17795946 ] Pavel Fadeev commented on SPARK-40894: -- I'll take it if you don't mind [~maxgekk]  > Introduce

[jira] [Updated] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Utkarsh Agarwal updated SPARK-46383: Description: `AccumulableInfo` is one of the top heap consumers in driver's heap dumps

[jira] [Updated] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Utkarsh Agarwal updated SPARK-46383: Description: `AccumulableInfo` is one of the top heap consumers in driver's heap dumps

[jira] [Updated] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Utkarsh Agarwal updated SPARK-46383: Attachment: screenshot-1.png > Reduce Driver Heap Usage by Reducing the Lifespan of

[jira] [Created] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

Utkarsh Agarwal created SPARK-46383: --- Summary: Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()` Key: SPARK-46383 URL: https://issues.apache.org/jira/browse/SPARK-46383

[jira] [Updated] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Utkarsh Agarwal updated SPARK-46383: Attachment: Screenshot 2023-11-06 at 3.56.26 PM.png > Reduce Driver Heap Usage by

[jira] [Commented] (SPARK-46365) Spark 3.5.0 Regression: Window Function Combination Yields Null Values

[ https://issues.apache.org/jira/browse/SPARK-46365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795932#comment-17795932 ] Josh Rosen commented on SPARK-46365: This issue appears to be fixed in

[jira] [Comment Edited] (SPARK-46349) Prevent Multiple SortOrders for an Expression

[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795922#comment-17795922 ] Majid Hajiheidari edited comment on SPARK-46349 at 12/12/23 8:21 PM: -

[jira] [Commented] (SPARK-46349) Prevent Multiple SortOrders for an Expression

[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795922#comment-17795922 ] Majid Hajiheidari commented on SPARK-46349: --- The PR was not working because it failed lots of

[jira] [Updated] (SPARK-46349) Prevent Multiple SortOrders for an Expression

[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Majid Hajiheidari updated SPARK-46349: -- Description: Hello everyone, This is my first contribution to the project. I welcome

[jira] [Updated] (SPARK-46349) Prevent Multiple SortOrders for an Expression

[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Majid Hajiheidari updated SPARK-46349: -- External issue URL: (was: https://github.com/apache/spark/pull/44283) > Prevent

[jira] [Updated] (SPARK-46349) Prevent Multiple SortOrders for an Expression

[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Majid Hajiheidari updated SPARK-46349: -- Summary: Prevent Multiple SortOrders for an Expression (was: Prevent SortOrder from

[jira] [Updated] (SPARK-46382) XML: Capture values interspersed between elements

[ https://issues.apache.org/jira/browse/SPARK-46382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46382: --- Labels: pull-request-available (was: ) > XML: Capture values interspersed between elements

[jira] [Created] (SPARK-46382) XML: Capture values interspersed between elements

Shujing Yang created SPARK-46382: Summary: XML: Capture values interspersed between elements Key: SPARK-46382 URL: https://issues.apache.org/jira/browse/SPARK-46382 Project: Spark Issue

[jira] [Updated] (SPARK-46381) Migrate sub-classes of AnalysisException to error classes

[ https://issues.apache.org/jira/browse/SPARK-46381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46381: --- Labels: pull-request-available (was: ) > Migrate sub-classes of AnalysisException to error

[jira] [Created] (SPARK-46381) Migrate sub-classes of AnalysisException to error classes

Max Gekk created SPARK-46381: Summary: Migrate sub-classes of AnalysisException to error classes Key: SPARK-46381 URL: https://issues.apache.org/jira/browse/SPARK-46381 Project: Spark Issue

[jira] [Resolved] (SPARK-46378) still remove Sort after converting Aggregate to Project

[ https://issues.apache.org/jira/browse/SPARK-46378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46378. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44310

[jira] [Resolved] (SPARK-46353) Refactor to improve `RegisterWorker` unit test

[ https://issues.apache.org/jira/browse/SPARK-46353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46353. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44284

[jira] [Resolved] (SPARK-46075) Refactor SparkConnectSessionManager to not use guava cache

[ https://issues.apache.org/jira/browse/SPARK-46075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46075. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43985

[jira] [Assigned] (SPARK-46075) Refactor SparkConnectSessionManager to not use guava cache

[ https://issues.apache.org/jira/browse/SPARK-46075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46075: Assignee: Juliusz Sompolski > Refactor SparkConnectSessionManager to not use guava cache

[jira] [Updated] (SPARK-46380) Replacing current time prior to inline table eval

[ https://issues.apache.org/jira/browse/SPARK-46380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46380: --- Labels: pull-request-available (was: ) > Replacing current time prior to inline table eval

[jira] [Commented] (SPARK-46362) calculation error

[ https://issues.apache.org/jira/browse/SPARK-46362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795741#comment-17795741 ] APeng Zhang commented on SPARK-46362: - [~paridee] This is not a defect of Spark, it's related to the

[jira] [Comment Edited] (SPARK-37913) Null Pointer Exception when Loading ML Pipeline Model with Custom Transformer

[ https://issues.apache.org/jira/browse/SPARK-37913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795697#comment-17795697 ] APeng Zhang edited comment on SPARK-37913 at 12/12/23 12:36 PM:

[jira] [Commented] (SPARK-45154) Pyspark DecisionTreeClassifier: results and tree structure in spark3 very different from that of the spark2 version on the same data and with the same hyperparameters.

[ https://issues.apache.org/jira/browse/SPARK-45154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795716#comment-17795716 ] APeng Zhang commented on SPARK-45154: - [~oumarnour] I think you need to set the _seed_ param of

[jira] [Commented] (SPARK-37913) Null Pointer Exception when Loading ML Pipeline Model with Custom Transformer

[ https://issues.apache.org/jira/browse/SPARK-37913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795697#comment-17795697 ] APeng Zhang commented on SPARK-37913: - I can reproduce this issue. h2. Solution: A simple approach

[jira] [Updated] (SPARK-46363) Improve the java text block with java15 feature

[ https://issues.apache.org/jira/browse/SPARK-46363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46363: --- Labels: pull-request-available (was: ) > Improve the java text block with java15 feature >

[jira] [Updated] (SPARK-46379) Reorganize `FrameInterpolateTests`

[ https://issues.apache.org/jira/browse/SPARK-46379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46379: --- Labels: pull-request-available (was: ) > Reorganize `FrameInterpolateTests` >

[jira] [Created] (SPARK-46379) Reorganize `FrameInterpolateTests`

Ruifeng Zheng created SPARK-46379: - Summary: Reorganize `FrameInterpolateTests` Key: SPARK-46379 URL: https://issues.apache.org/jira/browse/SPARK-46379 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: Apache Spark > Fix maven daily testing > --- > >

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: (was: Apache Spark) > Fix maven daily testing >

[jira] [Updated] (SPARK-46361) Add spark dataset chunk read API (python only)

[ https://issues.apache.org/jira/browse/SPARK-46361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-46361: --- Description: *Design doc:* h1.

[jira] [Assigned] (SPARK-46361) Add spark dataset chunk read API (python only)

[ https://issues.apache.org/jira/browse/SPARK-46361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-46361: -- Assignee: Weichen Xu > Add spark dataset chunk read API (python only) >