[jira] [Assigned] (SPARK-41111) Implement `DataFrame.show`

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: Ruifeng Zheng (was: Apache Spark) > Implement `DataFrame.show` > -

[jira] [Assigned] (SPARK-41111) Implement `DataFrame.show`

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: Apache Spark (was: Ruifeng Zheng) > Implement `DataFrame.show` > -

[jira] [Commented] (SPARK-41111) Implement `DataFrame.show`

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632148#comment-17632148 ] Apache Spark commented on SPARK-4: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632163#comment-17632163 ] Apache Spark commented on SPARK-39601: -- User 'pan3793' has created a pull request f

[jira] [Updated] (SPARK-41070) Performance issue when Spark SQL connects with TeraData

[ https://issues.apache.org/jira/browse/SPARK-41070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramakrishna updated SPARK-41070: Component/s: SQL > Performance issue when Spark SQL connects with TeraData >

[jira] [Updated] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated SPARK-41073: Description: In our cluster, zookeeper nearly crashed. I found the znodes of /zkdtsm/ZKDTSMRoot/Z

[jira] [Updated] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated SPARK-41073: Description: In our cluster, zookeeper nearly crashed. I found the znodes of /zkdtsm/ZKDTSMRoot/Z

[jira] [Updated] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated SPARK-41073: Description: In our cluster, zookeeper nearly crashed. I found the znodes of /zkdtsm/ZKDTSMRoot/Z

[jira] [Updated] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated SPARK-41073: Description: In our cluster, zookeeper nearly crashed. I found the znodes of /zkdtsm/ZKDTSMRoot/Z

[jira] [Created] (SPARK-41114) Support local data for LocalRelation

Deng Ziming created SPARK-41114: --- Summary: Support local data for LocalRelation Key: SPARK-41114 URL: https://issues.apache.org/jira/browse/SPARK-41114 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41044) Convert DATATYPE_MISMATCH.UNSPECIFIED_FRAME to INTERNAL_ERROR

[ https://issues.apache.org/jira/browse/SPARK-41044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41044: Assignee: BingKun Pan > Convert DATATYPE_MISMATCH.UNSPECIFIED_FRAME to INTERNAL_ERROR > -

[jira] [Resolved] (SPARK-41044) Convert DATATYPE_MISMATCH.UNSPECIFIED_FRAME to INTERNAL_ERROR

[ https://issues.apache.org/jira/browse/SPARK-41044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41044. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38555 [https://github.com

[jira] [Updated] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu updated SPARK-41073: Attachment: SPARK-41073.proposal.A.draft.001.patch > Spark ThriftServer generate huge amounts of D

[jira] [Resolved] (SPARK-41110) Implement `DataFrame.sparkSession` in Python client

[ https://issues.apache.org/jira/browse/SPARK-41110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41110. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38616 [https://

[jira] [Assigned] (SPARK-41110) Implement `DataFrame.sparkSession` in Python client

[ https://issues.apache.org/jira/browse/SPARK-41110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41110: - Assignee: Rui Wang > Implement `DataFrame.sparkSession` in Python client >

[jira] [Commented] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632209#comment-17632209 ] zhengchenyu commented on SPARK-41073: - We must add a valid credentials to jobConf.

[jira] [Commented] (SPARK-41072) Convert the internal error about failed stream to user-facing error

[ https://issues.apache.org/jira/browse/SPARK-41072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632259#comment-17632259 ] Apache Spark commented on SPARK-41072: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-41072) Convert the internal error about failed stream to user-facing error

[ https://issues.apache.org/jira/browse/SPARK-41072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41072: Assignee: (was: Apache Spark) > Convert the internal error about failed stream to use

[jira] [Assigned] (SPARK-41072) Convert the internal error about failed stream to user-facing error

[ https://issues.apache.org/jira/browse/SPARK-41072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41072: Assignee: Apache Spark > Convert the internal error about failed stream to user-facing er

[jira] [Commented] (SPARK-41114) Support local data for LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-41114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632273#comment-17632273 ] Bjørn Jørgensen commented on SPARK-41114: - [~dengziming] now you can a PR with [

[jira] [Commented] (SPARK-40559) Add applyInArrow to pyspark.sql.GroupedData

[ https://issues.apache.org/jira/browse/SPARK-40559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632304#comment-17632304 ] Apache Spark commented on SPARK-40559: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-40559) Add applyInArrow to pyspark.sql.GroupedData

[ https://issues.apache.org/jira/browse/SPARK-40559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40559: Assignee: (was: Apache Spark) > Add applyInArrow to pyspark.sql.GroupedData > ---

[jira] [Assigned] (SPARK-40559) Add applyInArrow to pyspark.sql.GroupedData

[ https://issues.apache.org/jira/browse/SPARK-40559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40559: Assignee: Apache Spark > Add applyInArrow to pyspark.sql.GroupedData > --

[jira] [Assigned] (SPARK-40372) Migrate failures of array type checks onto error classes

[ https://issues.apache.org/jira/browse/SPARK-40372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-40372: Assignee: Yang Jie > Migrate failures of array type checks onto error classes > -

[jira] [Resolved] (SPARK-40372) Migrate failures of array type checks onto error classes

[ https://issues.apache.org/jira/browse/SPARK-40372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40372. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38507 [https://github.com

[jira] [Assigned] (SPARK-41093) Remove netty-tcnative-classes from Spark dependencyList

[ https://issues.apache.org/jira/browse/SPARK-41093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-41093: Assignee: Cheng Pan > Remove netty-tcnative-classes from Spark dependencyList > -

[jira] [Resolved] (SPARK-41093) Remove netty-tcnative-classes from Spark dependencyList

[ https://issues.apache.org/jira/browse/SPARK-41093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-41093. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38596 [https://gi

[jira] [Updated] (SPARK-41093) Remove netty-tcnative-classes from Spark dependencyList

[ https://issues.apache.org/jira/browse/SPARK-41093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-41093: - Priority: Minor (was: Major) > Remove netty-tcnative-classes from Spark dependencyList > --

[jira] [Commented] (SPARK-40798) Alter partition should verify value

[ https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632413#comment-17632413 ] XiDuo You commented on SPARK-40798: --- [~rangareddy.av...@gmail.com] sure, just go ahead

[jira] [Commented] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632447#comment-17632447 ] Apache Spark commented on SPARK-38959: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632448#comment-17632448 ] Apache Spark commented on SPARK-38959: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-40875) Add .agg() to Connect DSL

[ https://issues.apache.org/jira/browse/SPARK-40875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632479#comment-17632479 ] Apache Spark commented on SPARK-40875: -- User 'grundprinzip' has created a pull requ

[jira] [Commented] (SPARK-40875) Add .agg() to Connect DSL

[ https://issues.apache.org/jira/browse/SPARK-40875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632480#comment-17632480 ] Apache Spark commented on SPARK-40875: -- User 'grundprinzip' has created a pull requ

[jira] [Updated] (SPARK-39405) NumPy Input Support in PySpark

[ https://issues.apache.org/jira/browse/SPARK-39405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39405: - Summary: NumPy Input Support in PySpark (was: NumPy input support in PySpark) > NumPy Input Su

[jira] [Updated] (SPARK-39405) NumPy input support in PySpark

[ https://issues.apache.org/jira/browse/SPARK-39405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39405: - Summary: NumPy input support in PySpark (was: NumPy input support in PySpark SQL) > NumPy inpu

[jira] [Commented] (SPARK-41096) Support reading parquet FIXED_LEN_BYTE_ARRAY type

[ https://issues.apache.org/jira/browse/SPARK-41096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632511#comment-17632511 ] Apache Spark commented on SPARK-41096: -- User 'kazuyukitanimura' has created a pull

[jira] [Assigned] (SPARK-41096) Support reading parquet FIXED_LEN_BYTE_ARRAY type

[ https://issues.apache.org/jira/browse/SPARK-41096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41096: Assignee: Apache Spark > Support reading parquet FIXED_LEN_BYTE_ARRAY type >

[jira] [Assigned] (SPARK-41096) Support reading parquet FIXED_LEN_BYTE_ARRAY type

[ https://issues.apache.org/jira/browse/SPARK-41096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41096: Assignee: (was: Apache Spark) > Support reading parquet FIXED_LEN_BYTE_ARRAY type > -

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-41053: --- Summary: Better Spark UI scalability and Driver stability for large applications (was: Supp

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-41053: --- Description: After SPARK-18085, the Spark history server(SHS) becomes more scalable for pro

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-41053: --- Attachment: Better_UI_Scalability.png > Better Spark UI scalability and Driver stability for

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-41053: --- Description: After SPARK-18085, the Spark history server(SHS) becomes more scalable for pro

[jira] [Updated] (SPARK-39405) NumPy Input Support in PySpark

[ https://issues.apache.org/jira/browse/SPARK-39405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39405: - Description: NumPy is the fundamental package for scientific computing with Python. It is very

[jira] [Created] (SPARK-41115) Add ClientType Enum to proto to indicate which client sends a request

Rui Wang created SPARK-41115: Summary: Add ClientType Enum to proto to indicate which client sends a request Key: SPARK-41115 URL: https://issues.apache.org/jira/browse/SPARK-41115 Project: Spark

[jira] [Commented] (SPARK-41115) Add ClientType Enum to proto to indicate which client sends a request

[ https://issues.apache.org/jira/browse/SPARK-41115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632554#comment-17632554 ] Apache Spark commented on SPARK-41115: -- User 'amaliujia' has created a pull request

[jira] [Commented] (SPARK-41115) Add ClientType Enum to proto to indicate which client sends a request

[ https://issues.apache.org/jira/browse/SPARK-41115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632555#comment-17632555 ] Apache Spark commented on SPARK-41115: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-41115) Add ClientType Enum to proto to indicate which client sends a request

[ https://issues.apache.org/jira/browse/SPARK-41115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41115: Assignee: (was: Apache Spark) > Add ClientType Enum to proto to indicate which client

[jira] [Assigned] (SPARK-41115) Add ClientType Enum to proto to indicate which client sends a request

[ https://issues.apache.org/jira/browse/SPARK-41115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41115: Assignee: Apache Spark > Add ClientType Enum to proto to indicate which client sends a re

[jira] [Commented] (SPARK-40809) Add as(alias: String) to connect DSL

[ https://issues.apache.org/jira/browse/SPARK-40809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632557#comment-17632557 ] Apache Spark commented on SPARK-40809: -- User 'grundprinzip' has created a pull requ

[jira] [Commented] (SPARK-40809) Add as(alias: String) to connect DSL

[ https://issues.apache.org/jira/browse/SPARK-40809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632558#comment-17632558 ] Apache Spark commented on SPARK-40809: -- User 'grundprinzip' has created a pull requ

[jira] [Commented] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632564#comment-17632564 ] Felipe commented on SPARK-40178: The same happens at Scala. There is a workaround: col("

[jira] [Commented] (SPARK-41114) Support local data for LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-41114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632573#comment-17632573 ] Rui Wang commented on SPARK-41114: -- cc [~ruifengz] > Support local data for LocalRelat

[jira] [Created] (SPARK-41116) Input relation can be optional for Project in Connect proto

Rui Wang created SPARK-41116: Summary: Input relation can be optional for Project in Connect proto Key: SPARK-41116 URL: https://issues.apache.org/jira/browse/SPARK-41116 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-41116) Input relation can be optional for Project in Connect proto

[ https://issues.apache.org/jira/browse/SPARK-41116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41116: Assignee: Apache Spark > Input relation can be optional for Project in Connect proto > --

[jira] [Commented] (SPARK-41116) Input relation can be optional for Project in Connect proto

[ https://issues.apache.org/jira/browse/SPARK-41116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632574#comment-17632574 ] Apache Spark commented on SPARK-41116: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-41116) Input relation can be optional for Project in Connect proto

[ https://issues.apache.org/jira/browse/SPARK-41116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41116: Assignee: (was: Apache Spark) > Input relation can be optional for Project in Connect