[jira] [Updated] (SPARK-39965) Skip PVC cleanup when driver doesn't own PVCs

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39965: -- Summary: Skip PVC cleanup when driver doesn't own PVCs (was: Spark on K8s delete pvc even tho

[jira] [Assigned] (SPARK-39965) Skip PVC cleanup when driver doesn't own PVCs

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39965: - Assignee: pralabhkumar > Skip PVC cleanup when driver doesn't own PVCs > --

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576558#comment-17576558 ] Apache Spark commented on SPARK-39965: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39965: Assignee: (was: Apache Spark) > Spark on K8s delete pvc even though it's not being us

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576557#comment-17576557 ] Apache Spark commented on SPARK-39965: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39965: Assignee: Apache Spark > Spark on K8s delete pvc even though it's not being used. > -

[jira] [Comment Edited] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576550#comment-17576550 ] Dongjoon Hyun edited comment on SPARK-39965 at 8/8/22 5:57 AM: ---

[jira] [Assigned] (SPARK-39973) Avoid noisy warnings logs when spark.scheduler.listenerbus.metrics.maxListenerClassesTimed = 0

[ https://issues.apache.org/jira/browse/SPARK-39973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39973: Assignee: Apache Spark > Avoid noisy warnings logs when > spark.scheduler.listenerbus.me

[jira] [Assigned] (SPARK-39973) Avoid noisy warnings logs when spark.scheduler.listenerbus.metrics.maxListenerClassesTimed = 0

[ https://issues.apache.org/jira/browse/SPARK-39973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39973: Assignee: (was: Apache Spark) > Avoid noisy warnings logs when > spark.scheduler.lis

[jira] [Commented] (SPARK-39973) Avoid noisy warnings logs when spark.scheduler.listenerbus.metrics.maxListenerClassesTimed = 0

[ https://issues.apache.org/jira/browse/SPARK-39973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576554#comment-17576554 ] Apache Spark commented on SPARK-39973: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-39764) Make PhysicalOperation the same as ScanOperation

[ https://issues.apache.org/jira/browse/SPARK-39764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39764: --- Assignee: Wenchen Fan > Make PhysicalOperation the same as ScanOperation >

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576552#comment-17576552 ] Dongjoon Hyun commented on SPARK-39965: --- FYI, I'll add you as a co-author for that

[jira] [Resolved] (SPARK-39764) Make PhysicalOperation the same as ScanOperation

[ https://issues.apache.org/jira/browse/SPARK-39764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39764. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37176 [https://gith

[jira] [Updated] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39965: -- Priority: Trivial (was: Minor) > Spark on K8s delete pvc even though it's not being used. > -

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576550#comment-17576550 ] Dongjoon Hyun commented on SPARK-39965: --- Actually, during Spark termination, we al

[jira] [Comment Edited] (SPARK-39979) IndexOutOfBoundsException on groupby + apply pandas grouped map udf function

[ https://issues.apache.org/jira/browse/SPARK-39979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575509#comment-17575509 ] Hyukjin Kwon edited comment on SPARK-39979 at 8/8/22 4:58 AM:

[jira] [Resolved] (SPARK-39953) Hudi spark-submits from EMR 5.33 to EMR 6.5

[ https://issues.apache.org/jira/browse/SPARK-39953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39953. -- Resolution: Incomplete no feedback from the reporter. > Hudi spark-submits from EMR 5.33 to

[jira] [Resolved] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39994. -- Resolution: Invalid > How to write (save) PySpark dataframe containing vector column? > --

[jira] [Updated] (SPARK-39993) Spark on Kubernetes doesn't filter data by date

[ https://issues.apache.org/jira/browse/SPARK-39993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39993: - Priority: Major (was: Blocker) > Spark on Kubernetes doesn't filter data by date >

[jira] [Commented] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576542#comment-17576542 ] Hyukjin Kwon commented on SPARK-39994: -- Okay, assuming from the stacktrace in the S

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39994: - Flags: (was: Important) > How to write (save) PySpark dataframe containing vector column? > --

[jira] [Commented] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576541#comment-17576541 ] Hyukjin Kwon commented on SPARK-39994: -- Can you show all the stacktrace of the erro

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39994: - Language: (was: English) > How to write (save) PySpark dataframe containing vector column? > -

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39994: - Fix Version/s: (was: 3.3.0) > How to write (save) PySpark dataframe containing vector column

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39994: - Target Version/s: (was: 3.3.0) > How to write (save) PySpark dataframe containing vector colum

[jira] [Commented] (SPARK-39995) PySpark installation doesn't support Scala 2.13 binaries

[ https://issues.apache.org/jira/browse/SPARK-39995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576540#comment-17576540 ] Hyukjin Kwon commented on SPARK-39995: -- Yeah, I think we should make it available.

[jira] [Resolved] (SPARK-39998) Can't set pyspark log level in log4j2.properties

[ https://issues.apache.org/jira/browse/SPARK-39998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39998. -- Resolution: Invalid > Can't set pyspark log level in log4j2.properties > -

[jira] [Commented] (SPARK-39828) Catalog.listTables() should respect currentCatalog

[ https://issues.apache.org/jira/browse/SPARK-39828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576520#comment-17576520 ] Apache Spark commented on SPARK-39828: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-39828) Catalog.listTables() should respect currentCatalog

[ https://issues.apache.org/jira/browse/SPARK-39828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576519#comment-17576519 ] Apache Spark commented on SPARK-39828: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576516#comment-17576516 ] pralabhkumar commented on SPARK-39965: -- [~dongjoon]  Thx for replying. We don't se

[jira] [Resolved] (SPARK-39988) LevelDBIterator not close after used in `RemoteBlockPushResolver`, `YarnShuffleService` and `ExternalShuffleBlockResolver`

[ https://issues.apache.org/jira/browse/SPARK-39988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39988. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37420 [https://

[jira] [Assigned] (SPARK-39988) LevelDBIterator not close after used in `RemoteBlockPushResolver`, `YarnShuffleService` and `ExternalShuffleBlockResolver`

[ https://issues.apache.org/jira/browse/SPARK-39988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39988: - Assignee: Yang Jie > LevelDBIterator not close after used in `RemoteBlockPushResolver`,

[jira] [Created] (SPARK-40002) Limit pushed down through window using ntile function

Bruce Robbins created SPARK-40002: - Summary: Limit pushed down through window using ntile function Key: SPARK-40002 URL: https://issues.apache.org/jira/browse/SPARK-40002 Project: Spark Issue

[jira] [Commented] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576462#comment-17576462 ] Apache Spark commented on SPARK-40001: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40001: Assignee: (was: Apache Spark) > Add config to make DEFAULT values in JSON tables mutu

[jira] [Assigned] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40001: Assignee: Apache Spark > Add config to make DEFAULT values in JSON tables mutually exclus

[jira] [Commented] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576461#comment-17576461 ] Apache Spark commented on SPARK-40001: -- User 'dtenedor' has created a pull request

[jira] [Created] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

Daniel created SPARK-40001: -- Summary: Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS Key: SPARK-40001 URL: https://issues.apache.org/jira/browse/SPARK-4

[jira] [Commented] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576449#comment-17576449 ] Apache Spark commented on SPARK-4: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: (was: Apache Spark) > Add config to toggle whether to automatically add def

[jira] [Commented] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576448#comment-17576448 ] Apache Spark commented on SPARK-4: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: Apache Spark > Add config to toggle whether to automatically add default values

[jira] [Created] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

Daniel created SPARK-4: -- Summary: Add config to toggle whether to automatically add default values for INSERTs without user-specified fields Key: SPARK-4 URL: https://issues.apache.org/jira/browse/SPARK-4