[jira] [Created] (SPARK-33233) CUBE/ROLLUP can't support UnresolvedOrdinal

angerszhu created SPARK-33233: - Summary: CUBE/ROLLUP can't support UnresolvedOrdinal Key: SPARK-33233 URL: https://issues.apache.org/jira/browse/SPARK-33233 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33233) CUBE/ROLLUP can't support UnresolvedOrdinal

[ https://issues.apache.org/jira/browse/SPARK-33233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220009#comment-17220009 ] angerszhu commented on SPARK-33233: --- Raise a pr soon > CUBE/ROLLUP can't support UnresolvedOrdinal >

[jira] [Commented] (SPARK-33229) UnsupportedOperationException when group by with cube

[ https://issues.apache.org/jira/browse/SPARK-33229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220007#comment-17220007 ] Apache Spark commented on SPARK-33229: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-33229) UnsupportedOperationException when group by with cube

[ https://issues.apache.org/jira/browse/SPARK-33229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33229: Assignee: (was: Apache Spark) > UnsupportedOperationException when group by with

[jira] [Assigned] (SPARK-33229) UnsupportedOperationException when group by with cube

[ https://issues.apache.org/jira/browse/SPARK-33229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33229: Assignee: Apache Spark > UnsupportedOperationException when group by with cube >

[jira] [Commented] (SPARK-33229) UnsupportedOperationException when group by with cube

[ https://issues.apache.org/jira/browse/SPARK-33229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220002#comment-17220002 ] angerszhu commented on SPARK-33229: --- Working on this > UnsupportedOperationException when group by

[jira] [Updated] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33220: -- Description: For some schedule behavior, we will `scheduleAtFixedRate` to schedule task or heartbeat

[jira] [Resolved] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-33232. -- Resolution: Invalid > ConcurrentAppendException while updating delta lake table >

[jira] [Commented] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219997#comment-17219997 ] Jungtaek Lim commented on SPARK-33232: -- The issue doesn't look to be specific to "Apache Spark".

[jira] [Resolved] (SPARK-24915) Calling SparkSession.createDataFrame with schema can throw exception

[ https://issues.apache.org/jira/browse/SPARK-24915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24915. -- Resolution: Cannot Reproduce This is fixed from Spark 3.0 and cannot be reproduced. >

[jira] [Resolved] (SPARK-18180) pyspark.sql.Row does not serialize well to json

[ https://issues.apache.org/jira/browse/SPARK-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18180. -- Resolution: Not A Bug > pyspark.sql.Row does not serialize well to json >

[jira] [Resolved] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

[ https://issues.apache.org/jira/browse/SPARK-32137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32137. -- Resolution: Cannot Reproduce > AttributeError: Can only use .dt accessor with datetimelike

[jira] [Commented] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

[ https://issues.apache.org/jira/browse/SPARK-32137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219992#comment-17219992 ] Hyukjin Kwon commented on SPARK-32137: -- It was fixed by Arrow dependency upgrade in Spark 3.0.0,

[jira] [Commented] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219990#comment-17219990 ] Hyukjin Kwon commented on SPARK-33232: -- This sounds specific to Databrick's instead of Apache

[jira] [Assigned] (SPARK-33003) Add type hints guideliness to the documentation

[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33003: Assignee: Maciej Szymkiewicz > Add type hints guideliness to the documentation >

[jira] [Resolved] (SPARK-33003) Add type hints guideliness to the documentation

[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33003. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30094

[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219974#comment-17219974 ] Apache Spark commented on SPARK-32084: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32084: Assignee: (was: Apache Spark) > Replace dictionary-based function definitions to

[jira] [Assigned] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32084: Assignee: Apache Spark > Replace dictionary-based function definitions to proper

[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219973#comment-17219973 ] Apache Spark commented on SPARK-32084: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219952#comment-17219952 ] Dongjoon Hyun commented on SPARK-33044: --- Thank you so much, [~shaneknapp]! :) > Add a Jenkins

[jira] [Updated] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khang Pham updated SPARK-33232: --- Description: I have two Spark Streaming job run concurrently.  * Stream join Job: join in Kafka

[jira] [Updated] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khang Pham updated SPARK-33232: --- Description: I have two Spark Streaming job run concurrently.  * Stream join Job: join in Kafka

[jira] [Updated] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khang Pham updated SPARK-33232: --- Description: I have two Spark Streaming job run concurrently.  * Stream join Job: join in Kafka

[jira] [Updated] (SPARK-33232) ConcurrentAppendException while updating delta lake table

[ https://issues.apache.org/jira/browse/SPARK-33232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khang Pham updated SPARK-33232: --- Description: I have two Spark Streaming job run concurrently.  * Stream join Job: join in Kafka

[jira] [Created] (SPARK-33232) ConcurrentAppendException while updating delta lake table

Khang Pham created SPARK-33232: -- Summary: ConcurrentAppendException while updating delta lake table Key: SPARK-33232 URL: https://issues.apache.org/jira/browse/SPARK-33232 Project: Spark Issue

[jira] [Assigned] (SPARK-33193) Hive ThriftServer JDBC Database MetaData API Behavior Auditing

[ https://issues.apache.org/jira/browse/SPARK-33193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33193: - Assignee: Kent Yao > Hive ThriftServer JDBC Database MetaData API Behavior Auditing >

[jira] [Resolved] (SPARK-33193) Hive ThriftServer JDBC Database MetaData API Behavior Auditing

[ https://issues.apache.org/jira/browse/SPARK-33193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33193. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30101

[jira] [Commented] (SPARK-32863) Full outer stream-stream join

[ https://issues.apache.org/jira/browse/SPARK-32863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219907#comment-17219907 ] Cheng Su commented on SPARK-32863: -- [~warrenzhu25] - Thanks for your interest, Yes, I already had a

[jira] [Commented] (SPARK-32863) Full outer stream-stream join

[ https://issues.apache.org/jira/browse/SPARK-32863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219872#comment-17219872 ] Zhongwei Zhu commented on SPARK-32863: -- [~chengsu] Have you already worked on this? I want to help

[jira] [Created] (SPARK-33231) Make podCreationTimeout configurable

Holden Karau created SPARK-33231: Summary: Make podCreationTimeout configurable Key: SPARK-33231 URL: https://issues.apache.org/jira/browse/SPARK-33231 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219840#comment-17219840 ] Bryan Cutler commented on SPARK-33213: -- Just a couple notes: The library and format versions are

[jira] [Updated] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-33230: --- Summary: FileOutputWriter jobs have duplicate JobIDs if launched in same second (was:

[jira] [Updated] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33230: -- Priority: Major (was: Minor) > FileOutputWriter to set jobConf

[jira] [Updated] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33230: -- Issue Type: Bug (was: Improvement) > FileOutputWriter to set jobConf

[jira] [Commented] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219742#comment-17219742 ] Apache Spark commented on SPARK-33095: -- User 'ScrapCodes' has created a pull request for this

[jira] [Commented] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219740#comment-17219740 ] Apache Spark commented on SPARK-33095: -- User 'ScrapCodes' has created a pull request for this

[jira] [Commented] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219727#comment-17219727 ] Rohit Mishra commented on SPARK-33220: -- [~angerszhuuu], Can you please add the description? >

[jira] [Commented] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219707#comment-17219707 ] Apache Spark commented on SPARK-33230: -- User 'steveloughran' has created a pull request for this

[jira] [Commented] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219705#comment-17219705 ] Apache Spark commented on SPARK-33230: -- User 'steveloughran' has created a pull request for this

[jira] [Assigned] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33230: Assignee: Apache Spark > FileOutputWriter to set jobConf

[jira] [Assigned] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33230: Assignee: (was: Apache Spark) > FileOutputWriter to set jobConf

[jira] [Updated] (SPARK-33230) FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to description.uuid

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-33230: --- Summary: FileOutputWriter to set jobConf "spark.sql.sources.writeJobUUID" to

[jira] [Assigned] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33228: Assignee: (was: Apache Spark) > Don't uncache data when replacing an existing view

[jira] [Assigned] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33228: Assignee: Apache Spark > Don't uncache data when replacing an existing view having the

[jira] [Commented] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219682#comment-17219682 ] Apache Spark commented on SPARK-33228: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-33230) re-instate "spark.sql.sources.writeJobUUID" as unique ID in FileOutputWriter jobs

Steve Loughran created SPARK-33230: -- Summary: re-instate "spark.sql.sources.writeJobUUID" as unique ID in FileOutputWriter jobs Key: SPARK-33230 URL: https://issues.apache.org/jira/browse/SPARK-33230

[jira] [Created] (SPARK-33229) UnsupportedOperationException when group by with cube

Yuming Wang created SPARK-33229: --- Summary: UnsupportedOperationException when group by with cube Key: SPARK-33229 URL: https://issues.apache.org/jira/browse/SPARK-33229 Project: Spark Issue

[jira] [Created] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

Takeshi Yamamuro created SPARK-33228: Summary: Don't uncache data when replacing an existing view having the same plan Key: SPARK-33228 URL: https://issues.apache.org/jira/browse/SPARK-33228

[jira] [Created] (SPARK-33227) Add Jar with Azure SAS token fails with URL encoded characters

James McShane created SPARK-33227: - Summary: Add Jar with Azure SAS token fails with URL encoded characters Key: SPARK-33227 URL: https://issues.apache.org/jira/browse/SPARK-33227 Project: Spark

[jira] [Resolved] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33104. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30133

[jira] [Assigned] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33104: Assignee: Hyukjin Kwon > Fix `YarnClusterSuite.yarn-cluster should respect conf

[jira] [Resolved] (SPARK-33145) In Execution web page, when `Succeeded Job` has many child url elements,they will extend over the edge of the page.

[ https://issues.apache.org/jira/browse/SPARK-33145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] akiyamaneko resolved SPARK-33145. - Fix Version/s: 3.1.0 Resolution: Fixed > In Execution web page, when `Succeeded Job` has

[jira] [Commented] (SPARK-31069) high cpu caused by chunksBeingTransferred in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-31069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219535#comment-17219535 ] Apache Spark commented on SPARK-31069: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-31069) high cpu caused by chunksBeingTransferred in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-31069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219534#comment-17219534 ] Apache Spark commented on SPARK-31069: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-33075) Only disable auto bucketed scan for cached query

[ https://issues.apache.org/jira/browse/SPARK-33075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219529#comment-17219529 ] Apache Spark commented on SPARK-33075: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33075) Only disable auto bucketed scan for cached query

[ https://issues.apache.org/jira/browse/SPARK-33075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33075: Assignee: (was: Apache Spark) > Only disable auto bucketed scan for cached query >

[jira] [Assigned] (SPARK-33075) Only disable auto bucketed scan for cached query

[ https://issues.apache.org/jira/browse/SPARK-33075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33075: Assignee: Apache Spark > Only disable auto bucketed scan for cached query >

[jira] [Commented] (SPARK-33075) Only disable auto bucketed scan for cached query

[ https://issues.apache.org/jira/browse/SPARK-33075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219527#comment-17219527 ] Apache Spark commented on SPARK-33075: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219497#comment-17219497 ] Yuming Wang commented on SPARK-33207: - Thank you [~chengsu]. > Reduce the number of tasks launched