[jira] [Created] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-35238: --- Summary: Add JindoFS SDK in cloud integration documents Key: SPARK-35238 URL: https://issues.apache.org/jira/browse/SPARK-35238 Project: Spark Issue Type: Docu

[jira] [Assigned] (SPARK-35169) Wrong result of min ANSI interval division by -1

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35169: --- Assignee: angerszhu > Wrong result of min ANSI interval division by -1 > --

[jira] [Resolved] (SPARK-35169) Wrong result of min ANSI interval division by -1

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35169. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32314 [https://gith

[jira] [Commented] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332991#comment-17332991 ] Apache Spark commented on SPARK-35238: -- User 'adrian-wang' has created a pull reque

[jira] [Assigned] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35238: Assignee: Apache Spark > Add JindoFS SDK in cloud integration documents > ---

[jira] [Assigned] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35238: Assignee: (was: Apache Spark) > Add JindoFS SDK in cloud integration documents >

[jira] [Updated] (SPARK-35237) In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on spark driver. It seems to related to prometheus

2021-04-27 Thread Keunhyun Oh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keunhyun Oh updated SPARK-35237: Summary: In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on

[jira] [Created] (SPARK-35240) Use CheckpointFileManager for checkpoint manipulation

2021-04-27 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-35240: --- Summary: Use CheckpointFileManager for checkpoint manipulation Key: SPARK-35240 URL: https://issues.apache.org/jira/browse/SPARK-35240 Project: Spark Issue Typ

[jira] [Created] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread ulysses you (Jira)
ulysses you created SPARK-35239: --- Summary: Coalesce shuffle partition should handle empty input RDD Key: SPARK-35239 URL: https://issues.apache.org/jira/browse/SPARK-35239 Project: Spark Issue

[jira] [Assigned] (SPARK-35240) Use CheckpointFileManager for checkpoint manipulation

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35240: Assignee: L. C. Hsieh (was: Apache Spark) > Use CheckpointFileManager for checkpoint man

[jira] [Commented] (SPARK-35240) Use CheckpointFileManager for checkpoint manipulation

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332995#comment-17332995 ] Apache Spark commented on SPARK-35240: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-35240) Use CheckpointFileManager for checkpoint manipulation

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35240: Assignee: Apache Spark (was: L. C. Hsieh) > Use CheckpointFileManager for checkpoint man

[jira] [Commented] (SPARK-35240) Use CheckpointFileManager for checkpoint manipulation

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332996#comment-17332996 ] Apache Spark commented on SPARK-35240: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-04-27 Thread Anthony (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333000#comment-17333000 ] Anthony commented on SPARK-18105: - We are seeing similar issues with Spark 3.0.1 as well

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-04-27 Thread Anthony (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333000#comment-17333000 ] Anthony edited comment on SPARK-18105 at 4/27/21, 7:26 AM: --- We

[jira] [Created] (SPARK-35241) Investigate to prefer vectorized hash map in hash aggregate selectively

2021-04-27 Thread Cheng Su (Jira)
Cheng Su created SPARK-35241: Summary: Investigate to prefer vectorized hash map in hash aggregate selectively Key: SPARK-35241 URL: https://issues.apache.org/jira/browse/SPARK-35241 Project: Spark

[jira] [Commented] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333016#comment-17333016 ] Apache Spark commented on SPARK-35239: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35239: Assignee: (was: Apache Spark) > Coalesce shuffle partition should handle empty input

[jira] [Assigned] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35239: Assignee: Apache Spark > Coalesce shuffle partition should handle empty input RDD >

[jira] [Assigned] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34979: Assignee: (was: Apache Spark) > Failed to install pyspark[sql] (due to pyarrow error)

[jira] [Created] (SPARK-35242) Support change default database for spark sql

2021-04-27 Thread hong dongdong (Jira)
hong dongdong created SPARK-35242: - Summary: Support change default database for spark sql Key: SPARK-35242 URL: https://issues.apache.org/jira/browse/SPARK-35242 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333039#comment-17333039 ] Apache Spark commented on SPARK-34979: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333038#comment-17333038 ] Yikun Jiang commented on SPARK-34979: - Note that arrow 4.0.0 has been published, and

[jira] [Assigned] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34979: Assignee: Apache Spark > Failed to install pyspark[sql] (due to pyarrow error) on aarch64

[jira] [Comment Edited] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333038#comment-17333038 ] Yikun Jiang edited comment on SPARK-34979 at 4/27/21, 8:34 AM: ---

[jira] [Updated] (SPARK-35242) Support change default database for spark sql

2021-04-27 Thread hong dongdong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hong dongdong updated SPARK-35242: -- Description: Spark catalog default database can only be 'default'. When we can not access 'def

[jira] [Updated] (SPARK-35242) Support change catalog default database for spark

2021-04-27 Thread hong dongdong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hong dongdong updated SPARK-35242: -- Summary: Support change catalog default database for spark (was: Support change default datab

[jira] [Commented] (SPARK-35242) Support change catalog default database for spark

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333055#comment-17333055 ] Apache Spark commented on SPARK-35242: -- User 'hddong' has created a pull request fo

[jira] [Assigned] (SPARK-35242) Support change catalog default database for spark

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35242: Assignee: Apache Spark > Support change catalog default database for spark >

[jira] [Assigned] (SPARK-35242) Support change catalog default database for spark

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35242: Assignee: (was: Apache Spark) > Support change catalog default database for spark > -

[jira] [Updated] (SPARK-35217) com.google.protobuf.Parser.parseFrom() method Can't use in spark

2021-04-27 Thread ShuDaoNan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ShuDaoNan updated SPARK-35217: -- Priority: Trivial (was: Major) > com.google.protobuf.Parser.parseFrom() method Can't use in spark >

[jira] [Resolved] (SPARK-35217) com.google.protobuf.Parser.parseFrom() method Can't use in spark

2021-04-27 Thread ShuDaoNan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ShuDaoNan resolved SPARK-35217. --- Resolution: Incomplete Amazon emr自带的protobuf-java-version.jar 版本过低导致 > com.google.protobuf.Parser.

[jira] [Commented] (SPARK-35228) Add expression ToHiveString for keep consistent between hive/spark format in df.show and transform

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333094#comment-17333094 ] Apache Spark commented on SPARK-35228: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35228) Add expression ToHiveString for keep consistent between hive/spark format in df.show and transform

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35228: Assignee: (was: Apache Spark) > Add expression ToHiveString for keep consistent betwe

[jira] [Assigned] (SPARK-35228) Add expression ToHiveString for keep consistent between hive/spark format in df.show and transform

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35228: Assignee: Apache Spark > Add expression ToHiveString for keep consistent between hive/spa

[jira] [Commented] (SPARK-35228) Add expression ToHiveString for keep consistent between hive/spark format in df.show and transform

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333096#comment-17333096 ] Apache Spark commented on SPARK-35228: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-35020) Group exception messages in catalyst/util

2021-04-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333101#comment-17333101 ] jiaan.geng commented on SPARK-35020: I'm working on. > Group exception messages in

[jira] [Issue Comment Deleted] (SPARK-35060) Group exception messages in sql/types

2021-04-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35060: --- Comment: was deleted (was: I'm working on.) > Group exception messages in sql/types > -

[jira] [Commented] (SPARK-34878) Test actual size of year-month and day-time intervals

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333106#comment-17333106 ] Apache Spark commented on SPARK-34878: -- User 'Peng-Lei' has created a pull request

[jira] [Assigned] (SPARK-34878) Test actual size of year-month and day-time intervals

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34878: Assignee: (was: Apache Spark) > Test actual size of year-month and day-time intervals

[jira] [Assigned] (SPARK-34878) Test actual size of year-month and day-time intervals

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34878: Assignee: Apache Spark > Test actual size of year-month and day-time intervals >

[jira] [Assigned] (SPARK-35020) Group exception messages in catalyst/util

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35020: Assignee: (was: Apache Spark) > Group exception messages in catalyst/util > -

[jira] [Assigned] (SPARK-35020) Group exception messages in catalyst/util

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35020: Assignee: Apache Spark > Group exception messages in catalyst/util >

[jira] [Commented] (SPARK-35020) Group exception messages in catalyst/util

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333131#comment-17333131 ] Apache Spark commented on SPARK-35020: -- User 'beliefer' has created a pull request

[jira] [Issue Comment Deleted] (SPARK-35020) Group exception messages in catalyst/util

2021-04-27 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35020: --- Comment: was deleted (was: I'm working on.) > Group exception messages in catalyst/util > -

[jira] [Assigned] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35239: --- Assignee: ulysses you > Coalesce shuffle partition should handle empty input RDD > ---

[jira] [Resolved] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35239. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32362 [https://gith

[jira] [Resolved] (SPARK-35091) Support ANSI intervals by date_part()

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35091. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32351 [https://gith

[jira] [Assigned] (SPARK-35091) Support ANSI intervals by date_part()

2021-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35091: --- Assignee: Kent Yao > Support ANSI intervals by date_part() > --

[jira] [Commented] (SPARK-35176) Raise TypeError in inappropriate type case rather than ValueError

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333246#comment-17333246 ] Apache Spark commented on SPARK-35176: -- User 'Yikun' has created a pull request for

[jira] [Created] (SPARK-35243) Support columnar execution on ANSI interval types

2021-04-27 Thread Max Gekk (Jira)
Max Gekk created SPARK-35243: Summary: Support columnar execution on ANSI interval types Key: SPARK-35243 URL: https://issues.apache.org/jira/browse/SPARK-35243 Project: Spark Issue Type: Sub-tas

[jira] [Assigned] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35238: Assignee: Adrian Wang > Add JindoFS SDK in cloud integration documents >

[jira] [Resolved] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35238. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32360 [https://gi

[jira] [Created] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-35244: --- Summary: invoke should throw the original exception Key: SPARK-35244 URL: https://issues.apache.org/jira/browse/SPARK-35244 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1756#comment-1756 ] Apache Spark commented on SPARK-35244: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35244: Assignee: Apache Spark (was: Wenchen Fan) > invoke should throw the original exception >

[jira] [Assigned] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35244: Assignee: Apache Spark (was: Wenchen Fan) > invoke should throw the original exception >

[jira] [Assigned] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35244: Assignee: Wenchen Fan (was: Apache Spark) > invoke should throw the original exception >

[jira] [Created] (SPARK-35245) DynamicFilter pushdown not working

2021-04-27 Thread jean-claude (Jira)
jean-claude created SPARK-35245: --- Summary: DynamicFilter pushdown not working Key: SPARK-35245 URL: https://issues.apache.org/jira/browse/SPARK-35245 Project: Spark Issue Type: Bug Co

[jira] [Created] (SPARK-35246) Disable intersects for all streaming queries

2021-04-27 Thread Jose Torres (Jira)
Jose Torres created SPARK-35246: --- Summary: Disable intersects for all streaming queries Key: SPARK-35246 URL: https://issues.apache.org/jira/browse/SPARK-35246 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35246) Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

2021-04-27 Thread Jose Torres (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Torres updated SPARK-35246: Summary: Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

[jira] [Commented] (SPARK-35246) Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333404#comment-17333404 ] Apache Spark commented on SPARK-35246: -- User 'jose-torres' has created a pull reque

[jira] [Assigned] (SPARK-35246) Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35246: Assignee: Apache Spark > Streaming-batch intersects are incorrectly allowed through > Un

[jira] [Assigned] (SPARK-35246) Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35246: Assignee: (was: Apache Spark) > Streaming-batch intersects are incorrectly allowed th

[jira] [Updated] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34979: - Priority: Trivial (was: Major) > Failed to install pyspark[sql] (due to pyarrow error) on aarch

[jira] [Updated] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34979: - Component/s: Documentation > Failed to install pyspark[sql] (due to pyarrow error) on aarch64 >

[jira] [Resolved] (SPARK-35150) Accelerate fallback BLAS with dev.ludovic.netlib

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35150. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32253 [https://gi

[jira] [Assigned] (SPARK-35150) Accelerate fallback BLAS with dev.ludovic.netlib

2021-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35150: Assignee: Ludovic Henry > Accelerate fallback BLAS with dev.ludovic.netlib >

[jira] [Created] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35247: -- Summary: Add `mask` built-in function as a data masking function Key: SPARK-35247 URL: https://issues.apache.org/jira/browse/SPARK-35247 Project: Spark I

[jira] [Updated] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35247: --- Issue Type: New Feature (was: Bug) > Add `mask` built-in function as a data masking functio

[jira] [Assigned] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35247: Assignee: Kousuke Saruta (was: Apache Spark) > Add `mask` built-in function as a data ma

[jira] [Assigned] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35247: Assignee: Apache Spark (was: Kousuke Saruta) > Add `mask` built-in function as a data ma

[jira] [Commented] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333548#comment-17333548 ] Apache Spark commented on SPARK-35247: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Daniel Dai (Jira)
Daniel Dai created SPARK-35248: -- Summary: Spark shall load system class first in IsolatedClientLoader Key: SPARK-35248 URL: https://issues.apache.org/jira/browse/SPARK-35248 Project: Spark Issu

[jira] [Assigned] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35248: Assignee: (was: Apache Spark) > Spark shall load system class first in IsolatedClient

[jira] [Assigned] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35248: Assignee: Apache Spark > Spark shall load system class first in IsolatedClientLoader > --

[jira] [Commented] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333554#comment-17333554 ] Apache Spark commented on SPARK-35248: -- User 'daijyc' has created a pull request fo

[jira] [Commented] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333555#comment-17333555 ] Apache Spark commented on SPARK-35248: -- User 'daijyc' has created a pull request fo

[jira] [Resolved] (SPARK-34979) Failed to install pyspark[sql] (due to pyarrow error) on aarch64

2021-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34979. -- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Fixed in [https:/

[jira] [Created] (SPARK-35249) to_timestamp can't parse 6 digit microsecond SSSSSS

2021-04-27 Thread t oo (Jira)
t oo created SPARK-35249: Summary: to_timestamp can't parse 6 digit microsecond SS Key: SPARK-35249 URL: https://issues.apache.org/jira/browse/SPARK-35249 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-35243) Support columnar execution on ANSI interval types

2021-04-27 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334393#comment-17334393 ] PengLei commented on SPARK-35243: - Working on this > Support columnar execution on ANSI

[jira] [Resolved] (SPARK-35236) Support archive files as resources for CREATE FUNCTION USING syntax

2021-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35236. -- Fix Version/s: 3.2.0 Resolution: Fixed Fixed in [https://github.com/apache/spark/pull/3

[jira] [Created] (SPARK-35250) SQL DataFrameReader mode is misdocumented

2021-04-27 Thread Tim McNamara (Jira)
Tim McNamara created SPARK-35250: Summary: SQL DataFrameReader mode is misdocumented Key: SPARK-35250 URL: https://issues.apache.org/jira/browse/SPARK-35250 Project: Spark Issue Type: Documen

[jira] [Updated] (SPARK-35250) SQL DataFrameReader mode is misdocumented

2021-04-27 Thread Tim McNamara (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim McNamara updated SPARK-35250: - Description: The unescapedQuoteHandling parameter of DataFrameReader isn't correctly documented

[jira] [Updated] (SPARK-35250) SQL DataFrameReader mode is misdocumented

2021-04-27 Thread Tim McNamara (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim McNamara updated SPARK-35250: - Description: The unescapedQuoteHandling parameter of DataFrameReader isn't correctly documented

[jira] [Resolved] (SPARK-35244) invoke should throw the original exception

2021-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35244. -- Fix Version/s: 3.2.0 Resolution: Fixed [https://github.com/apache/spark/pull/32370] >

[jira] [Resolved] (SPARK-35246) Streaming-batch intersects are incorrectly allowed through UnsupportedOperationsChecker

2021-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35246. -- Assignee: Jose Torres Resolution: Fixed Fixed in [https://github.com/apache/spark/pull/3

[jira] [Updated] (SPARK-35248) Spark shall load system class first in IsolatedClientLoader

2021-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35248: - Target Version/s: (was: 3.1.2) > Spark shall load system class first in IsolatedClientLoader >

[jira] [Updated] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

2021-04-27 Thread Tim McNamara (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim McNamara updated SPARK-35250: - Summary: SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented (was: SQL DataFr

[jira] [Created] (SPARK-35251) Improve LiveEntityHelpers.newAccumulatorInfos performace

2021-04-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-35251: --- Summary: Improve LiveEntityHelpers.newAccumulatorInfos performace Key: SPARK-35251 URL: https://issues.apache.org/jira/browse/SPARK-35251 Project: Spark Issue

[jira] [Created] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConfi parameter is null

2021-04-27 Thread lynn (Jira)
lynn created SPARK-35252: Summary: PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConfi parameter is null Key: SPARK-35252 URL: https://issues.apache.org/jira/browse/SPARK-35252 Project: Spark

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConfi parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of {code:java} // Implemention Class package com.lynn.spark.sql.v2 import org.apache.sp

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Summary: PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null (was: PartitionR

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" of {code:java} // Implemention Class package com.lynn.spar

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

[jira] [Updated] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2 sqlConf parameter is null

2021-04-27 Thread lynn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lynn updated SPARK-35252: - Description: The codes of "MyPartitionReaderFactory" : {code:java} // Implemention Class package com.lynn.spark.

  1   2   >