[jira] [Created] (SPARK-33535) export LANG to en_US.UTF-8 in jenkins test script

2020-11-24 Thread Yang Jie (Jira)
Yang Jie created SPARK-33535: Summary: export LANG to en_US.UTF-8 in jenkins test script Key: SPARK-33535 URL: https://issues.apache.org/jira/browse/SPARK-33535 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-33077) Spark 3 / Cats 2.2.0 classpath issue

2020-11-24 Thread Stijn De Haes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238155#comment-17238155 ] Stijn De Haes commented on SPARK-33077: --- [~srowen] I started to tackle the problem

[jira] [Assigned] (SPARK-33533) BasicConnectionProvider should consider case-sensitivity for properties.

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33533: Assignee: Apache Spark (was: Kousuke Saruta) > BasicConnectionProvider should consider c

[jira] [Assigned] (SPARK-33533) BasicConnectionProvider should consider case-sensitivity for properties.

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33533: Assignee: Kousuke Saruta (was: Apache Spark) > BasicConnectionProvider should consider c

[jira] [Commented] (SPARK-33533) BasicConnectionProvider should consider case-sensitivity for properties.

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238145#comment-17238145 ] Apache Spark commented on SPARK-33533: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-33534) Allow specifying a minimum number of bytes in a split of a file

2020-11-24 Thread Niels Basjes (Jira)
Niels Basjes created SPARK-33534: Summary: Allow specifying a minimum number of bytes in a split of a file Key: SPARK-33534 URL: https://issues.apache.org/jira/browse/SPARK-33534 Project: Spark

[jira] [Created] (SPARK-33533) BasicConnectionProvider should consider case-sensitivity for properties.

2020-11-24 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-33533: -- Summary: BasicConnectionProvider should consider case-sensitivity for properties. Key: SPARK-33533 URL: https://issues.apache.org/jira/browse/SPARK-33533 Project:

[jira] [Commented] (SPARK-33532) Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238091#comment-17238091 ] Apache Spark commented on SPARK-33532: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-33532) Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238088#comment-17238088 ] Apache Spark commented on SPARK-33532: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-33532) Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33532: Assignee: Apache Spark > Remove unreachable branch in SpecificParquetRecordReaderBase.ini

[jira] [Assigned] (SPARK-33532) Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33532: Assignee: (was: Apache Spark) > Remove unreachable branch in SpecificParquetRecordRea

[jira] [Updated] (SPARK-33532) Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33532: - Summary: Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method (was: Clean up

[jira] [Created] (SPARK-33532) Clean up Unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-11-24 Thread Yang Jie (Jira)
Yang Jie created SPARK-33532: Summary: Clean up Unreachable branch in SpecificParquetRecordReaderBase.initialize method Key: SPARK-33532 URL: https://issues.apache.org/jira/browse/SPARK-33532 Project: Spa

[jira] [Created] (SPARK-33531) [SQL] Avoid shuffle when calling CollectLimitExec#executeToIterator

2020-11-24 Thread Mori[A]rty (Jira)
Mori[A]rty created SPARK-33531: -- Summary: [SQL] Avoid shuffle when calling CollectLimitExec#executeToIterator Key: SPARK-33531 URL: https://issues.apache.org/jira/browse/SPARK-33531 Project: Spark

[jira] [Commented] (SPARK-33449) Add cache for Parquet Metadata

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238067#comment-17238067 ] Apache Spark commented on SPARK-33449: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-33449) Add cache for Parquet Metadata

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33449: Assignee: (was: Apache Spark) > Add cache for Parquet Metadata >

[jira] [Assigned] (SPARK-33449) Add cache for Parquet Metadata

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33449: Assignee: Apache Spark > Add cache for Parquet Metadata > --

[jira] [Created] (SPARK-33530) Support --archives option natively

2020-11-24 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33530: Summary: Support --archives option natively Key: SPARK-33530 URL: https://issues.apache.org/jira/browse/SPARK-33530 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-33514) Migrate TRUNCATE TABLE to new resolution framework

2020-11-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33514. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30457 [https://gith

[jira] [Assigned] (SPARK-33514) Migrate TRUNCATE TABLE to new resolution framework

2020-11-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33514: --- Assignee: Terry Kim > Migrate TRUNCATE TABLE to new resolution framework >

[jira] [Resolved] (SPARK-33528) File/Directory does not exist

2020-11-24 Thread zengxl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengxl resolved SPARK-33528. Resolution: Not A Problem > File/Directory does not exist > - > >

[jira] [Commented] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238000#comment-17238000 ] Apache Spark commented on SPARK-33529: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33529: Assignee: (was: Apache Spark) > Resolver of V2 partition specs doesn't handle __HIVE_

[jira] [Assigned] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33529: Assignee: Apache Spark > Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PAR

[jira] [Assigned] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33529: Assignee: Apache Spark > Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PAR

[jira] [Updated] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33529: --- Description: The partition value '__HIVE_DEFAULT_PARTITION__' should be handled as null - the same

[jira] [Created] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33529: -- Summary: Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__ Key: SPARK-33529 URL: https://issues.apache.org/jira/browse/SPARK-33529 Project: Spark

[jira] [Commented] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237982#comment-17237982 ] Apache Spark commented on SPARK-33526: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33526: Assignee: Apache Spark > Add config to control if cancel invoke interrupt task on thrifts

[jira] [Assigned] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33526: Assignee: (was: Apache Spark) > Add config to control if cancel invoke interrupt task

[jira] [Comment Edited] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

2020-11-24 Thread fanxin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237961#comment-17237961 ] fanxin edited comment on SPARK-31754 at 11/24/20, 9:07 AM: --- I

[jira] [Assigned] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32921: Assignee: Apache Spark > Extend MapOutputTracker to support tracking and serving the meta

[jira] [Assigned] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32921: Assignee: (was: Apache Spark) > Extend MapOutputTracker to support tracking and servi

[jira] [Commented] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2020-11-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237967#comment-17237967 ] Apache Spark commented on SPARK-32921: -- User 'Victsm' has created a pull request fo

[jira] [Comment Edited] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

2020-11-24 Thread fanxin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237961#comment-17237961 ] fanxin edited comment on SPARK-31754 at 11/24/20, 8:50 AM: --- I

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

2020-11-24 Thread fanxin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237961#comment-17237961 ] fanxin commented on SPARK-31754: I encountered the same NEP in Spark2.4.4. I have found

[jira] [Created] (SPARK-33528) File/Directory does not exist

2020-11-24 Thread zengxl (Jira)
zengxl created SPARK-33528: -- Summary: File/Directory does not exist Key: SPARK-33528 URL: https://issues.apache.org/jira/browse/SPARK-33528 Project: Spark Issue Type: Bug Components: SQL

[jira] [Assigned] (SPARK-33521) Universal type conversion of V2 partition values

2020-11-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33521: --- Assignee: Maxim Gekk > Universal type conversion of V2 partition values > -

[jira] [Resolved] (SPARK-33521) Universal type conversion of V2 partition values

2020-11-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33521. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30474 [https://gith

<    1   2