[jira] [Commented] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246357#comment-17246357 ] Apache Spark commented on SPARK-33071: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33536) Incorrect join results when joining twice with the same DF

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246359#comment-17246359 ] Apache Spark commented on SPARK-33536: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33536) Incorrect join results when joining twice with the same DF

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246360#comment-17246360 ] Apache Spark commented on SPARK-33536: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-33718) SharedState shall be an system-wide singleton

2020-12-09 Thread Kent Yao (Jira)
Kent Yao created SPARK-33718: Summary: SharedState shall be an system-wide singleton Key: SPARK-33718 URL: https://issues.apache.org/jira/browse/SPARK-33718 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33718) SharedState shall be an system-wide singleton

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33718: Assignee: (was: Apache Spark) > SharedState shall be an system-wide singleton >

[jira] [Updated] (SPARK-33719) Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance

2020-12-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-33719: --- Description: Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance. So

[jira] [Assigned] (SPARK-33719) Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33719: Assignee: Gengliang Wang (was: Apache Spark) > Add

[jira] [Assigned] (SPARK-33719) Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33719: Assignee: Apache Spark (was: Gengliang Wang) > Add

[jira] [Commented] (SPARK-33719) Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246377#comment-17246377 ] Apache Spark commented on SPARK-33719: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-33720) Support submit to k8s only with token

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246415#comment-17246415 ] Apache Spark commented on SPARK-33720: -- User 'hddong' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33720) Support submit to k8s only with token

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33720: Assignee: Apache Spark > Support submit to k8s only with token >

[jira] [Created] (SPARK-33719) Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance

2020-12-09 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-33719: -- Summary: Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance Key: SPARK-33719 URL: https://issues.apache.org/jira/browse/SPARK-33719

[jira] [Commented] (SPARK-33718) SharedState shall be an system-wide singleton

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246362#comment-17246362 ] Apache Spark commented on SPARK-33718: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33718) SharedState shall be an system-wide singleton

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33718: Assignee: Apache Spark > SharedState shall be an system-wide singleton >

[jira] [Commented] (SPARK-33718) SharedState shall be an system-wide singleton

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246363#comment-17246363 ] Apache Spark commented on SPARK-33718: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33655) Thrift server : FETCH_PRIOR does not cause to reiterate from start position.

2020-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33655. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30600

[jira] [Assigned] (SPARK-33655) Thrift server : FETCH_PRIOR does not cause to reiterate from start position.

2020-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33655: Assignee: Dooyoung Hwang > Thrift server : FETCH_PRIOR does not cause to reiterate from

[jira] [Updated] (SPARK-33605) Add GCS FS/connector config (dependencies?) akin to S3

2020-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-33605: -- Summary: Add GCS FS/connector config (dependencies?) akin to S3 (was: Add GCS FS/connector

[jira] [Updated] (SPARK-33720) Support submit to k8s only with token

2020-12-09 Thread hong dongdong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hong dongdong updated SPARK-33720: -- Summary: Support submit to k8s only with token (was: Support submit only with token) >

[jira] [Created] (SPARK-33720) Support submit only with token

2020-12-09 Thread hong dongdong (Jira)
hong dongdong created SPARK-33720: - Summary: Support submit only with token Key: SPARK-33720 URL: https://issues.apache.org/jira/browse/SPARK-33720 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-33720) Support submit to k8s only with token

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33720: Assignee: (was: Apache Spark) > Support submit to k8s only with token >

[jira] [Commented] (SPARK-33720) Support submit to k8s only with token

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246414#comment-17246414 ] Apache Spark commented on SPARK-33720: -- User 'hddong' has created a pull request for this issue:

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246717#comment-17246717 ] Maxim Gekk commented on SPARK-33571: > The behavior of the to be introduced in Spark 3.1 >

[jira] [Commented] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246722#comment-17246722 ] Apache Spark commented on SPARK-33722: -- User 'aokolnychyi' has created a pull request for this

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246725#comment-17246725 ] Dongjoon Hyun commented on SPARK-18105: --- Apache Spark 3.x is using lz4-java-1.7.1.jar and this

[jira] [Commented] (SPARK-32110) -0.0 vs 0.0 is inconsistent

2020-12-09 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246585#comment-17246585 ] Robert Joseph Evans commented on SPARK-32110: - Are we sure that this issue should be closed?

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246643#comment-17246643 ] Gabor Somogyi edited comment on SPARK-33635 at 12/9/20, 4:39 PM: -

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246640#comment-17246640 ] Gabor Somogyi commented on SPARK-33635: --- I've changed to SQL because you're not executing a

[jira] [Assigned] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33725: Assignee: Apache Spark (was: L. C. Hsieh) > Upgrade snappy-java to 1.1.8.2 >

[jira] [Commented] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246798#comment-17246798 ] Apache Spark commented on SPARK-33725: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33725: Assignee: L. C. Hsieh (was: Apache Spark) > Upgrade snappy-java to 1.1.8.2 >

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246804#comment-17246804 ] Apache Spark commented on SPARK-32920: -- User 'venkata91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32920: Assignee: (was: Apache Spark) > Add support in Spark driver to coordinate the

[jira] [Updated] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33725: -- Issue Type: Bug (was: Improvement) > Upgrade snappy-java to 1.1.8.2 >

[jira] [Updated] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33725: -- Affects Version/s: 3.1.0 2.4.7 3.0.1 > Upgrade

[jira] [Resolved] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33722. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30688

[jira] [Updated] (SPARK-33261) Allow people to extend the pod feature steps

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33261: -- Affects Version/s: (was: 3.1.0) 3.2.0 > Allow people to extend the

[jira] [Created] (SPARK-33724) Allow decommissioning script location to be configured

2020-12-09 Thread Holden Karau (Jira)
Holden Karau created SPARK-33724: Summary: Allow decommissioning script location to be configured Key: SPARK-33724 URL: https://issues.apache.org/jira/browse/SPARK-33724 Project: Spark Issue

[jira] [Created] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33725: --- Summary: Upgrade snappy-java to 1.1.8.2 Key: SPARK-33725 URL: https://issues.apache.org/jira/browse/SPARK-33725 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246803#comment-17246803 ] Apache Spark commented on SPARK-32920: -- User 'venkata91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32920: Assignee: Apache Spark > Add support in Spark driver to coordinate the finalization of

[jira] [Commented] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246720#comment-17246720 ] Apache Spark commented on SPARK-33722: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33722: Assignee: (was: Apache Spark) > Handle DELETE in ReplaceNullWithFalseInPredicate >

[jira] [Assigned] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33722: Assignee: Apache Spark > Handle DELETE in ReplaceNullWithFalseInPredicate >

[jira] [Updated] (SPARK-33261) Allow people to extend the pod feature steps

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33261: -- Parent: (was: SPARK-33005) Issue Type: Improvement (was: Sub-task) > Allow

[jira] [Commented] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246799#comment-17246799 ] Apache Spark commented on SPARK-33725: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246643#comment-17246643 ] Gabor Somogyi commented on SPARK-33635: --- {quote}I no longer believe this is a true regression in

[jira] [Created] (SPARK-33723) ANSI mode: Casting string to date should throw exception on parse error

2020-12-09 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-33723: -- Summary: ANSI mode: Casting string to date should throw exception on parse error Key: SPARK-33723 URL: https://issues.apache.org/jira/browse/SPARK-33723 Project:

[jira] [Commented] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

2020-12-09 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246678#comment-17246678 ] Thomas Graves commented on SPARK-33614: --- can you add a description here with more details.

[jira] [Assigned] (SPARK-33723) ANSI mode: Casting string to date should throw exception on parse error

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33723: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: Casting string to date should

[jira] [Commented] (SPARK-27554) org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

2020-12-09 Thread Emil Kleszcz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246543#comment-17246543 ] Emil Kleszcz commented on SPARK-27554: -- Dear [~1028344...@qq.com], could you please share what was

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2020-12-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246577#comment-17246577 ] Wenchen Fan commented on SPARK-18105: - Is this still an issue in Spark 3.x? cc [~dongjoon] > LZ4

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246633#comment-17246633 ] Gabor Somogyi commented on SPARK-33635: --- Since you're measuring speed I've ported the Kafka source

[jira] [Comment Edited] (SPARK-23086) Spark SQL cannot support high concurrency for lock in HiveMetastoreCatalog

2020-12-09 Thread Gustavo Martin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246641#comment-17246641 ] Gustavo Martin edited comment on SPARK-23086 at 12/9/20, 4:13 PM: --

[jira] [Assigned] (SPARK-33723) ANSI mode: Casting string to date should throw exception on parse error

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33723: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: Casting string to date should

[jira] [Commented] (SPARK-33723) ANSI mode: Casting string to date should throw exception on parse error

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246675#comment-17246675 ] Apache Spark commented on SPARK-33723: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-33723) ANSI mode: Casting string to date should throw exception on parse error

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246676#comment-17246676 ] Apache Spark commented on SPARK-33723: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246631#comment-17246631 ] Gabor Somogyi commented on SPARK-33635: --- BTW, I'm sure you know but using collect gathers all the

[jira] [Issue Comment Deleted] (SPARK-23086) Spark SQL cannot support high concurrency for lock in HiveMetastoreCatalog

2020-12-09 Thread Gustavo Martin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Martin updated SPARK-23086: --- Comment: was deleted (was: Why was this ticket closed? AFAIK the same synchronization

[jira] [Commented] (SPARK-23086) Spark SQL cannot support high concurrency for lock in HiveMetastoreCatalog

2020-12-09 Thread Gustavo Martin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246641#comment-17246641 ] Gustavo Martin commented on SPARK-23086: AFAIK the same synchronization point exists in Spark

[jira] [Commented] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246617#comment-17246617 ] Anton Okolnychyi commented on SPARK-33722: -- I'll submit a PR soon. > Handle DELETE in

[jira] [Created] (SPARK-33722) Handle DELETE in ReplaceNullWithFalseInPredicate

2020-12-09 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-33722: Summary: Handle DELETE in ReplaceNullWithFalseInPredicate Key: SPARK-33722 URL: https://issues.apache.org/jira/browse/SPARK-33722 Project: Spark

[jira] [Commented] (SPARK-23086) Spark SQL cannot support high concurrency for lock in HiveMetastoreCatalog

2020-12-09 Thread Gustavo Martin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246626#comment-17246626 ] Gustavo Martin commented on SPARK-23086: Why was this ticket closed? AFAIK the same

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246625#comment-17246625 ] Gabor Somogyi commented on SPARK-33635: --- [~david.wyles] try to turn off Kafka consumer caching.

[jira] [Updated] (SPARK-33635) Performance regression in Kafka read

2020-12-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-33635: -- Component/s: (was: Structured Streaming) SQL > Performance regression in

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246833#comment-17246833 ] Dongjoon Hyun commented on SPARK-33713: --- For master and branch-3.1, we don't need a build history.

[jira] [Commented] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246890#comment-17246890 ] Apache Spark commented on SPARK-33725: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246899#comment-17246899 ] Dongjoon Hyun commented on SPARK-33727: --- BTW, K8s IT succeed 4 hours ago in this run. So, I guess

[jira] [Assigned] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33727: Assignee: (was: Apache Spark) > `gpg: keyserver receive failed: No name` during K8s

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246930#comment-17246930 ] Apache Spark commented on SPARK-33727: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246929#comment-17246929 ] Apache Spark commented on SPARK-33727: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246808#comment-17246808 ] Shane Knapp commented on SPARK-33713: - hmm.  so while i heartily endorse this, there will be a

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246850#comment-17246850 ] Shane Knapp commented on SPARK-33713: - got it.  gonna rejigger my rejiggering of the JJB configs. >

[jira] [Created] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33727: - Summary: `gpg: keyserver receive failed: No name` during K8s IT Key: SPARK-33727 URL: https://issues.apache.org/jira/browse/SPARK-33727 Project: Spark

[jira] [Commented] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246891#comment-17246891 ] Apache Spark commented on SPARK-33725: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33728) Improve error messages during K8s integration test failure

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33728: Assignee: (was: Apache Spark) > Improve error messages during K8s integration test

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246810#comment-17246810 ] Shane Knapp commented on SPARK-33713: - fwiw, here's what the new and sexy build names will be:

[jira] [Comment Edited] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246810#comment-17246810 ] Shane Knapp edited comment on SPARK-33713 at 12/9/20, 8:42 PM: --- fwiw,

[jira] [Updated] (SPARK-33726) Duplicate field names causes wrong answers during aggregation

2020-12-09 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-33726: Labels: correctness (was: ) > Duplicate field names causes wrong answers during aggregation >

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246830#comment-17246830 ] Dongjoon Hyun commented on SPARK-33713: --- Oh, this was a request for only `master/branch-3.1`

[jira] [Assigned] (SPARK-33724) Allow decommissioning script location to be configured

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33724: Assignee: Holden Karau (was: Apache Spark) > Allow decommissioning script location to

[jira] [Assigned] (SPARK-33724) Allow decommissioning script location to be configured

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33724: Assignee: Apache Spark (was: Holden Karau) > Allow decommissioning script location to

[jira] [Commented] (SPARK-33724) Allow decommissioning script location to be configured

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246847#comment-17246847 ] Apache Spark commented on SPARK-33724: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33716) Decommissioning Race Condition during Pod Snapshot

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246846#comment-17246846 ] Apache Spark commented on SPARK-33716: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246858#comment-17246858 ] Sean R. Owen commented on SPARK-33713: -- I don't think we care much about build history, yes. Don't

[jira] [Resolved] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33725. --- Fix Version/s: 3.2.0 Resolution: Fixed This is resolved via

[jira] [Updated] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33727: -- Description: K8s IT fails with gpg: keyserver receive failed: No name. This seems to be

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246883#comment-17246883 ] Dongjoon Hyun commented on SPARK-33727: --- Hi, [~shaneknapp]. Could you try `gpg` command in the

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246884#comment-17246884 ] Dongjoon Hyun commented on SPARK-33727: --- cc [~hyukjin.kwon] > `gpg: keyserver receive failed: No

[jira] [Assigned] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33727: Assignee: Apache Spark > `gpg: keyserver receive failed: No name` during K8s IT >

[jira] [Commented] (SPARK-33724) Allow decommissioning script location to be configured

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246848#comment-17246848 ] Apache Spark commented on SPARK-33724: -- User 'holdenk' has created a pull request for this issue:

[jira] [Created] (SPARK-33728) Improve error messages during K8s integration test failure

2020-12-09 Thread Holden Karau (Jira)
Holden Karau created SPARK-33728: Summary: Improve error messages during K8s integration test failure Key: SPARK-33728 URL: https://issues.apache.org/jira/browse/SPARK-33728 Project: Spark

[jira] [Commented] (SPARK-33713) Remove `hive-2.3` post fix at master/branch-3.1 Jenkins job names

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246816#comment-17246816 ] Shane Knapp commented on SPARK-33713: - also, let me think about other sneaky ways of making this

[jira] [Comment Edited] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246894#comment-17246894 ] Shane Knapp edited comment on SPARK-33727 at 12/9/20, 11:30 PM: ok, this

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246894#comment-17246894 ] Shane Knapp commented on SPARK-33727: - ok, this is not a failure on the jenkins worker – it's

[jira] [Commented] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2020-12-09 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246923#comment-17246923 ] Holden Karau commented on SPARK-33727: -- So I've seen the keys.gnupg.net key server be flaky on my

[jira] [Commented] (SPARK-33728) Improve error messages during K8s integration test failure

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246924#comment-17246924 ] Apache Spark commented on SPARK-33728: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33728) Improve error messages during K8s integration test failure

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246925#comment-17246925 ] Apache Spark commented on SPARK-33728: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33728) Improve error messages during K8s integration test failure

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33728: Assignee: Apache Spark > Improve error messages during K8s integration test failure >

[jira] [Commented] (SPARK-33725) Upgrade snappy-java to 1.1.8.2

2020-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246931#comment-17246931 ] Apache Spark commented on SPARK-33725: -- User 'viirya' has created a pull request for this issue:

  1   2   >