[jira] [Updated] (SPARK-29497) Cannot assign instance of java.lang.invoke.SerializedLambda to field

2020-12-10 Thread Rob Russo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Russo updated SPARK-29497: -- Affects Version/s: 3.0.1 > Cannot assign instance of java.lang.invoke.SerializedLambda to field >

[jira] [Commented] (SPARK-33734) Spark Core

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247721#comment-17247721 ] Hyukjin Kwon commented on SPARK-33734: -- [~Bhure], please fix the JIRA title to reflect the issue

[jira] [Resolved] (SPARK-33734) Spark Core

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33734. -- Resolution: Incomplete > Spark Core > --- > > Key: SPARK-33734 >

[jira] [Assigned] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33752: Assignee: Apache Spark > Avoid the getSimpleMessage of AnalysisException adds semicolon

[jira] [Assigned] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33752: Assignee: (was: Apache Spark) > Avoid the getSimpleMessage of AnalysisException adds

[jira] [Commented] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247710#comment-17247710 ] Apache Spark commented on SPARK-33752: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly

2020-12-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33752: --- Summary: Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly (was: Avoid the

[jira] [Created] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException add semicolon repeatedly

2020-12-10 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-33752: -- Summary: Avoid the getSimpleMessage of AnalysisException add semicolon repeatedly Key: SPARK-33752 URL: https://issues.apache.org/jira/browse/SPARK-33752 Project: Spark

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-10 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247693#comment-17247693 ] Darcy Shen commented on SPARK-33044:

[jira] [Assigned] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33750: - Assignee: Dongjoon Hyun > Use `hadoop-3.2` distribution in

[jira] [Resolved] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33750. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30722

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-10 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247687#comment-17247687 ] Darcy Shen commented on SPARK-33044: Awesome! > Add a Jenkins build and test job for Scala 2.13 >

[jira] [Resolved] (SPARK-33527) Extend the function of decode so as consistent with mainstream databases

2020-12-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33527. - Fix Version/s: 3.2.0 Assignee: jiaan.geng Resolution: Fixed > Extend the

[jira] [Resolved] (SPARK-32910) Remove UninterruptibleThread usage from KafkaOffsetReader

2020-12-10 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-32910. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30668

[jira] [Assigned] (SPARK-32910) Remove UninterruptibleThread usage from KafkaOffsetReader

2020-12-10 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-32910: Assignee: Gabor Somogyi > Remove UninterruptibleThread usage from KafkaOffsetReader >

[jira] [Commented] (SPARK-33751) Migrate ALTER VIEW ... AS command to the new resolution framework

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247645#comment-17247645 ] Apache Spark commented on SPARK-33751: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33751) Migrate ALTER VIEW ... AS command to the new resolution framework

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33751: Assignee: (was: Apache Spark) > Migrate ALTER VIEW ... AS command to the new

[jira] [Commented] (SPARK-33751) Migrate ALTER VIEW ... AS command to the new resolution framework

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247644#comment-17247644 ] Apache Spark commented on SPARK-33751: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33751) Migrate ALTER VIEW ... AS command to the new resolution framework

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33751: Assignee: Apache Spark > Migrate ALTER VIEW ... AS command to the new resolution

[jira] [Assigned] (SPARK-33749) Exclude target directory in pycodestyle and flake8

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33749: Assignee: Hyukjin Kwon > Exclude target directory in pycodestyle and flake8 >

[jira] [Resolved] (SPARK-33749) Exclude target directory in pycodestyle and flake8

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33749. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Assigned] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33750: Assignee: (was: Apache Spark) > Use `hadoop-3.2` distribution in

[jira] [Commented] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247642#comment-17247642 ] Apache Spark commented on SPARK-33750: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33750: Assignee: Apache Spark > Use `hadoop-3.2` distribution in

[jira] [Created] (SPARK-33751) Migrate ALTER VIEW ... AS command to the new resolution framework

2020-12-10 Thread Terry Kim (Jira)
Terry Kim created SPARK-33751: - Summary: Migrate ALTER VIEW ... AS command to the new resolution framework Key: SPARK-33751 URL: https://issues.apache.org/jira/browse/SPARK-33751 Project: Spark

[jira] [Updated] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33750: -- Priority: Minor (was: Major) > Use `hadoop-3.2` distribution in

[jira] [Created] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33750: - Summary: Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite Key: SPARK-33750 URL: https://issues.apache.org/jira/browse/SPARK-33750 Project: Spark

[jira] [Updated] (SPARK-33750) Use `hadoop-3.2` distribution in HiveExternalCatalogVersionsSuite

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33750: -- Issue Type: Improvement (was: Task) > Use `hadoop-3.2` distribution in

[jira] [Resolved] (SPARK-30946) FileStreamSourceLog/FileStreamSinkLog: leverage UnsafeRow type to serialize/deserialize entry

2020-12-10 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-30946. -- Resolution: Won't Fix Closing as I don't see any support on this and I'm now in favor of one

[jira] [Closed] (SPARK-30946) FileStreamSourceLog/FileStreamSinkLog: leverage UnsafeRow type to serialize/deserialize entry

2020-12-10 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim closed SPARK-30946. > FileStreamSourceLog/FileStreamSinkLog: leverage UnsafeRow type to > serialize/deserialize entry >

[jira] [Updated] (SPARK-33749) Exclude target directory in pycodestyle and flake8

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33749: - Summary: Exclude target directory in pycodestyle and flake8 (was: Exclude more permissively in

[jira] [Assigned] (SPARK-33738) Add timeout info for ShuffleBlockFetcherIterator when get remote block

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33738: Assignee: (was: Apache Spark) > Add timeout info for ShuffleBlockFetcherIterator

[jira] [Assigned] (SPARK-33738) Add timeout info for ShuffleBlockFetcherIterator when get remote block

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33738: Assignee: Apache Spark > Add timeout info for ShuffleBlockFetcherIterator when get

[jira] [Commented] (SPARK-33738) Add timeout info for ShuffleBlockFetcherIterator when get remote block

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247623#comment-17247623 ] Apache Spark commented on SPARK-33738: -- User 'southernriver' has created a pull request for this

[jira] [Commented] (SPARK-33740) hadoop configs in hive-site.xml can overrides pre-existing hadoop ones

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247616#comment-17247616 ] Apache Spark commented on SPARK-33740: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33295) Upgrade ORC to 1.6.6

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33295. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30715

[jira] [Commented] (SPARK-33748) Respect PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247610#comment-17247610 ] Hyukjin Kwon commented on SPARK-33748: -- cc [~dongjoon] and [~eje] FYI > Respect PYSPARK_PYTHON and

[jira] [Commented] (SPARK-33749) Exclude more permissively in pycodestyle and flake8

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247609#comment-17247609 ] Apache Spark commented on SPARK-33749: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33749) Exclude more permissively in pycodestyle and flake8

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247608#comment-17247608 ] Apache Spark commented on SPARK-33749: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33749) Exclude more permissively in pycodestyle and flake8

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33749: Assignee: Apache Spark > Exclude more permissively in pycodestyle and flake8 >

[jira] [Assigned] (SPARK-33749) Exclude more permissively in pycodestyle and flake8

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33749: Assignee: (was: Apache Spark) > Exclude more permissively in pycodestyle and flake8

[jira] [Updated] (SPARK-33749) Exclude more permissively in pycodestyle and flake8

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33749: - Summary: Exclude more permissively in pycodestyle and flake8 (was: Exclude cloudpickle and

[jira] [Created] (SPARK-33749) Exclude cloudpickle and conf.py more permissively in pycodestyle and flake8

2020-12-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33749: Summary: Exclude cloudpickle and conf.py more permissively in pycodestyle and flake8 Key: SPARK-33749 URL: https://issues.apache.org/jira/browse/SPARK-33749 Project:

[jira] [Created] (SPARK-33748) Respect PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON

2020-12-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33748: Summary: Respect PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON Key: SPARK-33748 URL: https://issues.apache.org/jira/browse/SPARK-33748 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33599) Group exception messages in catalyst/analysis

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247606#comment-17247606 ] Apache Spark commented on SPARK-33599: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33599) Group exception messages in catalyst/analysis

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33599: Assignee: Apache Spark > Group exception messages in catalyst/analysis >

[jira] [Assigned] (SPARK-33599) Group exception messages in catalyst/analysis

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33599: Assignee: (was: Apache Spark) > Group exception messages in catalyst/analysis >

[jira] [Commented] (SPARK-33599) Group exception messages in catalyst/analysis

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247605#comment-17247605 ] Apache Spark commented on SPARK-33599: -- User 'beliefer' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33742. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30711

[jira] [Assigned] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33742: - Assignee: Maxim Gekk > Throw PartitionsAlreadyExistException from >

[jira] [Assigned] (SPARK-33747) Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33747: Assignee: Apache Spark > Avoid calling unregisterMapOutput when the map stage is being

[jira] [Assigned] (SPARK-33747) Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33747: Assignee: (was: Apache Spark) > Avoid calling unregisterMapOutput when the map stage

[jira] [Commented] (SPARK-33747) Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247592#comment-17247592 ] Apache Spark commented on SPARK-33747: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247576#comment-17247576 ] Hyukjin Kwon commented on SPARK-33730: -- I think we can change in 3.1 too since this is essentially

[jira] [Commented] (SPARK-33295) Upgrade ORC to 1.6.6

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247567#comment-17247567 ] Apache Spark commented on SPARK-33295: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33295) Upgrade ORC to 1.6.6

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33295: Assignee: Apache Spark (was: Dongjoon Hyun) > Upgrade ORC to 1.6.6 >

[jira] [Commented] (SPARK-33295) Upgrade ORC to 1.6.6

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247568#comment-17247568 ] Apache Spark commented on SPARK-33295: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33295) Upgrade ORC to 1.6.6

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33295: Assignee: Dongjoon Hyun (was: Apache Spark) > Upgrade ORC to 1.6.6 >

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-10 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247565#comment-17247565 ] Maciej Szymkiewicz commented on SPARK-33730: [~hyukjin.kwon] We're going to target this for

[jira] [Updated] (SPARK-33747) Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-10 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-33747: - Fix Version/s: 2.4.5 3.0.1 > Avoid calling unregisterMapOutput when the map stage is

[jira] [Created] (SPARK-33747) Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-10 Thread weixiuli (Jira)
weixiuli created SPARK-33747: Summary: Avoid calling unregisterMapOutput when the map stage is being rerunning. Key: SPARK-33747 URL: https://issues.apache.org/jira/browse/SPARK-33747 Project: Spark

[jira] [Commented] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-10 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247557#comment-17247557 ] Shane Knapp commented on SPARK-33746: - wow.  this was a rabbit hole of epic proportions, and after

[jira] [Commented] (SPARK-32398) Upgrade to scalatest 3.2.0 for Scala 2.13.3 compatibility

2020-12-10 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247545#comment-17247545 ] Sean R. Owen commented on SPARK-32398: -- Yes it did. It's just how it goes, I think. I continue to

[jira] [Commented] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-10 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247523#comment-17247523 ] Shane Knapp commented on SPARK-33746: - ...and here's what's actually going on:   {code:java} 

[jira] [Commented] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-10 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247521#comment-17247521 ] Shane Knapp commented on SPARK-33746: - yah, noticed this yesterday and took the worker offline... 

[jira] [Commented] (SPARK-33739) Jobs committed through the S3A Magic committer don't report the bytes written

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247497#comment-17247497 ] Apache Spark commented on SPARK-33739: -- User 'steveloughran' has created a pull request for this

[jira] [Assigned] (SPARK-33739) Jobs committed through the S3A Magic committer don't report the bytes written

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33739: Assignee: (was: Apache Spark) > Jobs committed through the S3A Magic committer don't

[jira] [Assigned] (SPARK-33739) Jobs committed through the S3A Magic committer don't report the bytes written

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33739: Assignee: Apache Spark > Jobs committed through the S3A Magic committer don't report the

[jira] [Commented] (SPARK-33739) Jobs committed through the S3A Magic committer don't report the bytes written

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247496#comment-17247496 ] Apache Spark commented on SPARK-33739: -- User 'steveloughran' has created a pull request for this

[jira] [Created] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-10 Thread Holden Karau (Jira)
Holden Karau created SPARK-33746: Summary: Minikube is failing to start on research-jenkins-worker-05 Key: SPARK-33746 URL: https://issues.apache.org/jira/browse/SPARK-33746 Project: Spark

[jira] [Created] (SPARK-33745) SparkR docker install requires LaTeX packages

2020-12-10 Thread Holden Karau (Jira)
Holden Karau created SPARK-33745: Summary: SparkR docker install requires LaTeX packages Key: SPARK-33745 URL: https://issues.apache.org/jira/browse/SPARK-33745 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32398) Upgrade to scalatest 3.2.0 for Scala 2.13.3 compatibility

2020-12-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247484#comment-17247484 ] Steve Loughran commented on SPARK-32398: Did scalatest really rename funsuite? That's just

[jira] [Commented] (SPARK-29497) Cannot assign instance of java.lang.invoke.SerializedLambda to field

2020-12-10 Thread Nathan Kronenfeld (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247482#comment-17247482 ] Nathan Kronenfeld commented on SPARK-29497: --- This is happening to us too.  We haven't figured

[jira] [Updated] (SPARK-33744) Canonicalization error in SortAggregate

2020-12-10 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-33744: --- Affects Version/s: 3.0.1 > Canonicalization error in SortAggregate >

[jira] [Updated] (SPARK-33744) Canonicalization error in SortAggregate

2020-12-10 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-33744: --- Affects Version/s: (was: 3.1.0) > Canonicalization error in SortAggregate >

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Created] (SPARK-33744) Canonicalization error in SortAggregate

2020-12-10 Thread Andy Grove (Jira)
Andy Grove created SPARK-33744: -- Summary: Canonicalization error in SortAggregate Key: SPARK-33744 URL: https://issues.apache.org/jira/browse/SPARK-33744 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Commented] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247433#comment-17247433 ] Apache Spark commented on SPARK-33743: -- User 'luxu1-ms' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33743: Assignee: (was: Apache Spark) > Change datatype mapping in JDBC mssqldialect:

[jira] [Assigned] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33743: Assignee: Apache Spark > Change datatype mapping in JDBC mssqldialect: DATETIME to

[jira] [Commented] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247434#comment-17247434 ] Apache Spark commented on SPARK-33743: -- User 'luxu1-ms' has created a pull request for this issue:

[jira] [Created] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2020-12-10 Thread Lu Xu (Jira)
Lu Xu created SPARK-33743: - Summary: Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2 Key: SPARK-33743 URL: https://issues.apache.org/jira/browse/SPARK-33743 Project: Spark Issue

[jira] [Commented] (SPARK-33737) Use an Informer+Lister API in the ExecutorPodWatcher

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247429#comment-17247429 ] Dongjoon Hyun commented on SPARK-33737: --- Although it sounds like a big change, yes, we may

[jira] [Updated] (SPARK-33737) Use an Informer+Lister API in the ExecutorPodWatcher

2020-12-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33737: -- Affects Version/s: (was: 3.0.2) 3.2.0 > Use an Informer+Lister API

[jira] [Commented] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247390#comment-17247390 ] Apache Spark commented on SPARK-33742: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247389#comment-17247389 ] Apache Spark commented on SPARK-33742: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33742: Assignee: (was: Apache Spark) > Throw PartitionsAlreadyExistException from >

[jira] [Assigned] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33742: Assignee: Apache Spark > Throw PartitionsAlreadyExistException from >

[jira] [Created] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33742: -- Summary: Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions() Key: SPARK-33742 URL: https://issues.apache.org/jira/browse/SPARK-33742

[jira] [Assigned] (SPARK-33741) Add minimum threshold speculation config

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33741: Assignee: (was: Apache Spark) > Add minimum threshold speculation config >

[jira] [Assigned] (SPARK-33741) Add minimum threshold speculation config

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33741: Assignee: Apache Spark > Add minimum threshold speculation config >

[jira] [Commented] (SPARK-33741) Add minimum threshold speculation config

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247378#comment-17247378 ] Apache Spark commented on SPARK-33741: -- User 'redsanket' has created a pull request for this issue:

[jira] [Created] (SPARK-33741) Add minimum threshold speculation config

2020-12-10 Thread Sanket Reddy (Jira)
Sanket Reddy created SPARK-33741: Summary: Add minimum threshold speculation config Key: SPARK-33741 URL: https://issues.apache.org/jira/browse/SPARK-33741 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32110) -0.0 vs 0.0 is inconsistent

2020-12-10 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247362#comment-17247362 ] Robert Joseph Evans commented on SPARK-32110: - Thanks [~tanelk] then resolving this is fine.

[jira] [Commented] (SPARK-32110) -0.0 vs 0.0 is inconsistent

2020-12-10 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247357#comment-17247357 ] Tanel Kiis commented on SPARK-32110: [~cloud_fan] fixed issue I mentioned in the first comment in

[jira] [Assigned] (SPARK-22769) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22769: Assignee: (was: Apache Spark) > When driver stopping, there is errors: "Could not

[jira] [Assigned] (SPARK-22769) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22769: Assignee: Apache Spark > When driver stopping, there is errors: "Could not find >

[jira] [Assigned] (SPARK-33740) hadoop configs in hive-site.xml can overrides pre-existing hadoop ones

2020-12-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33740: Assignee: (was: Apache Spark) > hadoop configs in hive-site.xml can overrides

  1   2   >