[jira] [Assigned] (SPARK-33765) Migrate UNCACHE TABLE to new resolution framework

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33765: Assignee: (was: Apache Spark) > Migrate UNCACHE TABLE to new resolution framework > -

[jira] [Commented] (SPARK-33765) Migrate UNCACHE TABLE to new resolution framework

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248283#comment-17248283 ] Apache Spark commented on SPARK-33765: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-33765) Migrate UNCACHE TABLE to new resolution framework

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33765: Assignee: Apache Spark > Migrate UNCACHE TABLE to new resolution framework >

[jira] [Created] (SPARK-33765) Migrate UNCACHE TABLE to new resolution framework

2020-12-11 Thread Terry Kim (Jira)
Terry Kim created SPARK-33765: - Summary: Migrate UNCACHE TABLE to new resolution framework Key: SPARK-33765 URL: https://issues.apache.org/jira/browse/SPARK-33765 Project: Spark Issue Type: Sub-t

[jira] [Commented] (SPARK-33762) Bump commons-codec to latest version.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248273#comment-17248273 ] Apache Spark commented on SPARK-33762: -- User 'n-marion' has created a pull request

[jira] [Assigned] (SPARK-33762) Bump commons-codec to latest version.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33762: Assignee: Apache Spark > Bump commons-codec to latest version. > ---

[jira] [Assigned] (SPARK-33762) Bump commons-codec to latest version.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33762: Assignee: (was: Apache Spark) > Bump commons-codec to latest version. >

[jira] [Commented] (SPARK-32526) Let sql/catalyst module tests pass for Scala 2.13

2020-12-11 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248259#comment-17248259 ] Darcy Shen commented on SPARK-32526: ExpressionEncoderSuite still fails to work: ht

[jira] [Assigned] (SPARK-33653) DSv2: REFRESH TABLE should recache the table itself

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33653: Assignee: (was: Apache Spark) > DSv2: REFRESH TABLE should recache the table itself >

[jira] [Assigned] (SPARK-33653) DSv2: REFRESH TABLE should recache the table itself

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33653: Assignee: Apache Spark > DSv2: REFRESH TABLE should recache the table itself > --

[jira] [Commented] (SPARK-33653) DSv2: REFRESH TABLE should recache the table itself

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248257#comment-17248257 ] Apache Spark commented on SPARK-33653: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248241#comment-17248241 ] Xinli Shang commented on SPARK-26345: - The Presto and Iceberg effort are not tied to

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread James R. Taylor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248236#comment-17248236 ] James R. Taylor commented on SPARK-26345: - Thanks for the update, [~sha...@uber.

[jira] [Resolved] (SPARK-33729) When refreshing cache, Spark should not use cached plan when recaching data

2020-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33729. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30699 [https://

[jira] [Assigned] (SPARK-33729) When refreshing cache, Spark should not use cached plan when recaching data

2020-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33729: - Assignee: Chao Sun > When refreshing cache, Spark should not use cached plan when recac

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248231#comment-17248231 ] Xinli Shang commented on SPARK-26345: - For the performance, there is an Eng Blog I f

[jira] [Assigned] (SPARK-33764) Make state store maintenance interval as SQL config

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33764: Assignee: L. C. Hsieh (was: Apache Spark) > Make state store maintenance interval as SQL

[jira] [Assigned] (SPARK-33764) Make state store maintenance interval as SQL config

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33764: Assignee: Apache Spark (was: L. C. Hsieh) > Make state store maintenance interval as SQL

[jira] [Commented] (SPARK-33764) Make state store maintenance interval as SQL config

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248217#comment-17248217 ] Apache Spark commented on SPARK-33764: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread James R. Taylor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248216#comment-17248216 ] James R. Taylor commented on SPARK-26345: - Any updates on this issue, [~zi]? Wou

[jira] [Created] (SPARK-33764) Make state store maintenance interval as SQL config

2020-12-11 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33764: --- Summary: Make state store maintenance interval as SQL config Key: SPARK-33764 URL: https://issues.apache.org/jira/browse/SPARK-33764 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.

2020-12-11 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248168#comment-17248168 ] Attila Zsolt Piros commented on SPARK-32617: I will have PR for this soon.

[jira] [Commented] (SPARK-33731) Standardize exception types

2020-12-11 Thread Shril Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248156#comment-17248156 ] Shril Kumar commented on SPARK-33731: - Most welcome [~hyukjin.kwon]. :) > Standardi

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-11 Thread Shril Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248155#comment-17248155 ] Shril Kumar commented on SPARK-33730: - [~hyukjin.kwon], thank you for your response.

[jira] [Created] (SPARK-33763) Add metrics for better tracking of dynamic allocation

2020-12-11 Thread Holden Karau (Jira)
Holden Karau created SPARK-33763: Summary: Add metrics for better tracking of dynamic allocation Key: SPARK-33763 URL: https://issues.apache.org/jira/browse/SPARK-33763 Project: Spark Issue T

[jira] [Commented] (SPARK-33752) Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248121#comment-17248121 ] Apache Spark commented on SPARK-33752: -- User 'n-marion' has created a pull request

[jira] [Updated] (SPARK-33762) Bump commons-codec to latest version.

2020-12-11 Thread Nicholas Marion (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Marion updated SPARK-33762: Description: Currently Spark pulls in commons-codec version 2.10 which was released 6 year

[jira] [Created] (SPARK-33762) Bump commons-codec to latest version.

2020-12-11 Thread Nicholas Marion (Jira)
Nicholas Marion created SPARK-33762: --- Summary: Bump commons-codec to latest version. Key: SPARK-33762 URL: https://issues.apache.org/jira/browse/SPARK-33762 Project: Spark Issue Type: Depe

[jira] [Commented] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248112#comment-17248112 ] Apache Spark commented on SPARK-22256: -- User 'dmcwhorter' has created a pull reques

[jira] [Commented] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248110#comment-17248110 ] Apache Spark commented on SPARK-22256: -- User 'dmcwhorter' has created a pull reques

[jira] [Created] (SPARK-33761) [k8s] Support fetching driver and executor pod templates from HCFS

2020-12-11 Thread Xuzhou Yin (Jira)
Xuzhou Yin created SPARK-33761: -- Summary: [k8s] Support fetching driver and executor pod templates from HCFS Key: SPARK-33761 URL: https://issues.apache.org/jira/browse/SPARK-33761 Project: Spark

[jira] [Commented] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-11 Thread Nicholas Marion (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248107#comment-17248107 ] Nicholas Marion commented on SPARK-33695: - [~dongjoon] , As a security issue, w

[jira] [Resolved] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

2020-12-11 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-33576. -- Resolution: Duplicate Going to resolve as a duplicate, but please reopen if you find it is dif

[jira] [Updated] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33742: -- Fix Version/s: (was: 3.2.0) 3.1.0 3.0.2

[jira] [Commented] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

2020-12-11 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248064#comment-17248064 ] Bryan Cutler commented on SPARK-33576: -- [~darshats] I believe the only current work

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248051#comment-17248051 ] Maciej Szymkiewicz edited comment on SPARK-33730 at 12/11/20, 5:30 PM: ---

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248051#comment-17248051 ] Maciej Szymkiewicz edited comment on SPARK-33730 at 12/11/20, 5:29 PM: ---

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248051#comment-17248051 ] Maciej Szymkiewicz commented on SPARK-33730: {quote}Eh, I think Maciej Szymk

[jira] [Created] (SPARK-33760) Extend Dynamic Partition Pruning Support to DataSources

2020-12-11 Thread Anoop Johnson (Jira)
Anoop Johnson created SPARK-33760: - Summary: Extend Dynamic Partition Pruning Support to DataSources Key: SPARK-33760 URL: https://issues.apache.org/jira/browse/SPARK-33760 Project: Spark Iss

[jira] [Commented] (SPARK-32398) Upgrade to scalatest 3.2.0 for Scala 2.13.3 compatibility

2020-12-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248021#comment-17248021 ] Steve Loughran commented on SPARK-32398: doesn't work with old versions though.

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-11 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248013#comment-17248013 ] Shane Knapp commented on SPARK-33044: - done > Add a Jenkins build and test job for

[jira] [Resolved] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-11 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp resolved SPARK-33746. - Resolution: Fixed > Minikube is failing to start on research-jenkins-worker-05 > ---

[jira] [Assigned] (SPARK-33759) docker entrypoint should using `spark-class` for spark executor

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33759: Assignee: (was: Apache Spark) > docker entrypoint should using `spark-class` for spar

[jira] [Commented] (SPARK-33759) docker entrypoint should using `spark-class` for spark executor

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248004#comment-17248004 ] Apache Spark commented on SPARK-33759: -- User 'dungdm93' has created a pull request

[jira] [Commented] (SPARK-33759) docker entrypoint should using `spark-class` for spark executor

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248006#comment-17248006 ] Apache Spark commented on SPARK-33759: -- User 'dungdm93' has created a pull request

[jira] [Assigned] (SPARK-33759) docker entrypoint should using `spark-class` for spark executor

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33759: Assignee: Apache Spark > docker entrypoint should using `spark-class` for spark executor

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-11 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248003#comment-17248003 ] Shane Knapp commented on SPARK-33044: - ah, ok...  i thought i just needed to run `./

[jira] [Updated] (SPARK-33759) docker entrypoint should using `spark-class` for spark executor

2020-12-11 Thread Dung Dang Minh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dung Dang Minh updated SPARK-33759: --- Summary: docker entrypoint should using `spark-class` for spark executor (was: spark execut

[jira] [Created] (SPARK-33759) spark executor should using `spark-class` in docker entrypoint

2020-12-11 Thread Dung Dang Minh (Jira)
Dung Dang Minh created SPARK-33759: -- Summary: spark executor should using `spark-class` in docker entrypoint Key: SPARK-33759 URL: https://issues.apache.org/jira/browse/SPARK-33759 Project: Spark

[jira] [Commented] (SPARK-33746) Minikube is failing to start on research-jenkins-worker-05

2020-12-11 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247998#comment-17247998 ] Shane Knapp commented on SPARK-33746: - k8s master build passed: [https://amplab.cs.

[jira] [Resolved] (SPARK-33757) Fix the R dependencies build error on GitHub Actions and AppVeyor

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33757. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution: Fixed

[jira] [Assigned] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33526: Assignee: ulysses you > Add config to control if cancel invoke interrupt task on thriftse

[jira] [Resolved] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33526. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30481 [https://gi

[jira] [Created] (SPARK-33758) Prune unnecessary output partitioning when the attribute is not part of output.

2020-12-11 Thread Prakhar Jain (Jira)
Prakhar Jain created SPARK-33758: Summary: Prune unnecessary output partitioning when the attribute is not part of output. Key: SPARK-33758 URL: https://issues.apache.org/jira/browse/SPARK-33758 Proje

[jira] [Updated] (SPARK-33757) Fix the R dependencies build error on GitHub Actions and AppVeyor

2020-12-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-33757: --- Description: R dependencies build error happens now. The reason seems that usethis package i

[jira] [Updated] (SPARK-33757) Fix the R dependencies build error on GitHub Actions and AppVeyor

2020-12-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-33757: --- Summary: Fix the R dependencies build error on GitHub Actions and AppVeyor (was: Fix the R

[jira] [Commented] (SPARK-33757) Fix the R dependencies build error on GitHub Actions

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247930#comment-17247930 ] Apache Spark commented on SPARK-33757: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-33757) Fix the R dependencies build error on GitHub Actions

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33757: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the R dependencies build error on Git

[jira] [Assigned] (SPARK-33757) Fix the R dependencies build error on GitHub Actions

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33757: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the R dependencies build error on Git

[jira] [Created] (SPARK-33757) Fix the R dependencies build error on GitHub Actions

2020-12-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-33757: -- Summary: Fix the R dependencies build error on GitHub Actions Key: SPARK-33757 URL: https://issues.apache.org/jira/browse/SPARK-33757 Project: Spark Issu

[jira] [Commented] (SPARK-33617) Avoid generating small files for INSERT INTO TABLE from VALUES

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247926#comment-17247926 ] Apache Spark commented on SPARK-33617: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-33617) Avoid generating small files for INSERT INTO TABLE from VALUES

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247925#comment-17247925 ] Apache Spark commented on SPARK-33617: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-33731) Standardize exception types

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247923#comment-17247923 ] Hyukjin Kwon commented on SPARK-33731: -- Sure, thanks [~shril]. I have some thoughts

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247922#comment-17247922 ] Hyukjin Kwon edited comment on SPARK-33730 at 12/11/20, 1:17 PM: -

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247922#comment-17247922 ] Hyukjin Kwon commented on SPARK-33730: -- Eh, I think [~zero323] is already working o

[jira] [Assigned] (SPARK-33748) Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33748: Assignee: (was: Apache Spark) > Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON envi

[jira] [Commented] (SPARK-33748) Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247914#comment-17247914 ] Apache Spark commented on SPARK-33748: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-33748) Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33748: Assignee: Apache Spark > Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment var

[jira] [Updated] (SPARK-33748) Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables

2020-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33748: - Summary: Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables (was: Respect P

[jira] [Assigned] (SPARK-33706) Require fully specified partition identifier in partitionExists()

2020-12-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33706: --- Assignee: Maxim Gekk > Require fully specified partition identifier in partitionExists() >

[jira] [Resolved] (SPARK-33706) Require fully specified partition identifier in partitionExists()

2020-12-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33706. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30667 [https://gith

[jira] [Assigned] (SPARK-33755) Allow creating orc table when row format separator is defined

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33755: Assignee: (was: Apache Spark) > Allow creating orc table when row format separator is

[jira] [Commented] (SPARK-33755) Allow creating orc table when row format separator is defined

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247894#comment-17247894 ] Apache Spark commented on SPARK-33755: -- User 'StefanXiepj' has created a pull reque

[jira] [Assigned] (SPARK-33755) Allow creating orc table when row format separator is defined

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33755: Assignee: (was: Apache Spark) > Allow creating orc table when row format separator is

[jira] [Assigned] (SPARK-33755) Allow creating orc table when row format separator is defined

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33755: Assignee: Apache Spark > Allow creating orc table when row format separator is defined >

[jira] [Assigned] (SPARK-33654) Migrate CACHE TABLE to new resolution framework

2020-12-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33654: --- Assignee: Terry Kim > Migrate CACHE TABLE to new resolution framework > ---

[jira] [Resolved] (SPARK-33654) Migrate CACHE TABLE to new resolution framework

2020-12-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33654. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30598 [https://gith

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247879#comment-17247879 ] David Wyles commented on SPARK-33635: - [~gsomogyi]  "try to turn off Kafka consumer

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247878#comment-17247878 ] David Wyles commented on SPARK-33635: - "Since you're measuring speed I've ported the

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:12 PM: -

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:11 PM: -

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:11 PM: -

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-11 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247875#comment-17247875 ] David Wyles commented on SPARK-33635: - I'll give all those a go and get back to you.

[jira] [Commented] (SPARK-33691) Support partition filters in ALTER TABLE DROP PARTITION

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247870#comment-17247870 ] Apache Spark commented on SPARK-33691: -- User 'StefanXiepj' has created a pull reque

[jira] [Commented] (SPARK-33691) Support partition filters in ALTER TABLE DROP PARTITION

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247871#comment-17247871 ] Apache Spark commented on SPARK-33691: -- User 'StefanXiepj' has created a pull reque

[jira] [Commented] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247866#comment-17247866 ] Apache Spark commented on SPARK-33742: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247850#comment-17247850 ] Apache Spark commented on SPARK-33742: -- User 'MaxGekk' has created a pull request f

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Shril Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247827#comment-17247827 ] Shril Kumar edited comment on SPARK-33730 at 12/11/20, 10:40 AM: -

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-11 Thread Shril Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247827#comment-17247827 ] Shril Kumar commented on SPARK-33730: - I was inspecting the use of the print stateme

[jira] [Commented] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247822#comment-17247822 ] Apache Spark commented on SPARK-33742: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33542) Group exception messages in catalyst/catalog

2020-12-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247818#comment-17247818 ] jiaan.geng commented on SPARK-33542: I'm working on. > Group exception messages in

[jira] [Updated] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-12-11 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-28367: -- Fix Version/s: 3.1.0 > Kafka connector infinite wait because metadata never updated >

[jira] [Closed] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-12-11 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi closed SPARK-28367. - > Kafka connector infinite wait because metadata never updated > ---

[jira] [Resolved] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-12-11 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-28367. --- Resolution: Fixed > Kafka connector infinite wait because metadata never updated > -

[jira] [Commented] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-12-11 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247813#comment-17247813 ] Gabor Somogyi commented on SPARK-28367: --- The issue solved in subtasks so closing t

[jira] [Assigned] (SPARK-33756) BytesToBytesMap's iterator hasNext method should be idempotent.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33756: Assignee: Apache Spark > BytesToBytesMap's iterator hasNext method should be idempotent.

[jira] [Commented] (SPARK-33756) BytesToBytesMap's iterator hasNext method should be idempotent.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247802#comment-17247802 ] Apache Spark commented on SPARK-33756: -- User 'advancedxy' has created a pull reques

[jira] [Assigned] (SPARK-33756) BytesToBytesMap's iterator hasNext method should be idempotent.

2020-12-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33756: Assignee: (was: Apache Spark) > BytesToBytesMap's iterator hasNext method should be i

[jira] [Resolved] (SPARK-33754) Update kubernetes/integration-tests/README.md to follow the default Hadoop profile updated

2020-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33754. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30726 [https://

[jira] [Updated] (SPARK-33755) Allow creating orc table when row format separator is defined

2020-12-11 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiepengjie updated SPARK-33755: --- Description: When creating table like this: {code:java} create table test_orc(c1 string) row format

  1   2   >