[jira] [Created] (SPARK-26533) Support query auto cancel on thriftserver

zhoukang created SPARK-26533: Summary: Support query auto cancel on thriftserver Key: SPARK-26533 URL: https://issues.apache.org/jira/browse/SPARK-26533 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733848#comment-16733848 ] Hyukjin Kwon commented on SPARK-26126: -- There are sometimes some reasons and it nee

[jira] [Commented] (SPARK-26339) Behavior of reading files that start with underscore is confusing

[ https://issues.apache.org/jira/browse/SPARK-26339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733844#comment-16733844 ] Apache Spark commented on SPARK-26339: -- User 'KeiichiHirobe' has created a pull req

[jira] [Commented] (SPARK-26339) Behavior of reading files that start with underscore is confusing

[ https://issues.apache.org/jira/browse/SPARK-26339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733846#comment-16733846 ] Apache Spark commented on SPARK-26339: -- User 'KeiichiHirobe' has created a pull req

[jira] [Comment Edited] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733836#comment-16733836 ] liupengcheng edited comment on SPARK-26126 at 1/4/19 5:41 AM:

[jira] [Commented] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733836#comment-16733836 ] liupengcheng commented on SPARK-26126: -- [~hyukjin.kwon] Yes, there is no actual pro

[jira] [Updated] (SPARK-26532) repartitionByRange reads source files twice

[ https://issues.apache.org/jira/browse/SPARK-26532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dias updated SPARK-26532: -- Component/s: SQL > repartitionByRange reads source files twice > -

[jira] [Updated] (SPARK-26532) repartitionByRange reads source files twice

[ https://issues.apache.org/jira/browse/SPARK-26532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dias updated SPARK-26532: -- Summary: repartitionByRange reads source files twice (was: repartitionByRange strategy reads source f

[jira] [Created] (SPARK-26532) repartitionByRange strategy reads source files twice

Mike Dias created SPARK-26532: - Summary: repartitionByRange strategy reads source files twice Key: SPARK-26532 URL: https://issues.apache.org/jira/browse/SPARK-26532 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-26532) repartitionByRange strategy reads source files twice

[ https://issues.apache.org/jira/browse/SPARK-26532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dias updated SPARK-26532: -- Attachment: repartitionByRange Stages.png repartition Stages.png > repartitionByRange

[jira] [Assigned] (SPARK-26530) Validate heartheat arguments in SparkSubmitArguments

[ https://issues.apache.org/jira/browse/SPARK-26530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26530: Assignee: Apache Spark > Validate heartheat arguments in SparkSubmitArguments > -

[jira] [Assigned] (SPARK-26530) Validate heartheat arguments in SparkSubmitArguments

[ https://issues.apache.org/jira/browse/SPARK-26530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26530: Assignee: (was: Apache Spark) > Validate heartheat arguments in SparkSubmitArguments

[jira] [Created] (SPARK-26531) How to setup Apache Spark to use local hard disk when data does not fit in RAM in local mode?

Lê Văn Thanh created SPARK-26531: Summary: How to setup Apache Spark to use local hard disk when data does not fit in RAM in local mode? Key: SPARK-26531 URL: https://issues.apache.org/jira/browse/SPARK-26531

[jira] [Created] (SPARK-26530) Validate heartheat arguments in SparkSubmitArguments

liupengcheng created SPARK-26530: Summary: Validate heartheat arguments in SparkSubmitArguments Key: SPARK-26530 URL: https://issues.apache.org/jira/browse/SPARK-26530 Project: Spark Issue Ty

[jira] [Comment Edited] (SPARK-26389) temp checkpoint folder at executor should be deleted on graceful shutdown

[ https://issues.apache.org/jira/browse/SPARK-26389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733815#comment-16733815 ] Fengyu Cao edited comment on SPARK-26389 at 1/4/19 4:03 AM:

[jira] [Commented] (SPARK-26389) temp checkpoint folder at executor should be deleted on graceful shutdown

[ https://issues.apache.org/jira/browse/SPARK-26389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733815#comment-16733815 ] Fengyu Cao commented on SPARK-26389: {quote}Temp checkpoint can be used in one-node

[jira] [Assigned] (SPARK-26529) Add logs for IOException when preparing local resource

[ https://issues.apache.org/jira/browse/SPARK-26529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26529: Assignee: Apache Spark > Add logs for IOException when preparing local resource > --

[jira] [Assigned] (SPARK-26529) Add logs for IOException when preparing local resource

[ https://issues.apache.org/jira/browse/SPARK-26529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26529: Assignee: (was: Apache Spark) > Add logs for IOException when preparing local resourc

[jira] [Comment Edited] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733799#comment-16733799 ] Hyukjin Kwon edited comment on SPARK-26075 at 1/4/19 3:27 AM:

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733799#comment-16733799 ] Hyukjin Kwon commented on SPARK-26075: -- [~bhadani.neeraj...@gmail.com] can you post

[jira] [Created] (SPARK-26529) Add logs for IOException when preparing local resource

liupengcheng created SPARK-26529: Summary: Add logs for IOException when preparing local resource Key: SPARK-26529 URL: https://issues.apache.org/jira/browse/SPARK-26529 Project: Spark Issue

[jira] [Commented] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733796#comment-16733796 ] Hyukjin Kwon commented on SPARK-26126: -- If there's no actual problem hapenning, I w

[jira] [Commented] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733795#comment-16733795 ] Hyukjin Kwon commented on SPARK-26126: -- It matters since JIRA's supposed to file an

[jira] [Updated] (SPARK-26528) FsHistoryProviderSuite failed in IDEA because not exist "spark.testing" property

[ https://issues.apache.org/jira/browse/SPARK-26528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-26528: --- Priority: Minor (was: Major) > FsHistoryProviderSuite failed in IDEA because not exist "spark.testi

[jira] [Assigned] (SPARK-26526) test case for SPARK-10316 is not valid any more

[ https://issues.apache.org/jira/browse/SPARK-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26526: --- Assignee: Liu, Linhong > test case for SPARK-10316 is not valid any more >

[jira] [Resolved] (SPARK-26526) test case for SPARK-10316 is not valid any more

[ https://issues.apache.org/jira/browse/SPARK-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26526. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23440 [https://gith

[jira] [Assigned] (SPARK-26528) FsHistoryProviderSuite failed in IDEA because not exist "spark.testing" property

[ https://issues.apache.org/jira/browse/SPARK-26528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26528: Assignee: Apache Spark > FsHistoryProviderSuite failed in IDEA because not exist "spark.t

[jira] [Assigned] (SPARK-26528) FsHistoryProviderSuite failed in IDEA because not exist "spark.testing" property

[ https://issues.apache.org/jira/browse/SPARK-26528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26528: Assignee: (was: Apache Spark) > FsHistoryProviderSuite failed in IDEA because not exi

[jira] [Created] (SPARK-26528) FsHistoryProviderSuite failed in IDEA because not exist "spark.testing" property

deshanxiao created SPARK-26528: -- Summary: FsHistoryProviderSuite failed in IDEA because not exist "spark.testing" property Key: SPARK-26528 URL: https://issues.apache.org/jira/browse/SPARK-26528 Project

[jira] [Assigned] (SPARK-26527) Let acquireUnrollMemory fail fast if required space exceeds memory limit

[ https://issues.apache.org/jira/browse/SPARK-26527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26527: - Assignee: SongYadong > Let acquireUnrollMemory fail fast if required space exceeds memory limit

[jira] [Reopened] (SPARK-26494) 【spark sql】Use spark to read oracle TIMESTAMP(6) WITH LOCAL TIME ZONE type can't be found,

[ https://issues.apache.org/jira/browse/SPARK-26494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 秦坤 reopened SPARK-26494: > 【spark sql】Use spark to read oracle TIMESTAMP(6) WITH LOCAL TIME ZONE type > can't be found, >

[jira] [Updated] (SPARK-26494) 【spark sql】Use spark to read oracle TIMESTAMP(6) WITH LOCAL TIME ZONE type can't be found,

[ https://issues.apache.org/jira/browse/SPARK-26494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 秦坤 updated SPARK-26494: --- Description: Use spark to read oracle TIMESTAMP(6) WITH LOCAL TIME ZONE type can't be found, When the data type is

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733740#comment-16733740 ] Saisai Shao commented on SPARK-26512: - Please list the problems you saw, any log or

[jira] [Commented] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733738#comment-16733738 ] liupengcheng commented on SPARK-26126: -- [~hyukjin.kwon] it's an issue, it's really

[jira] [Commented] (SPARK-26452) Suppressing exception in finally: Java heap space java.lang.OutOfMemoryError: Java heap space

[ https://issues.apache.org/jira/browse/SPARK-26452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733715#comment-16733715 ] Jungtaek Lim commented on SPARK-26452: -- Maybe better to let driver (force) shutting

[jira] [Created] (SPARK-26527) Let acquireUnrollMemory fail fast if required space exceeds memory limit

SongYadong created SPARK-26527: -- Summary: Let acquireUnrollMemory fail fast if required space exceeds memory limit Key: SPARK-26527 URL: https://issues.apache.org/jira/browse/SPARK-26527 Project: Spark

[jira] [Commented] (SPARK-26527) Let acquireUnrollMemory fail fast if required space exceeds memory limit

[ https://issues.apache.org/jira/browse/SPARK-26527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733708#comment-16733708 ] Apache Spark commented on SPARK-26527: -- User 'SongYadong' has created a pull reques

[jira] [Assigned] (SPARK-26527) Let acquireUnrollMemory fail fast if required space exceeds memory limit

[ https://issues.apache.org/jira/browse/SPARK-26527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26527: Assignee: (was: Apache Spark) > Let acquireUnrollMemory fail fast if required space e

[jira] [Assigned] (SPARK-26527) Let acquireUnrollMemory fail fast if required space exceeds memory limit

[ https://issues.apache.org/jira/browse/SPARK-26527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26527: Assignee: Apache Spark > Let acquireUnrollMemory fail fast if required space exceeds memo

[jira] [Assigned] (SPARK-26489) Use ConfigEntry for hardcoded configs for python/r categories.

[ https://issues.apache.org/jira/browse/SPARK-26489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26489: -- Assignee: Jungtaek Lim > Use ConfigEntry for hardcoded configs for python/r categorie

[jira] [Resolved] (SPARK-26489) Use ConfigEntry for hardcoded configs for python/r categories.

[ https://issues.apache.org/jira/browse/SPARK-26489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26489. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23428 [https:

[jira] [Commented] (SPARK-25873) Date corruption when Spark and Hive both are on different timezones

[ https://issues.apache.org/jira/browse/SPARK-25873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733558#comment-16733558 ] Pablo Langa Blanco commented on SPARK-25873: Hello, it seems duplicated with

[jira] [Issue Comment Deleted] (SPARK-25873) Date corruption when Spark and Hive both are on different timezones

[ https://issues.apache.org/jira/browse/SPARK-25873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Langa Blanco updated SPARK-25873: --- Comment: was deleted (was: Hi [~pawanlawale] Timestamp in hive is saved as a long v

[jira] [Commented] (SPARK-25873) Date corruption when Spark and Hive both are on different timezones

[ https://issues.apache.org/jira/browse/SPARK-25873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733538#comment-16733538 ] Pablo Langa Blanco commented on SPARK-25873: Hi [~pawanlawale] Timestamp in

[jira] [Assigned] (SPARK-26349) Pyspark should not accept insecure p4yj gateways

[ https://issues.apache.org/jira/browse/SPARK-26349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26349: Assignee: Apache Spark > Pyspark should not accept insecure p4yj gateways > -

[jira] [Assigned] (SPARK-26349) Pyspark should not accept insecure p4yj gateways

[ https://issues.apache.org/jira/browse/SPARK-26349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26349: Assignee: (was: Apache Spark) > Pyspark should not accept insecure p4yj gateways > --

[jira] [Commented] (SPARK-19261) Support `ALTER TABLE table_name ADD COLUMNS(..)` statement

[ https://issues.apache.org/jira/browse/SPARK-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733445#comment-16733445 ] suman gorantla commented on SPARK-19261: Please understand that we have command

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733431#comment-16733431 ] David Vrba commented on SPARK-25401: [~gwang3] could you please recommend someone wh

[jira] [Updated] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25253: - Fix Version/s: 2.3.3 > Refactor pyspark connection & authentication > --

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733428#comment-16733428 ] Dongjoon Hyun commented on SPARK-25253: --- Yep. Thanks, [~irashid]. :) > Refactor p

[jira] [Comment Edited] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733424#comment-16733424 ] Dongjoon Hyun edited comment on SPARK-25253 at 1/3/19 7:33 PM: ---

[jira] [Comment Edited] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733424#comment-16733424 ] Dongjoon Hyun edited comment on SPARK-25253 at 1/3/19 7:32 PM: ---

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733427#comment-16733427 ] Imran Rashid commented on SPARK-25253: -- in branch-2.2: https://github.com/apache/s

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733424#comment-16733424 ] Dongjoon Hyun commented on SPARK-25253: --- I don't see this in `branch-2.2`? > Refa

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733419#comment-16733419 ] Imran Rashid commented on SPARK-25253: -- [~hyukjin.kwon] yes, it looks like I did as

[jira] [Commented] (SPARK-26519) spark sql CHANGE COLUMN not working

[ https://issues.apache.org/jira/browse/SPARK-26519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733413#comment-16733413 ] Dongjoon Hyun commented on SPARK-26519: --- +1 for [~hyukjin.kwon]. [~sumanGorantla]

[jira] [Commented] (SPARK-22951) count() after dropDuplicates() on emptyDataFrame returns incorrect value

[ https://issues.apache.org/jira/browse/SPARK-22951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733328#comment-16733328 ] Dongjoon Hyun commented on SPARK-22951: --- This is backported to `branch-2.2` via h

[jira] [Updated] (SPARK-22951) count() after dropDuplicates() on emptyDataFrame returns incorrect value

[ https://issues.apache.org/jira/browse/SPARK-22951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22951: -- Fix Version/s: 2.2.3 > count() after dropDuplicates() on emptyDataFrame returns incorrect valu

[jira] [Commented] (SPARK-26385) YARN - Spark Stateful Structured streaming HDFS_DELEGATION_TOKEN not found in cache

[ https://issues.apache.org/jira/browse/SPARK-26385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733318#comment-16733318 ] Gabor Somogyi commented on SPARK-26385: --- Yeah and additional logs would be also go

[jira] [Commented] (SPARK-26257) SPIP: Interop Support for Spark Language Extensions

[ https://issues.apache.org/jira/browse/SPARK-26257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733297#comment-16733297 ] Tyson Condie commented on SPARK-26257: -- [~rxin] Any thoughts regarding this SPIP? I

[jira] [Resolved] (SPARK-26522) Auth secret error in RBackendAuthHandler

[ https://issues.apache.org/jira/browse/SPARK-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26522. Resolution: Not A Problem Yes, this is an issue with Livy that is fixed in the master bran

[jira] [Resolved] (SPARK-26517) Avoid duplicate test in ParquetSchemaPruningSuite

[ https://issues.apache.org/jira/browse/SPARK-26517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26517. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23427 [https://github.c

[jira] [Assigned] (SPARK-26447) Allow OrcColumnarBatchReader to return less partition columns

[ https://issues.apache.org/jira/browse/SPARK-26447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26447: --- Assignee: Gengliang Wang > Allow OrcColumnarBatchReader to return less partition columns >

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733209#comment-16733209 ] Gabor Somogyi commented on SPARK-26359: --- [~Tint] did the suggested workaround work

[jira] [Resolved] (SPARK-26447) Allow OrcColumnarBatchReader to return less partition columns

[ https://issues.apache.org/jira/browse/SPARK-26447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26447. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23387 [https://gith

[jira] [Commented] (SPARK-26389) temp checkpoint folder at executor should be deleted on graceful shutdown

[ https://issues.apache.org/jira/browse/SPARK-26389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733201#comment-16733201 ] Gabor Somogyi commented on SPARK-26389: --- Temp checkpoint can be used in one-node s

[jira] [Assigned] (SPARK-26517) Avoid duplicate test in ParquetSchemaPruningSuite

[ https://issues.apache.org/jira/browse/SPARK-26517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26517: - Assignee: Liang-Chi Hsieh > Avoid duplicate test in ParquetSchemaPruningSuite > ---

[jira] [Resolved] (SPARK-26501) Unexpected overriden of exitFn in SparkSubmitSuite

[ https://issues.apache.org/jira/browse/SPARK-26501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26501. --- Resolution: Fixed Assignee: liupengcheng Fix Version/s: 3.0.0 2.4.

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733173#comment-16733173 ] Neeraj Bhadani commented on SPARK-26075: [~hyukjin.kwon] I have verified with Sp

[jira] [Resolved] (SPARK-26514) Introduce a new conf to improve the task parallelism

[ https://issues.apache.org/jira/browse/SPARK-26514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26514. --- Resolution: Won't Fix > Introduce a new conf to improve the task parallelism > -

[jira] [Updated] (SPARK-26434) disallow ADAPTIVE_EXECUTION_ENABLED&CBO_ENABLED in org.apache.spark.sql.execution.streaming.StreamExecution#runStream, but logWarning in org.apache.spark.sql.streaming.S

[ https://issues.apache.org/jira/browse/SPARK-26434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-26434: -- Affects Version/s: (was: 2.4.0) 3.0.0 > disallow ADAPTIVE_EXECUTION

[jira] [Created] (SPARK-26526) test case for SPARK-10316 is not valid any more

Liu, Linhong created SPARK-26526: Summary: test case for SPARK-10316 is not valid any more Key: SPARK-26526 URL: https://issues.apache.org/jira/browse/SPARK-26526 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-26526) test case for SPARK-10316 is not valid any more

[ https://issues.apache.org/jira/browse/SPARK-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26526: Assignee: Apache Spark > test case for SPARK-10316 is not valid any more > --

[jira] [Assigned] (SPARK-26526) test case for SPARK-10316 is not valid any more

[ https://issues.apache.org/jira/browse/SPARK-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26526: Assignee: (was: Apache Spark) > test case for SPARK-10316 is not valid any more > ---

[jira] [Assigned] (SPARK-26454) While creating new UDF with JAR though UDF is created successfully, it throws IllegegalArgument Exception

[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26454: Assignee: (was: Apache Spark) > While creating new UDF with JAR though UDF is created

[jira] [Assigned] (SPARK-26454) While creating new UDF with JAR though UDF is created successfully, it throws IllegegalArgument Exception

[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26454: Assignee: Apache Spark > While creating new UDF with JAR though UDF is created successful

[jira] [Updated] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26525: - Description: Currently, spark would not release ShuffleBlockFetcherIterator until the whole tas

[jira] [Commented] (SPARK-26433) Tail method for spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-26433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732895#comment-16732895 ] Hyukjin Kwon commented on SPARK-26433: -- There are few potential workarounds. For in

[jira] [Updated] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26525: - Description: Currently, spark would not release ShuffleBlockFetcherIterator until the whole tas

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732882#comment-16732882 ] Hyukjin Kwon commented on SPARK-25253: -- [~irashid], did you manually backport this

[jira] [Assigned] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26525: Assignee: (was: Apache Spark) > Fast release memory of ShuffleBlockFetcherIterator >

[jira] [Commented] (SPARK-26502) Get rid of hiveResultString() in QueryExecution

[ https://issues.apache.org/jira/browse/SPARK-26502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732874#comment-16732874 ] Hyukjin Kwon commented on SPARK-26502: -- Can we update JIRA as well, [~maxgekk]? >

[jira] [Commented] (SPARK-26502) Get rid of hiveResultString() in QueryExecution

[ https://issues.apache.org/jira/browse/SPARK-26502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732873#comment-16732873 ] Hyukjin Kwon commented on SPARK-26502: -- Fixed in https://github.com/apache/spark/pu

[jira] [Assigned] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26525: Assignee: Apache Spark > Fast release memory of ShuffleBlockFetcherIterator > ---

[jira] [Resolved] (SPARK-26502) Get rid of hiveResultString() in QueryExecution

[ https://issues.apache.org/jira/browse/SPARK-26502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-26502. --- Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 > Get ri

[jira] [Assigned] (SPARK-26524) If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor w

[ https://issues.apache.org/jira/browse/SPARK-26524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26524: Assignee: Apache Spark > If the application directory fails to be created on the SPARK_WO

[jira] [Assigned] (SPARK-26524) If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor w

[ https://issues.apache.org/jira/browse/SPARK-26524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26524: Assignee: (was: Apache Spark) > If the application directory fails to be created on t

[jira] [Updated] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26525: - Description: Currently, spark would not release ShuffleBlockFetcherIterator until the whole tas

[jira] [Created] (SPARK-26525) Fast release memory of ShuffleBlockFetcherIterator

liupengcheng created SPARK-26525: Summary: Fast release memory of ShuffleBlockFetcherIterator Key: SPARK-26525 URL: https://issues.apache.org/jira/browse/SPARK-26525 Project: Spark Issue Type

[jira] [Created] (SPARK-26524) If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor wi

hantiantian created SPARK-26524: --- Summary: If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor will be allocated indefinitely.

[jira] [Commented] (SPARK-26522) Auth secret error in RBackendAuthHandler

[ https://issues.apache.org/jira/browse/SPARK-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732762#comment-16732762 ] Hyukjin Kwon commented on SPARK-26522: -- I think it's something Livy should support

[jira] [Assigned] (SPARK-26522) Auth secret error in RBackendAuthHandler

[ https://issues.apache.org/jira/browse/SPARK-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26522: Assignee: (was: Matt Cheah) > Auth secret error in RBackendAuthHandler >

[jira] [Updated] (SPARK-26522) Auth secret error in RBackendAuthHandler

[ https://issues.apache.org/jira/browse/SPARK-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26522: - Labels: (was: newbie) > Auth secret error in RBackendAuthHandler > ---

[jira] [Resolved] (SPARK-26519) spark sql CHANGE COLUMN not working

[ https://issues.apache.org/jira/browse/SPARK-26519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26519. -- Resolution: Duplicate > spark sql CHANGE COLUMN not working > -

[jira] [Commented] (SPARK-26519) spark sql CHANGE COLUMN not working

[ https://issues.apache.org/jira/browse/SPARK-26519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732758#comment-16732758 ] Hyukjin Kwon commented on SPARK-26519: -- It's a duplicate of SPARK-23890 or SPARK-24