[jira] [Updated] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25906: -- Description: This is a regression on Spark 2.4.0. *Spark 2.3.2* {code:java} $ cat test.scala

[jira] [Updated] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25906: -- Priority: Major (was: Critical) > spark-shell cannot handle `-i` option correctly > -

[jira] [Commented] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671169#comment-16671169 ] Dongjoon Hyun commented on SPARK-25906: --- Thank you for confirmation. The workaroun

[jira] [Commented] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671165#comment-16671165 ] Hyukjin Kwon commented on SPARK-25906: -- Yea, looks this is pretty much related with

[jira] [Updated] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25906: -- Priority: Critical (was: Major) > spark-shell cannot handle `-i` option correctly > -

[jira] [Comment Edited] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671161#comment-16671161 ] Dongjoon Hyun edited comment on SPARK-25906 at 11/1/18 6:31 AM: --

[jira] [Commented] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671161#comment-16671161 ] Dongjoon Hyun commented on SPARK-25906: --- Hi, [~dbtsai] . Could you take a look at

[jira] [Commented] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671159#comment-16671159 ] Dongjoon Hyun commented on SPARK-25906: --- cc [~cloud_fan] and [~smilegator] . Alth

[jira] [Created] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-25906: - Summary: spark-shell cannot handle `-i` option correctly Key: SPARK-25906 URL: https://issues.apache.org/jira/browse/SPARK-25906 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25906) spark-shell cannot handle `-i` option correctly

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25906: -- Description: This is a regression on Spark 2.4.0. *Spark 2.3.2* {code:java} $ cat test.scala

[jira] [Resolved] (SPARK-25884) Add TBLPROPERTIES and COMMENT, and use LOCATION when SHOW CREATE TABLE.

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25884. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22892 [https://gith

[jira] [Assigned] (SPARK-25672) Inferring schema from CSV string literal

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25672: Assignee: Maxim Gekk > Inferring schema from CSV string literal > ---

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671002#comment-16671002 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671000#comment-16671000 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671001#comment-16671001 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request

[jira] [Resolved] (SPARK-25672) Inferring schema from CSV string literal

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25672. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22666 [https://gi

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-10-31 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670904#comment-16670904 ] Jungtaek Lim commented on SPARK-20568: -- [~zsxwing] Yes I'll see the time slot and t

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670897#comment-16670897 ] Shixiong Zhu commented on SPARK-20568: -- [~kabhwan] I think this is pretty useful. D

[jira] [Resolved] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25899. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Flaky test: CoarseGrainedSchedu

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670777#comment-16670777 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670776#comment-16670776 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request fo

[jira] [Issue Comment Deleted] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25855: - Comment: was deleted (was: User 'squito' has created a pull request for this issue: https://gith

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670773#comment-16670773 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670738#comment-16670738 ] Apache Spark commented on SPARK-25855: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25892) AttributeReference.withMetadata method should have return type AttributeReference

2018-10-31 Thread kevin yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670735#comment-16670735 ] kevin yu commented on SPARK-25892: -- I am looking into this now. Kevin > AttributeRefer

[jira] [Created] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

2018-10-31 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25905: Summary: BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers Key: SPARK-25905 URL: https://issues.apache.org/jira/browse/SPARK-25905 Proje

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25825: Assignee: (was: Apache Spark) > Kerberos Support for Long Running Jobs in Kubernetes

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25825: Assignee: Apache Spark > Kerberos Support for Long Running Jobs in Kubernetes >

[jira] [Commented] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670719#comment-16670719 ] Apache Spark commented on SPARK-25825: -- User 'ifilonenko' has created a pull reques

[jira] [Commented] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670715#comment-16670715 ] Apache Spark commented on SPARK-25825: -- User 'ifilonenko' has created a pull reques

[jira] [Updated] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25827: - Description: There are a couple of issues with replicating & remote reads of large encrypted bl

[jira] [Assigned] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25904: Assignee: (was: Apache Spark) > Avoid allocating arrays too large for JVMs >

[jira] [Assigned] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25904: Assignee: Apache Spark > Avoid allocating arrays too large for JVMs > ---

[jira] [Commented] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670695#comment-16670695 ] Apache Spark commented on SPARK-25904: -- User 'squito' has created a pull request fo

[jira] [Created] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25904: Summary: Avoid allocating arrays too large for JVMs Key: SPARK-25904 URL: https://issues.apache.org/jira/browse/SPARK-25904 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-25900) When the the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25900: --- Summary: When the the page number is more than the total page size, then fall back to the first page (was:

[jira] [Updated] (SPARK-25900) If the the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25900: --- Summary: If the the page number is more than the total page size, then fall back to the first page (was: I

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670684#comment-16670684 ] Apache Spark commented on SPARK-25900: -- User 'shahidki31' has created a pull reques

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670682#comment-16670682 ] Apache Spark commented on SPARK-25900: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25900: Assignee: Apache Spark > If the When the page number is more than the total page size, th

[jira] [Assigned] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25900: Assignee: (was: Apache Spark) > If the When the page number is more than the total pa

[jira] [Created] (SPARK-25903) Flaky test: BarrierTaskContextSuite.throw exception on barrier() call timeout

2018-10-31 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25903: -- Summary: Flaky test: BarrierTaskContextSuite.throw exception on barrier() call timeout Key: SPARK-25903 URL: https://issues.apache.org/jira/browse/SPARK-25903 Pro

[jira] [Commented] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-10-31 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670665#comment-16670665 ] Bruce Robbins commented on SPARK-24758: --- This issue was introduced by commit  [b83

[jira] [Assigned] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25902: Assignee: Apache Spark > Support for dates with milliseconds in Arrow bindings >

[jira] [Assigned] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25902: Assignee: (was: Apache Spark) > Support for dates with milliseconds in Arrow bindings

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670600#comment-16670600 ] Apache Spark commented on SPARK-25902: -- User 'javierluraschi' has created a pull re

[jira] [Created] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Javier Luraschi (JIRA)
Javier Luraschi created SPARK-25902: --- Summary: Support for dates with milliseconds in Arrow bindings Key: SPARK-25902 URL: https://issues.apache.org/jira/browse/SPARK-25902 Project: Spark I

[jira] [Commented] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670583#comment-16670583 ] Apache Spark commented on SPARK-25901: -- User 'yogeshg' has created a pull request f

[jira] [Assigned] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25901: Assignee: Apache Spark > Barrier mode spawns a bunch of threads that get collected on gc

[jira] [Assigned] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25901: Assignee: (was: Apache Spark) > Barrier mode spawns a bunch of threads that get colle

[jira] [Updated] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yogesh garg updated SPARK-25901: Description: After a barrier job is terminated (successfully or interrupted), the accompanying th

[jira] [Comment Edited] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670573#comment-16670573 ] yogesh garg edited comment on SPARK-25901 at 10/31/18 7:06 PM: ---

[jira] [Updated] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yogesh garg updated SPARK-25901: Attachment: Screen Shot 2018-10-31 at 11.57.25 AM.png Screen Shot 2018-10-31 at 11.

[jira] [Created] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
yogesh garg created SPARK-25901: --- Summary: Barrier mode spawns a bunch of threads that get collected on gc Key: SPARK-25901 URL: https://issues.apache.org/jira/browse/SPARK-25901 Project: Spark

[jira] [Commented] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670573#comment-16670573 ] yogesh garg commented on SPARK-25901: - I am working on this task. > Barrier mode sp

[jira] [Assigned] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25815: Assignee: Apache Spark > Kerberos Support in Kubernetes resource manager (Client Mode) >

[jira] [Commented] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670565#comment-16670565 ] Apache Spark commented on SPARK-25815: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25815: Assignee: (was: Apache Spark) > Kerberos Support in Kubernetes resource manager (Clie

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670531#comment-16670531 ] shahid commented on SPARK-25900: Thanks. I am working on it > If the When the page numb

[jira] [Resolved] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

2018-10-31 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25860. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22857 [https://github.com/a

[jira] [Created] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-25900: Summary: If the When the page number is more than the total page size, then fall back to the first page Key: SPARK-25900 URL: https://issues.apache.org/jira/browse

[jira] [Resolved] (SPARK-23781) Merge YARN and Mesos token renewal code

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-23781. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22624 [https://gi

[jira] [Assigned] (SPARK-23781) Merge YARN and Mesos token renewal code

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-23781: Assignee: Marcelo Vanzin > Merge YARN and Mesos token renewal code >

[jira] [Resolved] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25855. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22881 [https:

[jira] [Assigned] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25855: -- Assignee: Imran Rashid > Don't use Erasure Coding for event log files > -

[jira] [Commented] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670445#comment-16670445 ] Apache Spark commented on SPARK-25899: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25899: Assignee: Shixiong Zhu (was: Apache Spark) > Flaky test: CoarseGrainedSchedulerBackendSu

[jira] [Commented] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670443#comment-16670443 ] Apache Spark commented on SPARK-25899: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25899: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test: CoarseGrainedSchedulerBackendSu

[jira] [Updated] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25899: - Issue Type: Test (was: Documentation) > Flaky test: CoarseGrainedSchedulerBackendSuite.compute

[jira] [Created] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25899: Summary: Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched Key: SPARK-25899 URL: https://issues.apache.org/jira/browse/SPARK-25

[jira] [Assigned] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25897: Assignee: Apache Spark > Cannot run k8s integration tests in sbt > --

[jira] [Commented] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670429#comment-16670429 ] Apache Spark commented on SPARK-25897: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25897: Assignee: (was: Apache Spark) > Cannot run k8s integration tests in sbt > ---

[jira] [Commented] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670430#comment-16670430 ] Apache Spark commented on SPARK-25897: -- User 'vanzin' has created a pull request fo

[jira] [Updated] (SPARK-25898) pandas_udf not catching errors early on

2018-10-31 Thread Kaushalya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaushalya updated SPARK-25898: -- Summary: pandas_udf not catching errors early on (was: pandas_udf ) > pandas_udf not catching errors

[jira] [Created] (SPARK-25898) pandas_udf

2018-10-31 Thread Kaushalya (JIRA)
Kaushalya created SPARK-25898: - Summary: pandas_udf Key: SPARK-25898 URL: https://issues.apache.org/jira/browse/SPARK-25898 Project: Spark Issue Type: Bug Components: SQL Affects V

[jira] [Created] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25897: -- Summary: Cannot run k8s integration tests in sbt Key: SPARK-25897 URL: https://issues.apache.org/jira/browse/SPARK-25897 Project: Spark Issue Type: Impro

[jira] [Assigned] (SPARK-25891) Upgrade to Py4J 0.10.8.1

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25891: - Assignee: Dongjoon Hyun > Upgrade to Py4J 0.10.8.1 > > >

[jira] [Resolved] (SPARK-25891) Upgrade to Py4J 0.10.8.1

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25891. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22901 [https://

[jira] [Resolved] (SPARK-24746) AWS S3 301 Moved Permanently error message even after setting fs.s3a.endpoint for bucket in Mumbai region.

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24746. Resolution: Information Provided In general, please use the mailing list for questions. B

[jira] [Resolved] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25206. - Resolution: Fixed Fix Version/s: 2.4.0 > wrong records are returned when Hive metastore s

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670339#comment-16670339 ] Wenchen Fan commented on SPARK-25206: - marked it as resolved in 2.4.0, since the tes

[jira] [Reopened] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-25206: - > wrong records are returned when Hive metastore schema and parquet schema are > in different lette

[jira] [Resolved] (SPARK-25893) Show a directional error message for unsupported Hive Metastore versions

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25893. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22902 [https://

[jira] [Assigned] (SPARK-25893) Show a directional error message for unsupported Hive Metastore versions

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25893: - Assignee: Dongjoon Hyun > Show a directional error message for unsupported Hive Metasto

[jira] [Commented] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670246#comment-16670246 ] Apache Spark commented on SPARK-25896: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25896: Assignee: Apache Spark (was: Wenchen Fan) > Accumulator should only be updated once for

[jira] [Assigned] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25896: Assignee: Wenchen Fan (was: Apache Spark) > Accumulator should only be updated once for

[jira] [Created] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-25896: --- Summary: Accumulator should only be updated once for each successful task in shuffle map stage Key: SPARK-25896 URL: https://issues.apache.org/jira/browse/SPARK-25896 P

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670222#comment-16670222 ] shahid commented on SPARK-25861: Sorry [~srowen], next time I will take care. > Remove

[jira] [Updated] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25861: -- Priority: Trivial (was: Minor) Issue Type: Improvement (was: Bug) This isn't a bug, even if it

[jira] [Assigned] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25895: Assignee: Apache Spark > No test to compare Zstd and Lz4 Compression Algorithm >

[jira] [Commented] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670113#comment-16670113 ] Apache Spark commented on SPARK-25895: -- User 'Udbhav30' has created a pull request

[jira] [Assigned] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25895: Assignee: (was: Apache Spark) > No test to compare Zstd and Lz4 Compression Algorithm

[jira] [Created] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Udbhav Agrawal (JIRA)
Udbhav Agrawal created SPARK-25895: -- Summary: No test to compare Zstd and Lz4 Compression Algorithm Key: SPARK-25895 URL: https://issues.apache.org/jira/browse/SPARK-25895 Project: Spark Iss

[jira] [Assigned] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25894: Assignee: (was: Apache Spark) > Include a count of the number of physical columns rea

[jira] [Assigned] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25894: Assignee: Apache Spark > Include a count of the number of physical columns read for a col

[jira] [Commented] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670068#comment-16670068 ] Apache Spark commented on SPARK-25894: -- User 'mallman' has created a pull request f

[jira] [Resolved] (SPARK-25886) Improve error message of `FailureSafeParser` and `from_avro` in FAILFAST mode

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25886. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22895 [https://gi

  1   2   >