[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-18 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656320#comment-16656320 ] Sunayan Saikia commented on SPARK-13478: [~vanzin]: If I want to 'spark-submit'

[jira] [Comment Edited] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-18 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656320#comment-16656320 ] Sunayan Saikia edited comment on SPARK-13478 at 10/19/18 6:15 AM:

[jira] [Created] (SPARK-25779) Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-18 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25779: Summary: Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite Key: SPARK-25779 URL: https://issues.apache.org/jira/browse/SPARK-25779

[jira] [Commented] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656314#comment-16656314 ] Wenchen Fan commented on SPARK-21402: - LGTM, thanks for doing it [~dongjoon] ! > Fi

[jira] [Commented] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access

2018-10-18 Thread Greg Senia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656308#comment-16656308 ] Greg Senia commented on SPARK-25778: Could we add a snippit like this.. I just teste

[jira] [Updated] (SPARK-25775) Race between end-of-task and completion iterator read lock release

2018-10-18 Thread Ablimit A. Keskin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ablimit A. Keskin updated SPARK-25775: -- Description: The following issue comes from a production Spark job where executors die

[jira] [Created] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access

2018-10-18 Thread Greg Senia (JIRA)
Greg Senia created SPARK-25778: -- Summary: WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access Key: SPARK-25778 URL: https://issues.apache.org/jira/browse/SPARK-25778 Project: Spark

[jira] [Commented] (SPARK-25777) Optimize DetermineTableStats if LogicalRelation already cached

2018-10-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656228#comment-16656228 ] Yuming Wang commented on SPARK-25777: - I'm working on. This needs to fix at least:

[jira] [Created] (SPARK-25777) Optimize DetermineTableStats if LogicalRelation already cached

2018-10-18 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25777: --- Summary: Optimize DetermineTableStats if LogicalRelation already cached Key: SPARK-25777 URL: https://issues.apache.org/jira/browse/SPARK-25777 Project: Spark

[jira] [Resolved] (SPARK-25493) CRLF Line Separators don't work in multiline CSVs

2018-10-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25493. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22503 [https://gi

[jira] [Assigned] (SPARK-25493) CRLF Line Separators don't work in multiline CSVs

2018-10-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25493: Assignee: Justin Uang > CRLF Line Separators don't work in multiline CSVs > -

[jira] [Commented] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656180#comment-16656180 ] Apache Spark commented on SPARK-25776: -- User '10110346' has created a pull request

[jira] [Assigned] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25776: Assignee: (was: Apache Spark) > The disk write buffer size must be greater than 12. >

[jira] [Commented] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656177#comment-16656177 ] Apache Spark commented on SPARK-25776: -- User '10110346' has created a pull request

[jira] [Assigned] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25776: Assignee: Apache Spark > The disk write buffer size must be greater than 12. > --

[jira] [Updated] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-25776: Description: In {color:#205081}{{UnsafeSorterSpillWriter.java}}{color}, when we write a record to a spill

[jira] [Updated] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-25776: Description: In {color:#205081}{{UnsafeSorterSpillWriter.java}}{color}, when we write a record to a spill

[jira] [Created] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-10-18 Thread liuxian (JIRA)
liuxian created SPARK-25776: --- Summary: The disk write buffer size must be greater than 12. Key: SPARK-25776 URL: https://issues.apache.org/jira/browse/SPARK-25776 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-25764) Avoid usage of deprecated methods in examples for BisectingKMeans

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25764. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22763 [https://gith

[jira] [Assigned] (SPARK-25764) Avoid usage of deprecated methods in examples for BisectingKMeans

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25764: --- Assignee: Marco Gaido > Avoid usage of deprecated methods in examples for BisectingKMeans >

[jira] [Updated] (SPARK-25740) Refactor DetermineTableStats to invalidate cache when some configuration changed

2018-10-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25740: Description: How to reproduce: {code:sql} # spark-sql create table t1 (a int) stored as parquet; c

[jira] [Resolved] (SPARK-25761) sparksql执行sql语句的时候,sql语句已经执行成功,但是从sparkui上看该语句还是没有执行完成,还是running状态。

2018-10-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25761. - Resolution: Invalid > sparksql执行sql语句的时候,sql语句已经执行成功,但是从sparkui上看该语句还是没有执行完成,还是running状态。 >

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656065#comment-16656065 ] Apache Spark commented on SPARK-24499: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656062#comment-16656062 ] Apache Spark commented on SPARK-24499: -- User 'dilipbiswal' has created a pull reque

[jira] [Updated] (SPARK-25775) Race between end-of-task and completion iterator read lock release

2018-10-18 Thread Ablimit A. Keskin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ablimit A. Keskin updated SPARK-25775: -- Affects Version/s: (was: 2.2.0) 2.2.2 > Race between end-of

[jira] [Updated] (SPARK-25775) Race between end-of-task and completion iterator read lock release

2018-10-18 Thread Ablimit A. Keskin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ablimit A. Keskin updated SPARK-25775: -- Description: The following issue comes from a production Spark job where executors die

[jira] [Created] (SPARK-25775) Race between end-of-task and completion iterator read lock release

2018-10-18 Thread Ablimit A. Keskin (JIRA)
Ablimit A. Keskin created SPARK-25775: - Summary: Race between end-of-task and completion iterator read lock release Key: SPARK-25775 URL: https://issues.apache.org/jira/browse/SPARK-25775 Project:

[jira] [Updated] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21402: -- Fix Version/s: 2.4.0 2.3.3 2.2.3 > Fix java array of str

[jira] [Resolved] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-21402. --- Resolution: Fixed Target Version/s: 2.2.3, 2.3.3, 2.4.0 > Fix java array of struct

[jira] [Updated] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21402: -- Target Version/s: (was: 2.2.3, 2.3.3, 2.4.0) > Fix java array of structs deserialization > -

[jira] [Assigned] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-21402: - Assignee: Vladimir Kuriatkov > Fix java array of structs deserialization >

[jira] [Assigned] (SPARK-25683) Updated the log for the firstTime event Drop occurs.

2018-10-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25683: -- Assignee: Shivu Sondur > Updated the log for the firstTime event Drop occurs. > -

[jira] [Resolved] (SPARK-25683) Updated the log for the firstTime event Drop occurs.

2018-10-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25683. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22677 [https:

[jira] [Created] (SPARK-25774) Eliminate query anomalies with empty partitions - TRUNCATE, SELECT DISTINCT, etc.

2018-10-18 Thread Steven Cardella (JIRA)
Steven Cardella created SPARK-25774: --- Summary: Eliminate query anomalies with empty partitions - TRUNCATE, SELECT DISTINCT, etc. Key: SPARK-25774 URL: https://issues.apache.org/jira/browse/SPARK-25774

[jira] [Assigned] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25773: Assignee: Apache Spark (was: Shixiong Zhu) > Cancel zombie tasks in a result stage when

[jira] [Assigned] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25773: Assignee: Shixiong Zhu (was: Apache Spark) > Cancel zombie tasks in a result stage when

[jira] [Commented] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655887#comment-16655887 ] Apache Spark commented on SPARK-25773: -- User 'zsxwing' has created a pull request f

[jira] [Commented] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655888#comment-16655888 ] Apache Spark commented on SPARK-25773: -- User 'zsxwing' has created a pull request f

[jira] [Updated] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25773: - Description: When a job finishes, there may be some zombie tasks still running due to stage retr

[jira] [Created] (SPARK-25773) Cancel zombie tasks in a result stage when the job finishes

2018-10-18 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25773: Summary: Cancel zombie tasks in a result stage when the job finishes Key: SPARK-25773 URL: https://issues.apache.org/jira/browse/SPARK-25773 Project: Spark

[jira] [Updated] (SPARK-21402) Fix java array of structs deserialization

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21402: -- Summary: Fix java array of structs deserialization (was: Java encoders - switch fields on col

[jira] [Commented] (SPARK-25772) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655862#comment-16655862 ] Apache Spark commented on SPARK-25772: -- User 'vofque' has created a pull request fo

[jira] [Assigned] (SPARK-25772) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25772: Assignee: (was: Apache Spark) > Java encoders - switch fields on collectAsList >

[jira] [Assigned] (SPARK-25772) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25772: Assignee: Apache Spark > Java encoders - switch fields on collectAsList > ---

[jira] [Commented] (SPARK-25772) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655861#comment-16655861 ] Apache Spark commented on SPARK-25772: -- User 'vofque' has created a pull request fo

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655855#comment-16655855 ] Dongjoon Hyun commented on SPARK-21402: --- We already committed with SPARK JIRA ID,

[jira] [Commented] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655852#comment-16655852 ] Apache Spark commented on SPARK-25771: -- User 'zsxwing' has created a pull request f

[jira] [Comment Edited] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655845#comment-16655845 ] Dongjoon Hyun edited comment on SPARK-21402 at 10/18/18 8:47 PM: -

[jira] [Assigned] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25771: Assignee: Apache Spark (was: Shixiong Zhu) > Fix improper synchronization in PythonWorke

[jira] [Assigned] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25771: Assignee: Shixiong Zhu (was: Apache Spark) > Fix improper synchronization in PythonWorke

[jira] [Commented] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655850#comment-16655850 ] Apache Spark commented on SPARK-25771: -- User 'zsxwing' has created a pull request f

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655845#comment-16655845 ] Dongjoon Hyun commented on SPARK-21402: --- I cloned this JIRA issue. Can we do like

[jira] [Created] (SPARK-25772) Java encoders - switch fields on collectAsList

2018-10-18 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-25772: - Summary: Java encoders - switch fields on collectAsList Key: SPARK-25772 URL: https://issues.apache.org/jira/browse/SPARK-25772 Project: Spark Issue Type:

[jira] [Created] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-18 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25771: Summary: Fix improper synchronization in PythonWorkerFactory Key: SPARK-25771 URL: https://issues.apache.org/jira/browse/SPARK-25771 Project: Spark Issue Typ

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655825#comment-16655825 ] Apache Spark commented on SPARK-24499: -- User 'gatorsmile' has created a pull reques

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655824#comment-16655824 ] Apache Spark commented on SPARK-24499: -- User 'gatorsmile' has created a pull reques

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655776#comment-16655776 ] Xiao Li commented on SPARK-24499: - Feel free to create the extra tasks to improve the do

[jira] [Resolved] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24499. - Resolution: Fixed Assignee: Yuanjian Li Fix Version/s: 2.4.0 Target Version/

[jira] [Updated] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24499: Summary: Split the page of sql-programming-guide.html to multiple separate pages (was: Documentation impr

[jira] [Updated] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24499: Component/s: (was: Spark Core) > Split the page of sql-programming-guide.html to multiple separate pag

[jira] [Commented] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655711#comment-16655711 ] Huaxin Gao commented on SPARK-25769: I will work on this. Thanks! > UnresolvedAttri

[jira] [Created] (SPARK-25770) support SparkDataFrame pretty print

2018-10-18 Thread Weiqiang Zhuang (JIRA)
Weiqiang Zhuang created SPARK-25770: --- Summary: support SparkDataFrame pretty print Key: SPARK-25770 URL: https://issues.apache.org/jira/browse/SPARK-25770 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-25758) Deprecate BisectingKMeans compute cost

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25758: -- Issue Type: Task (was: Improvement) > Deprecate BisectingKMeans compute cost > --

[jira] [Resolved] (SPARK-25758) Deprecate BisectingKMeans compute cost

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25758. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22756 [https://

[jira] [Assigned] (SPARK-25758) Deprecate BisectingKMeans compute cost

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25758: - Assignee: Marco Gaido > Deprecate BisectingKMeans compute cost > --

[jira] [Resolved] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25682. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22681 [https:

[jira] [Assigned] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25682: -- Assignee: Marcelo Vanzin > Docker images generated from dev build and from dist tarba

[jira] [Commented] (SPARK-25766) AMCredentialRenewer can leak FS clients

2018-10-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655574#comment-16655574 ] Marcelo Vanzin commented on SPARK-25766: That code doesn't exist anymore in 2.4.

[jira] [Resolved] (SPARK-25763) Use more `@contextmanager` to ensure clean-up each test.

2018-10-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25763. -- Resolution: Fixed Assignee: Takuya Ueshin Fix Version/s: 3.0.0 fixed in https:

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655530#comment-16655530 ] Marco Gaido commented on SPARK-25767: - Your conversion of a Java array in a Scala Se

[jira] [Resolved] (SPARK-25760) Set AddJarCommand return empty

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25760. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22747 [https://

[jira] [Assigned] (SPARK-25760) Set AddJarCommand return empty

2018-10-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25760: - Assignee: Yuming Wang > Set AddJarCommand return empty > --

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655445#comment-16655445 ] Thomas Brugiere commented on SPARK-25767: - Please can you provide an example? I

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2018-10-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655442#comment-16655442 ] Yuming Wang commented on SPARK-21725: - [~owen.omalley] [~ste...@apache.org] I found

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655440#comment-16655440 ] Marco Gaido commented on SPARK-25767: - So I tracked down the issue. The problem is t

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655428#comment-16655428 ] Thomas Brugiere commented on SPARK-25767: - It looks like it's not stopping the p

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655418#comment-16655418 ] Marco Gaido commented on SPARK-25767: - It is interesting, I can reproduce with the J

[jira] [Commented] (SPARK-25665) Refactor ObjectHashAggregateExecBenchmark to use main method

2018-10-18 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655416#comment-16655416 ] Peter Toth commented on SPARK-25665: I will submit a PR but I've run into SPARK-2576

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655391#comment-16655391 ] Thomas Brugiere commented on SPARK-25767: - If it can help, I have created a gith

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655383#comment-16655383 ] Apache Spark commented on SPARK-21402: -- User 'vofque' has created a pull request fo

[jira] [Updated] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-18 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-25768: --- Description:  The following doesn't work since SPARK-18186 so it is a regression. {code:java} test("

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655382#comment-16655382 ] Apache Spark commented on SPARK-21402: -- User 'vofque' has created a pull request fo

[jira] [Assigned] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21402: Assignee: (was: Apache Spark) > Java encoders - switch fields on collectAsList >

[jira] [Assigned] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21402: Assignee: Apache Spark > Java encoders - switch fields on collectAsList > ---

[jira] [Updated] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-18 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-25768: --- Description:  The following doesn't work since SPARK-18168 so it is a regression. {code:java} test("

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655380#comment-16655380 ] Apache Spark commented on SPARK-21402: -- User 'vofque' has created a pull request fo

[jira] [Updated] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-18 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-25768: --- Description:   The following doesn't work since -SPARK-18186:(- {code:java} test("constant argument

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655374#comment-16655374 ] Thomas Brugiere commented on SPARK-25767: - Hi Marco, I have noticed the bug usi

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655369#comment-16655369 ] Wenchen Fan commented on SPARK-20760: - has this been fixed by https://issues.apache.

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655366#comment-16655366 ] Marco Gaido commented on SPARK-25767: - I tried on current master branch but I wasn't

[jira] [Updated] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21402: Fix Version/s: (was: 2.4.0) > Java encoders - switch fields on collectAsList > ---

[jira] [Reopened] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-21402: - Assignee: (was: Vladimir Kuriatkov) > Java encoders - switch fields on collectAsList > ---

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655322#comment-16655322 ] Wenchen Fan commented on SPARK-21402: - oh this ticket reports the bug for map type.

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655314#comment-16655314 ] Wenchen Fan commented on SPARK-21402: - Can you create a ticket for the map case? Bas

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-18 Thread Vladimir Kuriatkov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655295#comment-16655295 ] Vladimir Kuriatkov commented on SPARK-21402: I guess, only array case has be

[jira] [Updated] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-25769: Description: {{UnresolvedAttribute.sql()}} output is incorrectly escaped for nested column

[jira] [Updated] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-25769: Description:  This issue affects dynamic SQL generation that relies on {{sql()}}. {code:ja

[jira] [Created] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-25769: --- Summary: UnresolvedAttribute.sql() incorrectly escapes nested columns Key: SPARK-25769 URL: https://issues.apache.org/jira/browse/SPARK-25769 Project: Spark

[jira] [Commented] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655255#comment-16655255 ] Apache Spark commented on SPARK-25768: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25768: Assignee: (was: Apache Spark) > Constant argument expecting Hive UDAFs doesn't work >

  1   2   >