[jira] [Commented] (SPARK-12452) Add exception details to TaskCompletionListener/TaskContext

2016-08-30 Thread Jagadeesan A S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451171#comment-15451171 ] Jagadeesan A S commented on SPARK-12452: [~srowen] any suggestion from your side..? can i take it

[jira] [Resolved] (SPARK-15985) Reduce runtime overhead of a program that reads an primitive array in Dataset

2016-08-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15985. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13704

[jira] [Updated] (SPARK-15985) Reduce runtime overhead of a program that reads an primitive array in Dataset

2016-08-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15985: Assignee: Kazuaki Ishizaki > Reduce runtime overhead of a program that reads an primitive array in

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-08-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451091#comment-15451091 ] Joseph K. Bradley commented on SPARK-5992: -- Awesome, thanks! I made some comments, but it looks

[jira] [Created] (SPARK-17325) Inconsistent Spillable threshold and AppendOnlyMap growing threshold may trigger out-of-memory errors

2016-08-30 Thread Lijie Xu (JIRA)
Lijie Xu created SPARK-17325: Summary: Inconsistent Spillable threshold and AppendOnlyMap growing threshold may trigger out-of-memory errors Key: SPARK-17325 URL: https://issues.apache.org/jira/browse/SPARK-17325

[jira] [Assigned] (SPARK-17324) Remove Direct Usage of HiveClient in InsertIntoHiveTable

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17324: Assignee: (was: Apache Spark) > Remove Direct Usage of HiveClient in

[jira] [Assigned] (SPARK-17324) Remove Direct Usage of HiveClient in InsertIntoHiveTable

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17324: Assignee: Apache Spark > Remove Direct Usage of HiveClient in InsertIntoHiveTable >

[jira] [Commented] (SPARK-17324) Remove Direct Usage of HiveClient in InsertIntoHiveTable

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450913#comment-15450913 ] Apache Spark commented on SPARK-17324: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-17324) Remove Direct Usage of HiveClient in InsertIntoHiveTable

2016-08-30 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17324: --- Summary: Remove Direct Usage of HiveClient in InsertIntoHiveTable Key: SPARK-17324 URL: https://issues.apache.org/jira/browse/SPARK-17324 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17318. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-15575) Remove breeze from dependencies?

2016-08-30 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450836#comment-15450836 ] Vladimir Feinberg commented on SPARK-15575: --- Some of the biggest issues with Breeze perf I've

[jira] [Commented] (SPARK-17323) ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450742#comment-15450742 ] Apache Spark commented on SPARK-17323: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17323) ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17323: Assignee: Wenchen Fan (was: Apache Spark) > ALTER VIEW AS should keep the previous table

[jira] [Assigned] (SPARK-17323) ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17323: Assignee: Apache Spark (was: Wenchen Fan) > ALTER VIEW AS should keep the previous table

[jira] [Created] (SPARK-17323) ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

2016-08-30 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-17323: --- Summary: ALTER VIEW AS should keep the previous table properties, comment, create_time, etc. Key: SPARK-17323 URL: https://issues.apache.org/jira/browse/SPARK-17323

[jira] [Commented] (SPARK-16283) Implement percentile_approx SQL function

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450696#comment-15450696 ] Apache Spark commented on SPARK-16283: -- User 'clockfly' has created a pull request for this issue:

[jira] [Commented] (SPARK-14742) Redirect spark-ec2 doc to new location

2016-08-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450632#comment-15450632 ] Shivaram Venkataraman commented on SPARK-14742: --- To follow up my guess was right - there is

[jira] [Commented] (SPARK-14742) Redirect spark-ec2 doc to new location

2016-08-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450626#comment-15450626 ] Nicholas Chammas commented on SPARK-14742: -- {quote} Otherwise the only way to get to this link

[jira] [Commented] (SPARK-14742) Redirect spark-ec2 doc to new location

2016-08-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450619#comment-15450619 ] Shivaram Venkataraman commented on SPARK-14742: --- Couple of things I noticed - It seems to

[jira] [Commented] (SPARK-14155) Hide UserDefinedType in Spark 2.0

2016-08-30 Thread Robert Conrad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450612#comment-15450612 ] Robert Conrad commented on SPARK-14155: --- [~r...@databricks.com] echoing Maciej above - is there any

[jira] [Commented] (SPARK-14742) Redirect spark-ec2 doc to new location

2016-08-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450602#comment-15450602 ] Nicholas Chammas commented on SPARK-14742: -- http://spark.apache.org/docs/latest/ec2-scripts.html

[jira] [Created] (SPARK-17322) 'ANY n' clause for SQL queries to increase the ease of use of WHERE clause predicates

2016-08-30 Thread Suman Somasundar (JIRA)
Suman Somasundar created SPARK-17322: Summary: 'ANY n' clause for SQL queries to increase the ease of use of WHERE clause predicates Key: SPARK-17322 URL: https://issues.apache.org/jira/browse/SPARK-17322

[jira] [Assigned] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17321: Assignee: (was: Apache Spark) > YARN shuffle service should use good disk from

[jira] [Commented] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450424#comment-15450424 ] Apache Spark commented on SPARK-17321: -- User 'zhaoyunjiong' has created a pull request for this

[jira] [Assigned] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17321: Assignee: Apache Spark > YARN shuffle service should use good disk from

[jira] [Created] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

2016-08-30 Thread yunjiong zhao (JIRA)
yunjiong zhao created SPARK-17321: - Summary: YARN shuffle service should use good disk from yarn.nodemanager.local-dirs Key: SPARK-17321 URL: https://issues.apache.org/jira/browse/SPARK-17321

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450270#comment-15450270 ] Apache Spark commented on SPARK-17243: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Resolved] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-17243. --- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.1.0 > Spark 2.0

[jira] [Commented] (SPARK-17313) Support spark-shell on cluster mode

2016-08-30 Thread Ewan Leith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450194#comment-15450194 ] Ewan Leith commented on SPARK-17313: I think Apache Zeppelin and Spark Notebook both cover this

[jira] [Assigned] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17320: Assignee: Apache Spark > Spark Mesos module not building on PRs >

[jira] [Updated] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-17320: --- Fix Version/s: (was: 2.0.1) > Spark Mesos module not building on PRs >

[jira] [Updated] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-17320: --- Affects Version/s: (was: 2.0.0) 2.1.0 > Spark Mesos module not

[jira] [Commented] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450113#comment-15450113 ] Apache Spark commented on SPARK-17320: -- User 'mgummelt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17320: Assignee: (was: Apache Spark) > Spark Mesos module not building on PRs >

[jira] [Created] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17320: --- Summary: Spark Mesos module not building on PRs Key: SPARK-17320 URL: https://issues.apache.org/jira/browse/SPARK-17320 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17318: Assignee: (was: Apache Spark) > Fix flaky test: o.a.s.repl.ReplSuite replicating

[jira] [Commented] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450101#comment-15450101 ] Apache Spark commented on SPARK-17318: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17318: Assignee: Apache Spark > Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of

[jira] [Commented] (SPARK-17319) Move addJar from HiveSessionState to HiveExternalCatalog

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450094#comment-15450094 ] Apache Spark commented on SPARK-17319: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17319) Move addJar from HiveSessionState to HiveExternalCatalog

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17319: Assignee: (was: Apache Spark) > Move addJar from HiveSessionState to

[jira] [Assigned] (SPARK-17319) Move addJar from HiveSessionState to HiveExternalCatalog

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17319: Assignee: Apache Spark > Move addJar from HiveSessionState to HiveExternalCatalog >

[jira] [Created] (SPARK-17319) Move addJar from HiveSessionState to HiveExternalCatalog

2016-08-30 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17319: --- Summary: Move addJar from HiveSessionState to HiveExternalCatalog Key: SPARK-17319 URL: https://issues.apache.org/jira/browse/SPARK-17319 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17318: - Description: There are a lot of failures recently:

[jira] [Created] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-17318: Summary: Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl Key: SPARK-17318 URL: https://issues.apache.org/jira/browse/SPARK-17318

[jira] [Resolved] (SPARK-16456) Reuse the uncorrelated scalar subqueries with the same logical plan in a query

2016-08-30 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16456. --- Resolution: Duplicate > Reuse the uncorrelated scalar subqueries with the same

[jira] [Resolved] (SPARK-16419) EnsureRequirements adds extra Sort to already sorted cached table

2016-08-30 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16419. --- Resolution: Duplicate > EnsureRequirements adds extra Sort to already sorted cached

[jira] [Resolved] (SPARK-17314) Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl

2016-08-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17314. -- Resolution: Fixed Fix Version/s: 2.1.0 > Use Netty's DefaultThreadFactory to enable its

[jira] [Resolved] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17304. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14871

[jira] [Commented] (SPARK-17317) Add package vignette to SparkR

2016-08-30 Thread Junyang Qian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450013#comment-15450013 ] Junyang Qian commented on SPARK-17317: -- WIP > Add package vignette to SparkR >

[jira] [Created] (SPARK-17317) Add package vignette to SparkR

2016-08-30 Thread Junyang Qian (JIRA)
Junyang Qian created SPARK-17317: Summary: Add package vignette to SparkR Key: SPARK-17317 URL: https://issues.apache.org/jira/browse/SPARK-17317 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17316: Assignee: Apache Spark > Don't block StandaloneSchedulerBackend.executorRemoved >

[jira] [Assigned] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17316: Assignee: (was: Apache Spark) > Don't block

[jira] [Commented] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449985#comment-15449985 ] Apache Spark commented on SPARK-17316: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

2016-08-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-17316: Summary: Don't block StandaloneSchedulerBackend.executorRemoved Key: SPARK-17316 URL: https://issues.apache.org/jira/browse/SPARK-17316 Project: Spark Issue

[jira] [Commented] (SPARK-17315) Add Kolmogorov-Smirnov Test to SparkR

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449889#comment-15449889 ] Apache Spark commented on SPARK-17315: -- User 'junyangq' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17315) Add Kolmogorov-Smirnov Test to SparkR

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17315: Assignee: (was: Apache Spark) > Add Kolmogorov-Smirnov Test to SparkR >

[jira] [Assigned] (SPARK-17315) Add Kolmogorov-Smirnov Test to SparkR

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17315: Assignee: Apache Spark > Add Kolmogorov-Smirnov Test to SparkR >

[jira] [Created] (SPARK-17315) Add Kolmogorov-Smirnov Test to SparkR

2016-08-30 Thread Junyang Qian (JIRA)
Junyang Qian created SPARK-17315: Summary: Add Kolmogorov-Smirnov Test to SparkR Key: SPARK-17315 URL: https://issues.apache.org/jira/browse/SPARK-17315 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

2016-08-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449830#comment-15449830 ] Steve Loughran commented on SPARK-17307: I think this a subset of SPARK-7481, where I am doing

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-30 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449790#comment-15449790 ] Alex Bozarth commented on SPARK-17243: -- [~ste...@apache.org] [~tgraves] The issues you mentioned are

[jira] [Commented] (SPARK-17313) Support spark-shell on cluster mode

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449684#comment-15449684 ] Sean Owen commented on SPARK-17313: --- The problem is: how are you going to interact with a shell on your

[jira] [Resolved] (SPARK-17312) Support spark-shell on cluster mode

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17312. --- Resolution: Duplicate > Support spark-shell on cluster mode > --- >

[jira] [Reopened] (SPARK-17312) Support spark-shell on cluster mode

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-17312: --- > Support spark-shell on cluster mode > --- > > Key:

[jira] [Assigned] (SPARK-17314) Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17314: Assignee: Shixiong Zhu (was: Apache Spark) > Use Netty's DefaultThreadFactory to enable

[jira] [Assigned] (SPARK-17314) Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17314: Assignee: Apache Spark (was: Shixiong Zhu) > Use Netty's DefaultThreadFactory to enable

[jira] [Commented] (SPARK-17314) Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449670#comment-15449670 ] Apache Spark commented on SPARK-17314: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-17314) Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl

2016-08-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-17314: Summary: Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl Key: SPARK-17314 URL: https://issues.apache.org/jira/browse/SPARK-17314 Project: Spark

[jira] [Reopened] (SPARK-17313) Support spark-shell on cluster mode

2016-08-30 Thread Mahmoud Elgamal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahmoud Elgamal reopened SPARK-17313: - > Support spark-shell on cluster mode > --- > >

[jira] [Closed] (SPARK-17312) Support spark-shell on cluster mode

2016-08-30 Thread Mahmoud Elgamal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahmoud Elgamal closed SPARK-17312. --- Resolution: Fixed > Support spark-shell on cluster mode >

[jira] [Resolved] (SPARK-17313) Support spark-shell on cluster mode

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17313. Resolution: Duplicate > Support spark-shell on cluster mode >

[jira] [Updated] (SPARK-17312) Support spark-shell on cluster mode

2016-08-30 Thread Mahmoud Elgamal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahmoud Elgamal updated SPARK-17312: Issue Type: New Feature (was: Bug) > Support spark-shell on cluster mode >

[jira] [Created] (SPARK-17312) Support spark-shell on cluster mode

2016-08-30 Thread Mahmoud Elgamal (JIRA)
Mahmoud Elgamal created SPARK-17312: --- Summary: Support spark-shell on cluster mode Key: SPARK-17312 URL: https://issues.apache.org/jira/browse/SPARK-17312 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-17313) Support spark-shell on cluster mode

2016-08-30 Thread Mahmoud Elgamal (JIRA)
Mahmoud Elgamal created SPARK-17313: --- Summary: Support spark-shell on cluster mode Key: SPARK-17313 URL: https://issues.apache.org/jira/browse/SPARK-17313 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-17306) Memory leak in QuantileSummaries

2016-08-30 Thread Tim Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449512#comment-15449512 ] Tim Hunter commented on SPARK-17306: [~srowen] yes I had a discussion yesterday with [~clockfly]. The

[jira] [Commented] (SPARK-16402) JDBC source: Implement save API

2016-08-30 Thread Dragisa Krsmanovic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449509#comment-15449509 ] Dragisa Krsmanovic commented on SPARK-16402: Any progress on this ? > JDBC source: Implement

[jira] [Resolved] (SPARK-6460) Implement OpensslAesCtrCryptoCodec to enable encrypted shuffle algorithms which openssl provides

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6460. --- Resolution: Duplicate This was covered by SPARK-5682. > Implement OpensslAesCtrCryptoCodec

[jira] [Resolved] (SPARK-10771) Implement the shuffle encryption with AES-CTR crypto using JCE key provider.

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10771. Resolution: Duplicate This was covered by SPARK-5682. > Implement the shuffle encryption

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449425#comment-15449425 ] Dongjoon Hyun commented on SPARK-17299: --- Hi, [~jbeard] and [~srowen]. For the compatibility, it

[jira] [Resolved] (SPARK-12333) Support shuffle spill encryption in Spark

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12333. Resolution: Fixed This should be covered by SPARK-5682, reopen if that's not correct. >

[jira] [Resolved] (SPARK-5682) Add encrypted shuffle in spark

2016-08-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5682. --- Resolution: Fixed Assignee: Ferdinand Xu Fix Version/s: 2.1.0 > Add encrypted

[jira] [Commented] (SPARK-7445) StringIndexer should handle binary labels properly

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449390#comment-15449390 ] Sean Owen commented on SPARK-7445: -- I think that if you want a specific mapping from labels to integers

[jira] [Commented] (SPARK-7445) StringIndexer should handle binary labels properly

2016-08-30 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449353#comment-15449353 ] Ruben Janssen commented on SPARK-7445: -- [~mengxr] Could you please update on this? > StringIndexer

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449250#comment-15449250 ] Thomas Graves commented on SPARK-17243: --- I agree, there are a ton of ways to improve the history

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448732#comment-15448732 ] Steve Loughran commented on SPARK-17243: One thing to consider here whether there are any ways to

[jira] [Assigned] (SPARK-17311) Standardize Python-Java MLlib API to accept optional long seeds in all cases

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17311: Assignee: Apache Spark (was: Sean Owen) > Standardize Python-Java MLlib API to accept

[jira] [Commented] (SPARK-17311) Standardize Python-Java MLlib API to accept optional long seeds in all cases

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448700#comment-15448700 ] Apache Spark commented on SPARK-17311: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17311) Standardize Python-Java MLlib API to accept optional long seeds in all cases

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17311: Assignee: Sean Owen (was: Apache Spark) > Standardize Python-Java MLlib API to accept

[jira] [Resolved] (SPARK-16832) CrossValidator and TrainValidationSplit are not random without seed

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16832. --- Resolution: Won't Fix Per [~mengxr] I think this is WontFix, and the least confusing thing to do for

[jira] [Created] (SPARK-17311) Standardize Python-Java MLlib API to accept optional long seeds in all cases

2016-08-30 Thread Sean Owen (JIRA)
Sean Owen created SPARK-17311: - Summary: Standardize Python-Java MLlib API to accept optional long seeds in all cases Key: SPARK-17311 URL: https://issues.apache.org/jira/browse/SPARK-17311 Project:

[jira] [Commented] (SPARK-17310) Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side

2016-08-30 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448672#comment-15448672 ] Andrew Duffy commented on SPARK-17310: -- +1 to this, see comments on

[jira] [Assigned] (SPARK-17308) Replace all pattern match on boolean value by if/else block.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17308: Assignee: (was: Apache Spark) > Replace all pattern match on boolean value by if/else

[jira] [Assigned] (SPARK-17308) Replace all pattern match on boolean value by if/else block.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17308: Assignee: Apache Spark > Replace all pattern match on boolean value by if/else block. >

[jira] [Commented] (SPARK-17308) Replace all pattern match on boolean value by if/else block.

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448671#comment-15448671 ] Apache Spark commented on SPARK-17308: -- User 'shiv4nsh' has created a pull request for this issue:

[jira] [Resolved] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17264. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-17276) Stop environment parameters flooding Jenkins build output

2016-08-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17276: -- Assignee: Xin Ren > Stop environment parameters flooding Jenkins build output >

[jira] [Created] (SPARK-17310) Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side

2016-08-30 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-17310: Summary: Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side Key: SPARK-17310 URL: https://issues.apache.org/jira/browse/SPARK-17310

[jira] [Assigned] (SPARK-17309) ALTER VIEW should throw exception if view not exist

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17309: Assignee: Apache Spark (was: Wenchen Fan) > ALTER VIEW should throw exception if view

[jira] [Assigned] (SPARK-17309) ALTER VIEW should throw exception if view not exist

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17309: Assignee: Wenchen Fan (was: Apache Spark) > ALTER VIEW should throw exception if view

[jira] [Commented] (SPARK-17309) ALTER VIEW should throw exception if view not exist

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448588#comment-15448588 ] Apache Spark commented on SPARK-17309: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-17180) Unable to Alter the Temporary View Using ALTER VIEW command

2016-08-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448587#comment-15448587 ] Apache Spark commented on SPARK-17180: -- User 'cloud-fan' has created a pull request for this issue:

  1   2   >