[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Description: Testcase --- {code} import org.apache.spark.sql.expressions.Aggregator import

[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Description: Testcase --- {code} import org.apache.spark.sql.expressions.Aggregator import

[jira] [Commented] (SPARK-12703) Spark KMeans Documentation Python Api

2016-01-14 Thread Anton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098247#comment-15098247 ] Anton commented on SPARK-12703: --- The result is correct now, thanks! > Spark KMeans Documentation Python

[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Description: Testcase --- {code} import org.apache.spark.sql.expressions.Aggregator import

[jira] [Resolved] (SPARK-12784) Spark UI IndexOutOfBoundsException with dynamic allocation

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12784. -- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 1.5.3

[jira] [Updated] (SPARK-12784) Spark UI IndexOutOfBoundsException with dynamic allocation

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12784: - Affects Version/s: 1.6.0 > Spark UI IndexOutOfBoundsException with dynamic allocation >

[jira] [Resolved] (SPARK-12771) Improve code generation for CaseWhen

2016-01-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12771. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > Improve code

[jira] [Commented] (SPARK-8540) KMeans-based outlier detection

2016-01-14 Thread Rakesh Chalasani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098516#comment-15098516 ] Rakesh Chalasani commented on SPARK-8540: - [~josephkb] is this JIRA still of interest? >

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-01-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098546#comment-15098546 ] Reynold Xin commented on SPARK-11857: - Oh yes :) Let me know if you want to deprecate it and submit

[jira] [Commented] (SPARK-11570) ambiguous hostname resolving during startup

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097941#comment-15097941 ] Sean Owen commented on SPARK-11570: --- No, I meant SPARK_LOCAL_HOSTNAME. Have a look at the other JIRA

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097987#comment-15097987 ] Daniel Darabos commented on SPARK-11293: > so should be reopened or not? is there still a memory

[jira] [Reopened] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos reopened SPARK-11293: > Spillable collections leak shuffle memory > - > >

[jira] [Commented] (SPARK-12822) Change default build to Hadoop 2.7

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097935#comment-15097935 ] Sean Owen commented on SPARK-12822: --- At least 2.6, yes. I support this. Though the risk here is

[jira] [Commented] (SPARK-12803) Consider adding ability to profile specific instances of executors in spark

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097933#comment-15097933 ] Sean Owen commented on SPARK-12803: --- You wouldn't know which task executes on which executor anyway

[jira] [Commented] (SPARK-12803) Consider adding ability to profile specific instances of executors in spark

2016-01-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097836#comment-15097836 ] Rajesh Balamohan commented on SPARK-12803: -- Letting the profiler agent run on all executors and

[jira] [Commented] (SPARK-12497) thriftServer does not support semicolon in sql

2016-01-14 Thread Ajesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097992#comment-15097992 ] Ajesh Kumar commented on SPARK-12497: - I think this should be treated as a hive issue rather than

[jira] [Resolved] (SPARK-9844) File appender race condition during SparkWorker shutdown

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9844. -- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 Resolved by

[jira] [Updated] (SPARK-9844) File appender race condition during SparkWorker shutdown

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9844: - Assignee: Bryan Cutler > File appender race condition during SparkWorker shutdown >

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Description: Below is a simple `pyspark` script that tries to split an RDD into a dictionary

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Affects Version/s: 1.5.2 > Failure to maintain consistent RDD references in pyspark >

[jira] [Created] (SPARK-12823) Cannot create UDF with StructType input

2016-01-14 Thread Frank Rosner (JIRA)
Frank Rosner created SPARK-12823: Summary: Cannot create UDF with StructType input Key: SPARK-12823 URL: https://issues.apache.org/jira/browse/SPARK-12823 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12824: -- Component/s: PySpark > Failure to maintain consistent RDD references in pyspark >

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Description: Below is a simple {{pyspark}} script that tries to split an RDD into a dictionary

[jira] [Updated] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-11293: --- Affects Version/s: 1.6.0 > Spillable collections leak shuffle memory >

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098011#comment-15098011 ] Daniel Darabos commented on SPARK-11293: > so add 1.6.0 as affected version... Done. >

[jira] [Created] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
Paul Shearer created SPARK-12824: Summary: Failure to maintain consistent RDD references in pyspark Key: SPARK-12824 URL: https://issues.apache.org/jira/browse/SPARK-12824 Project: Spark

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Description: Below is a simple {{pyspark}} script that tries to split an RDD into a dictionary

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-01-14 Thread Romi Kuntsman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098008#comment-15098008 ] Romi Kuntsman commented on SPARK-11293: --- so add 1.6.0 as affected version... > Spillable

[jira] [Commented] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

2016-01-14 Thread Jayadevan M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098027#comment-15098027 ] Jayadevan M commented on SPARK-12262: - Yea Agreed. This workaround is working fine. But why FORMATTED

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12824: -- Description: Below is a simple `pyspark` script that tries to split an RDD into a dictionary

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Description: Below is a simple `pyspark` script that tries to split an RDD into a dictionary

[jira] [Updated] (SPARK-12555) DatasetAggregatorSuite fails on big-endian platforms

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Description: Testcase --- import org.apache.spark.sql.expressions.Aggregator import

[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Summary: Datasets: data is corrupted when input data is reordered (was: DatasetAggregatorSuite

[jira] [Created] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

2016-01-14 Thread Alex Nederlof (JIRA)
Alex Nederlof created SPARK-12825: - Summary: Spark-submit Jar URL loading fails on redirect Key: SPARK-12825 URL: https://issues.apache.org/jira/browse/SPARK-12825 Project: Spark Issue Type:

[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Description: Testcase --- import org.apache.spark.sql.expressions.Aggregator import

[jira] [Commented] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

2016-01-14 Thread Jayadevan M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098139#comment-15098139 ] Jayadevan M commented on SPARK-12825: - Can you tell the full command. > Spark-submit Jar URL

[jira] [Updated] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Paul Shearer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Shearer updated SPARK-12824: - Description: Below is a simple {{pyspark}} script that tries to split an RDD into a dictionary

[jira] [Updated] (SPARK-12555) Datasets: data is corrupted when input data is reordered

2016-01-14 Thread Tim Preece (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Preece updated SPARK-12555: --- Environment: ALL platforms on 1.6 (was: ALL platforms ( although test only explicitly fails on Big

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-01-14 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098355#comment-15098355 ] Iulian Dragos commented on SPARK-11857: --- Sure, but that seems to be done already... by you:

[jira] [Created] (SPARK-12828) support natural join

2016-01-14 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-12828: --- Summary: support natural join Key: SPARK-12828 URL: https://issues.apache.org/jira/browse/SPARK-12828 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-12783) Dataset map serialization error

2016-01-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101034#comment-15101034 ] Wenchen Fan commented on SPARK-12783: - hi [~babloo80], can you try to change `Map` to

[jira] [Commented] (SPARK-12829) Turn Java style checker on

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101094#comment-15101094 ] Apache Spark commented on SPARK-12829: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12829) Turn Java style checker on

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12829: Assignee: Reynold Xin (was: Apache Spark) > Turn Java style checker on >

[jira] [Created] (SPARK-12829) Turn Java style checker on

2016-01-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12829: --- Summary: Turn Java style checker on Key: SPARK-12829 URL: https://issues.apache.org/jira/browse/SPARK-12829 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-7703) Task failure caused by block fetch failure in BlockManager.doGetRemote() when using TorrentBroadcast

2016-01-14 Thread Hailong Wen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101119#comment-15101119 ] Hailong Wen commented on SPARK-7703: Note that this issue is exactly the same with SPARK-9591 and

[jira] [Commented] (SPARK-9591) Job failed for exception during getting Broadcast variable

2016-01-14 Thread Hailong Wen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101123#comment-15101123 ] Hailong Wen commented on SPARK-9591: SPARK-7703 is also caused by this and it is marked as a duplicate

[jira] [Commented] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099069#comment-15099069 ] Shixiong Zhu commented on SPARK-12826: -- This line is weird: 16/01/14 18:23:30 INFO Worker:

[jira] [Updated] (SPARK-11559) Make `runs` no effect in k-means

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11559: -- Target Version/s: 2.0.0 (was: ) > Make `runs` no effect in k-means >

[jira] [Updated] (SPARK-12363) PowerIterationClustering test case failed if we deprecated KMeans.setRuns

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-12363: -- Target Version/s: 2.0.0 > PowerIterationClustering test case failed if we deprecated

[jira] [Resolved] (SPARK-12174) Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another location when Exception throw"

2016-01-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12174. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Slow test:

[jira] [Assigned] (SPARK-12828) support natural join

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12828: Assignee: (was: Apache Spark) > support natural join > > >

[jira] [Assigned] (SPARK-12828) support natural join

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12828: Assignee: Apache Spark > support natural join > > >

[jira] [Resolved] (SPARK-12813) Eliminate serialization for back to back operations

2016-01-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-12813. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10747

[jira] [Commented] (SPARK-12828) support natural join

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101029#comment-15101029 ] Apache Spark commented on SPARK-12828: -- User 'adrian-wang' has created a pull request for this

[jira] [Assigned] (SPARK-12829) Turn Java style checker on

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12829: Assignee: Apache Spark (was: Reynold Xin) > Turn Java style checker on >

[jira] [Updated] (SPARK-12691) Multiple unionAll on Dataframe goes growingly slow.

2016-01-14 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Liang updated SPARK-12691: Component/s: (was: Spark Core) SQL > Multiple unionAll on Dataframe goes

[jira] [Resolved] (SPARK-7703) Task failure caused by block fetch failure in BlockManager.doGetRemote() when using TorrentBroadcast

2016-01-14 Thread Hailong Wen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hailong Wen resolved SPARK-7703. Resolution: Fixed Fix Version/s: 1.6.0 Already fixed in 1.6.0. > Task failure caused by

[jira] [Assigned] (SPARK-12174) Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another location when Exception throw"

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12174: Assignee: (was: Apache Spark) > Slow test: BlockManagerSuite."SPARK-9591:

[jira] [Assigned] (SPARK-12174) Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another location when Exception throw"

2016-01-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-12174: -- Assignee: Josh Rosen > Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another

[jira] [Commented] (SPARK-12174) Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another location when Exception throw"

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099020#comment-15099020 ] Apache Spark commented on SPARK-12174: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12174) Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from another location when Exception throw"

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12174: Assignee: Apache Spark > Slow test: BlockManagerSuite."SPARK-9591: getRemoteBytes from

[jira] [Commented] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099039#comment-15099039 ] Shixiong Zhu commented on SPARK-12825: -- Is "redirect" implemented with JavaScript, or 301/302 http

[jira] [Created] (SPARK-12827) Configurable bind address for WebUI

2016-01-14 Thread Zee Chen (JIRA)
Zee Chen created SPARK-12827: Summary: Configurable bind address for WebUI Key: SPARK-12827 URL: https://issues.apache.org/jira/browse/SPARK-12827 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099211#comment-15099211 ] Marcelo Vanzin commented on SPARK-12826: Sounds related to SPARK-12308. > Spark Workers do not

[jira] [Assigned] (SPARK-11560) Optimize KMeans implementation

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11560: Assignee: Apache Spark > Optimize KMeans implementation > --

[jira] [Assigned] (SPARK-11560) Optimize KMeans implementation

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11560: Assignee: (was: Apache Spark) > Optimize KMeans implementation >

[jira] [Commented] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Alan Braithwaite (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099079#comment-15099079 ] Alan Braithwaite commented on SPARK-12826: -- Yes, we set it to listen to the any address because

[jira] [Commented] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099099#comment-15099099 ] Shixiong Zhu commented on SPARK-12826: -- I have a hack. In onDisconnected {code} override def

[jira] [Updated] (SPARK-10388) Public dataset loader interface

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10388: -- Assignee: (was: Xiangrui Meng) > Public dataset loader interface >

[jira] [Commented] (SPARK-10388) Public dataset loader interface

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099118#comment-15099118 ] Xiangrui Meng commented on SPARK-10388: --- [~zjffdu] Thanks for posting the design doc! There might

[jira] [Updated] (SPARK-11559) Make `runs` no effect in k-means

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11559: -- Assignee: Yanbo Liang > Make `runs` no effect in k-means > >

[jira] [Updated] (SPARK-11559) Make `runs` no effect in k-means

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11559: -- Shepherd: Xiangrui Meng > Make `runs` no effect in k-means >

[jira] [Updated] (SPARK-11559) Make `runs` no effect in k-means

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11559: -- Description: We deprecated `runs` in Spark 1.6 (SPARK-11358). In 2.0, we can either remove

[jira] [Commented] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

2016-01-14 Thread Xiu (Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099007#comment-15099007 ] Xiu (Joe) Guo commented on SPARK-12262: --- You might want to check out this JIRA:

[jira] [Closed] (SPARK-7903) PythonUDT shouldn't get serialized on the Scala side

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7903. Resolution: Duplicate > PythonUDT shouldn't get serialized on the Scala side >

[jira] [Updated] (SPARK-10388) Public dataset loader interface

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10388: -- Shepherd: Xiangrui Meng > Public dataset loader interface > --- >

[jira] [Commented] (SPARK-11219) Make Parameter Description Format Consistent in PySpark.MLlib

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099258#comment-15099258 ] Xiangrui Meng commented on SPARK-11219: --- + [~davies] and [~joshrosen] for Python style discussion

[jira] [Commented] (SPARK-12822) Change default build to Hadoop 2.7

2016-01-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098620#comment-15098620 ] Marcelo Vanzin commented on SPARK-12822: Hmm... if 2.2 is still supported, I'd rather keep the

[jira] [Assigned] (SPARK-12799) Simplify various string output for expressions

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12799: Assignee: Apache Spark > Simplify various string output for expressions >

[jira] [Assigned] (SPARK-12799) Simplify various string output for expressions

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12799: Assignee: (was: Apache Spark) > Simplify various string output for expressions >

[jira] [Commented] (SPARK-12799) Simplify various string output for expressions

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098710#comment-15098710 ] Apache Spark commented on SPARK-12799: -- User 'liancheng' has created a pull request for this issue:

[jira] [Resolved] (SPARK-12821) Style checker should run when some configuration files for style are modified but any source files are not.

2016-01-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12821. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10754

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

2016-01-14 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098734#comment-15098734 ] Zhan Zhang commented on SPARK-5159: --- This issue is definitely broken. But fixing it needs a complete

[jira] [Commented] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098738#comment-15098738 ] Maciej Szymkiewicz commented on SPARK-12824: ??It seems that all the keys in the dictionary

[jira] [Updated] (SPARK-12821) Style checker should run when some configuration files for style are modified but any source files are not.

2016-01-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12821: --- Assignee: Kousuke Saruta > Style checker should run when some configuration files for style are

[jira] [Commented] (SPARK-10809) Single-document topicDistributions method for LocalLDAModel

2016-01-14 Thread Crawdaddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098737#comment-15098737 ] Crawdaddy commented on SPARK-10809: --- With a 100K document / 200K feature model with K = 250, even this

[jira] [Comment Edited] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098738#comment-15098738 ] Maciej Szymkiewicz edited comment on SPARK-12824 at 1/14/16 7:51 PM: -

[jira] [Comment Edited] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

2016-01-14 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098738#comment-15098738 ] Maciej Szymkiewicz edited comment on SPARK-12824 at 1/14/16 7:55 PM: -

[jira] [Resolved] (SPARK-12829) Turn Java style checker on

2016-01-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12829. - Resolution: Fixed Fix Version/s: 2.0.0 > Turn Java style checker on >

[jira] [Assigned] (SPARK-12830) Java style: disallow trailing whitespaces

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12830: Assignee: Apache Spark (was: Reynold Xin) > Java style: disallow trailing whitespaces >

[jira] [Created] (SPARK-12830) Java style: disallow trailing whitespaces

2016-01-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12830: --- Summary: Java style: disallow trailing whitespaces Key: SPARK-12830 URL: https://issues.apache.org/jira/browse/SPARK-12830 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12830) Java style: disallow trailing whitespaces

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101211#comment-15101211 ] Apache Spark commented on SPARK-12830: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12830) Java style: disallow trailing whitespaces

2016-01-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12830: Assignee: Reynold Xin (was: Apache Spark) > Java style: disallow trailing whitespaces >

[jira] [Created] (SPARK-12831) akka.remote.OversizedPayloadException on DirectTaskResult

2016-01-14 Thread Brett Stime (JIRA)
Brett Stime created SPARK-12831: --- Summary: akka.remote.OversizedPayloadException on DirectTaskResult Key: SPARK-12831 URL: https://issues.apache.org/jira/browse/SPARK-12831 Project: Spark

[jira] [Created] (SPARK-12832) spark on mesos dispacher need a constraints

2016-01-14 Thread astralidea (JIRA)
astralidea created SPARK-12832: -- Summary: spark on mesos dispacher need a constraints Key: SPARK-12832 URL: https://issues.apache.org/jira/browse/SPARK-12832 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-12708) Sorting task error in Stages Page when yarn mode

2016-01-14 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-12708: --- Assignee: Koyo Yoshida > Sorting task error in Stages Page when yarn mode >

[jira] [Resolved] (SPARK-12708) Sorting task error in Stages Page when yarn mode

2016-01-14 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-12708. Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Sorting task

[jira] [Comment Edited] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099069#comment-15099069 ] Shixiong Zhu edited comment on SPARK-12826 at 1/14/16 10:47 PM: The issue

[jira] [Comment Edited] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

2016-01-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15099069#comment-15099069 ] Shixiong Zhu edited comment on SPARK-12826 at 1/14/16 10:46 PM: This line

[jira] [Issue Comment Deleted] (SPARK-7751) Add @Since annotation to stable and experimental methods in MLlib

2016-01-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7751: - Comment: was deleted (was: User 'petz2000' has created a pull request for this issue:

  1   2   >