[jira] [Updated] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

2016-09-08 Thread Xing Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Shi updated SPARK-17465: - Description: After updating Spark from 1.5.0 to 1.6.0, I found that it seems to have a memory leak on my

[jira] [Updated] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

2016-09-08 Thread Xing Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Shi updated SPARK-17465: - Description: After updating Spark from 1.5.0 to 1.6.0, I found that it seems to have a memory leak on my

[jira] [Updated] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

2016-09-08 Thread Xing Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Shi updated SPARK-17465: - Affects Version/s: 1.6.0 1.6.1 1.6.2 Description:

[jira] [Created] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to a memory leak proportional to total number of tasks

2016-09-08 Thread Xing Shi (JIRA)
Xing Shi created SPARK-17465: Summary: Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to a memory leak proportional to total number of tasks Key: SPARK-17465 URL: https://issues.ap

[jira] [Issue Comment Deleted] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17245: -- Comment: was deleted (was: please check the issue https://issues.apache.org/jira/browse/SPARK-17447 thi

[jira] [Resolved] (SPARK-17448) There should a limit of k in mllib.Kmeans

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17448. --- Resolution: Not A Problem > There should a limit of k in mllib.Kmeans > -

[jira] [Updated] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17447: -- Flags: (was: Patch,Important) Please also read https://cwiki.apache.org/confluence/display/SPARK/Con

[jira] [Updated] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17447: -- Labels: (was: performance) > performance improvement in Partitioner.DefaultPartitioner > ---

[jira] [Updated] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17447: -- Priority: Trivial (was: Major) It's not going to be a bottleneck though; the size of this array is hun

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475943#comment-15475943 ] Shivaram Venkataraman commented on SPARK-17428: --- I think there are bunch of

[jira] [Commented] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-09-08 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475860#comment-15475860 ] Peng Meng commented on SPARK-6160: -- hi [~GayathriMurali], are you still working on this,

[jira] [Assigned] (SPARK-17464) SparkR spark.als arguments reg should be 0.1 by default

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17464: Assignee: (was: Apache Spark) > SparkR spark.als arguments reg should be 0.1 by defaul

[jira] [Commented] (SPARK-17464) SparkR spark.als arguments reg should be 0.1 by default

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475849#comment-15475849 ] Apache Spark commented on SPARK-17464: -- User 'yanboliang' has created a pull request

[jira] [Assigned] (SPARK-17464) SparkR spark.als arguments reg should be 0.1 by default

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17464: Assignee: Apache Spark > SparkR spark.als arguments reg should be 0.1 by default > ---

[jira] [Created] (SPARK-17464) SparkR spark.als arguments reg should be 0.1 by default

2016-09-08 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-17464: --- Summary: SparkR spark.als arguments reg should be 0.1 by default Key: SPARK-17464 URL: https://issues.apache.org/jira/browse/SPARK-17464 Project: Spark Issue T

[jira] [Commented] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-09-08 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475769#comment-15475769 ] Peng Meng commented on SPARK-6160: -- Hi [~josephkb], I have some discussion with [~srowen]

[jira] [Commented] (SPARK-15509) R MLlib algorithms should support input columns "features" and "label"

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475758#comment-15475758 ] WangJianfei commented on SPARK-15509: - please check my issue https://issues.apache.or

[jira] [Commented] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475756#comment-15475756 ] WangJianfei commented on SPARK-17245: - please check the issue https://issues.apache.o

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475754#comment-15475754 ] WangJianfei commented on SPARK-17387: - please check https://issues.apache.org/jira/br

[jira] [Commented] (SPARK-17449) Relation between heartbeatInterval and network timeout

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475727#comment-15475727 ] Yang Liang commented on SPARK-17449: Sorry , let me clarify it . The relation between

[jira] [Updated] (SPARK-17449) Relation between heartbeatInterval and network timeout

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Liang updated SPARK-17449: --- Description: $ spark-shell --master yarn --conf spark.executor.heartbeatInterval=20s --num-executors

[jira] [Updated] (SPARK-17449) executorTimeoutMs configure error

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Liang updated SPARK-17449: --- Description: $ spark-shell --master yarn --conf spark.executor.heartbeatInterval=20s --num-executors

[jira] [Updated] (SPARK-17449) Relation between

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Liang updated SPARK-17449: --- Summary: Relation between (was: executorTimeoutMs configure error) > Relation between > --

[jira] [Updated] (SPARK-17449) Relation between heartbeatInterval and network timeout

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Liang updated SPARK-17449: --- Summary: Relation between heartbeatInterval and network timeout (was: Relation between ) > Relation

[jira] [Commented] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-09-08 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475709#comment-15475709 ] Peng Meng commented on SPARK-6160: -- hi Joseph K. Bradley > ChiSqSelector should keep tes

[jira] [Issue Comment Deleted] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-09-08 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Meng updated SPARK-6160: - Comment: was deleted (was: hi Joseph K. Bradley) > ChiSqSelector should keep test statistic info > --

[jira] [Updated] (SPARK-17449) executorTimeoutMs configure error

2016-09-08 Thread Yang Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Liang updated SPARK-17449: --- Description: $ spark-shell --master yarn --conf spark.executor.heartbeatInterval=20s --num-executors

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475664#comment-15475664 ] Jeff Zhang commented on SPARK-17428: Found another elegant way to specify version, us

[jira] [Comment Edited] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475607#comment-15475607 ] WangJianfei edited comment on SPARK-17447 at 9/9/16 2:28 AM: -

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475645#comment-15475645 ] Jeff Zhang commented on SPARK-17428: I just link the jira of python virtualenv. It s

[jira] [Comment Edited] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475607#comment-15475607 ] WangJianfei edited comment on SPARK-17447 at 9/9/16 2:12 AM: -

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475630#comment-15475630 ] Jeff Zhang commented on SPARK-17428: Source code url needs to be specified for versio

[jira] [Commented] (SPARK-17448) There should a limit of k in mllib.Kmeans

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475619#comment-15475619 ] WangJianfei commented on SPARK-17448: - Mabye we can limit k according to the number o

[jira] [Comment Edited] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475612#comment-15475612 ] Felix Cheung edited comment on SPARK-17428 at 9/9/16 1:59 AM: -

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475612#comment-15475612 ] Felix Cheung commented on SPARK-17428: -- I don't think I see a way to specify a versi

[jira] [Commented] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread WangJianfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475607#comment-15475607 ] WangJianfei commented on SPARK-17447: - you can look this source code as below: we ca

[jira] [Comment Edited] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475593#comment-15475593 ] Sun Rui edited comment on SPARK-17428 at 9/9/16 1:52 AM: - I don't

[jira] [Comment Edited] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475593#comment-15475593 ] Sun Rui edited comment on SPARK-17428 at 9/9/16 1:50 AM: - I don't

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475593#comment-15475593 ] Sun Rui commented on SPARK-17428: - I don't understand the meaning of exact version contro

[jira] [Created] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17463: -- Summary: Serialization of accumulators in heartbeats is not thread-safe Key: SPARK-17463 URL: https://issues.apache.org/jira/browse/SPARK-17463 Project: Spark I

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475536#comment-15475536 ] Josh Rosen commented on SPARK-17463: [~zsxwing], FYI, since you're good at these type

[jira] [Created] (SPARK-17462) Check for places within MLlib which should use VersionUtils to parse Spark version strings

2016-09-08 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-17462: - Summary: Check for places within MLlib which should use VersionUtils to parse Spark version strings Key: SPARK-17462 URL: https://issues.apache.org/jira/browse/SPARK-174

[jira] [Resolved] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-09-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-15487. -- Resolution: Fixed Fix Version/s: 2.1.0 > Spark Master UI to reverse proxy Application an

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475418#comment-15475418 ] Apache Spark commented on SPARK-17387: -- User 'BryanCutler' has created a pull reques

[jira] [Updated] (SPARK-17460) Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative

2016-09-08 Thread Chris Perluss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Perluss updated SPARK-17460: -- Affects Version/s: 2.0.0 > Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes b

[jira] [Closed] (SPARK-17461) Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative

2016-09-08 Thread Chris Perluss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Perluss closed SPARK-17461. - Resolution: Duplicate Duplicate of SPARK-17460 > Dataset.joinWith causes OutOfMemory due to logi

[jira] [Created] (SPARK-17461) Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative

2016-09-08 Thread Chris Perluss (JIRA)
Chris Perluss created SPARK-17461: - Summary: Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative Key: SPARK-17461 URL: https://issues.apache.org/jira/browse/SPARK-17461 P

[jira] [Created] (SPARK-17460) Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative

2016-09-08 Thread Chris Perluss (JIRA)
Chris Perluss created SPARK-17460: - Summary: Dataset.joinWith causes OutOfMemory due to logicalPlan sizeInBytes being negative Key: SPARK-17460 URL: https://issues.apache.org/jira/browse/SPARK-17460 P

[jira] [Created] (SPARK-17459) Add Linear Discriminant to dimensionality reduction algorithms

2016-09-08 Thread Joshua Howard (JIRA)
Joshua Howard created SPARK-17459: - Summary: Add Linear Discriminant to dimensionality reduction algorithms Key: SPARK-17459 URL: https://issues.apache.org/jira/browse/SPARK-17459 Project: Spark

[jira] [Resolved] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17405. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15016 [https://github.

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475253#comment-15475253 ] Marcelo Vanzin commented on SPARK-17387: Yeah, that's what I mean. Running the py

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475242#comment-15475242 ] Bryan Cutler commented on SPARK-17387: -- [~vanzin] you said if you use PySpark you co

[jira] [Updated] (SPARK-17458) Alias specified for aggregates in a pivot are not honored

2016-09-08 Thread Ravi Somepalli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Somepalli updated SPARK-17458: --- Description: When using pivot and multiple aggregations we need to alias to avoid special ch

[jira] [Updated] (SPARK-17458) Alias specified for aggregates in a pivot are not honored

2016-09-08 Thread Ravi Somepalli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Somepalli updated SPARK-17458: --- Description: When using pivot and multiple aggregations we need to alias to avoid special ch

[jira] [Created] (SPARK-17458) Alias specified for aggregates in a pivot are not honored

2016-09-08 Thread Ravi Somepalli (JIRA)
Ravi Somepalli created SPARK-17458: -- Summary: Alias specified for aggregates in a pivot are not honored Key: SPARK-17458 URL: https://issues.apache.org/jira/browse/SPARK-17458 Project: Spark

[jira] [Updated] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2016-09-08 Thread Nic Eggert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nic Eggert updated SPARK-17455: --- Priority: Minor (was: Major) > IsotonicRegression takes non-polynomial time for some inputs > --

[jira] [Assigned] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17455: Assignee: (was: Apache Spark) > IsotonicRegression takes non-polynomial time for some

[jira] [Assigned] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17455: Assignee: Apache Spark > IsotonicRegression takes non-polynomial time for some inputs > --

[jira] [Commented] (SPARK-17302) Cannot set non-Spark SQL session variables in hive-site.xml, spark-defaults.conf, or using --conf

2016-09-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475156#comment-15475156 ] Ryan Blue commented on SPARK-17302: --- In 1.6.x, Spark pulled session config for Hive fro

[jira] [Commented] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475158#comment-15475158 ] Apache Spark commented on SPARK-17455: -- User 'neggert' has created a pull request fo

[jira] [Updated] (SPARK-17457) Spark SQL shows poor performance for group by and sort by on multiple columns

2016-09-08 Thread Sabyasachi Nayak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sabyasachi Nayak updated SPARK-17457: - Description: In one of the use case when we are running one hive query with Tez it is tak

[jira] [Updated] (SPARK-17457) Spark SQL shows poor performance for group by and sort by on multiple columns

2016-09-08 Thread Sabyasachi Nayak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sabyasachi Nayak updated SPARK-17457: - Summary: Spark SQL shows poor performance for group by and sort by on multiple columns

[jira] [Created] (SPARK-17457) Spark SQL shows poor performance for group by on multiple columns

2016-09-08 Thread Sabyasachi Nayak (JIRA)
Sabyasachi Nayak created SPARK-17457: Summary: Spark SQL shows poor performance for group by on multiple columns Key: SPARK-17457 URL: https://issues.apache.org/jira/browse/SPARK-17457 Project: S

[jira] [Updated] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17405: --- Assignee: Eric Liang > Simple aggregation query OOMing after SPARK-16525 > --

[jira] [Assigned] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17405: Assignee: (was: Apache Spark) > Simple aggregation query OOMing after SPARK-16525 > --

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475040#comment-15475040 ] Apache Spark commented on SPARK-17405: -- User 'ericl' has created a pull request for

[jira] [Assigned] (SPARK-17456) Utility for parsing Spark versions

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17456: Assignee: Apache Spark (was: Joseph K. Bradley) > Utility for parsing Spark versions > --

[jira] [Commented] (SPARK-17456) Utility for parsing Spark versions

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475041#comment-15475041 ] Apache Spark commented on SPARK-17456: -- User 'jkbradley' has created a pull request

[jira] [Assigned] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17405: Assignee: Apache Spark > Simple aggregation query OOMing after SPARK-16525 > -

[jira] [Assigned] (SPARK-17456) Utility for parsing Spark versions

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17456: Assignee: Joseph K. Bradley (was: Apache Spark) > Utility for parsing Spark versions > --

[jira] [Commented] (SPARK-12452) Add exception details to TaskCompletionListener/TaskContext

2016-09-08 Thread Neelesh Shastry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475027#comment-15475027 ] Neelesh Shastry commented on SPARK-12452: - This was originally filed for 1.5.2, w

[jira] [Commented] (SPARK-16525) Enable Row Based HashMap in HashAggregateExec

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475023#comment-15475023 ] Apache Spark commented on SPARK-16525: -- User 'ericl' has created a pull request for

[jira] [Commented] (SPARK-17446) no total size for data source tables in InMemoryCatalog

2016-09-08 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474995#comment-15474995 ] Zhenhua Wang commented on SPARK-17446: -- Ok, I've added the description. Thanks. >

[jira] [Updated] (SPARK-17446) no total size for data source tables in InMemoryCatalog

2016-09-08 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17446: - Description: For data source table in InMemoryCatalog, it's catalogTable.storage.locationUri is N

[jira] [Commented] (SPARK-17456) Utility for parsing Spark versions

2016-09-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474917#comment-15474917 ] Joseph K. Bradley commented on SPARK-17456: --- Linking a JIRA which will require

[jira] [Created] (SPARK-17456) Utility for parsing Spark versions

2016-09-08 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-17456: - Summary: Utility for parsing Spark versions Key: SPARK-17456 URL: https://issues.apache.org/jira/browse/SPARK-17456 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-11035) Launcher: allow apps to be launched in-process

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11035: Assignee: (was: Apache Spark) > Launcher: allow apps to be launched in-process > -

[jira] [Assigned] (SPARK-11035) Launcher: allow apps to be launched in-process

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11035: Assignee: Apache Spark > Launcher: allow apps to be launched in-process >

[jira] [Commented] (SPARK-11035) Launcher: allow apps to be launched in-process

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474872#comment-15474872 ] Apache Spark commented on SPARK-11035: -- User 'kishorvpatil' has created a pull reque

[jira] [Created] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2016-09-08 Thread Nic Eggert (JIRA)
Nic Eggert created SPARK-17455: -- Summary: IsotonicRegression takes non-polynomial time for some inputs Key: SPARK-17455 URL: https://issues.apache.org/jira/browse/SPARK-17455 Project: Spark Iss

[jira] [Commented] (SPARK-16445) Multilayer Perceptron Classifier wrapper in SparkR

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474688#comment-15474688 ] Apache Spark commented on SPARK-16445: -- User 'keypointt' has created a pull request

[jira] [Created] (SPARK-17454) Add option to specify Mesos resource offer constraints

2016-09-08 Thread Chris Bannister (JIRA)
Chris Bannister created SPARK-17454: --- Summary: Add option to specify Mesos resource offer constraints Key: SPARK-17454 URL: https://issues.apache.org/jira/browse/SPARK-17454 Project: Spark

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

2016-09-08 Thread Josh Elser (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474616#comment-15474616 ] Josh Elser commented on SPARK-17445: bq. I think one part you're missing, Josh, is th

[jira] [Updated] (SPARK-17453) Broadcast block already exists in MemoryStore

2016-09-08 Thread Chris Bannister (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Bannister updated SPARK-17453: Description: Whilst doing a broadcast join we reliably hit this exception, the code worked

[jira] [Created] (SPARK-17453) Broadcast block already exists in MemoryStore

2016-09-08 Thread Chris Bannister (JIRA)
Chris Bannister created SPARK-17453: --- Summary: Broadcast block already exists in MemoryStore Key: SPARK-17453 URL: https://issues.apache.org/jira/browse/SPARK-17453 Project: Spark Issue Typ

[jira] [Commented] (SPARK-17428) SparkR executors/workers support virtualenv

2016-09-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474587#comment-15474587 ] Felix Cheung commented on SPARK-17428: -- Agree with above. And to be clear, packrat i

[jira] [Commented] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

2016-09-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474557#comment-15474557 ] Thomas Graves commented on SPARK-17321: --- so there are 2 possible things here: 1) Y

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

2016-09-08 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474543#comment-15474543 ] Matei Zaharia commented on SPARK-17445: --- I think one part you're missing, Josh, is

[jira] [Assigned] (SPARK-17429) spark sql length(1) return error

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17429: Assignee: (was: Apache Spark) > spark sql length(1) return error > ---

[jira] [Assigned] (SPARK-17429) spark sql length(1) return error

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17429: Assignee: Apache Spark > spark sql length(1) return error > --

[jira] [Commented] (SPARK-17429) spark sql length(1) return error

2016-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474532#comment-15474532 ] Apache Spark commented on SPARK-17429: -- User 'cenyuhai' has created a pull request f

[jira] [Commented] (SPARK-17450) spark sql rownumber OOM

2016-09-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474517#comment-15474517 ] Herman van Hovell commented on SPARK-17450: --- You could try. You would also have

[jira] [Commented] (SPARK-17450) spark sql rownumber OOM

2016-09-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474518#comment-15474518 ] Herman van Hovell commented on SPARK-17450: --- You could try. You would also have

[jira] [Issue Comment Deleted] (SPARK-17450) spark sql rownumber OOM

2016-09-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17450: -- Comment: was deleted (was: You could try. You would also have to add the follow-up by d

[jira] [Comment Edited] (SPARK-17450) spark sql rownumber OOM

2016-09-08 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474430#comment-15474430 ] cen yuhai edited comment on SPARK-17450 at 9/8/16 5:07 PM: --- hi,

[jira] [Commented] (SPARK-17450) spark sql rownumber OOM

2016-09-08 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474430#comment-15474430 ] cen yuhai commented on SPARK-17450: --- hi,herman, can i merge your pr for native spark wi

[jira] [Commented] (SPARK-17443) SparkLauncher should allow stoppingApplication and need not rely on SparkSubmit binary

2016-09-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474386#comment-15474386 ] Marcelo Vanzin commented on SPARK-17443: The second bullet is actually SPARK-1103

[jira] [Commented] (SPARK-17446) no total size for data source tables in InMemoryCatalog

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474385#comment-15474385 ] Sean Owen commented on SPARK-17446: --- [~ZenWzh] there is no detail at all here. Please d

[jira] [Commented] (SPARK-17449) executorTimeoutMs configure error

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474381#comment-15474381 ] Sean Owen commented on SPARK-17449: --- Sorry, I don't see the problem? the configured tim

[jira] [Commented] (SPARK-17447) performance improvement in Partitioner.DefaultPartitioner

2016-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474375#comment-15474375 ] Sean Owen commented on SPARK-17447: --- Why don't they need to be sorted? > performance i

  1   2   >