[jira] [Updated] (SPARK-4604) Make MatrixFactorizationModel constructor public

[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4604: - Fix Version/s: (was: 1.2.0) > Make MatrixFactorizationModel constructor public > -

[jira] [Reopened] (SPARK-4583) GradientBoostedTrees error logging should use loss being minimized

[ https://issues.apache.org/jira/browse/SPARK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-4583: -- > GradientBoostedTrees error logging should use loss being minimized > -

[jira] [Updated] (SPARK-4583) GradientBoostedTrees error logging should use loss being minimized

[ https://issues.apache.org/jira/browse/SPARK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4583: - Fix Version/s: (was: 1.2.0) > GradientBoostedTrees error logging should use loss being minimiz

[jira] [Commented] (SPARK-4583) GradientBoostedTrees error logging should use loss being minimized

[ https://issues.apache.org/jira/browse/SPARK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225866#comment-14225866 ] Apache Spark commented on SPARK-4583: - User 'mengxr' has created a pull request for th

[jira] [Reopened] (SPARK-4604) Make MatrixFactorizationModel constructor public

[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-4604: -- > Make MatrixFactorizationModel constructor public > ---

[jira] [Commented] (SPARK-4604) Make MatrixFactorizationModel constructor public

[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225862#comment-14225862 ] Apache Spark commented on SPARK-4604: - User 'mengxr' has created a pull request for th

[jira] [Closed] (SPARK-4594) Improvement the broadcast for HiveConf

[ https://issues.apache.org/jira/browse/SPARK-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo closed SPARK-4594. -- Resolution: Fixed > Improvement the broadcast for HiveConf > -- > >

[jira] [Updated] (SPARK-4612) Configuration object gets created for every task even if not new file/jar is added

[ https://issues.apache.org/jira/browse/SPARK-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4612: - Target Version/s: 1.3.0 > Configuration object gets created for every task even if not new file/ja

[jira] [Updated] (SPARK-4612) Configuration object gets created for every task even if not new file/jar is added

[ https://issues.apache.org/jira/browse/SPARK-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4612: - Priority: Critical (was: Major) > Configuration object gets created for every task even if not ne

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225790#comment-14225790 ] zzc commented on SPARK-2468: Great. I will test it later. > Netty-based block server / client

[jira] [Updated] (SPARK-3936) Remove auto join elimination and introduce aggregateMessages

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3936: --- Description: This is actually a ticket with two separate problems: 1. Remove auto join elimination 2

[jira] [Commented] (SPARK-3936) Remove auto join elimination and introduce aggregateMessages

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225788#comment-14225788 ] Reynold Xin commented on SPARK-3936: BTW message from Taobao: 根据团队小伙伴的测试，[SPARK-3936]

[jira] [Updated] (SPARK-3936) Remove auto join elimination and introduce aggregateMessages

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3936: --- Summary: Remove auto join elimination and introduce aggregateMessages (was: Incorrect result in Graph

[jira] [Updated] (SPARK-3385) Improve shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3385: --- Assignee: (was: Reynold Xin) > Improve shuffle performance > --- > >

[jira] [Updated] (SPARK-3385) Improve shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3385: --- Target Version/s: 1.3.0 (was: 1.2.0) > Improve shuffle performance > --- > >

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225778#comment-14225778 ] Reynold Xin commented on SPARK-2468: Glad that we are able to resolve this! > Netty-b

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225773#comment-14225773 ] Lianhui Wang commented on SPARK-2468: - [~adav] that 's very great. With patch #3465, I

[jira] [Commented] (SPARK-3779) yarn spark.yarn.applicationMaster.waitTries config should be changed to a time period

[ https://issues.apache.org/jira/browse/SPARK-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225762#comment-14225762 ] Apache Spark commented on SPARK-3779: - User 'sryza' has created a pull request for thi

[jira] [Commented] (SPARK-4618) Make foreign DDL commands options case-insensitive

[ https://issues.apache.org/jira/browse/SPARK-4618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225749#comment-14225749 ] Apache Spark commented on SPARK-4618: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-4618) Make foreign DDL commands options case-insensitive

wangfei created SPARK-4618: -- Summary: Make foreign DDL commands options case-insensitive Key: SPARK-4618 URL: https://issues.apache.org/jira/browse/SPARK-4618 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4516. Resolution: Fixed Fix Version/s: 1.2.0 > Netty off-heap memory use causes executors t

[jira] [Commented] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225743#comment-14225743 ] Aaron Davidson commented on SPARK-4516: --- About my last point, [~rxin], [~pwendell],

[jira] [Created] (SPARK-4617) Fix spark.yarn.applicationMaster.waitTries doc

Sandy Ryza created SPARK-4617: - Summary: Fix spark.yarn.applicationMaster.waitTries doc Key: SPARK-4617 URL: https://issues.apache.org/jira/browse/SPARK-4617 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4516: - Assignee: Aaron Davidson > Netty off-heap memory use causes executors to be killed by OS > ---

[jira] [Commented] (SPARK-3588) Gaussian Mixture Model clustering

[ https://issues.apache.org/jira/browse/SPARK-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225738#comment-14225738 ] Meethu Mathew commented on SPARK-3588: -- [~mengxr] We are happy to collaborate with [~

[jira] [Updated] (SPARK-4451) force to kill process after 5 seconds

[ https://issues.apache.org/jira/browse/SPARK-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic updated SPARK-4451: --- Description: When we stop history server or thrift server, sometimes they were not totally st

[jira] [Commented] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225731#comment-14225731 ] Apache Spark commented on SPARK-4516: - User 'aarondav' has created a pull request for

[jira] [Comment Edited] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225728#comment-14225728 ] Daniel Erenrich edited comment on SPARK-4001 at 11/26/14 4:39 AM: --

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225728#comment-14225728 ] Daniel Erenrich commented on SPARK-4001: I was about to start coding something lik

[jira] [Updated] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4584: -- Assignee: Marcelo Vanzin (was: Sandy Ryza) > 2x Performance regression for Spark-on-YARN >

[jira] [Created] (SPARK-4616) SPARK_CONF_DIR is not effective in spark-submit

leo.luan created SPARK-4616: --- Summary: SPARK_CONF_DIR is not effective in spark-submit Key: SPARK-4616 URL: https://issues.apache.org/jira/browse/SPARK-4616 Project: Spark Issue Type: Bug Affec

[jira] [Updated] (SPARK-2062) VertexRDD.apply does not use the mergeFunc

[ https://issues.apache.org/jira/browse/SPARK-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2062: - Target Version/s: 1.1.1, 1.2.0 > VertexRDD.apply does not use the mergeFunc >

[jira] [Updated] (SPARK-4115) [GraphX] add overrided count for EdgeRDD

[ https://issues.apache.org/jira/browse/SPARK-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4115: - Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1) > [GraphX] add overrided count for EdgeRDD >

[jira] [Resolved] (SPARK-4604) Make MatrixFactorizationModel constructor public

[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4604. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3459 [https://githu

[jira] [Updated] (SPARK-3716) Change partitionStrategy to utilize PartitionStrategy.fromString(_) to match edgeStorageLevel and vertexStorageLevel syntax in Analytics.scala

[ https://issues.apache.org/jira/browse/SPARK-3716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3716: - Target Version/s: 1.1.1, 1.2.0 > Change partitionStrategy to utilize PartitionStrategy.fromString(_) to ma

[jira] [Updated] (SPARK-3635) Find Strongly Connected Components with Graphx has a small bug

[ https://issues.apache.org/jira/browse/SPARK-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3635: - Target Version/s: 1.1.1, 1.2.0 > Find Strongly Connected Components with Graphx has a small bug >

[jira] [Updated] (SPARK-4249) A problem of EdgePartitionBuilder in Graphx

[ https://issues.apache.org/jira/browse/SPARK-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4249: - Target Version/s: 1.1.1, 1.2.0, 1.0.3 > A problem of EdgePartitionBuilder in Graphx >

[jira] [Commented] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225700#comment-14225700 ] Aaron Davidson commented on SPARK-4516: --- It turns out there was a real bug which cau

[jira] [Resolved] (SPARK-4583) GradientBoostedTrees error logging should use loss being minimized

[ https://issues.apache.org/jira/browse/SPARK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4583. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3439 [https://githu

[jira] [Updated] (SPARK-4308) SQL operation state is not properly set when exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-4308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4308: - Assignee: Cheng Lian > SQL operation state is not properly set when exception is thrown >

[jira] [Updated] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3704: - Target Version/s: 1.1.1, 1.2.0 Fix Version/s: 1.1.1 > the types not match adding value form spark r

[jira] [Updated] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

[ https://issues.apache.org/jira/browse/SPARK-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3791: - Fix Version/s: 1.1.1 > HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call >

[jira] [Updated] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

[ https://issues.apache.org/jira/browse/SPARK-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3791: - Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1) > HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call >

[jira] [Commented] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225687#comment-14225687 ] Andrew Or commented on SPARK-3704: -- Backported into 1.1.1 through https://github.com/apac

[jira] [Updated] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

[ https://issues.apache.org/jira/browse/SPARK-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3791: - Assignee: Cheng Lian > HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call >

[jira] [Commented] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

[ https://issues.apache.org/jira/browse/SPARK-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225686#comment-14225686 ] Andrew Or commented on SPARK-3791: -- Backported into 1.1.1 through https://github.com/apac

[jira] [Updated] (SPARK-3708) Backticks aren't handled correctly in aliases

[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3708: - Summary: Backticks aren't handled correctly in aliases (was: Backticks aren't handled correctly is aliase

[jira] [Commented] (SPARK-3834) Backticks not correctly handled in subquery aliases

[ https://issues.apache.org/jira/browse/SPARK-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225684#comment-14225684 ] Andrew Or commented on SPARK-3834: -- Also resolved by https://github.com/apache/spark/pull

[jira] [Updated] (SPARK-3834) Backticks not correctly handled in subquery aliases

[ https://issues.apache.org/jira/browse/SPARK-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3834: - Fix Version/s: 1.1.1 > Backticks not correctly handled in subquery aliases > -

[jira] [Updated] (SPARK-3834) Backticks not correctly handled in subquery aliases

[ https://issues.apache.org/jira/browse/SPARK-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3834: - Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) > Backticks not correctly handled in subquery aliases > -

[jira] [Updated] (SPARK-3708) Backticks aren't handled correctly is aliases

[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3708: - Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) > Backticks aren't handled correctly is aliases > ---

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225681#comment-14225681 ] Aaron Davidson commented on SPARK-2468: --- Hey guys, I finally got a chance to run a m

[jira] [Updated] (SPARK-3708) Backticks aren't handled correctly is aliases

[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3708: - Fix Version/s: 1.1.1 > Backticks aren't handled correctly is aliases > ---

[jira] [Commented] (SPARK-3708) Backticks aren't handled correctly is aliases

[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225682#comment-14225682 ] Andrew Or commented on SPARK-3708: -- This is back ported into 1.1.1 through https://github

[jira] [Commented] (SPARK-4614) Slight API changes in Matrix and Matrices

[ https://issues.apache.org/jira/browse/SPARK-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225663#comment-14225663 ] Apache Spark commented on SPARK-4614: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-4611) Implement the efficient vector norm

[ https://issues.apache.org/jira/browse/SPARK-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225657#comment-14225657 ] Apache Spark commented on SPARK-4611: - User 'dbtsai' has created a pull request for th

[jira] [Commented] (SPARK-2458) Make failed application log visible on History Server

[ https://issues.apache.org/jira/browse/SPARK-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225652#comment-14225652 ] Apache Spark commented on SPARK-2458: - User 'tsudukim' has created a pull request for

[jira] [Updated] (SPARK-4615) Cannot disconnect from spark-shell

[ https://issues.apache.org/jira/browse/SPARK-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Omans updated SPARK-4615: Description: When running spark-shell using the `v1.2.0-snapshot1` tag, using the instructions at:

[jira] [Updated] (SPARK-4615) Cannot disconnect from spark-shell

[ https://issues.apache.org/jira/browse/SPARK-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Omans updated SPARK-4615: Description: When running spark-shell using the `v1.2.0-snapshot1` tag, using the instructions at:

[jira] [Created] (SPARK-4615) Cannot disconnect from spark-shell

Thomas Omans created SPARK-4615: --- Summary: Cannot disconnect from spark-shell Key: SPARK-4615 URL: https://issues.apache.org/jira/browse/SPARK-4615 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225645#comment-14225645 ] Apache Spark commented on SPARK-4537: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-4614) Slight API changes in Matrix and Matrices

Xiangrui Meng created SPARK-4614: Summary: Slight API changes in Matrix and Matrices Key: SPARK-4614 URL: https://issues.apache.org/jira/browse/SPARK-4614 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-4161) Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4161: - Target Version/s: 1.1.2, 1.2.1 (was: 1.1.1, 1.2.0) > Spark shell class path is not correctly set if "spar

[jira] [Updated] (SPARK-3884) If deploy mode is cluster, --driver-memory shouldn't apply to client JVM

[ https://issues.apache.org/jira/browse/SPARK-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3884: - Target Version/s: 1.1.2, 1.2.1 (was: 1.1.1, 1.2.0) > If deploy mode is cluster, --driver-memory shouldn't

[jira] [Updated] (SPARK-3677) pom.xml and SparkBuild.scala are wrong : Scalastyle is never applyed to the sources under yarn/common

[ https://issues.apache.org/jira/browse/SPARK-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3677: - Target Version/s: 1.1.2, 1.2.1 (was: 1.1.1, 1.2.0) > pom.xml and SparkBuild.scala are wrong : Scalastyle

[jira] [Updated] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4452: - Target Version/s: 1.3.0, 1.1.2, 1.2.1 (was: 1.1.1, 1.2.0, 1.3.0) > Shuffle data structures can starve oth

[jira] [Commented] (SPARK-3546) InputStream of ManagedBuffer is not closed and causes running out of file descriptor

[ https://issues.apache.org/jira/browse/SPARK-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225638#comment-14225638 ] Andrew Or commented on SPARK-3546: -- Hey [~sarutak] do we want this for branch-1.1? If so

[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225636#comment-14225636 ] Andrew Or commented on SPARK-3995: -- Hey [~mengxr] do we want this in branch-1.1? If so we

[jira] [Updated] (SPARK-3632) ConnectionManager can run out of receive threads with authentication on

[ https://issues.apache.org/jira/browse/SPARK-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3632: - Target Version/s: 1.2.0, 1.1.2 (was: 1.1.1) > ConnectionManager can run out of receive threads with authe

[jira] [Reopened] (SPARK-3632) ConnectionManager can run out of receive threads with authentication on

[ https://issues.apache.org/jira/browse/SPARK-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-3632: -- This is not fully resolved; we still need to back port it to branch-1.1 > ConnectionManager can run out of

[jira] [Updated] (SPARK-4514) ComplexFutureAction does not preserve job group IDs

[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4514: - Target Version/s: 1.1.2, 1.2.1 (was: 1.1.1, 1.2.0) > ComplexFutureAction does not preserve job group IDs

[jira] [Updated] (SPARK-4468) Wrong Parquet filters are created for all inequality predicates with literals on the left hand side

[ https://issues.apache.org/jira/browse/SPARK-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4468: - Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1, 1.2.0, 1.0.3) > Wrong Parquet filters are created for all ine

[jira] [Updated] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4613: --- Assignee: Cheng Lian > Make JdbcRDD easier to use from Java >

[jira] [Commented] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225632#comment-14225632 ] Patrick Wendell commented on SPARK-4613: Yeah the only other tricky bit is the cla

[jira] [Updated] (SPARK-4468) Wrong Parquet filters are created for all inequality predicates with literals on the left hand side

[ https://issues.apache.org/jira/browse/SPARK-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4468: - Target Version/s: 1.1.1, 1.2.0, 1.0.3 (was: 1.2.0, 1.0.3, 1.1.2) > Wrong Parquet filters are created for

[jira] [Updated] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3628: - Target Version/s: 0.9.3, 1.0.3, 1.1.2, 1.2.1 (was: 1.1.1, 0.9.3, 1.0.3, 1.2.1) > Don't apply accumulator

[jira] [Commented] (SPARK-2445) MesosExecutorBackend crashes in fine grained mode

[ https://issues.apache.org/jira/browse/SPARK-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225630#comment-14225630 ] Greg Bowyer commented on SPARK-2445: My last comment is wrong, my build system did som

[jira] [Commented] (SPARK-4516) Netty off-heap memory use causes executors to be killed by OS

[ https://issues.apache.org/jira/browse/SPARK-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225627#comment-14225627 ] Apache Spark commented on SPARK-4516: - User 'aarondav' has created a pull request for

[jira] [Updated] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4613: --- Description: We might eventually deprecate it, but for now it would be nice to expose a Java w

[jira] [Commented] (SPARK-4608) Reorganize StreamingContext implicit to improve API convenience

[ https://issues.apache.org/jira/browse/SPARK-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225621#comment-14225621 ] Apache Spark commented on SPARK-4608: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225615#comment-14225615 ] Matei Zaharia commented on SPARK-4613: -- BTW the strawman for this would be a version

[jira] [Created] (SPARK-4613) Make JdbcRDD easier to use from Java

Matei Zaharia created SPARK-4613: Summary: Make JdbcRDD easier to use from Java Key: SPARK-4613 URL: https://issues.apache.org/jira/browse/SPARK-4613 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225603#comment-14225603 ] Nishkam Ravi commented on SPARK-4584: - I would recommend working with JavaWordCount. W

[jira] [Comment Edited] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225589#comment-14225589 ] maji2014 edited comment on SPARK-4314 at 11/26/14 2:05 AM: --- Disc

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225593#comment-14225593 ] Andrew Or commented on SPARK-4584: -- It may not be, but I just wanted to get a sense of ho

[jira] [Updated] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maji2014 updated SPARK-4314: Description: [Reproduce] 1. Run HdfsWordCount interface, such as "ssc.textFileStream(args(0))" 2. Upload f

[jira] [Comment Edited] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225589#comment-14225589 ] maji2014 edited comment on SPARK-4314 at 11/26/14 1:59 AM: --- Disc

[jira] [Comment Edited] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225589#comment-14225589 ] maji2014 edited comment on SPARK-4314 at 11/26/14 1:57 AM: --- Disc

[jira] [Commented] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225589#comment-14225589 ] maji2014 commented on SPARK-4314: - Discription for fileStream in textFileStream method is

[jira] [Updated] (SPARK-4611) Implement the efficient vector norm

[ https://issues.apache.org/jira/browse/SPARK-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4611: - Assignee: DB Tsai > Implement the efficient vector norm > --- > >

[jira] [Commented] (SPARK-4612) Configuration object gets created for every task even if not new file/jar is added

[ https://issues.apache.org/jira/browse/SPARK-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225574#comment-14225574 ] Apache Spark commented on SPARK-4612: - User 'tdas' has created a pull request for this

[jira] [Created] (SPARK-4612) Configuration object gets created for every task even if not new file/jar is added

Tathagata Das created SPARK-4612: Summary: Configuration object gets created for every task even if not new file/jar is added Key: SPARK-4612 URL: https://issues.apache.org/jira/browse/SPARK-4612 Proj

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225561#comment-14225561 ] Saisai Shao commented on SPARK-4537: Thanks TD, I'm going to fix this issue. > Add 'p

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225567#comment-14225567 ] Nishkam Ravi commented on SPARK-4584: - Around 2GB in the map stage and 1.5GB in the co

[jira] [Commented] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225566#comment-14225566 ] xukun commented on SPARK-4598: -- i think you can sort the task data first and then paginate th

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225563#comment-14225563 ] Nishkam Ravi commented on SPARK-4584: - [~andrewor14] Just curious, why is that relevan

[jira] [Commented] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225557#comment-14225557 ] WangTaoTheTonic commented on SPARK-4598: I think pagination is a better idea. Allo

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225554#comment-14225554 ] Andrew Or commented on SPARK-4584: -- Hey [~nravi] how much data are you shuffling? Can you

[jira] [Commented] (SPARK-4609) Job can not finish if there is one bad slave in clusters

[ https://issues.apache.org/jira/browse/SPARK-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225543#comment-14225543 ] Davies Liu commented on SPARK-4609: --- We already have blacklist for executors, so we coul

[jira] [Comment Edited] (SPARK-2445) MesosExecutorBackend crashes in fine grained mode

[ https://issues.apache.org/jira/browse/SPARK-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225540#comment-14225540 ] Greg Bowyer edited comment on SPARK-2445 at 11/26/14 1:21 AM: --

  1   2   3   >