[jira] [Updated] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-1476: --- Fix Version/s: 1.1.0 > 2GB limit in spark for blocks > -

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967419#comment-13967419 ] Mridul Muralidharan commented on SPARK-1476: WIP Proposal: - All references t

[jira] [Created] (SPARK-1476) 2GB limit in spark for blocks

Mridul Muralidharan created SPARK-1476: -- Summary: 2GB limit in spark for blocks Key: SPARK-1476 URL: https://issues.apache.org/jira/browse/SPARK-1476 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1057: --- Priority: Major (was: Minor) > Remove Fastutil > --- > > Key: SP

[jira] [Resolved] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1057. Resolution: Fixed > Remove Fastutil > --- > > Key: SPARK-1057 >

[jira] [Updated] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1057: --- Fix Version/s: 1.0.0 > Remove Fastutil > --- > > Key: SPARK-1057

[jira] [Updated] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1057: --- Assignee: Sean Owen (was: Evan Chan) > Remove Fastutil > --- > >

[jira] [Updated] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1057: --- Summary: Remove Fastutil (was: Upgrade to fastutil-6.5.x) > Remove Fastutil > --

[jira] [Commented] (SPARK-1057) Remove Fastutil

[ https://issues.apache.org/jira/browse/SPARK-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967413#comment-13967413 ] Patrick Wendell commented on SPARK-1057: I changed the title of the JIRA to reflec

[jira] [Updated] (SPARK-1466) Pyspark doesn't check if gateway process launches correctly

[ https://issues.apache.org/jira/browse/SPARK-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1466: --- Priority: Blocker (was: Major) > Pyspark doesn't check if gateway process launches correctly

[jira] [Updated] (SPARK-1328) Current implementation of Standard Deviation in MLUtils may cause catastrophic cancellation, and loss precision.

[ https://issues.apache.org/jira/browse/SPARK-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1328: --- Assignee: Xusen Yin > Current implementation of Standard Deviation in MLUtils may cause > ca

[jira] [Resolved] (SPARK-1328) Current implementation of Standard Deviation in MLUtils may cause catastrophic cancellation, and loss precision.

[ https://issues.apache.org/jira/browse/SPARK-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1328. Resolution: Fixed > Current implementation of Standard Deviation in MLUtils may cause > ca

[jira] [Updated] (SPARK-1475) Draining event logging queue before stopping event logger

[ https://issues.apache.org/jira/browse/SPARK-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-1475: - Description: When stopping SparkListenerBus, its event queue needs to be drained. And this needs to happe

[jira] [Updated] (SPARK-1475) Draining event logging queue before stopping event logger

[ https://issues.apache.org/jira/browse/SPARK-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-1475: - Description: When stopping SparkListenerBus, its event queue needs to be drained. And this needs to happe

[jira] [Commented] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn

[ https://issues.apache.org/jira/browse/SPARK-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967234#comment-13967234 ] Kan Zhang commented on SPARK-1407: -- I opened SPARK-1475 to track the above PR. > EventLo

[jira] [Resolved] (SPARK-1475) Draining event logging queue before stopping event logger

[ https://issues.apache.org/jira/browse/SPARK-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang resolved SPARK-1475. -- Resolution: Fixed > Draining event logging queue before stopping event logger > ---

[jira] [Commented] (SPARK-1475) Draining event logging queue before stopping event logger

[ https://issues.apache.org/jira/browse/SPARK-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967228#comment-13967228 ] Kan Zhang commented on SPARK-1475: -- When event queue is not drained, users may observe si

[jira] [Created] (SPARK-1475) Draining event logging queue before stopping event logger

Kan Zhang created SPARK-1475: Summary: Draining event logging queue before stopping event logger Key: SPARK-1475 URL: https://issues.apache.org/jira/browse/SPARK-1475 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-1355) Switch website to the Apache CMS

[ https://issues.apache.org/jira/browse/SPARK-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967196#comment-13967196 ] Matei Zaharia commented on SPARK-1355: -- I have to say this was pretty good, I avoided

[jira] [Commented] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967193#comment-13967193 ] Mridul Muralidharan commented on SPARK-1453: The timeout gets hit only when we

[jira] [Commented] (SPARK-542) Cache Miss when machine have multiple hostname

[ https://issues.apache.org/jira/browse/SPARK-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967185#comment-13967185 ] Mridul Muralidharan commented on SPARK-542: --- Spark uses only hostnames - not ip's

[jira] [Updated] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated SPARK-1474: - Summary: Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark c

[jira] [Commented] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark-master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967175#comment-13967175 ] Xuan Gong commented on SPARK-1474: -- Verified that spark application *works* in yarn-clien

[jira] [Commented] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark-master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967173#comment-13967173 ] Xuan Gong commented on SPARK-1474: -- use mvn to Compile the spark code from spark master b

[jira] [Created] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark-master branch with hadoop-2.2.0

Xuan Gong created SPARK-1474: Summary: Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark-master branch with hadoop-2.2.0 Key: SPARK-1474 URL: https://issues.apache.org/jira

[jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967166#comment-13967166 ] Xuan Gong commented on SPARK-1465: -- https://github.com/apache/spark/pull/396 > Spark com

[jira] [Updated] (SPARK-1473) Feature selection for high dimensional datasets

[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Zendejas updated SPARK-1473: Description: For classification tasks involving large feature spaces in the order of tens o

[jira] [Created] (SPARK-1473) Feature selection for high dimensional datasets

Ignacio Zendejas created SPARK-1473: --- Summary: Feature selection for high dimensional datasets Key: SPARK-1473 URL: https://issues.apache.org/jira/browse/SPARK-1473 Project: Spark Issue Typ

[jira] [Created] (SPARK-1472) Go through YARN api used in Spark to make sure we aren't using Private Apis

Thomas Graves created SPARK-1472: Summary: Go through YARN api used in Spark to make sure we aren't using Private Apis Key: SPARK-1472 URL: https://issues.apache.org/jira/browse/SPARK-1472 Project: Sp

[jira] [Resolved] (SPARK-1241) Support sliding in RDD

[ https://issues.apache.org/jira/browse/SPARK-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1241. -- Resolution: Fixed Fix Version/s: 1.0.0 > Support sliding in RDD > --

[jira] [Resolved] (SPARK-1225) ROC AUC and Average Precision for Binary classification models

[ https://issues.apache.org/jira/browse/SPARK-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1225. -- Resolution: Fixed Fix Version/s: 1.0.0 > ROC AUC and Average Precision for Binary classi

[jira] [Commented] (SPARK-1225) ROC AUC and Average Precision for Binary classification models

[ https://issues.apache.org/jira/browse/SPARK-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966987#comment-13966987 ] Matei Zaharia commented on SPARK-1225: -- Included in https://github.com/apache/spark/p

[jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966887#comment-13966887 ] Xuan Gong commented on SPARK-1465: -- [~tgraves] bq. For Spark I think we are better off j

[jira] [Resolved] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

[ https://issues.apache.org/jira/browse/SPARK-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1417. -- Resolution: Fixed Fix Version/s: 1.0.0 > Spark on Yarn - spark UI link from resourcemana

[jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966570#comment-13966570 ] Thomas Graves commented on SPARK-1465: -- https://issues.apache.org/jira/browse/YARN-19

[jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966545#comment-13966545 ] Thomas Graves commented on SPARK-1465: -- You can't just update spark to the new api as

[jira] [Commented] (SPARK-1426) Make MLlib work with NumPy versions older than 1.7

[ https://issues.apache.org/jira/browse/SPARK-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966388#comment-13966388 ] Sandeep Singh commented on SPARK-1426: -- https://github.com/apache/spark/pull/391 > M

[jira] [Assigned] (SPARK-1426) Make MLlib work with NumPy versions older than 1.7

[ https://issues.apache.org/jira/browse/SPARK-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh reassigned SPARK-1426: Assignee: Sandeep Singh > Make MLlib work with NumPy versions older than 1.7 >

[jira] [Commented] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966332#comment-13966332 ] Mridul Muralidharan commented on SPARK-1391: Another place where this is relev