[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

2014-05-15 Thread Erik J. Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993742#comment-13993742 ] Erik J. Erlandson commented on SPARK-1473: -- I'm fairly new to Spark, and

[jira] [Resolved] (SPARK-1688) PySpark throws unhelpful exception when pyspark cannot be loaded

2014-05-15 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1688. --- Resolution: Fixed PySpark throws unhelpful exception when pyspark cannot be loaded

[jira] [Updated] (SPARK-1824) Python examples still take in master

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1824: - Fix Version/s: 1.0.1 Python examples still take in master --

[jira] [Resolved] (SPARK-1788) Upgrade Parquet to 1.4.3

2014-05-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1788. - Resolution: Fixed Upgrade Parquet to 1.4.3

[jira] [Updated] (SPARK-1635) Java API docs do not show annotation.

2014-05-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1635: - Priority: Minor (was: Major) Java API docs do not show annotation.

[jira] [Commented] (SPARK-1696) RowMatrix.dspr is not using parameter alpha for DenseVector

2014-05-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998127#comment-13998127 ] Xiangrui Meng commented on SPARK-1696: -- Thanks! I sent a PR:

[jira] [Updated] (SPARK-1758) failing test org.apache.spark.JavaAPISuite.wholeTextFiles

2014-05-15 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-1758: Attachment: SPARK-1758.patch failing test org.apache.spark.JavaAPISuite.wholeTextFiles

[jira] [Commented] (SPARK-1779) Warning when spark.storage.memoryFraction is not between 0 and 1

2014-05-15 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993901#comment-13993901 ] Erik Erlandson commented on SPARK-1779: --- I'll volunteer to take this, can somebody

[jira] [Updated] (SPARK-1436) Compression code broke in-memory store

2014-05-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1436: --- Description: Try run the following code: {code} package org.apache.spark.sql import

[jira] [Created] (SPARK-1842) update scala-logging-slf4j to version 2.1.2

2014-05-15 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1842: -- Summary: update scala-logging-slf4j to version 2.1.2 Key: SPARK-1842 URL: https://issues.apache.org/jira/browse/SPARK-1842 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-1757) Support saving null primitives with .saveAsParquetFile()

2014-05-15 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-1757: - Summary: Support saving null primitives with .saveAsParquetFile() Key: SPARK-1757 URL: https://issues.apache.org/jira/browse/SPARK-1757 Project: Spark Issue Type:

[jira] [Created] (SPARK-1767) Prefer HDFS-cached replicas when scheduling data-local tasks

2014-05-15 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-1767: - Summary: Prefer HDFS-cached replicas when scheduling data-local tasks Key: SPARK-1767 URL: https://issues.apache.org/jira/browse/SPARK-1767 Project: Spark Issue

[jira] [Created] (SPARK-1780) Non-existent SPARK_DAEMON_OPTS is referred to in a few places

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1780: Summary: Non-existent SPARK_DAEMON_OPTS is referred to in a few places Key: SPARK-1780 URL: https://issues.apache.org/jira/browse/SPARK-1780 Project: Spark Issue

[jira] [Updated] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

2014-05-15 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1775: - Fix Version/s: 0.9.2 Unneeded lock in ShuffleMapTask.deserializeInfo

[jira] [Commented] (SPARK-1433) Upgrade Mesos dependency to 0.17.0

2014-05-15 Thread Timothy St. Clair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993896#comment-13993896 ] Timothy St. Clair commented on SPARK-1433: -- Likely want to aim higher at this

[jira] [Updated] (SPARK-1631) App name set in SparkConf (not in JVM properties) not respected by Yarn backend

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1631: --- Priority: Blocker (was: Major) App name set in SparkConf (not in JVM properties) not

[jira] [Commented] (SPARK-1767) Prefer HDFS-cached replicas when scheduling data-local tasks

2014-05-15 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993909#comment-13993909 ] Sandy Ryza commented on SPARK-1767: --- Currently, RDDs only support a single level of

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993108#comment-13993108 ] Thomas Graves commented on SPARK-1755: -- I believe this is a dup of SPARK-1664

[jira] [Resolved] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1775. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 707

[jira] [Created] (SPARK-1786) Kryo Serialization Error in GraphX

2014-05-15 Thread Joseph E. Gonzalez (JIRA)
Joseph E. Gonzalez created SPARK-1786: - Summary: Kryo Serialization Error in GraphX Key: SPARK-1786 URL: https://issues.apache.org/jira/browse/SPARK-1786 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1786) Kryo Serialization Error in GraphX

2014-05-15 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-1786: -- Description: The following code block will generate a serialization error when run in

[jira] [Created] (SPARK-1840) SparkListenerBus prints out scary error message when terminating normally

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1840: Summary: SparkListenerBus prints out scary error message when terminating normally Key: SPARK-1840 URL: https://issues.apache.org/jira/browse/SPARK-1840 Project: Spark

[jira] [Updated] (SPARK-1840) SparkListenerBus prints out scary error message when terminating normally

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1840: - Description: This is because the Scala's NonLocalReturnControl (which extends ControlThrowable) is

[jira] [Updated] (SPARK-1769) Executor loss can cause race condition in Pool

2014-05-15 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-1769: -- Description: Loss of executors (in this case due to OOMs) exposes a race condition in

[jira] [Updated] (SPARK-1764) EOF reached before Python server acknowledged

2014-05-15 Thread Bouke van der Bijl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bouke van der Bijl updated SPARK-1764: -- Description: I'm getting EOF reached before Python server acknowledged while using

[jira] [Commented] (SPARK-1767) Prefer HDFS-cached replicas when scheduling data-local tasks

2014-05-15 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997761#comment-13997761 ] Aaron Davidson commented on SPARK-1767: --- One simple workaround to this is to just

[jira] [Commented] (SPARK-1575) failing tests with master branch

2014-05-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996955#comment-13996955 ] Sean Owen commented on SPARK-1575: -- For what it's worth, I no longer see this failure I

[jira] [Updated] (SPARK-1841) update scalatest to version 2.1.5

2014-05-15 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1841: --- Description: scalatest 1.9.* not support Scala 2.11 (was: scalatest 1.9.* not Scala 2.11) update

[jira] [Updated] (SPARK-1778) Add 'limit' transformation to SchemaRDD.

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1778: --- Assignee: Takuya Ueshin Add 'limit' transformation to SchemaRDD.

[jira] [Created] (SPARK-1770) repartition and coalesce(shuffle=true) put objects with the same key in the same bucket

2014-05-15 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-1770: Summary: repartition and coalesce(shuffle=true) put objects with the same key in the same bucket Key: SPARK-1770 URL: https://issues.apache.org/jira/browse/SPARK-1770

[jira] [Closed] (SPARK-1838) On a YARN cluster, Spark doesn't run on local mode

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1838. Resolution: Not a Problem Looks like I accidentally set SPARK_YARN_MODE to true manually, which directly

[jira] [Commented] (SPARK-1787) Build failure on JDK8 :: SBT fails to load build configuration file

2014-05-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998492#comment-13998492 ] Sean Owen commented on SPARK-1787: -- Duplicate of

[jira] [Created] (SPARK-1843) Provide a simpler alternative to assemble-deps

2014-05-15 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-1843: -- Summary: Provide a simpler alternative to assemble-deps Key: SPARK-1843 URL: https://issues.apache.org/jira/browse/SPARK-1843 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-1696) RowMatrix.dspr is not using parameter alpha for DenseVector

2014-05-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1696. -- Resolution: Fixed Fix Version/s: 1.0.0 RowMatrix.dspr is not using parameter alpha for

[jira] [Commented] (SPARK-1836) REPL $outer type mismatch causes lookup() and equals() problems

2014-05-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998312#comment-13998312 ] Michael Armbrust commented on SPARK-1836: - This sounds like it could be related to

[jira] [Updated] (SPARK-1825) Windows Spark fails to work with Linux YARN

2014-05-15 Thread Taeyun Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taeyun Kim updated SPARK-1825: -- Affects Version/s: 1.0.0 Windows Spark fails to work with Linux YARN

[jira] [Created] (SPARK-1771) CoarseGrainedSchedulerBackend is not resilient to Akka restarts

2014-05-15 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-1771: - Summary: CoarseGrainedSchedulerBackend is not resilient to Akka restarts Key: SPARK-1771 URL: https://issues.apache.org/jira/browse/SPARK-1771 Project: Spark

[jira] [Resolved] (SPARK-1494) Hive Dependencies being checked by MIMA

2014-05-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1494. - Resolution: Fixed Hive Dependencies being checked by MIMA

[jira] [Updated] (SPARK-1764) EOF reached before Python server acknowledged

2014-05-15 Thread Bouke van der Bijl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bouke van der Bijl updated SPARK-1764: -- Description: I'm getting EOF reached before Python server acknowledged while using

[jira] [Created] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1823: Summary: ExternalAppendOnlyMap can still OOM if one key is very large Key: SPARK-1823 URL: https://issues.apache.org/jira/browse/SPARK-1823 Project: Spark Issue

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Description: In YARN client mode, --name is ignored because the deploy mode is client, and the name is

[jira] [Resolved] (SPARK-1833) Have an empty SparkContext constructor instead of relying on new SparkContext(new SparkConf())

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1833. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 774

[jira] [Commented] (SPARK-1605) Improve mllib.linalg.Vector

2014-05-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998106#comment-13998106 ] Xiangrui Meng commented on SPARK-1605: -- `toBreeze` exposes a breeze type. We might

[jira] [Resolved] (SPARK-1646) ALS micro-optimisation

2014-05-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1646. -- Resolution: Implemented Fix Version/s: 1.0.0 PR:

[jira] [Resolved] (SPARK-1840) SparkListenerBus prints out scary error message when terminating normally

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1840. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 783

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

2014-05-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998494#comment-13998494 ] Sean Owen commented on SPARK-1473: -- I believe these types of thing were more the goals of

[jira] [Created] (SPARK-1838) On a YARN cluster, Spark doesn't run on local mode

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1838: Summary: On a YARN cluster, Spark doesn't run on local mode Key: SPARK-1838 URL: https://issues.apache.org/jira/browse/SPARK-1838 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1827) LICENSE and NOTICE files need a refresh to contain transitive dependency info

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1827. Resolution: Fixed Assignee: Sean Owen Fixed by:

[jira] [Created] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1755: Summary: Spark-submit --name does not resolve to application name on YARN Key: SPARK-1755 URL: https://issues.apache.org/jira/browse/SPARK-1755 Project: Spark

[jira] [Created] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

2014-05-15 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1760: -- Summary: mvn -Dsuites=* test throw an ClassNotFoundException Key: SPARK-1760 URL: https://issues.apache.org/jira/browse/SPARK-1760 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-1664) spark-submit --name doesn't work in yarn-client mode

2014-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1664. Resolution: Duplicate spark-submit --name doesn't work in yarn-client mode

[jira] [Commented] (SPARK-1770) repartition and coalesce(shuffle=true) put objects with the same key in the same bucket

2014-05-15 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993908#comment-13993908 ] Aaron Davidson commented on SPARK-1770: --- Ah, that PR seems unrelated. repartition

[jira] [Commented] (SPARK-1821) Document History Server

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996755#comment-13996755 ] Andrew Or commented on SPARK-1821: -- Yes, it should be documented under monitoring.html in