[jira] [Created] (SPARK-2014) Make PySpark store RDDs in MEMORY_ONLY_SER with compression by default

2014-06-03 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-2014: Summary: Make PySpark store RDDs in MEMORY_ONLY_SER with compression by default Key: SPARK-2014 URL: https://issues.apache.org/jira/browse/SPARK-2014 Project: Spark

[jira] [Created] (SPARK-2013) Add Python pickleFile to programming guide

2014-06-03 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-2013: Summary: Add Python pickleFile to programming guide Key: SPARK-2013 URL: https://issues.apache.org/jira/browse/SPARK-2013 Project: Spark Issue Type: Document

[jira] [Commented] (SPARK-1977) mutable.BitSet in ALS not serializable with KryoSerializer

2014-06-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017400#comment-14017400 ] Xiangrui Meng commented on SPARK-1977: -- I cannot reproduce this error in v1.0.0. Ther

[jira] [Resolved] (SPARK-1817) RDD zip erroneous when partitions do not divide RDD count

2014-06-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1817. Resolution: Fixed Fix Version/s: 1.1.0 > RDD zip erroneous when partitions do not divide RDD

[jira] [Resolved] (SPARK-1979) Error message needs to be updated when user specifies --arg ( which is no longer required to run spark submit jobs).Spark documentation needs to be updated to reflect th

2014-06-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1979. Resolution: Not a Problem > Error message needs to be updated when user specifies --arg ( w

[jira] [Commented] (SPARK-1979) Error message needs to be updated when user specifies --arg ( which is no longer required to run spark submit jobs).Spark documentation needs to be updated to reflect t

2014-06-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017393#comment-14017393 ] Patrick Wendell commented on SPARK-1979: The documentation is up to date. You are

[jira] [Commented] (SPARK-1552) GraphX performs type comparison incorrectly

2014-06-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017340#comment-14017340 ] Ankur Dave commented on SPARK-1552: --- Proposed fix: https://github.com/apache/spark/pull/

[jira] [Commented] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017322#comment-14017322 ] Guoqiang Li commented on SPARK-1930: [~pwendell] I'm not familiar with python, where

[jira] [Comment Edited] (SPARK-1940) Enable rolling of executor logs (stdout / stderr)

2014-06-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010413#comment-14010413 ] Tathagata Das edited comment on SPARK-1940 at 6/4/14 1:50 AM: --

[jira] [Updated] (SPARK-1941) Update streamlib to 2.7.0 and use HyperLogLogPlus instead of HyperLogLog

2014-06-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1941: - Assignee: Reynold Xin > Update streamlib to 2.7.0 and use HyperLogLogPlus instead of HyperLogLog

[jira] [Resolved] (SPARK-1941) Update streamlib to 2.7.0 and use HyperLogLogPlus instead of HyperLogLog

2014-06-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1941. -- Resolution: Implemented PR: https://github.com/apache/spark/pull/897 > Update streamlib to 2.7

[jira] [Created] (SPARK-2012) PySpark StatCounter with numpy arrays

2014-06-03 Thread Jeremy Freeman (JIRA)
Jeremy Freeman created SPARK-2012: - Summary: PySpark StatCounter with numpy arrays Key: SPARK-2012 URL: https://issues.apache.org/jira/browse/SPARK-2012 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017310#comment-14017310 ] Patrick Wendell commented on SPARK-1790: I closed mine in favor of yours - thanks!

[jira] [Updated] (SPARK-2011) Eliminate duplicate join in Pregel

2014-06-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-2011: -- Description: In the iteration loop, Pregel currently performs an innerJoin to apply messages to vertice

[jira] [Updated] (SPARK-1161) Add saveAsObjectFile and SparkContext.objectFile in Python

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1161: - Fix Version/s: 1.1.0 > Add saveAsObjectFile and SparkContext.objectFile in Python > -

[jira] [Resolved] (SPARK-1161) Add saveAsObjectFile and SparkContext.objectFile in Python

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1161. -- Resolution: Fixed Merged this in -- thanks Kan! > Add saveAsObjectFile and SparkContext.object

[jira] [Created] (SPARK-2011) Eliminate duplicate join in Pregel

2014-06-03 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-2011: - Summary: Eliminate duplicate join in Pregel Key: SPARK-2011 URL: https://issues.apache.org/jira/browse/SPARK-2011 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1704) java.lang.AssertionError: assertion failed: No plan for ExplainCommand (Project [*])

2014-06-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017292#comment-14017292 ] Yin Huai commented on SPARK-1704: - [~marmbrus] I am attaching the link to the PR. > java.

[jira] [Comment Edited] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Sujeet Varakhedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017276#comment-14017276 ] Sujeet Varakhedi edited comment on SPARK-1790 at 6/4/14 12:06 AM: --

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Sujeet Varakhedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017276#comment-14017276 ] Sujeet Varakhedi commented on SPARK-1790: - Hey Patrick. Yeah my patch is pretty mu

[jira] [Created] (SPARK-2010) Support for nested data in PySpark SQL

2014-06-03 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2010: --- Summary: Support for nested data in PySpark SQL Key: SPARK-2010 URL: https://issues.apache.org/jira/browse/SPARK-2010 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-2009) Key not found exception when slow receiver starts

2014-06-03 Thread Vadim Chekan (JIRA)
Vadim Chekan created SPARK-2009: --- Summary: Key not found exception when slow receiver starts Key: SPARK-2009 URL: https://issues.apache.org/jira/browse/SPARK-2009 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Sujeet Varakhedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017248#comment-14017248 ] Sujeet Varakhedi commented on SPARK-1790: - https://github.com/apache/spark/pull/96

[jira] [Updated] (SPARK-2008) Enhance spark-ec2 to be able to add and remove slaves to an existing cluster

2014-06-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2008: Priority: Minor (was: Major) > Enhance spark-ec2 to be able to add and remove slaves to an

[jira] [Created] (SPARK-2008) Enhance spark-ec2 to be able to add and remove slaves to an existing cluster

2014-06-03 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2008: --- Summary: Enhance spark-ec2 to be able to add and remove slaves to an existing cluster Key: SPARK-2008 URL: https://issues.apache.org/jira/browse/SPARK-2008 Proj

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017217#comment-14017217 ] Patrick Wendell commented on SPARK-1790: Hey [~sujeetv] someone asked me to look a

[jira] [Comment Edited] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017217#comment-14017217 ] Patrick Wendell edited comment on SPARK-1790 at 6/3/14 10:21 PM: ---

[jira] [Resolved] (SPARK-1991) Support custom StorageLevels for vertices and edges

2014-06-03 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-1991. --- Resolution: Fixed Fix Version/s: 1.1.0 Resolved by https://github.com/apache/spark/pull/946 >

[jira] [Closed] (SPARK-2006) Spark Sql example throws ClassCastException: Long -> Int

2014-06-03 Thread Kevin Tham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Tham closed SPARK-2006. - > Spark Sql example throws ClassCastException: Long -> Int >

[jira] [Commented] (SPARK-2007) Spark on YARN picks up hadoop log4j.properties even if SPARK_LOG4J_CONF is set

2014-06-03 Thread Issac Buenrostro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017160#comment-14017160 ] Issac Buenrostro commented on SPARK-2007: - This can be solved by changing the name

[jira] [Resolved] (SPARK-2006) Spark Sql example throws ClassCastException: Long -> Int

2014-06-03 Thread Kevin Tham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Tham resolved SPARK-2006. --- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/949 > Spark Sql example throws Class

[jira] [Commented] (SPARK-2006) Spark Sql example throws ClassCastException: Long -> Int

2014-06-03 Thread Kevin Tham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017162#comment-14017162 ] Kevin Tham commented on SPARK-2006: --- Actually there was an earlier pull request that jus

[jira] [Updated] (SPARK-1969) Public available online summarizer for mean, variance, min, and max

2014-06-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-1969: --- Description: Basically, it moves the private ColumnStatisticsAggregator class from RowMatrix to public avail

[jira] [Created] (SPARK-2007) Spark on YARN picks up hadoop log4j.properties even if SPARK_LOG4J_CONF is set

2014-06-03 Thread Issac Buenrostro (JIRA)
Issac Buenrostro created SPARK-2007: --- Summary: Spark on YARN picks up hadoop log4j.properties even if SPARK_LOG4J_CONF is set Key: SPARK-2007 URL: https://issues.apache.org/jira/browse/SPARK-2007 Pr

[jira] [Updated] (SPARK-1969) Public available online summarizer for mean, variance, min, and max

2014-06-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-1969: --- Description: It basically moved the private ColumnStatisticsAggregator class from RowMatrix to public availa

[jira] [Updated] (SPARK-1969) Public available online summarizer for mean, variance, min, and max

2014-06-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-1969: --- Description: Basically, it moves the private ColumnStatisticsAggregator class from RowMatrix to public avail

[jira] [Updated] (SPARK-1468) The hash method used by partitionBy in Pyspark doesn't deal with None correctly.

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1468: - Affects Version/s: 1.0.0 > The hash method used by partitionBy in Pyspark doesn't deal with None

[jira] [Updated] (SPARK-1468) The hash method used by partitionBy in Pyspark doesn't deal with None correctly.

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1468: - Assignee: Erik Selin > The hash method used by partitionBy in Pyspark doesn't deal with None > c

[jira] [Resolved] (SPARK-1468) The hash method used by partitionBy in Pyspark doesn't deal with None correctly.

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1468. -- Resolution: Fixed > The hash method used by partitionBy in Pyspark doesn't deal with None > co

[jira] [Commented] (SPARK-2006) Spark Sql example throws ClassCastException: Long -> Int

2014-06-03 Thread Kevin Tham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017107#comment-14017107 ] Kevin Tham commented on SPARK-2006: --- https://github.com/apache/spark/pull/954 > Spark S

[jira] [Created] (SPARK-2006) Spark Sql example throws ClassCastException: Long -> Int

2014-06-03 Thread Kevin Tham (JIRA)
Kevin Tham created SPARK-2006: - Summary: Spark Sql example throws ClassCastException: Long -> Int Key: SPARK-2006 URL: https://issues.apache.org/jira/browse/SPARK-2006 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1992) Support for Pivotal HD in the Maven build

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1992: - Fix Version/s: 1.1.0 > Support for Pivotal HD in the Maven build > --

[jira] [Resolved] (SPARK-1992) Support for Pivotal HD in the Maven build

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1992. -- Resolution: Fixed > Support for Pivotal HD in the Maven build > ---

[jira] [Updated] (SPARK-1992) Support for Pivotal HD in the Maven build

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1992: - Issue Type: Improvement (was: Bug) > Support for Pivotal HD in the Maven build > ---

[jira] [Updated] (SPARK-1992) Support for Pivotal HD in the Maven build

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1992: - Fix Version/s: 1.0.1 > Support for Pivotal HD in the Maven build > --

[jira] [Updated] (SPARK-1992) Support for Pivotal HD in the Maven build

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1992: - Assignee: Christian Tzolov > Support for Pivotal HD in the Maven build >

[jira] [Resolved] (SPARK-1912) Compression memory issue during reduce

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1912. -- Resolution: Fixed > Compression memory issue during reduce > --

[jira] [Updated] (SPARK-1912) Compression memory issue during reduce

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1912: - Assignee: Wenchen Fan > Compression memory issue during reduce >

[jira] [Updated] (SPARK-1912) Compression memory issue during reduce

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1912: - Fix Version/s: 1.1.0 > Compression memory issue during reduce > -

[jira] [Created] (SPARK-2005) Investigate linux container-based solution

2014-06-03 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-2005: Summary: Investigate linux container-based solution Key: SPARK-2005 URL: https://issues.apache.org/jira/browse/SPARK-2005 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-2004) QA Automation

2014-06-03 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-2004: Summary: QA Automation Key: SPARK-2004 URL: https://issues.apache.org/jira/browse/SPARK-2004 Project: Spark Issue Type: Umbrella Components: Build,

[jira] [Resolved] (SPARK-2001) Remove docs/spark-debugger.md from master since it is obsolete

2014-06-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2001. Resolution: Fixed Target Version/s: 1.0.1, 1.1.0 (was: 1.1.0) > Remove docs/spark-debugg

[jira] [Updated] (SPARK-1468) The hash method used by partitionBy in Pyspark doesn't deal with None correctly.

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1468: - Fix Version/s: 1.0.1 0.9.2 > The hash method used by partitionBy in Pyspark do

[jira] [Created] (SPARK-2003) SparkContext(SparkConf) doesn't work in pyspark

2014-06-03 Thread Diana Carroll (JIRA)
Diana Carroll created SPARK-2003: Summary: SparkContext(SparkConf) doesn't work in pyspark Key: SPARK-2003 URL: https://issues.apache.org/jira/browse/SPARK-2003 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1942) Stop clearing spark.driver.port in unit tests

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1942. -- Resolution: Fixed > Stop clearing spark.driver.port in unit tests > ---

[jira] [Updated] (SPARK-1942) Stop clearing spark.driver.port in unit tests

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1942: - Fix Version/s: 1.1.0 > Stop clearing spark.driver.port in unit tests > --

[jira] [Updated] (SPARK-1942) Stop clearing spark.driver.port in unit tests

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1942: - Assignee: Syed A. Hashmi > Stop clearing spark.driver.port in unit tests > --

[jira] [Updated] (SPARK-1942) Stop clearing spark.driver.port in unit tests

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1942: - Priority: Trivial (was: Major) > Stop clearing spark.driver.port in unit tests > ---

[jira] [Created] (SPARK-2002) Race condition in accessing cache locations in DAGScheduler

2014-06-03 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-2002: Summary: Race condition in accessing cache locations in DAGScheduler Key: SPARK-2002 URL: https://issues.apache.org/jira/browse/SPARK-2002 Project: Spark Is

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017001#comment-14017001 ] Matei Zaharia commented on SPARK-1790: -- It's fine to skip the check right now; I thin

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

2014-06-03 Thread Sujeet Varakhedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016991#comment-14016991 ] Sujeet Varakhedi commented on SPARK-1790: - Sorry I just got to this yesterday. Cur

[jira] [Commented] (SPARK-2001) Remove docs/spark-debugger.md from master since it is obsolete

2014-06-03 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016963#comment-14016963 ] Henry Saputra commented on SPARK-2001: -- See https://github.com/apache/spark/pull/953

[jira] [Commented] (SPARK-2001) Remove docs/spark-debugger.md from master since it is obsolete

2014-06-03 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016948#comment-14016948 ] Henry Saputra commented on SPARK-2001: -- Working on this > Remove docs/spark-debugger

[jira] [Created] (SPARK-2001) Remove docs/spark-debugger.md from master since it is obsolete

2014-06-03 Thread Henry Saputra (JIRA)
Henry Saputra created SPARK-2001: Summary: Remove docs/spark-debugger.md from master since it is obsolete Key: SPARK-2001 URL: https://issues.apache.org/jira/browse/SPARK-2001 Project: Spark

[jira] [Commented] (SPARK-1999) UI : StorageLevel in storage tab and RDD Storage Info never changes

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016895#comment-14016895 ] Chen Chao commented on SPARK-1999: -- I have fixed and tested fine. Please assign it to me

[jira] [Comment Edited] (SPARK-1999) UI : StorageLevel in storage tab and RDD Storage Info never changes

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016385#comment-14016385 ] Chen Chao edited comment on SPARK-1999 at 6/3/14 3:41 PM: -- https:

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016790#comment-14016790 ] Guoqiang Li commented on SPARK-2000: Oh, I'm sorry. [~pwendell] You can help modify i

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Chao updated SPARK-2000: - Priority: Major (was: Blocker) > cannot connect to cluster in Standalone mode when run spark-shell in on

[jira] [Comment Edited] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016634#comment-14016634 ] Chen Chao edited comment on SPARK-2000 at 6/3/14 3:33 PM: -- actual

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016634#comment-14016634 ] Chen Chao commented on SPARK-2000: -- actually , I have already posted PR : https://github.

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2000: --- Priority: Blocker (was: Major) > cannot connect to cluster in Standalone mode when run spark-shell i

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016609#comment-14016609 ] Guoqiang Li commented on SPARK-2000: Ok, I have to reproduce the issue. > cannot conn

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2000: --- Fix Version/s: 1.1.0 1.0.1 > cannot connect to cluster in Standalone mode when run

[jira] [Comment Edited] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016602#comment-14016602 ] Chen Chao edited comment on SPARK-2000 at 6/3/14 3:24 PM: -- you ju

[jira] [Comment Edited] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016602#comment-14016602 ] Chen Chao edited comment on SPARK-2000 at 6/3/14 3:24 PM: -- you ju

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016602#comment-14016602 ] Chen Chao commented on SPARK-2000: -- you just run spark-shell and can connect to the clust

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016600#comment-14016600 ] Guoqiang Li commented on SPARK-2000: I tested, have not found the problem. {{cat conf/

[jira] [Comment Edited] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016592#comment-14016592 ] Chen Chao edited comment on SPARK-2000 at 6/3/14 3:11 PM: -- just r

[jira] [Comment Edited] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016592#comment-14016592 ] Chen Chao edited comment on SPARK-2000 at 6/3/14 3:09 PM: -- just r

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Chao updated SPARK-2000: - Summary: cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node wit

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016592#comment-14016592 ] Chen Chao commented on SPARK-2000: -- just run spark-shell in standalone model on one the o

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016535#comment-14016535 ] Guoqiang Li commented on SPARK-2000: How to reproduce the issue? > cannot connect to

[jira] [Commented] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016496#comment-14016496 ] sun.sam commented on SPARK-1998: I'm new here, thanks for reminding me of pulling request.

[jira] [Created] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node

2014-06-03 Thread Chen Chao (JIRA)
Chen Chao created SPARK-2000: Summary: cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node Key: SPARK-2000 URL: https://issues.apache.org/jira/browse/SPARK-2000 Pr

[jira] [Commented] (SPARK-1999) UI : StorageLevel in storage tab and RDD Storage Info never changes

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016385#comment-14016385 ] Chen Chao commented on SPARK-1999: -- PR : https://issues.apache.org/jira/browse/SPARK-1999

[jira] [Comment Edited] (SPARK-1999) UI : StorageLevel in storage tab and RDD Storage Info never changes

2014-06-03 Thread Chen Chao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016385#comment-14016385 ] Chen Chao edited comment on SPARK-1999 at 6/3/14 11:16 AM: --- http

[jira] [Created] (SPARK-1999) UI : StorageLevel in storage tab and RDD Storage Info never changes

2014-06-03 Thread Chen Chao (JIRA)
Chen Chao created SPARK-1999: Summary: UI : StorageLevel in storage tab and RDD Storage Info never changes Key: SPARK-1999 URL: https://issues.apache.org/jira/browse/SPARK-1999 Project: Spark I

[jira] [Resolved] (SPARK-1974) Most examples fail at startup because spark.master is not set

2014-06-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1974. -- Resolution: Not a Problem Fix Version/s: (was: 1.0.1) Decision was to not modify examples, b

[jira] [Commented] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016327#comment-14016327 ] Sean Owen commented on SPARK-1998: -- (Can you make a pull request versus a patch here? thi

[jira] [Commented] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016317#comment-14016317 ] sun.sam commented on SPARK-1998: it seems not completed fixed by the patch in spark-1916

[jira] [Updated] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sun.sam updated SPARK-1998: --- Attachment: (was: patch) > SparkFlumeEvent with body bigger than 1020 bytes are not read properly > -

[jira] [Updated] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sun.sam updated SPARK-1998: --- Attachment: patch.diff > SparkFlumeEvent with body bigger than 1020 bytes are not read properly > ---

[jira] [Updated] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sun.sam updated SPARK-1998: --- Attachment: patch > SparkFlumeEvent with body bigger than 1020 bytes are not read properly >

[jira] [Created] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

2014-06-03 Thread sun.sam (JIRA)
sun.sam created SPARK-1998: -- Summary: SparkFlumeEvent with body bigger than 1020 bytes are not read properly Key: SPARK-1998 URL: https://issues.apache.org/jira/browse/SPARK-1998 Project: Spark Iss