[jira] [Created] (SPARK-1385) Use existing code-path for JSON de/serialization of BlockId

2014-04-01 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1385: Summary: Use existing code-path for JSON de/serialization of BlockId Key: SPARK-1385 URL: https://issues.apache.org/jira/browse/SPARK-1385 Project: Spark Issue

[jira] [Created] (SPARK-1538) SparkUI forgets about all persisted RDD's not associated with stages

2014-04-18 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1538: Summary: SparkUI forgets about all persisted RDD's not associated with stages Key: SPARK-1538 URL: https://issues.apache.org/jira/browse/SPARK-1538 Project: Spark

[jira] [Updated] (SPARK-1538) SparkUI forgets about all persisted RDD's not directly associated with stages

2014-04-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1538: - Summary: SparkUI forgets about all persisted RDD's not directly associated with stages (was: SparkUI

[jira] [Updated] (SPARK-1538) SparkUI forgets about all persisted RDD's not directly associated with the Stage

2014-04-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1538: - Summary: SparkUI forgets about all persisted RDD's not directly associated with the Stage (was: SparkUI

[jira] [Created] (SPARK-1615) Very subtle race condition in SparkListenerSuite

2014-04-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1615: Summary: Very subtle race condition in SparkListenerSuite Key: SPARK-1615 URL: https://issues.apache.org/jira/browse/SPARK-1615 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1615) Very subtle race condition in SparkListenerSuite

2014-04-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1615: - Description: Much of SparkListenerSuite relies on LiveListenerBus's waitUntilEmpty() method. As the

[jira] [Updated] (SPARK-1641) Spark submit warning tells the user to use spark-submit

2014-04-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1641: - Description: $ bin/spark-submit ... Spark assembly has been built with Hive, including Datanucleus jars

[jira] [Commented] (SPARK-1641) Spark submit warning tells the user to use spark-submit

2014-04-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982112#comment-13982112 ] Andrew Or commented on SPARK-1641: -- Ah yes I didn't notice the other one. Me neither, so

[jira] [Closed] (SPARK-1641) Spark submit warning tells the user to use spark-submit

2014-04-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1641. Resolution: Duplicate Duplicates SPARK-1534 Spark submit warning tells the user to use spark-submit

[jira] [Created] (SPARK-1688) PySpark throws unhelpful exception when pyspark cannot be loaded

2014-04-30 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1688: Summary: PySpark throws unhelpful exception when pyspark cannot be loaded Key: SPARK-1688 URL: https://issues.apache.org/jira/browse/SPARK-1688 Project: Spark

[jira] [Created] (SPARK-1735) Make distribution script has missing profiles for special hadoop versions

2014-05-05 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1735: Summary: Make distribution script has missing profiles for special hadoop versions Key: SPARK-1735 URL: https://issues.apache.org/jira/browse/SPARK-1735 Project: Spark

[jira] [Created] (SPARK-1774) SparkSubmit --jars not working for yarn-client

2014-05-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1774: Summary: SparkSubmit --jars not working for yarn-client Key: SPARK-1774 URL: https://issues.apache.org/jira/browse/SPARK-1774 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993218#comment-13993218 ] Andrew Or edited comment on SPARK-1763 at 5/8/14 10:40 PM: ---

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993197#comment-13993197 ] Andrew Or commented on SPARK-1755: -- https://github.com/apache/spark/pull/699

[jira] [Commented] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993198#comment-13993198 ] Andrew Or commented on SPARK-1763: -- This is only for python. There is a seemingly related

[jira] [Closed] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1763. Resolution: Cannot Reproduce SparkSubmit arguments do not propagate to python files on YARN

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993196#comment-13993196 ] Andrew Or commented on SPARK-1755: -- You are correct. However, it also doesn't work for

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Affects Version/s: (was: 1.0.0) 0.9.1 Spark-submit --name does not resolve

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992495#comment-13992495 ] Andrew Or commented on SPARK-1755: -- SPARK-1631 is concerned with fixing the case in which

[jira] [Commented] (SPARK-1792) Missing Spark-Shell Configure Options

2014-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994803#comment-13994803 ] Andrew Or commented on SPARK-1792: -- Spark shell calls spark submit, which loads

[jira] [Created] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

2014-05-12 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1763: Summary: SparkSubmit arguments do not propagate to python files on YARN Key: SPARK-1763 URL: https://issues.apache.org/jira/browse/SPARK-1763 Project: Spark Issue

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Fix Version/s: (was: 1.0.1) 1.0.0 Spark-submit --name does not resolve to

[jira] [Commented] (SPARK-1780) Non-existent SPARK_DAEMON_OPTS is referred to in a few places

2014-05-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995902#comment-13995902 ] Andrew Or commented on SPARK-1780: -- https://github.com/apache/spark/pull/751

[jira] [Created] (SPARK-1808) bin/pyspark does not load default configuration properties

2014-05-13 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1808: Summary: bin/pyspark does not load default configuration properties Key: SPARK-1808 URL: https://issues.apache.org/jira/browse/SPARK-1808 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2014-05-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Affects Version/s: 1.0.0 ExternalAppendOnlyMap can still OOM if one key is very large

[jira] [Commented] (SPARK-1769) Executor loss can cause race condition in Pool

2014-05-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997079#comment-13997079 ] Andrew Or commented on SPARK-1769: -- https://github.com/apache/spark/pull/762 Executor

[jira] [Updated] (SPARK-1824) Python examples still take in master

2014-05-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1824: - Affects Version/s: 1.0.0 Python examples still take in master --

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2014-05-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Fix Version/s: 1.1.0 ExternalAppendOnlyMap can still OOM if one key is very large

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2014-05-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Description: If the values for one key do not collectively fit into memory, then the map will still OOM

[jira] [Created] (SPARK-1761) Add broadcast information on SparkUI storage tab

2014-05-14 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1761: Summary: Add broadcast information on SparkUI storage tab Key: SPARK-1761 URL: https://issues.apache.org/jira/browse/SPARK-1761 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Description: In YARN client mode, --name is ignored because the deploy mode is client, and the name is

[jira] [Updated] (SPARK-1824) Python examples still take in master

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1824: - Fix Version/s: 1.0.1 Python examples still take in master --

[jira] [Created] (SPARK-1780) Non-existent SPARK_DAEMON_OPTS is referred to in a few places

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1780: Summary: Non-existent SPARK_DAEMON_OPTS is referred to in a few places Key: SPARK-1780 URL: https://issues.apache.org/jira/browse/SPARK-1780 Project: Spark Issue

[jira] [Created] (SPARK-1840) SparkListenerBus prints out scary error message when terminating normally

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1840: Summary: SparkListenerBus prints out scary error message when terminating normally Key: SPARK-1840 URL: https://issues.apache.org/jira/browse/SPARK-1840 Project: Spark

[jira] [Updated] (SPARK-1840) SparkListenerBus prints out scary error message when terminating normally

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1840: - Description: This is because the Scala's NonLocalReturnControl (which extends ControlThrowable) is

[jira] [Closed] (SPARK-1838) On a YARN cluster, Spark doesn't run on local mode

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1838. Resolution: Not a Problem Looks like I accidentally set SPARK_YARN_MODE to true manually, which directly

[jira] [Created] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1823: Summary: ExternalAppendOnlyMap can still OOM if one key is very large Key: SPARK-1823 URL: https://issues.apache.org/jira/browse/SPARK-1823 Project: Spark Issue

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Description: In YARN client mode, --name is ignored because the deploy mode is client, and the name is

[jira] [Created] (SPARK-1838) On a YARN cluster, Spark doesn't run on local mode

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1838: Summary: On a YARN cluster, Spark doesn't run on local mode Key: SPARK-1838 URL: https://issues.apache.org/jira/browse/SPARK-1838 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

2014-05-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1755: Summary: Spark-submit --name does not resolve to application name on YARN Key: SPARK-1755 URL: https://issues.apache.org/jira/browse/SPARK-1755 Project: Spark

[jira] [Commented] (SPARK-1821) Document History Server

2014-05-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996755#comment-13996755 ] Andrew Or commented on SPARK-1821: -- Yes, it should be documented under monitoring.html in

[jira] [Updated] (SPARK-1850) Bad exception if multiple jars exist when running PySpark

2014-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1850: - Description: {code} Found multiple Spark assembly jars in

[jira] [Created] (SPARK-1850) Bad exception if multiple jars exist when running PySpark

2014-05-16 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1850: Summary: Bad exception if multiple jars exist when running PySpark Key: SPARK-1850 URL: https://issues.apache.org/jira/browse/SPARK-1850 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1850) Bad exception if multiple jars exist when running PySpark

2014-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1850: - Description: {code} Found multiple Spark assembly jars in

[jira] [Updated] (SPARK-1824) Python examples still take in master

2014-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1824: - Issue Type: Improvement (was: Bug) Python examples still take in master

[jira] [Commented] (SPARK-1808) bin/pyspark does not load default configuration properties

2014-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13999468#comment-13999468 ] Andrew Or commented on SPARK-1808: -- https://github.com/apache/spark/pull/799

[jira] [Created] (SPARK-1894) The default ec2 set-up ignores --driver-class-path

2014-05-20 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1894: Summary: The default ec2 set-up ignores --driver-class-path Key: SPARK-1894 URL: https://issues.apache.org/jira/browse/SPARK-1894 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1894) The default ec2 set-up ignores --driver-class-path

2014-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1894: - Description: If the user sets up an ec2 cluster, the scripts automatically add the following lines to

[jira] [Updated] (SPARK-1894) The default ec2 set-up ignores --driver-class-path

2014-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1894: - Description: If the user sets up an ec2 cluster, the scripts automatically add the following lines to

[jira] [Created] (SPARK-1896) MASTER masks spark.master in spark-shell

2014-05-21 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1896: Summary: MASTER masks spark.master in spark-shell Key: SPARK-1896 URL: https://issues.apache.org/jira/browse/SPARK-1896 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1897) Spark shell --jars (or spark.jars) doesn't work

2014-05-21 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1897: Summary: Spark shell --jars (or spark.jars) doesn't work Key: SPARK-1897 URL: https://issues.apache.org/jira/browse/SPARK-1897 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1900) Fix running PySpark files on YARN

2014-05-22 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1900: Summary: Fix running PySpark files on YARN Key: SPARK-1900 URL: https://issues.apache.org/jira/browse/SPARK-1900 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1900) Fix running PySpark files on YARN

2014-05-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1900: - Description: If I run the following on a YARN cluster ``` bin/spark-submit sheep.py --master yarn-client

[jira] [Updated] (SPARK-1900) Fix running PySpark files on YARN

2014-05-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1900: - Description: If I run the following on a YARN cluster {code} bin/spark-submit sheep.py --master

[jira] [Updated] (SPARK-1900) Fix running PySpark files on YARN

2014-05-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1900: - Description: If I run the following on a YARN cluster {code} bin/spark-submit sheep.py --master

[jira] [Closed] (SPARK-1904) ZooKeeper URI in spark-env.sh no longer working w/ bin/pyspark

2014-05-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1904. Resolution: Not a Problem ZooKeeper URI in spark-env.sh no longer working w/ bin/pyspark

[jira] [Created] (SPARK-1918) PySpark shell --py-files does not work for zip files

2014-05-23 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1918: Summary: PySpark shell --py-files does not work for zip files Key: SPARK-1918 URL: https://issues.apache.org/jira/browse/SPARK-1918 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1918) PySpark shell --py-files does not work for zip files

2014-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007874#comment-14007874 ] Andrew Or commented on SPARK-1918: -- https://github.com/apache/spark/pull/853 PySpark

[jira] [Created] (SPARK-1919) In Windows, Spark shell cannot load classes in spark.jars (--jars)

2014-05-23 Thread Andrew Or (JIRA)
Andrew Or created SPARK-1919: Summary: In Windows, Spark shell cannot load classes in spark.jars (--jars) Key: SPARK-1919 URL: https://issues.apache.org/jira/browse/SPARK-1919 Project: Spark

[jira] [Updated] (SPARK-2105) SparkUI doesn't remove active stages that failed

2014-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2105: - Affects Version/s: 1.0.0 SparkUI doesn't remove active stages that failed

[jira] [Updated] (SPARK-2105) SparkUI doesn't remove active stages that failed

2014-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2105: - Component/s: Spark Core SparkUI doesn't remove active stages that failed

[jira] [Created] (SPARK-2105) SparkUI doesn't remove active stages that failed

2014-06-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2105: Summary: SparkUI doesn't remove active stages that failed Key: SPARK-2105 URL: https://issues.apache.org/jira/browse/SPARK-2105 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2105) SparkUI doesn't remove active stages that failed

2014-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2105: - Description: If a stage fails because its tasks cannot be serialized, for instance, the failed stage

[jira] [Created] (SPARK-2144) SparkUI Executors tab displays incorrect RDD blocks

2014-06-13 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2144: Summary: SparkUI Executors tab displays incorrect RDD blocks Key: SPARK-2144 URL: https://issues.apache.org/jira/browse/SPARK-2144 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1201) Do not materialize partitions whenever possible in BlockManager

2014-06-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031425#comment-14031425 ] Andrew Or commented on SPARK-1201: -- Depends on when we release 1.0.1. I am actually

[jira] [Updated] (SPARK-2147) Master UI forgets about Executors when application exits cleanly

2014-06-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2147: - Description: When an application exits cleanly, the Master will remove all executors from the

[jira] [Created] (SPARK-2161) UI should remember executors that have been removed

2014-06-16 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2161: Summary: UI should remember executors that have been removed Key: SPARK-2161 URL: https://issues.apache.org/jira/browse/SPARK-2161 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1777) Pass cached blocks directly to disk if memory is not large enough

2014-06-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1777: - Attachment: spark-1777-design-doc.pdf Pass cached blocks directly to disk if memory is not large enough

[jira] [Updated] (SPARK-2242) Running sc.parallelize(..).count() hangs pyspark

2014-06-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2242: - Priority: Blocker (was: Critical) Running sc.parallelize(..).count() hangs pyspark

[jira] [Created] (SPARK-2258) Worker UI displays zombie executors

2014-06-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2258: Summary: Worker UI displays zombie executors Key: SPARK-2258 URL: https://issues.apache.org/jira/browse/SPARK-2258 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2258) Worker UI displays zombie executors

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2258: - Attachment: Screen Shot 2014-06-24 at 9.23.18 AM.png Worker UI displays zombie executors

[jira] [Created] (SPARK-2259) Spark submit documentation for --deploy-mode is highly misleading

2014-06-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2259: Summary: Spark submit documentation for --deploy-mode is highly misleading Key: SPARK-2259 URL: https://issues.apache.org/jira/browse/SPARK-2259 Project: Spark

[jira] [Created] (SPARK-2260) Spark submit standalone-cluster mode is broken

2014-06-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2260: Summary: Spark submit standalone-cluster mode is broken Key: SPARK-2260 URL: https://issues.apache.org/jira/browse/SPARK-2260 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1911) Warn users that jars should be built with Java 6 for YARN

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Summary: Warn users that jars should be built with Java 6 for YARN (was: Warn users that jars should be

[jira] [Reopened] (SPARK-1911) Warn users that jars should be built with Java 6 for YARN

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-1911: -- Looks like we still haven't fixed this. At the very least we should explicitly add this warning in the

[jira] [Updated] (SPARK-1911) Warn users if their jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Fix Version/s: (was: 1.0.0) 1.1.0 Warn users if their jars are not built with

[jira] [Updated] (SPARK-1911) Warn users if their jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Affects Version/s: 1.1.0 Warn users if their jars are not built with Java 6

[jira] [Updated] (SPARK-1911) Warn users if their jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Description: Currently we warn users only in make-distribution.sh, but most users build the jars

[jira] [Updated] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Summary: Warn users if their assembly jars are not built with Java 6 (was: Warn users if their jars are

[jira] [Updated] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Description: The root cause of the problem is detailed in:

[jira] [Updated] (SPARK-1911) Warn users if their jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Summary: Warn users if their jars are not built with Java 6 (was: Warn users that jars should be built

[jira] [Updated] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1911: - Description: The root cause of the problem is detailed in:

[jira] [Commented] (SPARK-2259) Spark submit documentation for --deploy-mode is highly misleading

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042811#comment-14042811 ] Andrew Or commented on SPARK-2259: -- https://github.com/apache/spark/pull/1200 Spark

[jira] [Updated] (SPARK-2266) Log page on Worker UI displays Some(app-id)

2014-06-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2266: - Attachment: Screen Shot 2014-06-24 at 5.07.54 PM.png Log page on Worker UI displays Some(app-id)

[jira] [Updated] (SPARK-2258) Worker UI displays zombie executors

2014-06-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2258: - Fix Version/s: 1.1.0 Worker UI displays zombie executors ---

[jira] [Resolved] (SPARK-2258) Worker UI displays zombie executors

2014-06-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-2258. -- Resolution: Fixed https://github.com/apache/spark/pull/1213 Worker UI displays zombie executors

[jira] [Created] (SPARK-2296) Refactor util.JsonProtocol for evolvability

2014-06-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2296: Summary: Refactor util.JsonProtocol for evolvability Key: SPARK-2296 URL: https://issues.apache.org/jira/browse/SPARK-2296 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2300) PySpark shell hides stderr output

2014-06-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2300: Summary: PySpark shell hides stderr output Key: SPARK-2300 URL: https://issues.apache.org/jira/browse/SPARK-2300 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-2300) PySpark shell hides stderr output

2014-06-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-2300. -- Resolution: Fixed PySpark shell hides stderr output -

[jira] [Updated] (SPARK-2307) SparkUI Storage page cached statuses incorrect

2014-06-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2307: - Attachment: Screen Shot 2014-06-27 at 11.09.54 AM.png SparkUI Storage page cached statuses incorrect

[jira] [Updated] (SPARK-2307) SparkUI Storage page cached statuses incorrect

2014-06-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2307: - Description: See attached: the executor has 512MB, but somehow it has cached (279 + 27 + 279 + 27) =

[jira] [Commented] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

2014-06-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046374#comment-14046374 ] Andrew Or commented on SPARK-1392: -- and by fixed by SPARK-1777 he means

[jira] [Updated] (SPARK-2340) Resolve paths properly for History Server

2014-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2340: - Description: We resolved paths for --jars and --files for spark submit when releasing Spark 1.0. The

[jira] [Created] (SPARK-2340) Resolve paths properly for History Server

2014-07-01 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2340: Summary: Resolve paths properly for History Server Key: SPARK-2340 URL: https://issues.apache.org/jira/browse/SPARK-2340 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2340) Resolve paths properly for event logging / history server

2014-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2340: - Summary: Resolve paths properly for event logging / history server (was: Resolve paths properly for

[jira] [Commented] (SPARK-1850) Bad exception if multiple jars exist when running PySpark

2014-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050318#comment-14050318 ] Andrew Or commented on SPARK-1850: -- Ye, I will change it. Bad exception if multiple

[jira] [Closed] (SPARK-1850) Bad exception if multiple jars exist when running PySpark

2014-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1850. Resolution: Fixed Bad exception if multiple jars exist when running PySpark

[jira] [Created] (SPARK-2349) Fix NPE in ExternalAppendOnlyMap

2014-07-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2349: Summary: Fix NPE in ExternalAppendOnlyMap Key: SPARK-2349 URL: https://issues.apache.org/jira/browse/SPARK-2349 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-2350) Master throws NPE

2014-07-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2350: Summary: Master throws NPE Key: SPARK-2350 URL: https://issues.apache.org/jira/browse/SPARK-2350 Project: Spark Issue Type: Bug Components: Spark Core

[jira] [Updated] (SPARK-2350) Master throws NPE

2014-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2350: - Description: ... if we launch a driver and there are more waiting drivers to be launched. This is

  1   2   3   4   5   6   7   8   9   10   >