[jira] [Closed] (SPARK-3071) Increase default driver memory

2015-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3071. Resolution: Fixed Fix Version/s: 1.5.0 > Increase default driver memory > ---

[jira] [Closed] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8687. Resolution: Fixed Assignee: SaintBacchus Fix Version/s: 1.5.0 Target Version/s:

[jira] [Updated] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8687: - Fix Version/s: 1.4.2 > Spark on yarn-client mode can't send `spark.yarn.credentials.file` to > executor.

[jira] [Updated] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-07-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8687: - Target Version/s: 1.5.0, 1.4.2 (was: 1.5.0) > Spark on yarn-client mode can't send `spark.yarn.credential

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Summary: Published POMs are no longer effective POMs (was: Pusblished POMs are no longer effective POMs)

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Assignee: Andrew Or > Published POMs are no longer effective POMs > --

[jira] [Commented] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612220#comment-14612220 ] Andrew Or commented on SPARK-8781: -- It was needed because of a bug in the maven shade plu

[jira] [Commented] (SPARK-2319) Number of tasks on executors become negative after executor failures

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612248#comment-14612248 ] Andrew Or commented on SPARK-2319: -- [~KaiXinXIaoLei] the negative active tasks is fixed i

[jira] [Updated] (SPARK-8581) Simplify and clean up the checkpointing code

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8581: - Target Version/s: 1.5.0 (was: 1.5.0, 1.4.2) > Simplify and clean up the checkpointing code >

[jira] [Closed] (SPARK-8581) Simplify and clean up the checkpointing code

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8581. Resolution: Fixed Fix Version/s: 1.5.0 > Simplify and clean up the checkpointing code > -

[jira] [Updated] (SPARK-8584) Better exception message if invalid checkpoint dir is specified

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8584: - Target Version/s: 1.5.0 (was: 1.5.0, 1.4.2) > Better exception message if invalid checkpoint dir is speci

[jira] [Closed] (SPARK-8584) Better exception message if invalid checkpoint dir is specified

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8584. Resolution: Fixed Fix Version/s: 1.5.0 > Better exception message if invalid checkpoint dir is specif

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Affects Version/s: (was: 1.4.1) 1.4.0 Target Version/s: 1.3.2, 1.4.1, 1.5.0

[jira] [Commented] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612313#comment-14612313 ] Andrew Or commented on SPARK-8781: -- Thanks for catching this [~k.shaposhni...@gmail.com].

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Priority: Blocker (was: Major) > Published POMs are no longer effective POMs > --

[jira] [Updated] (SPARK-8578) Should ignore user defined output committer when appending data

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8578: - Fix Version/s: (was: 1.4.2) 1.4.1 > Should ignore user defined output committer whe

[jira] [Updated] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8746: - Target Version/s: 1.4.1, 1.5.0 (was: 1.4.1) > Need to update download link for Hive 0.13.1 jars (HiveComp

[jira] [Updated] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8746: - Fix Version/s: (was: 1.4.2) 1.4.1 > Need to update download link for Hive 0.13.1 ja

[jira] [Updated] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8687: - Fix Version/s: (was: 1.4.2) 1.4.1 > Spark on yarn-client mode can't send `spark.yar

[jira] [Updated] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8687: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > Spark on yarn-client mode can't send `spark.yarn.cre

[jira] [Updated] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8787: - Fix Version/s: (was: 1.4.2) 1.4.1 > Change the parameter order of @deprecated in p

[jira] [Updated] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8754: - Fix Version/s: (was: 1.4.2) 1.4.1 > YarnClientSchedulerBackend doesn't stop gracefu

[jira] [Updated] (SPARK-7820) Java8-tests suite compile error under SBT

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7820: - Fix Version/s: (was: 1.4.2) 1.4.1 > Java8-tests suite compile error under SBT > ---

[jira] [Updated] (SPARK-8769) toLocalIterator should mention it results in many jobs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8769: - Fix Version/s: (was: 1.4.2) 1.4.1 > toLocalIterator should mention it results in ma

[jira] [Updated] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8754: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > YarnClientSchedulerBackend doesn't stop gracefully i

[jira] [Updated] (SPARK-8769) toLocalIterator should mention it results in many jobs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8769: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > toLocalIterator should mention it results in many jo

[jira] [Updated] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8766: - Fix Version/s: 1.4.1 > DataFrame Python API should work with column which has non-ascii character in > it

[jira] [Updated] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3444: - Fix Version/s: (was: 1.4.0) 1.4.1 > Provide a way to easily change the log level in

[jira] [Updated] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8766: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > DataFrame Python API should work with column which h

[jira] [Updated] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8766: - Target Version/s: 1.5.0, 1.4.2 (was: 1.3.2, 1.5.0, 1.4.2) > DataFrame Python API should work with column

[jira] [Updated] (SPARK-8621) crosstab exception when one of the value is empty

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8621: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > crosstab exception when one of the value is empty >

[jira] [Updated] (SPARK-8535) PySpark : Can't create DataFrame from Pandas dataframe with no explicit column name

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8535: - Fix Version/s: 1.4.1 > PySpark : Can't create DataFrame from Pandas dataframe with no explicit > column n

[jira] [Updated] (SPARK-8621) crosstab exception when one of the value is empty

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8621: - Fix Version/s: (was: 1.4.2) 1.4.1 > crosstab exception when one of the value is emp

[jira] [Updated] (SPARK-8736) GBTRegressionModel thresholds prediction but should not

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8736: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0, 1.4.2) > GBTRegressionModel thresholds prediction but should

[jira] [Updated] (SPARK-8372) History server shows incorrect information for application not started

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8372: - Fix Version/s: (was: 1.4.2) 1.4.1 > History server shows incorrect information for

[jira] [Updated] (SPARK-8563) Bug that IndexedRowMatrix.computeSVD() yields the U with wrong numCols

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8563: - Target Version/s: 1.0.3, 1.1.2, 1.2.3, 1.3.2, 1.4.1, 1.5.0 (was: 1.0.3, 1.1.2, 1.2.3, 1.3.2, 1.5.0, 1.4.2

[jira] [Updated] (SPARK-8563) Bug that IndexedRowMatrix.computeSVD() yields the U with wrong numCols

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8563: - Fix Version/s: (was: 1.4.2) 1.4.1 > Bug that IndexedRowMatrix.computeSVD() yields t

[jira] [Updated] (SPARK-8736) GBTRegressionModel thresholds prediction but should not

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8736: - Fix Version/s: (was: 1.4.2) 1.4.1 > GBTRegressionModel thresholds prediction but sh

[jira] [Updated] (SPARK-8628) Race condition in AbstractSparkSQLParser.parse

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8628: - Fix Version/s: (was: 1.4.2) 1.4.1 > Race condition in AbstractSparkSQLParser.parse

[jira] [Updated] (SPARK-8535) PySpark : Can't create DataFrame from Pandas dataframe with no explicit column name

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8535: - Fix Version/s: 1.3.2 > PySpark : Can't create DataFrame from Pandas dataframe with no explicit > column n

[jira] [Closed] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8781. Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 1.3.2 > Published

[jira] [Updated] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1564: - Fix Version/s: 1.5.0 > Add JavaScript into Javadoc to turn ::Experimental:: and such into badges > ---

[jira] [Closed] (SPARK-7835) Refactor HeartbeatReceiverSuite for coverage and clean up

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7835. Resolution: Fixed Fix Version/s: 1.5.0 > Refactor HeartbeatReceiverSuite for coverage and clean up >

[jira] [Created] (SPARK-8819) Spark doesn't compile with maven 3.3.x

2015-07-03 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8819: Summary: Spark doesn't compile with maven 3.3.x Key: SPARK-8819 URL: https://issues.apache.org/jira/browse/SPARK-8819 Project: Spark Issue Type: Bug Compon

[jira] [Updated] (SPARK-8819) Spark doesn't compile with maven 3.3.x

2015-07-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8819: - Description: Simple reproduction: Install maven 3.3.3 and run "build/mvn clean package -DskipTests" This

[jira] [Closed] (SPARK-5222) YARN client and cluster modes have different app name behaviors

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5222. Resolution: Won't Fix > YARN client and cluster modes have different app name behaviors > --

[jira] [Closed] (SPARK-8016) YARN cluster / client modes have different app names for python

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8016. Resolution: Won't Fix See resolution for SPARK-5012 to see why this is a won't fix. > YARN cluster / client

[jira] [Comment Edited] (SPARK-8016) YARN cluster / client modes have different app names for python

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615458#comment-14615458 ] Andrew Or edited comment on SPARK-8016 at 7/6/15 6:42 PM: -- See re

[jira] [Closed] (SPARK-8656) Spark Standalone master json API's worker number is not match web UI number

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8656. Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: 1.5.0 > Spark Standalone master js

[jira] [Closed] (SPARK-6707) Mesos Scheduler should allow the user to specify constraints based on slave attributes

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6707. Resolution: Fixed Fix Version/s: 1.5.0 > Mesos Scheduler should allow the user to specify constraints

[jira] [Updated] (SPARK-8656) Spark Standalone master json API's worker number is not match web UI number

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8656: - Assignee: thegiive > Spark Standalone master json API's worker number is not match web UI number > ---

[jira] [Updated] (SPARK-6707) Mesos Scheduler should allow the user to specify constraints based on slave attributes

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6707: - Assignee: Ankur Chauhan > Mesos Scheduler should allow the user to specify constraints based on slave > a

[jira] [Commented] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616076#comment-14616076 ] Andrew Or commented on SPARK-8781: -- This was later reverted in master, branch-1.4 and bra

[jira] [Created] (SPARK-8910) MiMa test is flaky because it starts a SQLContext

2015-07-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8910: Summary: MiMa test is flaky because it starts a SQLContext Key: SPARK-8910 URL: https://issues.apache.org/jira/browse/SPARK-8910 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8910) MiMa test is flaky because it starts a SQLContext

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8910: - Description: I've seen this a many times on GitHub. At the very least we should disable the SparkUI to pr

[jira] [Updated] (SPARK-8910) MiMa test is flaky because it starts a SQLContext

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8910: - Description: I've seen this many times on GitHub. At the very least we should disable the SparkUI to prev

[jira] [Created] (SPARK-8911) Local mode endless heartbeat warnings

2015-07-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8911: Summary: Local mode endless heartbeat warnings Key: SPARK-8911 URL: https://issues.apache.org/jira/browse/SPARK-8911 Project: Spark Issue Type: Bug Compone

[jira] [Created] (SPARK-8939) YARN EC2

2015-07-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8939: Summary: YARN EC2 Key: SPARK-8939 URL: https://issues.apache.org/jira/browse/SPARK-8939 Project: Spark Issue Type: Bug Components: EC2 Affects Version

[jira] [Updated] (SPARK-8939) YARN EC2 does not

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8939: - Description: I just set it up from scratch using the spark-ec2 script. Then I ran {code} bin/spark-shell -

[jira] [Updated] (SPARK-8939) YARN EC2 default setting fails with IllegalArgumentException

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8939: - Summary: YARN EC2 default setting fails with IllegalArgumentException (was: YARN EC2 does not ) > YARN E

[jira] [Updated] (SPARK-8939) YARN EC2 does not

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8939: - Environment: (was: I just set it up from scratch using the spark-ec2 script. Then I ran {code} bin/spa

[jira] [Updated] (SPARK-8939) YARN EC2 does not

2015-07-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8939: - Summary: YARN EC2 does not (was: YARN EC2 ) > YARN EC2 does not > -- > >

[jira] [Commented] (SPARK-8939) YARN EC2 default setting fails with IllegalArgumentException

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621021#comment-14621021 ] Andrew Or commented on SPARK-8939: -- [~shivaram] > YARN EC2 default setting fails with Il

[jira] [Created] (SPARK-8958) Dynamic allocation: change cached executor timeout to infinity

2015-07-09 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8958: Summary: Dynamic allocation: change cached executor timeout to infinity Key: SPARK-8958 URL: https://issues.apache.org/jira/browse/SPARK-8958 Project: Spark Issue T

[jira] [Commented] (SPARK-8939) YARN EC2 default setting fails with IllegalArgumentException

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621117#comment-14621117 ] Andrew Or commented on SPARK-8939: -- It's just the default. No particular reason from my u

[jira] [Updated] (SPARK-8953) SPARK_EXECUTOR_CORES is not read in SparkSubmit

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8953: - Summary: SPARK_EXECUTOR_CORES is not read in SparkSubmit (was: SPARK_EXECUTOR_CORES has no effect to dyn

[jira] [Updated] (SPARK-8953) SPARK_EXECUTOR_CORES has no effect to dynamic executor allocation function

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8953: - Affects Version/s: 1.0.0 > SPARK_EXECUTOR_CORES has no effect to dynamic executor allocation function > -

[jira] [Updated] (SPARK-8953) SPARK_EXECUTOR_CORES is not read in SparkSubmit

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8953: - Priority: Trivial (was: Minor) > SPARK_EXECUTOR_CORES is not read in SparkSubmit > --

[jira] [Closed] (SPARK-8953) SPARK_EXECUTOR_CORES has no effect to dynamic executor allocation function

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8953. Resolution: Fixed Assignee: meiyoula Fix Version/s: 1.5.0 Target Version/s: 1.5.

[jira] [Updated] (SPARK-7419) Flaky test: o.a.s.streaming.CheckpointSuite

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7419: - Assignee: Shixiong Zhu (was: Tathagata Das) > Flaky test: o.a.s.streaming.CheckpointSuite > -

[jira] [Closed] (SPARK-7419) Flaky test: o.a.s.streaming.CheckpointSuite

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7419. Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Target Version/s: 1.4.2,

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621196#comment-14621196 ] Andrew Or commented on SPARK-2017: -- Partially resolved by https://github.com/apache/spark

[jira] [Updated] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2017: - Fix Version/s: 1.5.0 1.4.2 > web ui stage page becomes unresponsive when the number of

[jira] [Comment Edited] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621196#comment-14621196 ] Andrew Or edited comment on SPARK-2017 at 7/9/15 8:24 PM: -- Resolv

[jira] [Closed] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2017. Resolution: Fixed Assignee: Andrew Or Target Version/s: 1.4.2, 1.5.0 > web ui stage pa

[jira] [Updated] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2017: - Affects Version/s: 1.0.0 > web ui stage page becomes unresponsive when the number of tasks is large >

[jira] [Updated] (SPARK-6287) Add support for dynamic allocation in the Mesos coarse-grained scheduler

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6287: - Affects Version/s: 1.2.0 > Add support for dynamic allocation in the Mesos coarse-grained scheduler >

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files for drivers launched with Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Priority: Critical (was: Minor) > Support cleaning up shuffle files for drivers launched with Mesos > ---

[jira] [Closed] (SPARK-6287) Add support for dynamic allocation in the Mesos coarse-grained scheduler

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6287. Resolution: Fixed Assignee: Iulian Dragos Fix Version/s: 1.5.0 Target Version/s:

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files when using shuffle service in Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Summary: Support cleaning up shuffle files when using shuffle service in Mesos (was: Support cleaning up

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files when using shuffle service in Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Affects Version/s: 1.5.0 > Support cleaning up shuffle files when using shuffle service in Mesos > ---

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files when using shuffle service in Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Target Version/s: 1.5.0 > Support cleaning up shuffle files when using shuffle service in Mesos >

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files when using shuffle service in Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Assignee: Timothy Chen > Support cleaning up shuffle files when using shuffle service in Mesos > -

[jira] [Updated] (SPARK-8873) Support cleaning up shuffle files when using shuffle service in Mesos

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8873: - Affects Version/s: (was: 1.5.0) 1.2.0 > Support cleaning up shuffle files when

[jira] [Commented] (SPARK-6287) Add support for dynamic allocation in the Mesos coarse-grained scheduler

2015-07-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621204#comment-14621204 ] Andrew Or commented on SPARK-6287: -- Note that this feature is not complete. The shuffle f

[jira] [Closed] (SPARK-8958) Dynamic allocation: change cached executor timeout to infinity

2015-07-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8958. Resolution: Fixed Fix Version/s: 1.5.0 > Dynamic allocation: change cached executor timeout to infini

[jira] [Closed] (SPARK-8675) Executors created by LocalBackend won't get the same classpath as other executor backends

2015-07-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8675. Resolution: Fixed Assignee: Min Zhou Fix Version/s: 1.5.0 Target Version/s: 1.5.

[jira] [Created] (SPARK-8987) Increase test coverage of DAGScheduler

2015-07-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8987: Summary: Increase test coverage of DAGScheduler Key: SPARK-8987 URL: https://issues.apache.org/jira/browse/SPARK-8987 Project: Spark Issue Type: Bug Compon

[jira] [Created] (SPARK-9025) Storage tab shows no blocks for cached RDDs

2015-07-13 Thread Andrew Or (JIRA)
Andrew Or created SPARK-9025: Summary: Storage tab shows no blocks for cached RDDs Key: SPARK-9025 URL: https://issues.apache.org/jira/browse/SPARK-9025 Project: Spark Issue Type: Bug C

[jira] [Updated] (SPARK-8582) Optimize checkpointing to avoid computing an RDD twice

2015-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8582: - Assignee: Shixiong Zhu Target Version/s: 1.6.0 (was: ) > Optimize checkpointing to avoid comp

[jira] [Updated] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10986: -- Affects Version/s: 1.5.2 Target Version/s: 1.6.0 > ClassNotFoundException when running on Client m

[jira] [Updated] (SPARK-7970) Optimize code for SQL queries fired on Union of RDDs (closure cleaner)

2015-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7970: - Assignee: Nitin Goyal Target Version/s: 1.6.0 > Optimize code for SQL queries fired on Union o

[jira] [Resolved] (SPARK-11112) DAG visualization: display RDD callsite

2015-11-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-2. --- Resolution: Fixed Fix Version/s: 1.6.0 > DAG visualization: display RDD callsite > ---

[jira] [Updated] (SPARK-10790) Dynamic Allocation does not request any executors if first stage needs less than or equal to spark.dynamicAllocation.initialExecutors

2015-11-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10790: -- Affects Version/s: (was: 1.5.1) 1.5.0 > Dynamic Allocation does not request

[jira] [Updated] (SPARK-10790) Dynamic Allocation does not request any executors if first stage needs less than or equal to spark.dynamicAllocation.initialExecutors

2015-11-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10790: -- Affects Version/s: (was: 1.5.0) 1.5.1 > Dynamic Allocation does not request

[jira] [Resolved] (SPARK-10471) Mesos Scheduler should reject offers not matching constraints for longer period of time

2015-11-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10471. --- Resolution: Fixed Assignee: Felix Bechstein Fix Version/s: 1.6.0 Target V

[jira] [Resolved] (SPARK-11296) FutureAction.onComplete consumes a thread while waiting for job

2015-11-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-11296. --- Resolution: Duplicate > FutureAction.onComplete consumes a thread while waiting for job > ---

[jira] [Updated] (SPARK-10444) Remove duplication in Mesos schedulers

2015-11-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10444: -- Target Version/s: 1.7.0 (was: 1.6.0) > Remove duplication in Mesos schedulers > --

[jira] [Resolved] (SPARK-10918) Task failed because executor kill by driver

2015-11-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10918. --- Resolution: Duplicate > Task failed because executor kill by driver > ---

[jira] [Updated] (SPARK-9552) Dynamic allocation kills busy executors on race condition

2015-11-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9552: - Summary: Dynamic allocation kills busy executors on race condition (was: Add force control for killExecut

<    3   4   5   6   7   8   9   10   11   12   >