[jira] [Updated] (SPARK-1384) spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs

[ https://issues.apache.org/jira/browse/SPARK-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1384: - Description: I've found an issue with the spark-shell in yarn-client mode in the 0.9.1 rc3

[jira] [Created] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn

Thomas Graves created SPARK-1407: Summary: EventLogging to HDFS doesn't work properly on yarn Key: SPARK-1407 URL: https://issues.apache.org/jira/browse/SPARK-1407 Project: Spark Issue Type:

[jira] [Created] (SPARK-1408) Modify Spark on Yarn to point to the history server when app finishes

Thomas Graves created SPARK-1408: Summary: Modify Spark on Yarn to point to the history server when app finishes Key: SPARK-1408 URL: https://issues.apache.org/jira/browse/SPARK-1408 Project: Spark

[jira] [Assigned] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

[ https://issues.apache.org/jira/browse/SPARK-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1417: Assignee: Thomas Graves Spark on Yarn - spark UI link from resourcemanager is broken

[jira] [Commented] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn

[ https://issues.apache.org/jira/browse/SPARK-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963025#comment-13963025 ] Thomas Graves commented on SPARK-1407: -- Note that the logging also doesn't work on

[jira] [Commented] (SPARK-1049) spark on yarn - yarn-client mode doesn't always exit properly

[ https://issues.apache.org/jira/browse/SPARK-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963409#comment-13963409 ] Thomas Graves commented on SPARK-1049: -- Yes it was, go ahead and resolve it. spark

[jira] [Assigned] (SPARK-1049) spark on yarn - yarn-client mode doesn't always exit properly

[ https://issues.apache.org/jira/browse/SPARK-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1049: Assignee: Thomas Graves spark on yarn - yarn-client mode doesn't always exit properly

[jira] [Commented] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn

[ https://issues.apache.org/jira/browse/SPARK-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963456#comment-13963456 ] Thomas Graves commented on SPARK-1407: -- That was the problem. I modified SparkHdfsLR

[jira] [Closed] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn

[ https://issues.apache.org/jira/browse/SPARK-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-1407. Resolution: Not a Problem Assignee: Thomas Graves EventLogging to HDFS doesn't work

[jira] [Created] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

Thomas Graves created SPARK-1453: Summary: Improve the way Spark on Yarn waits for executors before starting Key: SPARK-1453 URL: https://issues.apache.org/jira/browse/SPARK-1453 Project: Spark

[jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966570#comment-13966570 ] Thomas Graves commented on SPARK-1465: --

[jira] [Resolved] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

[ https://issues.apache.org/jira/browse/SPARK-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1417. -- Resolution: Fixed Fix Version/s: 1.0.0 Spark on Yarn - spark UI link from

[jira] [Created] (SPARK-1472) Go through YARN api used in Spark to make sure we aren't using Private Apis

Thomas Graves created SPARK-1472: Summary: Go through YARN api used in Spark to make sure we aren't using Private Apis Key: SPARK-1472 URL: https://issues.apache.org/jira/browse/SPARK-1472 Project:

[jira] [Commented] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968374#comment-13968374 ] Thomas Graves commented on SPARK-1453: -- Actually there are 2 timeouts. The one you

[jira] [Comment Edited] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968465#comment-13968465 ] Thomas Graves edited comment on SPARK-1474 at 4/14/14 4:06 PM:

[jira] [Assigned] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1474: Assignee: Thomas Graves Can not Launch spark application by YARN Client with

[jira] [Created] (SPARK-1491) maven hadoop-provided profile fails to build

Thomas Graves created SPARK-1491: Summary: maven hadoop-provided profile fails to build Key: SPARK-1491 URL: https://issues.apache.org/jira/browse/SPARK-1491 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1492) running-on-yarn doc should use spark-submit script for examples

Thomas Graves created SPARK-1492: Summary: running-on-yarn doc should use spark-submit script for examples Key: SPARK-1492 URL: https://issues.apache.org/jira/browse/SPARK-1492 Project: Spark

[jira] [Created] (SPARK-1502) Spark on Yarn: add config option to not include yarn/mapred cluster classpath

Thomas Graves created SPARK-1502: Summary: Spark on Yarn: add config option to not include yarn/mapred cluster classpath Key: SPARK-1502 URL: https://issues.apache.org/jira/browse/SPARK-1502 Project:

[jira] [Created] (SPARK-1507) Spark on Yarn: Add support for user to specify # cores for ApplicationMaster

Thomas Graves created SPARK-1507: Summary: Spark on Yarn: Add support for user to specify # cores for ApplicationMaster Key: SPARK-1507 URL: https://issues.apache.org/jira/browse/SPARK-1507 Project:

[jira] [Resolved] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1465. -- Resolution: Fixed Spark compilation is broken with the latest hadoop-2.4.0 release

[jira] [Created] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

Thomas Graves created SPARK-1528: Summary: Spark on Yarn: Add option for user to specify additional namenodes to get tokens from Key: SPARK-1528 URL: https://issues.apache.org/jira/browse/SPARK-1528

[jira] [Created] (SPARK-1534) spark-submit for yarn prints warnings even though calling as expected

Thomas Graves created SPARK-1534: Summary: spark-submit for yarn prints warnings even though calling as expected Key: SPARK-1534 URL: https://issues.apache.org/jira/browse/SPARK-1534 Project: Spark

[jira] [Assigned] (SPARK-1490) Add kerberos support to the HistoryServer

[ https://issues.apache.org/jira/browse/SPARK-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1490: Assignee: Thomas Graves Add kerberos support to the HistoryServer

[jira] [Commented] (SPARK-1472) Go through YARN api used in Spark to make sure we aren't using Private Apis

[ https://issues.apache.org/jira/browse/SPARK-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13975609#comment-13975609 ] Thomas Graves commented on SPARK-1472: -- So it looks like its currently impossible to

[jira] [Created] (SPARK-1557) Set permissions on event log files/directories

Thomas Graves created SPARK-1557: Summary: Set permissions on event log files/directories Key: SPARK-1557 URL: https://issues.apache.org/jira/browse/SPARK-1557 Project: Spark Issue Type:

[jira] [Created] (SPARK-1569) Spark on Yarn, authentication broken by pr299

Thomas Graves created SPARK-1569: Summary: Spark on Yarn, authentication broken by pr299 Key: SPARK-1569 URL: https://issues.apache.org/jira/browse/SPARK-1569 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1588) SPARK_JAVA_OPTS and SPARK_YARN_USER_ENV are not getting propagated

[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1588: - Summary: SPARK_JAVA_OPTS and SPARK_YARN_USER_ENV are not getting propagated (was:

[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS and SPARK_YARN_USER_ENV are not getting propagated

[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978846#comment-13978846 ] Thomas Graves commented on SPARK-1588: -- Mridul put up pr:

[jira] [Comment Edited] (SPARK-1588) SPARK_JAVA_OPTS and SPARK_YARN_USER_ENV are not getting propagated

[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978846#comment-13978846 ] Thomas Graves edited comment on SPARK-1588 at 4/23/14 8:27 PM:

[jira] [Commented] (SPARK-1604) Couldn't run spark-submit with yarn cluster mode when using deps jar

[ https://issues.apache.org/jira/browse/SPARK-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1398#comment-1398 ] Thomas Graves commented on SPARK-1604: -- if its just in that particular sequence of

[jira] [Commented] (SPARK-1557) Set permissions on event log files/directories

[ https://issues.apache.org/jira/browse/SPARK-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980213#comment-13980213 ] Thomas Graves commented on SPARK-1557: -- https://github.com/apache/spark/pull/538

[jira] [Resolved] (SPARK-1490) Add kerberos support to the HistoryServer

[ https://issues.apache.org/jira/browse/SPARK-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1490. -- Resolution: Fixed Fix Version/s: 1.0.0 Add kerberos support to the HistoryServer

[jira] [Commented] (SPARK-1626) Update Spark YARN docs to use spark-submit

[ https://issues.apache.org/jira/browse/SPARK-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981325#comment-13981325 ] Thomas Graves commented on SPARK-1626: -- this is dup of SPARK-1492 Update Spark YARN

[jira] [Reopened] (SPARK-1625) Ensure all legacy YARN options are supported with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-1625: -- These aren't the only things broken. One big issue is that authentication isn't being passed

[jira] [Commented] (SPARK-1639) Some tidying of Spark on YARN code

[ https://issues.apache.org/jira/browse/SPARK-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984318#comment-13984318 ] Thomas Graves commented on SPARK-1639: -- https://github.com/apache/spark/pull/561

[jira] [Resolved] (SPARK-1557) Set permissions on event log files/directories

[ https://issues.apache.org/jira/browse/SPARK-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1557. -- Resolution: Fixed Fix Version/s: 1.0.0 Set permissions on event log files/directories

[jira] [Created] (SPARK-1664) spark-submit --name doesn't work in yarn-client mode

Thomas Graves created SPARK-1664: Summary: spark-submit --name doesn't work in yarn-client mode Key: SPARK-1664 URL: https://issues.apache.org/jira/browse/SPARK-1664 Project: Spark Issue

[jira] [Created] (SPARK-1665) add a config to replace SPARK_YARN_USER_ENV

Thomas Graves created SPARK-1665: Summary: add a config to replace SPARK_YARN_USER_ENV Key: SPARK-1665 URL: https://issues.apache.org/jira/browse/SPARK-1665 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1664) spark-submit --name doesn't work in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1664: - Issue Type: Sub-task (was: Bug) Parent: SPARK-1652 spark-submit --name doesn't work in

[jira] [Closed] (SPARK-1625) Ensure all legacy YARN options are supported with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-1625. Resolution: Fixed Ensure all legacy YARN options are supported with spark-submit

[jira] [Commented] (SPARK-1625) Ensure all legacy YARN options are supported with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984519#comment-13984519 ] Thomas Graves commented on SPARK-1625: -- I'll create a separate jira for that.

[jira] [Updated] (SPARK-1569) Spark on Yarn, authentication broken by pr299

[ https://issues.apache.org/jira/browse/SPARK-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1569: - Issue Type: Sub-task (was: Bug) Parent: SPARK-1652 Spark on Yarn, authentication

[jira] [Commented] (SPARK-1569) Spark on Yarn, authentication broken by pr299

[ https://issues.apache.org/jira/browse/SPARK-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984713#comment-13984713 ] Thomas Graves commented on SPARK-1569: -- quote @tgravescs ah I see, you're right. I

[jira] [Created] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

Thomas Graves created SPARK-1718: Summary: pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat Key: SPARK-1718 URL: https://issues.apache.org/jira/browse/SPARK-1718

[jira] [Updated] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1719: - Issue Type: Sub-task (was: Bug) Parent: SPARK-1652 spark.executor.extraLibraryPath

[jira] [Created] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

Thomas Graves created SPARK-1719: Summary: spark.executor.extraLibraryPath isn't applied on yarn Key: SPARK-1719 URL: https://issues.apache.org/jira/browse/SPARK-1719 Project: Spark Issue

[jira] [Commented] (SPARK-1676) HDFS FileSystems continually pile up in the FS cache

[ https://issues.apache.org/jira/browse/SPARK-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989566#comment-13989566 ] Thomas Graves commented on SPARK-1676: -- https://github.com/apache/spark/pull/621

[jira] [Assigned] (SPARK-1676) HDFS FileSystems continually pile up in the FS cache

[ https://issues.apache.org/jira/browse/SPARK-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1676: Assignee: Thomas Graves HDFS FileSystems continually pile up in the FS cache

[jira] [Resolved] (SPARK-1676) HDFS FileSystems continually pile up in the FS cache

[ https://issues.apache.org/jira/browse/SPARK-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1676. -- Resolution: Fixed Fix Version/s: 0.9.2 1.0.0 HDFS FileSystems

[jira] [Commented] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989658#comment-13989658 ] Thomas Graves commented on SPARK-1718: -- No, see discussion on

[jira] [Commented] (SPARK-1569) Spark on Yarn, authentication broken by pr299

[ https://issues.apache.org/jira/browse/SPARK-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989825#comment-13989825 ] Thomas Graves commented on SPARK-1569: -- https://github.com/apache/spark/pull/649

[jira] [Commented] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989854#comment-13989854 ] Thomas Graves commented on SPARK-1718: -- I can build the jar on my mac, copy it over

[jira] [Commented] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989903#comment-13989903 ] Thomas Graves commented on SPARK-1718: -- I am running with jdk7 and building with

[jira] [Resolved] (SPARK-1665) add a config to replace SPARK_YARN_USER_ENV

[ https://issues.apache.org/jira/browse/SPARK-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1665. -- Resolution: Duplicate dup of SPARK-1680 add a config to replace SPARK_YARN_USER_ENV

[jira] [Updated] (SPARK-1191) Convert configs to use SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1191: - Priority: Major (was: Blocker) Convert configs to use SparkConf

[jira] [Updated] (SPARK-1768) History Server enhancements

[ https://issues.apache.org/jira/browse/SPARK-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1768: - Assignee: Marcelo Vanzin History Server enhancements ---

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993108#comment-13993108 ] Thomas Graves commented on SPARK-1755: -- I believe this is a dup of SPARK-1664

[jira] [Commented] (SPARK-1514) Standardize process for creating Spark packages

[ https://issues.apache.org/jira/browse/SPARK-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002193#comment-14002193 ] Thomas Graves commented on SPARK-1514: -- What was the resolution here? is there a

[jira] [Updated] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1518: - Assignee: (was: Thomas Graves) Spark master doesn't compile against hadoop-common trunk

[jira] [Created] (SPARK-1890) add modify acls to the web UI for the kill button

Thomas Graves created SPARK-1890: Summary: add modify acls to the web UI for the kill button Key: SPARK-1890 URL: https://issues.apache.org/jira/browse/SPARK-1890 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-1533) The (kill) button in the web UI is visible to everyone.

[ https://issues.apache.org/jira/browse/SPARK-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003328#comment-14003328 ] Thomas Graves edited comment on SPARK-1533 at 5/20/14 2:21 PM:

[jira] [Created] (SPARK-1891) Add admin acls to the Web UI

Thomas Graves created SPARK-1891: Summary: Add admin acls to the Web UI Key: SPARK-1891 URL: https://issues.apache.org/jira/browse/SPARK-1891 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1898) In deploy.yarn.Client, use YarnClient rather than YarnClientImpl

[ https://issues.apache.org/jira/browse/SPARK-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007100#comment-14007100 ] Thomas Graves commented on SPARK-1898: -- That is correct, as long as you don't modify

[jira] [Created] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

Thomas Graves created SPARK-2037: Summary: yarn client mode doesn't support spark.yarn.max.executor.failures Key: SPARK-2037 URL: https://issues.apache.org/jira/browse/SPARK-2037 Project: Spark

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025244#comment-14025244 ] Thomas Graves commented on SPARK-1291: -- https://github.com/apache/spark/pull/1002

[jira] [Resolved] (SPARK-1522) YARN ClientBase will throw a NPE if there is no YARN application specific classpath.

[ https://issues.apache.org/jira/browse/SPARK-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1522. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Bernardo Gomez Palacio YARN

[jira] [Resolved] (SPARK-1978) In some cases, spark-yarn does not automatically restart the failed container

[ https://issues.apache.org/jira/browse/SPARK-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1978. -- Resolution: Fixed In some cases, spark-yarn does not automatically restart the failed

[jira] [Resolved] (SPARK-1639) Some tidying of Spark on YARN code

[ https://issues.apache.org/jira/browse/SPARK-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1639. -- Resolution: Fixed Fix Version/s: 1.1.0 Some tidying of Spark on YARN code

[jira] [Created] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

Thomas Graves created SPARK-2111: Summary: pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1 Key: SPARK-2111 URL: https://issues.apache.org/jira/browse/SPARK-2111 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1740) Pyspark cancellation kills unrelated pyspark workers

[ https://issues.apache.org/jira/browse/SPARK-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029416#comment-14029416 ] Thomas Graves commented on SPARK-1740: -- On yarn when I run pyspark it kills the

[jira] [Updated] (SPARK-1740) Pyspark cancellation kills unrelated pyspark workers

[ https://issues.apache.org/jira/browse/SPARK-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1740: - Priority: Critical (was: Major) Pyspark cancellation kills unrelated pyspark workers

[jira] [Commented] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029422#comment-14029422 ] Thomas Graves commented on SPARK-1718: -- So this actually appears to be an issue

[jira] [Commented] (SPARK-2080) Yarn: history UI link missing, wrong reported user

[ https://issues.apache.org/jira/browse/SPARK-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029828#comment-14029828 ] Thomas Graves commented on SPARK-2080: -- Note that this is only fixing it for running

[jira] [Resolved] (SPARK-2080) Yarn: history UI link missing, wrong reported user

[ https://issues.apache.org/jira/browse/SPARK-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2080. -- Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Yarn: history UI link

[jira] [Created] (SPARK-2139) spark.yarn.dist.* configs are not documented

Thomas Graves created SPARK-2139: Summary: spark.yarn.dist.* configs are not documented Key: SPARK-2139 URL: https://issues.apache.org/jira/browse/SPARK-2139 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2051: - Issue Type: Improvement (was: Bug) spark.yarn.dist.* configs are not supported in yarn-cluster

[jira] [Resolved] (SPARK-2139) spark.yarn.dist.* configs are not documented

[ https://issues.apache.org/jira/browse/SPARK-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2139. -- Resolution: Duplicate spark.yarn.dist.* configs are not documented

[jira] [Commented] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030612#comment-14030612 ] Thomas Graves commented on SPARK-2051: -- Note that the configs weren't originally

[jira] [Updated] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2140: - Fix Version/s: 1.1.0 1.0.1 yarn stable client doesn't properly handle

[jira] [Created] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

Thomas Graves created SPARK-2140: Summary: yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM Key: SPARK-2140 URL: https://issues.apache.org/jira/browse/SPARK-2140 Project: Spark

[jira] [Resolved] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1951. -- Resolution: Invalid As pointed by Sean out spark-submit changed the default scheme to be

[jira] [Comment Edited] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030900#comment-14030900 ] Thomas Graves edited comment on SPARK-1951 at 6/13/14 5:55 PM:

[jira] [Resolved] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1930. -- Resolution: Fixed Fix Version/s: 1.1.0 The Container is running beyond physical memory

[jira] [Created] (SPARK-2165) spark on yarn: add support for setting maxAppAttempts in the ApplicationSubmissionContext

Thomas Graves created SPARK-2165: Summary: spark on yarn: add support for setting maxAppAttempts in the ApplicationSubmissionContext Key: SPARK-2165 URL: https://issues.apache.org/jira/browse/SPARK-2165

[jira] [Updated] (SPARK-1907) spark-submit: add exec at the end of the script

[ https://issues.apache.org/jira/browse/SPARK-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1907: - Assignee: Colin Patrick McCabe spark-submit: add exec at the end of the script

[jira] [Created] (SPARK-2167) spark-submit should return exit code based on failure/success

Thomas Graves created SPARK-2167: Summary: spark-submit should return exit code based on failure/success Key: SPARK-2167 URL: https://issues.apache.org/jira/browse/SPARK-2167 Project: Spark

[jira] [Resolved] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2051. -- Resolution: Fixed Fix Version/s: 1.1.0 spark.yarn.dist.* configs are not supported in

[jira] [Created] (SPARK-2224) allow running tests for one sub module

Thomas Graves created SPARK-2224: Summary: allow running tests for one sub module Key: SPARK-2224 URL: https://issues.apache.org/jira/browse/SPARK-2224 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038971#comment-14038971 ] Thomas Graves commented on SPARK-2224: -- Thanks, wasn't away of -pl.. doesn't seem to

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038976#comment-14038976 ] Thomas Graves commented on SPARK-2223: -- Some of our nightly builds are taking 50

[jira] [Resolved] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2224. -- Resolution: Invalid allow running tests for one sub module

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039081#comment-14039081 ] Thomas Graves commented on SPARK-2224: -- well that worked as in it actually ran in the

[jira] [Assigned] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1528: Assignee: Thomas Graves Spark on Yarn: Add option for user to specify additional

[jira] [Commented] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039186#comment-14039186 ] Thomas Graves commented on SPARK-1528: -- https://github.com/apache/spark/pull/1159

[jira] [Resolved] (SPARK-1395) Cannot launch jobs on Yarn cluster with local: scheme in SPARK_JAR

[ https://issues.apache.org/jira/browse/SPARK-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1395. -- Resolution: Fixed Fix Version/s: (was: 1.0.0) 1.1.0 Cannot

[jira] [Assigned] (SPARK-1890) add modify acls to the web UI for the kill button

[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1890: Assignee: Thomas Graves add modify acls to the web UI for the kill button

[jira] [Assigned] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1891: Assignee: Thomas Graves Add admin acls to the Web UI

[jira] [Commented] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042516#comment-14042516 ] Thomas Graves commented on SPARK-1891: -- https://github.com/apache/spark/pull/1196

[jira] [Resolved] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1718. -- Resolution: Duplicate pyspark doesn't work with assembly jar containing over 65536 files/dirs

  1   2   3   4   5   6   7   8   9   10   >