[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127188#comment-14127188 ] Thomas Graves commented on SPARK-3174: -- Since you mention the graceful decommission a

[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

[ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127204#comment-14127204 ] Thomas Graves commented on SPARK-3445: -- Yahoo moving to 2.x and I'm +1 for this propo

[jira] [Comment Edited] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127188#comment-14127188 ] Thomas Graves edited comment on SPARK-3174 at 9/9/14 4:51 PM: --

[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

[ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127318#comment-14127318 ] Thomas Graves commented on SPARK-3445: -- We are in progress of moving and the timeline

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127446#comment-14127446 ] Thomas Graves commented on SPARK-3174: -- {quote} Yeah so how about we create a sub-tas

[jira] [Resolved] (SPARK-3286) Cannot view ApplicationMaster UI when Yarn’s url scheme is https

[ https://issues.apache.org/jira/browse/SPARK-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3286. -- Resolution: Fixed Fix Version/s: 1.2.0 > Cannot view ApplicationMaster UI when Yarn’s url

[jira] [Created] (SPARK-3475) dev/merge_spark_pr.py fails on mac

Thomas Graves created SPARK-3475: Summary: dev/merge_spark_pr.py fails on mac Key: SPARK-3475 URL: https://issues.apache.org/jira/browse/SPARK-3475 Project: Spark Issue Type: Bug C

[jira] [Resolved] (SPARK-1713) YarnAllocationHandler starts a thread for every executor it runs

[ https://issues.apache.org/jira/browse/SPARK-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1713. -- Resolution: Fixed Fix Version/s: 1.2.0 > YarnAllocationHandler starts a thread for every

[jira] [Resolved] (SPARK-3475) dev/merge_spark_pr.py fails on mac

[ https://issues.apache.org/jira/browse/SPARK-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3475. -- Resolution: Invalid worked fine on another pr so not sure what happened on this one. I'll look

[jira] [Created] (SPARK-3476) Yarn ClientBase.validateArgs memory checks wrong

Thomas Graves created SPARK-3476: Summary: Yarn ClientBase.validateArgs memory checks wrong Key: SPARK-3476 URL: https://issues.apache.org/jira/browse/SPARK-3476 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-3476) Yarn ClientBase.validateArgs memory checks wrong

[ https://issues.apache.org/jira/browse/SPARK-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129026#comment-14129026 ] Thomas Graves commented on SPARK-3476: -- note https://github.com/apache/spark/pull/225

[jira] [Updated] (SPARK-3477) Clean up code in Yarn Client / ClientBase

[ https://issues.apache.org/jira/browse/SPARK-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3477: - Description: With the addition of new features and supporting multiple versions of yarn the code

[jira] [Resolved] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2140. -- Resolution: Fixed Fix Version/s: (was: 1.0.1) (was: 1.

[jira] [Commented] (SPARK-2668) Add variable of yarn log directory for reference from the log4j configuration

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130063#comment-14130063 ] Thomas Graves commented on SPARK-2668: -- note that the mapreduce one was changed in h

[jira] [Updated] (SPARK-3374) Spark on Yarn config cleanup

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-3492 > Spark on Yarn config cleanup

[jira] [Resolved] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1891. -- Resolution: Fixed > Add admin acls to the Web UI > > >

[jira] [Resolved] (SPARK-2558) Mention --queue argument in YARN documentation

[ https://issues.apache.org/jira/browse/SPARK-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2558. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.0, 1.0

[jira] [Created] (SPARK-3508) annotate the Spark configs to indicate which ones are meant for the end user

Thomas Graves created SPARK-3508: Summary: annotate the Spark configs to indicate which ones are meant for the end user Key: SPARK-3508 URL: https://issues.apache.org/jira/browse/SPARK-3508 Project: S

[jira] [Resolved] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3014. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Sandy Ryza > Log a more informa

[jira] [Resolved] (SPARK-3456) YarnAllocator can lose container requests to RM

[ https://issues.apache.org/jira/browse/SPARK-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3456. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Thomas Graves > YarnAllocator c

[jira] [Resolved] (SPARK-3410) The priority of shutdownhook for ApplicationMaster should not be integer literal

[ https://issues.apache.org/jira/browse/SPARK-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3410. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta > The priority o

[jira] [Resolved] (SPARK-3177) Yarn-alpha ClientBaseSuite Unit test failed

[ https://issues.apache.org/jira/browse/SPARK-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3177. -- Resolution: Fixed Fix Version/s: (was: 1.1.1) 1.2.0 > Yarn-alpha C

[jira] [Resolved] (SPARK-3557) Yarn client config prioritization is backwards

[ https://issues.apache.org/jira/browse/SPARK-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3557. -- Resolution: Duplicate > Yarn client config prioritization is backwards > ---

[jira] [Commented] (SPARK-3557) Yarn client config prioritization is backwards

[ https://issues.apache.org/jira/browse/SPARK-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138993#comment-14138993 ] Thomas Graves commented on SPARK-3557: -- This is a dup of SPARK-2872, although this ha

[jira] [Commented] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138996#comment-14138996 ] Thomas Graves commented on SPARK-2872: -- adding description from spark-3557 as it expl

[jira] [Created] (SPARK-3596) Support changing the yarn client monitor interval

Thomas Graves created SPARK-3596: Summary: Support changing the yarn client monitor interval Key: SPARK-3596 URL: https://issues.apache.org/jira/browse/SPARK-3596 Project: Spark Issue Type:

[jira] [Created] (SPARK-3607) ConnectionManager threads.max configs on the thread pools don't work

Thomas Graves created SPARK-3607: Summary: ConnectionManager threads.max configs on the thread pools don't work Key: SPARK-3607 URL: https://issues.apache.org/jira/browse/SPARK-3607 Project: Spark

[jira] [Created] (SPARK-3627) spark on yarn reports success even though job fails

Thomas Graves created SPARK-3627: Summary: spark on yarn reports success even though job fails Key: SPARK-3627 URL: https://issues.apache.org/jira/browse/SPARK-3627 Project: Spark Issue Type:

[jira] [Created] (SPARK-3632) ConnectionManager can run out of receive threads with authentication on

Thomas Graves created SPARK-3632: Summary: ConnectionManager can run out of receive threads with authentication on Key: SPARK-3632 URL: https://issues.apache.org/jira/browse/SPARK-3632 Project: Spark

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143438#comment-14143438 ] Thomas Graves commented on SPARK-3627: -- We could make this a separate issue, but I've

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143523#comment-14143523 ] Thomas Graves commented on SPARK-3627: -- this might be the same as SPARK-3293 > spark

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143877#comment-14143877 ] Thomas Graves commented on SPARK-2848: -- [~vanzin] [~pwendell] I think the fix version

[jira] [Resolved] (SPARK-2668) Add variable of yarn log directory for reference from the log4j configuration

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2668. -- Resolution: Fixed Fix Version/s: 1.2.0 > Add variable of yarn log directory for reference

[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

[ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14144946#comment-14144946 ] Thomas Graves commented on SPARK-3445: -- Any other comments or objections to this? >

[jira] [Created] (SPARK-3657) yarn alpha YarnRMClientImpl throws NPE appMasterRequest.setTrackingUrl starting spark-shell

Thomas Graves created SPARK-3657: Summary: yarn alpha YarnRMClientImpl throws NPE appMasterRequest.setTrackingUrl starting spark-shell Key: SPARK-3657 URL: https://issues.apache.org/jira/browse/SPARK-3657

[jira] [Resolved] (SPARK-3477) Clean up code in Yarn Client / ClientBase

[ https://issues.apache.org/jira/browse/SPARK-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3477. -- Resolution: Fixed Fix Version/s: 1.2.0 > Clean up code in Yarn Client / ClientBase >

[jira] [Resolved] (SPARK-3304) ApplicationMaster's Finish status is wrong when uncaught exception is thrown from ReporterThread

[ https://issues.apache.org/jira/browse/SPARK-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3304. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta > ApplicationMas

[jira] [Created] (SPARK-3678) Yarn app name reported in RM is different between cluster and client mode

Thomas Graves created SPARK-3678: Summary: Yarn app name reported in RM is different between cluster and client mode Key: SPARK-3678 URL: https://issues.apache.org/jira/browse/SPARK-3678 Project: Spar

[jira] [Commented] (SPARK-3678) Yarn app name reported in RM is different between cluster and client mode

[ https://issues.apache.org/jira/browse/SPARK-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147835#comment-14147835 ] Thomas Graves commented on SPARK-3678: -- Also note that the spark-submit --name optio

[jira] [Commented] (SPARK-3293) yarn's web show "SUCCEEDED" when the driver throw a exception in yarn-client

[ https://issues.apache.org/jira/browse/SPARK-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150008#comment-14150008 ] Thomas Graves commented on SPARK-3293: -- I'm going to fix this under SPARK-3627 > yar

[jira] [Closed] (SPARK-3293) yarn's web show "SUCCEEDED" when the driver throw a exception in yarn-client

[ https://issues.apache.org/jira/browse/SPARK-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-3293. Resolution: Duplicate > yarn's web show "SUCCEEDED" when the driver throw a exception in yarn-client

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150013#comment-14150013 ] Thomas Graves commented on SPARK-3627: -- I'm going to go through all the exit and stat

[jira] [Assigned] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3627: Assignee: Thomas Graves > spark on yarn reports success even though job fails > ---

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152308#comment-14152308 ] Thomas Graves commented on SPARK-3732: -- I think you should just change the name of th

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152409#comment-14152409 ] Thomas Graves commented on SPARK-3732: -- I understand your usecase and need for it, b

[jira] [Comment Edited] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152409#comment-14152409 ] Thomas Graves edited comment on SPARK-3732 at 9/29/14 10:19 PM:

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153558#comment-14153558 ] Thomas Graves commented on SPARK-3627: -- [~andrewor] Why was the affects version chan

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153777#comment-14153777 ] Thomas Graves commented on SPARK-3627: -- As far as I am aware the common cases worked

[jira] [Updated] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3627: - Affects Version/s: (was: 1.1.0) 1.2.0 > spark on yarn reports success e

[jira] [Resolved] (SPARK-3768) Modify default YARN memory_overhead-- from an additive constant to a multiplier

[ https://issues.apache.org/jira/browse/SPARK-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3768. -- Resolution: Fixed > Modify default YARN memory_overhead-- from an additive constant to a > mult

[jira] [Created] (SPARK-3768) Modify default YARN memory_overhead-- from an additive constant to a multiplier

Thomas Graves created SPARK-3768: Summary: Modify default YARN memory_overhead-- from an additive constant to a multiplier Key: SPARK-3768 URL: https://issues.apache.org/jira/browse/SPARK-3768 Project

[jira] [Created] (SPARK-3778) newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn

Thomas Graves created SPARK-3778: Summary: newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn Key: SPARK-3778 URL: https://issues.apache.org/jira/browse/SPARK-3778 Project: Spar

[jira] [Created] (SPARK-3779) yarn spark.yarn.applicationMaster.waitTries config should be changed to a time period

Thomas Graves created SPARK-3779: Summary: yarn spark.yarn.applicationMaster.waitTries config should be changed to a time period Key: SPARK-3779 URL: https://issues.apache.org/jira/browse/SPARK-3779 P

[jira] [Created] (SPARK-3780) YarnAllocator should look at the container completed diagnostic message

Thomas Graves created SPARK-3780: Summary: YarnAllocator should look at the container completed diagnostic message Key: SPARK-3780 URL: https://issues.apache.org/jira/browse/SPARK-3780 Project: Spark

[jira] [Assigned] (SPARK-3778) newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn

[ https://issues.apache.org/jira/browse/SPARK-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3778: Assignee: Thomas Graves > newAPIHadoopRDD doesn't properly pass credentials for secure hdfs

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160939#comment-14160939 ] Thomas Graves commented on SPARK-3174: -- Good write up Andrew. Just to make sure, the

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160988#comment-14160988 ] Thomas Graves commented on SPARK-3174: -- Perhaps I misread it then because the proposa

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161891#comment-14161891 ] Thomas Graves commented on SPARK-3797: -- Nice write up Sandy. Just to point out on

[jira] [Resolved] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3627. -- Resolution: Fixed Fix Version/s: 1.2.0 > spark on yarn reports success even though job fa

[jira] [Resolved] (SPARK-3837) Warn when YARN is killing containers for exceeding memory limits

[ https://issues.apache.org/jira/browse/SPARK-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3837. -- Resolution: Duplicate duplicate of SPARK-3780 > Warn when YARN is killing containers for exceed

[jira] [Updated] (SPARK-3780) YarnAllocator should look at the container completed diagnostic message

[ https://issues.apache.org/jira/browse/SPARK-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3780: - Issue Type: Improvement (was: Bug) > YarnAllocator should look at the container completed diagnos

[jira] [Resolved] (SPARK-3788) Yarn dist cache code is not friendly to HDFS HA, Federation

[ https://issues.apache.org/jira/browse/SPARK-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3788. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Yarn dist cache code is

[jira] [Created] (SPARK-3848) yarn alpha doesn't build on master

Thomas Graves created SPARK-3848: Summary: yarn alpha doesn't build on master Key: SPARK-3848 URL: https://issues.apache.org/jira/browse/SPARK-3848 Project: Spark Issue Type: Bug Co

[jira] [Updated] (SPARK-3848) yarn alpha doesn't build on master

[ https://issues.apache.org/jira/browse/SPARK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3848: - Assignee: Kousuke Saruta (was: Thomas Graves) > yarn alpha doesn't build on master >

[jira] [Resolved] (SPARK-3848) yarn alpha doesn't build on master

[ https://issues.apache.org/jira/browse/SPARK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3848. -- Resolution: Fixed Fix Version/s: 1.2.0 > yarn alpha doesn't build on master > ---

[jira] [Commented] (SPARK-3877) The exit code of spark-submit is still 0 when an yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165124#comment-14165124 ] Thomas Graves commented on SPARK-3877: -- this looks like a dup of SPARK-2167. Or actu

[jira] [Commented] (SPARK-3877) The exit code of spark-submit is still 0 when an yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174999#comment-14174999 ] Thomas Graves commented on SPARK-3877: -- [~vanzin] I agree. The user code should be e

[jira] [Resolved] (SPARK-3979) Yarn backend's default file replication should match HDFS's default one

[ https://issues.apache.org/jira/browse/SPARK-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3979. -- Resolution: Fixed Fix Version/s: 1.2.0 > Yarn backend's default file replication should m

[jira] [Resolved] (SPARK-3900) ApplicationMaster's shutdown hook fails and IllegalStateException is thrown.

[ https://issues.apache.org/jira/browse/SPARK-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3900. -- Resolution: Fixed Fix Version/s: 1.2.0 > ApplicationMaster's shutdown hook fails and Ille

[jira] [Commented] (SPARK-4098) use appUIAddress instead of appUIHostPort in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185126#comment-14185126 ] Thomas Graves commented on SPARK-4098: -- it was changed to deal with when yarn url sch

[jira] [Resolved] (SPARK-4116) Delete the abandoned log4j-spark-container.properties

[ https://issues.apache.org/jira/browse/SPARK-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-4116. -- Resolution: Fixed Fix Version/s: 1.2.0 > Delete the abandoned log4j-spark-container.prope

[jira] [Resolved] (SPARK-4095) [YARN][Minor]extract val isLaunchingDriver in ClientBase

[ https://issues.apache.org/jira/browse/SPARK-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-4095. -- Resolution: Fixed Fix Version/s: 1.2.0 > [YARN][Minor]extract val isLaunchingDriver in Cl

[jira] [Created] (SPARK-4117) Spark on Yarn handle AM being told command from RM

Thomas Graves created SPARK-4117: Summary: Spark on Yarn handle AM being told command from RM Key: SPARK-4117 URL: https://issues.apache.org/jira/browse/SPARK-4117 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-4098) use appUIAddress instead of appUIHostPort in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-4098. -- Resolution: Fixed Fix Version/s: 1.2.0 > use appUIAddress instead of appUIHostPort in yar

[jira] [Updated] (SPARK-7846) Use different way to pass spark.yarn.keytab and spark.yarn.principal in different modes

[ https://issues.apache.org/jira/browse/SPARK-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-7846: - Assignee: Tao Wang > Use different way to pass spark.yarn.keytab and spark.yarn.principal in > di

[jira] [Updated] (SPARK-7524) add configs for keytab and principal, move originals to internal

[ https://issues.apache.org/jira/browse/SPARK-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-7524: - Assignee: Tao Wang > add configs for keytab and principal, move originals to internal > --

[jira] [Closed] (SPARK-7524) add configs for keytab and principal, move originals to internal

[ https://issues.apache.org/jira/browse/SPARK-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-7524. Resolution: Fixed Fix Version/s: 1.5.0 > add configs for keytab and principal, move originals

[jira] [Closed] (SPARK-7846) Use different way to pass spark.yarn.keytab and spark.yarn.principal in different modes

[ https://issues.apache.org/jira/browse/SPARK-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-7846. Resolution: Fixed Fix Version/s: 1.5.0 > Use different way to pass spark.yarn.keytab and spar

[jira] [Created] (SPARK-8569) Documentation for SPARK_HISTORY_OPTS unclear

Thomas Graves created SPARK-8569: Summary: Documentation for SPARK_HISTORY_OPTS unclear Key: SPARK-8569 URL: https://issues.apache.org/jira/browse/SPARK-8569 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-8574) org/apache/spark/unsafe doesn't honor the java source/target versions

Thomas Graves created SPARK-8574: Summary: org/apache/spark/unsafe doesn't honor the java source/target versions Key: SPARK-8574 URL: https://issues.apache.org/jira/browse/SPARK-8574 Project: Spark

[jira] [Assigned] (SPARK-8574) org/apache/spark/unsafe doesn't honor the java source/target versions

[ https://issues.apache.org/jira/browse/SPARK-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-8574: Assignee: Thomas Graves > org/apache/spark/unsafe doesn't honor the java source/target vers

[jira] [Created] (SPARK-8593) History Server doesn't show complete application when one attempt inprogress

Thomas Graves created SPARK-8593: Summary: History Server doesn't show complete application when one attempt inprogress Key: SPARK-8593 URL: https://issues.apache.org/jira/browse/SPARK-8593 Project: S

[jira] [Created] (SPARK-8594) History Server doesn't show complete application when one attempt inprogress

Thomas Graves created SPARK-8594: Summary: History Server doesn't show complete application when one attempt inprogress Key: SPARK-8594 URL: https://issues.apache.org/jira/browse/SPARK-8594 Project: S

[jira] [Closed] (SPARK-8594) History Server doesn't show complete application when one attempt inprogress

[ https://issues.apache.org/jira/browse/SPARK-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-8594. Resolution: Duplicate accidentally filed same jira twice > History Server doesn't show complete app

[jira] [Created] (SPARK-8617) Handle history files better

Thomas Graves created SPARK-8617: Summary: Handle history files better Key: SPARK-8617 URL: https://issues.apache.org/jira/browse/SPARK-8617 Project: Spark Issue Type: Improvement C

[jira] [Resolved] (SPARK-8574) org/apache/spark/unsafe doesn't honor the java source/target versions

[ https://issues.apache.org/jira/browse/SPARK-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-8574. -- Resolution: Fixed Fix Version/s: 1.4.1 > org/apache/spark/unsafe doesn't honor the java s

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603220#comment-14603220 ] Thomas Graves commented on SPARK-1476: -- we have a lot of jira about the 2G limit. I'

[jira] [Closed] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-1476. Resolution: Duplicate > 2GB limit in spark for blocks > - > >

[jira] [Commented] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603223#comment-14603223 ] Thomas Graves commented on SPARK-1391: -- duping this to SPARK-6235. If something is m

[jira] [Closed] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-1391. Resolution: Duplicate > BlockManager cannot transfer blocks larger than 2G in size > ---

[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608280#comment-14608280 ] Thomas Graves commented on SPARK-6735: -- Pull request was up but didn't have time to d

[jira] [Commented] (SPARK-6951) History server slow startup if the event log directory is large

[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608287#comment-14608287 ] Thomas Graves commented on SPARK-6951: -- This actually happens more then just at start

[jira] [Commented] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955209#comment-13955209 ] Thomas Graves commented on SPARK-1350: -- I assume you are referring to the one in Clie

[jira] [Resolved] (SPARK-1232) maven hadoop 0.23 yarn-alpha build broken

[ https://issues.apache.org/jira/browse/SPARK-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1232. -- Resolution: Fixed Fix Version/s: 1.0.0 > maven hadoop 0.23 yarn-alpha build broken > ---

[jira] [Resolved] (SPARK-1345) spark on yarn 0.23 using maven doesn't build

[ https://issues.apache.org/jira/browse/SPARK-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1345. -- Resolution: Fixed Fix Version/s: 1.0.0 > spark on yarn 0.23 using maven doesn't build >

[jira] [Commented] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955443#comment-13955443 ] Thomas Graves commented on SPARK-1350: -- I agree it is weird, lets bring this back up

[jira] [Created] (SPARK-1384) spark-shell on yarn doesn't always work with secure hdfs

Thomas Graves created SPARK-1384: Summary: spark-shell on yarn doesn't always work with secure hdfs Key: SPARK-1384 URL: https://issues.apache.org/jira/browse/SPARK-1384 Project: Spark Issue

[jira] [Updated] (SPARK-1384) spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs

[ https://issues.apache.org/jira/browse/SPARK-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1384: - Description: I've found an issue with the spark-shell in yarn-client mode in the 0.9.1 rc3 rele

[jira] [Updated] (SPARK-1384) spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs

[ https://issues.apache.org/jira/browse/SPARK-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1384: - Summary: spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs (was: spar

[jira] [Assigned] (SPARK-1384) spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs

[ https://issues.apache.org/jira/browse/SPARK-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1384: Assignee: Thomas Graves > spark-shell on yarn on spark 0.9 branch doesn't always work with

<    4   5   6   7   8   9   10   11   >