[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130916#comment-14130916 ] Andrew Or commented on SPARK-3490: -- Fixed in https://github.com/apache/spark/pull/2363 >

[jira] [Closed] (SPARK-1047) Ability to disable the spark ui server (unit tests)

[ https://issues.apache.org/jira/browse/SPARK-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1047. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Ability to disable the spark ui ser

[jira] [Updated] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3490: - Fix Version/s: 1.2.0 1.1.1 > Alleviate port collisions during tests > -

[jira] [Closed] (SPARK-3429) Don't include the empty string "" as a defaultAclUser

[ https://issues.apache.org/jira/browse/SPARK-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3429. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Andrew Ash https:/

[jira] [Commented] (SPARK-2482) Resolve sbt warnings during build

[ https://issues.apache.org/jira/browse/SPARK-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131005#comment-14131005 ] Andrew Or commented on SPARK-2482: -- Fixed in https://github.com/apache/spark/pull/1330 >

[jira] [Closed] (SPARK-2482) Resolve sbt warnings during build

[ https://issues.apache.org/jira/browse/SPARK-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2482. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Guoqiang Li Target Version/s: 1

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131844#comment-14131844 ] Andrew Or commented on SPARK-3490: -- This still needs to be back ported into branch-1.1 >

[jira] [Updated] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3490: - Fix Version/s: (was: 1.1.1) > Alleviate port collisions during tests > ---

[jira] [Closed] (SPARK-3498) Block always replicated to the same node

[ https://issues.apache.org/jira/browse/SPARK-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3498. Resolution: Duplicate Assignee: Tathagata Das This is the cause of SPARK-3495, and the fix for both is

[jira] [Closed] (SPARK-3465) Task metrics are not aggregated correctly in local mode

[ https://issues.apache.org/jira/browse/SPARK-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3465. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Task metrics are not aggregated cor

[jira] [Updated] (SPARK-3187) Refactor and cleanup Yarn allocator code

[ https://issues.apache.org/jira/browse/SPARK-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3187: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-3492 > Refactor and cleanup Yarn allocator

[jira] [Commented] (SPARK-3492) Clean up Yarn integration code

[ https://issues.apache.org/jira/browse/SPARK-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132147#comment-14132147 ] Andrew Or commented on SPARK-3492: -- Thanks, I've added it to the list. > Clean up Yarn i

[jira] [Commented] (SPARK-3107) Don't pass null jar to executor in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132164#comment-14132164 ] Andrew Or commented on SPARK-3107: -- I see. Yes, setting it to an empty value is semantica

[jira] [Closed] (SPARK-1906) spark-submit doesn't send master URL to Driver in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1906. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Andrew Or This is fixed in https://github.c

[jira] [Closed] (SPARK-1908) Support local app jar in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1908. Resolution: Won't Fix Assignee: Andrew Or As described in the comment of https://github.com/apache/spa

[jira] [Closed] (SPARK-1909) "--jars" is not supported in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1909. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Andrew Or Also fixed in https://github.com/

[jira] [Closed] (SPARK-1131) Better document the --args option for yarn-standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1131. Resolution: Fixed --args is now deprecated. We use --arg instead. > Better document the --args option for y

[jira] [Updated] (SPARK-3464) Graceful decommission of executors

[ https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3464: - Target Version/s: 1.2.0 Affects Version/s: 1.1.0 Assignee: Andrew Or > Graceful decommis

[jira] [Closed] (SPARK-2714) DAGScheduler should log jobid when runJob finishes

[ https://issues.apache.org/jira/browse/SPARK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2714. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 > DAGScheduler should log jo

[jira] [Updated] (SPARK-1201) Do not materialize partitions whenever possible in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1201: - Target Version/s: 1.2.0 Fix Version/s: (was: 1.2.0) > Do not materialize partitions whenever po

[jira] [Updated] (SPARK-1761) Add broadcast information on SparkUI storage tab

[ https://issues.apache.org/jira/browse/SPARK-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1761: - Target Version/s: 1.2.0 Fix Version/s: (was: 1.2.0) > Add broadcast information on SparkUI stor

[jira] [Updated] (SPARK-1762) Add functionality to pin RDDs in cache

[ https://issues.apache.org/jira/browse/SPARK-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1762: - Target Version/s: 1.2.0 Fix Version/s: (was: 1.2.0) > Add functionality to pin RDDs in cache >

[jira] [Updated] (SPARK-2761) Merge similar code paths in ExternalSorter and EAOM

[ https://issues.apache.org/jira/browse/SPARK-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2761: - Fix Version/s: (was: 1.2.0) > Merge similar code paths in ExternalSorter and EAOM > --

[jira] [Created] (SPARK-3557) Yarn client config prioritization is backwards

Andrew Or created SPARK-3557: Summary: Yarn client config prioritization is backwards Key: SPARK-3557 URL: https://issues.apache.org/jira/browse/SPARK-3557 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136632#comment-14136632 ] Andrew Or commented on SPARK-3490: -- Backported. Closing this. > Alleviate port collision

[jira] [Commented] (SPARK-3560) In yarn-cluster mode, jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136637#comment-14136637 ] Andrew Or commented on SPARK-3560: -- [~sandyr] So is the fix simply to not set `spark.jars

[jira] [Updated] (SPARK-3560) In yarn-cluster mode, jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3560: - Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1) > In yarn-cluster mode, jars are distributed through multiple

[jira] [Updated] (SPARK-3560) In yarn-cluster mode, jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3560: - Affects Version/s: 1.1.0 > In yarn-cluster mode, jars are distributed through multiple mechanisms. > -

[jira] [Updated] (SPARK-3571) Spark standalone cluster mode doesn't work.

[ https://issues.apache.org/jira/browse/SPARK-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3571: - Assignee: Kousuke Saruta > Spark standalone cluster mode doesn't work. > -

[jira] [Updated] (SPARK-3564) Display App ID on HistoryPage

[ https://issues.apache.org/jira/browse/SPARK-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3564: - Assignee: Kousuke Saruta > Display App ID on HistoryPage > - > >

[jira] [Updated] (SPARK-3567) appId field in SparkDeploySchedulerBackend should be volatile

[ https://issues.apache.org/jira/browse/SPARK-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3567: - Target Version/s: 1.2.0 (was: 1.1.1, 1.2.0) > appId field in SparkDeploySchedulerBackend should be volati

[jira] [Updated] (SPARK-3567) appId field in SparkDeploySchedulerBackend should be volatile

[ https://issues.apache.org/jira/browse/SPARK-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3567: - Fix Version/s: 1.2.0 > appId field in SparkDeploySchedulerBackend should be volatile > ---

[jira] [Updated] (SPARK-3567) appId field in SparkDeploySchedulerBackend should be volatile

[ https://issues.apache.org/jira/browse/SPARK-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3567: - Assignee: Kousuke Saruta > appId field in SparkDeploySchedulerBackend should be volatile > ---

[jira] [Created] (SPARK-3591) Yarn cluster mode fires but does not forget

Andrew Or created SPARK-3591: Summary: Yarn cluster mode fires but does not forget Key: SPARK-3591 URL: https://issues.apache.org/jira/browse/SPARK-3591 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3591) Yarn cluster mode fires but does not forget

[ https://issues.apache.org/jira/browse/SPARK-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3591: - Target Version/s: 1.2.0 > Yarn cluster mode fires but does not forget > --

[jira] [Updated] (SPARK-3340) Deprecate ADD_JARS and ADD_FILES

[ https://issues.apache.org/jira/browse/SPARK-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3340: - Assignee: (was: Andrew Or) > Deprecate ADD_JARS and ADD_FILES > > >

[jira] [Closed] (SPARK-3560) In yarn-cluster mode, the same jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3560. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Fixed by https://github.com/apache/sp

[jira] [Reopened] (SPARK-3560) In yarn-cluster mode, the same jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-3560: -- Assignee: Min Shen Reopening just to reassign. Closing right afterwards, please disregard. > In yarn-

[jira] [Closed] (SPARK-3560) In yarn-cluster mode, the same jars are distributed through multiple mechanisms.

[ https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3560. Resolution: Fixed > In yarn-cluster mode, the same jars are distributed through multiple > mechanisms. > --

[jira] [Updated] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3535: - Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1) > Spark on Mesos not correctly setting heap overhead > --

[jira] [Commented] (SPARK-3610) History server log name should not be based on user input

[ https://issues.apache.org/jira/browse/SPARK-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143709#comment-14143709 ] Andrew Or commented on SPARK-3610: -- Hi all, I don't have the time to fix this, but this i

[jira] [Updated] (SPARK-3606) Spark-on-Yarn AmIpFilter does not work with Yarn HA.

[ https://issues.apache.org/jira/browse/SPARK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3606: - Affects Version/s: 1.1.0 > Spark-on-Yarn AmIpFilter does not work with Yarn HA. >

[jira] [Created] (SPARK-3651) Consolidate executor maps in CoarseGrainedSchedulerBackend

Andrew Or created SPARK-3651: Summary: Consolidate executor maps in CoarseGrainedSchedulerBackend Key: SPARK-3651 URL: https://issues.apache.org/jira/browse/SPARK-3651 Project: Spark Issue Type:

[jira] [Created] (SPARK-3653) SPARK_{DRIVER|EXECUTOR}_MEMORY is ignored in cluster mode

Andrew Or created SPARK-3653: Summary: SPARK_{DRIVER|EXECUTOR}_MEMORY is ignored in cluster mode Key: SPARK-3653 URL: https://issues.apache.org/jira/browse/SPARK-3653 Project: Spark Issue Type: I

[jira] [Closed] (SPARK-3653) SPARK_{DRIVER|EXECUTOR}_MEMORY is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3653. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > SPARK_{DRIVER|EXECUTOR}_MEMORY is i

[jira] [Created] (SPARK-3661) spark.driver.memory is ignored in cluster mode

Andrew Or created SPARK-3661: Summary: spark.driver.memory is ignored in cluster mode Key: SPARK-3661 URL: https://issues.apache.org/jira/browse/SPARK-3661 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-1853) Show Streaming application code context (file, line number) in Spark Stages UI

[ https://issues.apache.org/jira/browse/SPARK-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1853. Resolution: Fixed Fix Version/s: 1.1.1 Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) > Show

[jira] [Updated] (SPARK-3682) Add helpful warnings to the UI

[ https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3682: - Target Version/s: 1.2.0 Affects Version/s: 1.1.0 > Add helpful warnings to the UI > -

[jira] [Updated] (SPARK-3678) Yarn app name reported in RM is different between cluster and client mode

[ https://issues.apache.org/jira/browse/SPARK-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3678: - Affects Version/s: (was: 1.2.0) 1.1.0 > Yarn app name reported in RM is differe

[jira] [Created] (SPARK-3684) Can't configure local dirs in Yarn mode

Andrew Or created SPARK-3684: Summary: Can't configure local dirs in Yarn mode Key: SPARK-3684 URL: https://issues.apache.org/jira/browse/SPARK-3684 Project: Spark Issue Type: Bug Compo

[jira] [Created] (SPARK-3685) Spark's local dir scheme is not configurable

Andrew Or created SPARK-3685: Summary: Spark's local dir scheme is not configurable Key: SPARK-3685 URL: https://issues.apache.org/jira/browse/SPARK-3685 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3685) Spark's local dir scheme is not configurable

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147206#comment-14147206 ] Andrew Or commented on SPARK-3685: -- Note that this is not meaningful unless we also chang

[jira] [Updated] (SPARK-3661) spark.driver.memory is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Priority: Critical (was: Major) > spark.driver.memory is ignored in cluster mode > --

[jira] [Closed] (SPARK-3639) Kinesis examples set master as local

[ https://issues.apache.org/jira/browse/SPARK-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3639. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Aniket

[jira] [Closed] (SPARK-3476) Yarn ClientBase.validateArgs memory checks wrong

[ https://issues.apache.org/jira/browse/SPARK-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3476. Resolution: Fixed Assignee: Andrew Or > Yarn ClientBase.validateArgs memory checks wrong > ---

[jira] [Commented] (SPARK-3685) Spark's local dir scheme is not configurable

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151372#comment-14151372 ] Andrew Or commented on SPARK-3685: -- [~farrellee] One of the concerns for SPARK-3174 is th

[jira] [Updated] (SPARK-3685) Spark's local dir should accept only local paths

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3685: - Summary: Spark's local dir should accept only local paths (was: Spark's local dir scheme is not configura

[jira] [Updated] (SPARK-3685) Spark's local dir should accept only local paths

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3685: - Component/s: Spark Core > Spark's local dir should accept only local paths > -

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152070#comment-14152070 ] Andrew Or commented on SPARK-3685: -- Yeah there will be a design doc soon on the possible

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152228#comment-14152228 ] Andrew Or commented on SPARK-3685: -- Not sure if I fully understand what you mean. If I'm

[jira] [Closed] (SPARK-3734) DriverRunner should not read SPARK_HOME from submitter's environment

[ https://issues.apache.org/jira/browse/SPARK-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3734. Resolution: Fixed > DriverRunner should not read SPARK_HOME from submitter's environment > -

[jira] [Closed] (SPARK-3734) DriverRunner should not read SPARK_HOME from submitter's environment

[ https://issues.apache.org/jira/browse/SPARK-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3734. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target Version/s: 1.1.1,

[jira] [Reopened] (SPARK-3734) DriverRunner should not read SPARK_HOME from submitter's environment

[ https://issues.apache.org/jira/browse/SPARK-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-3734: -- > DriverRunner should not read SPARK_HOME from submitter's environment > ---

[jira] [Updated] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3627: - Affects Version/s: (was: 1.2.0) 1.1.0 > spark on yarn reports success even thou

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153758#comment-14153758 ] Andrew Or commented on SPARK-3627: -- Did this problem not exist in 1.1? I vaguely remember

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153830#comment-14153830 ] Andrew Or commented on SPARK-3627: -- Ok, sounds good. > spark on yarn reports success eve

[jira] [Closed] (SPARK-3755) Do not bind port 1 - 1024 to server in spark

[ https://issues.apache.org/jira/browse/SPARK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3755. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: wangfei > Do not bind port 1 - 1024 to serv

[jira] [Updated] (SPARK-2626) Stop SparkContext in all examples

[ https://issues.apache.org/jira/browse/SPARK-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2626: - Assignee: Sean Owen > Stop SparkContext in all examples > - > >

[jira] [Created] (SPARK-3767) Support wildcard in Spark properties

Andrew Or created SPARK-3767: Summary: Support wildcard in Spark properties Key: SPARK-3767 URL: https://issues.apache.org/jira/browse/SPARK-3767 Project: Spark Issue Type: New Feature

[jira] [Closed] (SPARK-3755) Do not bind port 1 - 1024 to server in spark

[ https://issues.apache.org/jira/browse/SPARK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3755. Resolution: Fixed Fix Version/s: 1.1.1 > Do not bind port 1 - 1024 to server in spark > -

[jira] [Closed] (SPARK-3759) SparkSubmitDriverBootstrapper should return exit code of driver process

[ https://issues.apache.org/jira/browse/SPARK-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3759. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Eric Ei

[jira] [Closed] (SPARK-2058) SPARK_CONF_DIR should override all present configs

[ https://issues.apache.org/jira/browse/SPARK-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2058. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > SPARK_CONF_DIR should override all

[jira] [Updated] (SPARK-2058) SPARK_CONF_DIR should override all present configs

[ https://issues.apache.org/jira/browse/SPARK-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2058: - Assignee: Eugen Cepoi > SPARK_CONF_DIR should override all present configs > -

[jira] [Commented] (SPARK-2058) SPARK_CONF_DIR should override all present configs

[ https://issues.apache.org/jira/browse/SPARK-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158197#comment-14158197 ] Andrew Or commented on SPARK-2058: -- To give a quick update, this change has not made it t

[jira] [Updated] (SPARK-3696) Do not override user-defined conf_dir in spark-config.sh

[ https://issues.apache.org/jira/browse/SPARK-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3696: - Assignee: WangTaoTheTonic > Do not override user-defined conf_dir in spark-config.sh > ---

[jira] [Closed] (SPARK-3696) Do not override user-defined conf_dir in spark-config.sh

[ https://issues.apache.org/jira/browse/SPARK-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3696. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target Version/s: 1.1.1,

[jira] [Closed] (SPARK-2778) Add unit tests for Yarn integration

[ https://issues.apache.org/jira/browse/SPARK-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2778. Resolution: Fixed Target Version/s: 1.2.0 > Add unit tests for Yarn integration >

[jira] [Updated] (SPARK-3710) YARN integration test is flaky

[ https://issues.apache.org/jira/browse/SPARK-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3710: - Affects Version/s: 1.2.0 > YARN integration test is flaky > -- > >

[jira] [Closed] (SPARK-3710) YARN integration test is flaky

[ https://issues.apache.org/jira/browse/SPARK-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3710. Resolution: Fixed Fix Version/s: 1.2.0 > YARN integration test is flaky > ---

[jira] [Commented] (SPARK-3710) YARN integration test is flaky

[ https://issues.apache.org/jira/browse/SPARK-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158326#comment-14158326 ] Andrew Or commented on SPARK-3710: -- https://github.com/apache/spark/pull/2605 > YARN int

[jira] [Closed] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3535. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Spark on Mesos not correctly settin

[jira] [Updated] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3535: - Assignee: Brenden Matthews > Spark on Mesos not correctly setting heap overhead >

[jira] [Updated] (SPARK-3775) Not suitable error message in spark-shell.cmd

[ https://issues.apache.org/jira/browse/SPARK-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3775: - Affects Version/s: 1.1.0 > Not suitable error message in spark-shell.cmd > ---

[jira] [Closed] (SPARK-3775) Not suitable error message in spark-shell.cmd

[ https://issues.apache.org/jira/browse/SPARK-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3775. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Masayos

[jira] [Closed] (SPARK-3774) typo comment in bin/utils.sh

[ https://issues.apache.org/jira/browse/SPARK-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3774. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Masayos

[jira] [Updated] (SPARK-3606) Spark-on-Yarn AmIpFilter does not work with Yarn HA.

[ https://issues.apache.org/jira/browse/SPARK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3606: - Fix Version/s: 1.2.0 > Spark-on-Yarn AmIpFilter does not work with Yarn HA. >

[jira] [Updated] (SPARK-3606) Spark-on-Yarn AmIpFilter does not work with Yarn HA.

[ https://issues.apache.org/jira/browse/SPARK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3606: - Target Version/s: 1.1.1, 1.2.0 Affects Version/s: (was: 1.2.0) > Spark-on-Yarn AmIpFilter does no

[jira] [Updated] (SPARK-3606) Spark-on-Yarn AmIpFilter does not work with Yarn HA.

[ https://issues.apache.org/jira/browse/SPARK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3606: - Affects Version/s: 1.2.0 > Spark-on-Yarn AmIpFilter does not work with Yarn HA. >

[jira] [Closed] (SPARK-3763) The example of building with sbt should be "sbt assembly" instead of "sbt compile"

[ https://issues.apache.org/jira/browse/SPARK-3763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3763. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta > The example of building wi

[jira] [Updated] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3597: - Assignee: Brenden Matthews > MesosSchedulerBackend does not implement `killTask` > ---

[jira] [Closed] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3597. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.1.1, 1.2.0 > MesosSchedulerBacke

[jira] [Commented] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159591#comment-14159591 ] Andrew Or commented on SPARK-3597: -- Fixed by https://github.com/apache/spark/pull/2453 >

[jira] [Assigned] (SPARK-3797) Integrate shuffle service in YARN's pluggable shuffle

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-3797: Assignee: Andrew Or > Integrate shuffle service in YARN's pluggable shuffle > -

[jira] [Updated] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3174: - Attachment: dynamic-scaling-executors-10-6-14.pdf > Provide elastic scaling within a Spark application > -

[jira] [Updated] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

[ https://issues.apache.org/jira/browse/SPARK-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3822: - Issue Type: Sub-task (was: Bug) Parent: SPARK-3174 > Expose a mechanism for SparkContext to ask f

[jira] [Created] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

Andrew Or created SPARK-3822: Summary: Expose a mechanism for SparkContext to ask for / remove Yarn containers Key: SPARK-3822 URL: https://issues.apache.org/jira/browse/SPARK-3822 Project: Spark

[jira] [Updated] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3174: - Description: A common complaint with Spark in a multi-tenant environment is that applications have a fixe

[jira] [Updated] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3174: - Description: A common complaint with Spark in a multi-tenant environment is that applications have a fixe

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160914#comment-14160914 ] Andrew Or commented on SPARK-3174: -- Hi all, I have attached an updated design doc that de

[jira] [Created] (SPARK-3825) Log more information when unrolling a block fails

Andrew Or created SPARK-3825: Summary: Log more information when unrolling a block fails Key: SPARK-3825 URL: https://issues.apache.org/jira/browse/SPARK-3825 Project: Spark Issue Type: Bug

<    1   2   3   4   5   6   7   8   9   10   >