Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220022092
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220020756
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220022604
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala
---
@@ -0,0 +1,106 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220021564
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220023313
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala
---
@@ -0,0 +1,106 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220021363
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220021145
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -86,6 +86,13 @@ private[spark] class Client
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220022949
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala
---
@@ -0,0 +1,106 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220021593
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220024661
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala
---
@@ -0,0 +1,93 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220021504
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220023729
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -140,10 +140,19 @@ private[yarn] class
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220022522
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala
---
@@ -0,0 +1,106 @@
+/*
+ * Licensed
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r220024168
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeValidatorSuite.scala
---
@@ -0,0 +1,150
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016885
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016802
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220013308
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -143,13 +143,28 @@ trait Logging {
// overriding the root logger's
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016494
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016445
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016032
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220016668
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220012951
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2421,11 +2425,13 @@ class SparkContext(config: SparkConf) extends
Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220013762
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -192,7 +207,16 @@ private[spark] object Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220015083
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220015864
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220012966
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2421,11 +2425,13 @@ class SparkContext(config: SparkConf) extends
Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220015442
--- Diff:
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala ---
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220014994
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -192,7 +207,16 @@ private[spark] object Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22504#discussion_r220014946
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2328,7 +2328,16 @@ private[spark] object Utils extends Logging {
* configure
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22511
LGTM. I went back and took a look at the related changes, and agree with
Imran that this is basically the same thing that 2.3 did; so no perf
regression, just higher memory usage than in the mmap
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22362
You need to add the mapping of old configs to new configs in
`SparkConf.configsWithAlternatives`.
---
-
To unsubscribe, e-mail
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22525
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22526
Merging to master / 2.4 / 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22325
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21816
That could help, yes.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21816
I know you pinged me, but I'm really not familiar with how Spark standalone
is deployed, and I don't even know who's maintaining this code these days
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22504
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
merging to master / 2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22381
I don't really have an opinion about the metrics or names or things like
that; but not being familiar with the internals of this metric system, I'd just
like to make sure that updating these metrics
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r219236374
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -386,6 +386,7 @@ final class ShuffleBlockFetcherIterator
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22460
To speed things up might be worth it to hack run-test.py to only run core
tests (or DistributedSuite).
---
-
To unsubscribe, e
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22460#discussion_r219011355
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/client/TransportResponseHandler.java
---
@@ -211,6 +211,10 @@ public void handle
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r218991156
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +445,36 @@ final class ShuffleBlockFetcherIterator
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r218990774
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +445,36 @@ final class ShuffleBlockFetcherIterator
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r218989956
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,49 +444,34 @@ final class ShuffleBlockFetcherIterator
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22452
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22444
> so any server restart results in hours of downtime, just from scanning.
Well, that's why 2.3 supports caching things on disk. Also, 2.4 has
SPARK-6951 which should make this a lot fas
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22441#discussion_r218166616
--- Diff: dev/create-release/release-build.sh ---
@@ -183,8 +188,17 @@ if [[ "$1" == "package" ]]; then
# Updated f
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21756
Only you can close your own PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22325
Or maybe the test needs to be updated. Either way, not a reason not to
merge your changes with the existing code
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22325
If the test fails with your change you're introducing a change of behavior
that breaks things, and you should fix it.
I don't see why that is a reason for not merging both code paths
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22399#discussion_r216896517
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/AbstractAppHandle.java ---
@@ -72,11 +74,7 @@ public void stop() {
@Override
public
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22289
> I failed to find documentation about the RM adding its own Hadoop config
files to the AM/executors' classpath
See `Client.getYarnAppClasspath` and `Client.getMRAppClassp
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22341
There was a trivial conflict in 2.3, I fixed it manually.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22341
Merging to master, 2.4 and 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216741392
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -538,6 +538,10 @@ private class LiveRDD(val info: RDDInfo) extends
LiveEntity
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22384
Looks fine.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22213
Merging to master / 2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216523989
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216523824
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -538,6 +538,14 @@ private class LiveRDD(val info: RDDInfo) extends
LiveEntity
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216513137
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -538,6 +538,14 @@ private class LiveRDD(val info: RDDInfo) extends
LiveEntity
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22383
We can't do this until we decide whether the next version will be a major
version or not. This breaks backwards compatibility. (See the bug I duplicated
your bug
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216406188
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216404844
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -23,6 +23,7 @@ import java.util.function.Function
import
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216405553
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216407616
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -538,6 +538,14 @@ private class LiveRDD(val info: RDDInfo) extends
LiveEntity
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216406902
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21398
I didn't make the original change so I'm refraining from making a decision
about whether this should go in. Perhaps another fix is desired, perhaps the
current behavior is desired, but either way
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216116705
--- Diff: core/src/test/java/org/apache/spark/ExecutorPluginSuite.java ---
@@ -68,7 +68,9 @@ public void testPluginClassDoesNotExist() {
} catch
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216116693
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -242,14 +242,17 @@ private[spark] class Executor(
// Notify
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22362#discussion_r216108106
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -199,8 +199,8 @@ private[deploy] class SparkSubmitArguments(args
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22362#discussion_r216107597
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -199,8 +199,8 @@ private[deploy] class SparkSubmitArguments(args
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22362#discussion_r216107505
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -520,6 +520,10 @@ private[spark] class SparkSubmit extends Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22362#discussion_r216107385
--- Diff: R/pkg/R/sparkR.R ---
@@ -624,8 +624,8 @@ sparkConfToSubmitOps[["spark.driver.extraClassPath"]]
<- "-
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106676
--- Diff: core/src/test/java/org/apache/spark/ExecutorPluginSuite.java ---
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106455
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -219,6 +239,18 @@ private[spark] class Executor(
heartbeater.shutdown
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106482
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -219,6 +239,18 @@ private[spark] class Executor(
heartbeater.shutdown
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106778
--- Diff: core/src/test/java/org/apache/spark/ExecutorPluginSuite.java ---
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106909
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -240,6 +240,16 @@ private[spark] object Utils extends Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216105914
--- Diff: core/src/main/java/org/apache/spark/ExecutorPlugin.java ---
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106874
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -240,6 +240,16 @@ private[spark] object Utils extends Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216106374
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -136,6 +136,26 @@ private[spark] class Executor(
// for fetching remote
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216094096
--- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ---
@@ -1205,6 +1205,39 @@ class UtilsSuite extends SparkFunSuite
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216092017
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2062,8 +2086,10 @@ private[spark] object Utils extends Logging {
try
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216094051
--- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ---
@@ -1205,6 +1205,39 @@ class UtilsSuite extends SparkFunSuite
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216091436
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2052,6 +2051,31 @@ private[spark] object Utils extends Logging
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216093949
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -1144,6 +1144,48 @@ class SparkSubmitSuite
conf1.get
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r216093880
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -1144,6 +1144,48 @@ class SparkSubmitSuite
conf1.get
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21669
> The clusters Kerberos config file pointed the KDC to an IP not visible to
my K8S pod
In the case where the launcher is fetching delegation tokens and providing
them to the dri
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
it's ok to keep them in the same suite where they are used.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22289
`spark.hadoop.*` is not a good name. That's a special prefix in Spark that
modifies any Hadoop `Configuration` object that Spark instantiates. That's the
easy one.
The hard one is that your
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21849#discussion_r216058998
--- Diff:
core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java ---
@@ -227,6 +220,82 @@ public void testInProcessLauncherDoesNotKillJvm
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21849#discussion_r216058203
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/ChildProcAppHandle.java ---
@@ -46,6 +47,25 @@ public synchronized void disconnect
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21849#discussion_r216059482
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/OutputRedirector.java ---
@@ -61,6 +62,10 @@ private void redirect() {
while ((line
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21849#discussion_r216058842
--- Diff: project/MimaExcludes.scala ---
@@ -106,7 +106,10 @@ object MimaExcludes {
ProblemFilters.exclude[InheritedNewAbstractMethodProblem
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216054443
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -219,6 +236,13 @@ private[spark] class Executor(
heartbeater.shutdown
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22192#discussion_r216049652
--- Diff: core/src/test/java/org/apache/spark/ExecutorPluginSuite.java ---
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
701 - 800 of 11074 matches
Mail list logo
