Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9221#issuecomment-151103830
Adding some tests here could be an interesting little project. the main
thing to do is to have some config file in test/resources, which git is set up
to ignore
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9221#issuecomment-151276071
If you use env vars, you'd better make sure they don't ever leak into test
reports (e.g no logging of env vars in an executor)
---
If your project is set up
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9221#issuecomment-151291754
...so if I submit a patch which modifies that package and runs a test which
prints your env vars to stdout, I've got your keys.
---
If your project is set up
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-151145666
Thomas: been through the hive versions and they didn't pull anything,
simply added another overloaded method before the existing three. Probably a
latent bug
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-151140385
Thomas, I'm not sure that they actually pulled something, but instead
overloaded the get() operation, which confused the reflection logic.
---
If your project
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43004016
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43003603
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-151173854
-Updated patch pulls the `val token=` and `fail` code from the tests. It
also pulls out the logic for deciding what to do with exceptions into its own
class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42945007
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42944993
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,104 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42944947
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42944952
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42945056
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,28 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42897746
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -1337,55 +1337,8 @@ object Client extends Logging {
conf
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42897540
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -322,8 +322,10 @@ private[spark] class Client(
// multiple times
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42866102
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -1337,55 +1337,8 @@ object Client extends Logging {
conf
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-150579863
-just pushed up a version with @dougb's suggestion; the check for cluster
mode is made before trying to obtain either hive or hbase tokens.
Note
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42866295
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-150688417
Updated patch
1. Moves all arg validation up before any hive class reflection
1. gets all hive class methods before attempting to call any
1. calls
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-150700891
I've tightened this up and am happy with the code now, except I'm not sure
that NoSuchMethodExceptions should be downgraded. It will hide the situation
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-150689362
-Will do. Then I can combine everything nicely.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-150658089
Patch with revert posted
For reviewers, here some things to consider
1. Which exceptions should trigger log & skip for token, vs w
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9168#discussion_r42779515
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -130,6 +130,21 @@ class SparkHadoopUtil extends Logging
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9221#issuecomment-150151459
Looking @ the twitter4j poms, their dependencies pull in a
com.googlecode.kryo -given the kryo pain its a possible troublespot. However,
that com.googlecode
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42784907
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9232
Stevel/patches/spark 11265 hive tokens
This is just the first "it compiles" stage of the patch: it doesn't contain
any fixes
I've just pulled the current code out of the y
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42667047
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42668737
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42669119
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,765 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42674559
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9168#discussion_r42604617
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -130,6 +130,21 @@ class SparkHadoopUtil extends Logging
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9168#discussion_r42482462
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -130,6 +132,20 @@ class SparkHadoopUtil extends Logging
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9168#issuecomment-149531922
solution 4: fix HDFS. Don't be afraid of filing reports & submitting
patches there; it's a better long-term strategy than trying to work around
things downst
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8766#issuecomment-149554897
If people are happy with that, I'll do that
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42537310
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42538300
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42539302
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42531200
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42540050
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42539027
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r42525776
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,1048 @@
+/*
+ * Licensed
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9182
SPARK-1537: pure service API and test service
This is purely the yarn/src/main and yarn/src/test bits of the YARN ATS
integration: the extension model to load and run implementations
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-149595658
Test failed as there was a class `StubApplicationReport` which had been
deleted but the deletion not pushed up. The YARN `ApplicationReport` class has
grown
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9168#discussion_r42392020
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -130,6 +132,20 @@ class SparkHadoopUtil extends Logging
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9144#issuecomment-148662454
It's been in 1.x. In fact, it's been in 0.19 when the {{fs.permission}} one
was first tagged as deprecated
[fs/permission/AccessControlException.java](https
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-147698799
I've been thinking about this âhow about I do the following
# have move the history in under yarn/src/history, as discussed above
# also as discussed
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-147749231
I've pushed out a rebase of this against master.
1. The asynchronous publishing of events with retries and, eventually
dropped events, appears
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-147770129
(this probably isn't going to compile BTW)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-146959330
Jerry, the code is where it is to isolate it across versions; it started
off where it was in the branch 1.2 era, where the split was common, alpha and
stable
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-145818164
@tgravescs: ATS v1. What we got from the ATS team was a commitment to not
break that API at some ws/v1/ endpoint they'll continue to offer.
However, V2
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8766#issuecomment-141033106
test failure is `org.apache.spark.streaming.CheckpointSuite.recovery with
invertible reduceByKeyAndWindow operation`; appears unrelated.
---
If your project
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8766#issuecomment-141160203
Marcelo, I'm not sure I trust nanoTime any more: I did some reading on the
topic and don't think code can be confident it works (i.e. is monotonic and
consistent
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8766#discussion_r39622243
--- Diff: core/src/main/scala/org/apache/spark/util/Clock.scala ---
@@ -21,12 +21,22 @@ package org.apache.spark.util
* An interface to represent
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8766#discussion_r39622785
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -26,7 +26,7 @@ import com.codahale.metrics.{Gauge, MetricRegistry
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8766#discussion_r39622560
--- Diff: core/src/main/scala/org/apache/spark/util/Clock.scala ---
@@ -44,7 +54,7 @@ private[spark] class SystemClock extends Clock
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8766#discussion_r39622552
--- Diff: core/src/main/scala/org/apache/spark/util/Clock.scala ---
@@ -65,3 +75,18 @@ private[spark] class SystemClock extends Clock {
-1
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8766#discussion_r39622461
--- Diff: core/src/main/scala/org/apache/spark/util/Clock.scala ---
@@ -65,3 +75,18 @@ private[spark] class SystemClock extends Clock {
-1
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39633153
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnEventListener.scala
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39635426
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39635411
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39638971
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39634546
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39637541
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39635159
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39638560
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39638651
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39638893
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39637695
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39638905
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39643031
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39643240
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39643548
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39644786
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39645121
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39645278
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39653822
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39653693
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39654135
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39654285
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39654447
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39654834
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39655137
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39653540
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39653626
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39653690
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39672695
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala
---
@@ -37,9 +38,12 @@ private[spark] class
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-140853826
Wow âthanks for putting in the effort! I've gone through the work and
will also review the rest of the code to see if there are any implications for
the rest
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39657485
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryService.scala
---
@@ -0,0 +1,981 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39657710
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39658639
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39658764
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39658896
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39659260
--- Diff:
yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnTimelineUtils.scala
---
@@ -0,0 +1,720 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39659390
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnExtensionService.scala
---
@@ -0,0 +1,198 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/8744#discussion_r39660218
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnExtensionService.scala
---
@@ -0,0 +1,198 @@
+/*
+ * Licensed
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-140319344
I came across this problem (and filed the JIRA) when I was trying to set a
properties file up,
{code}
sbin/spark-history-server --properties-file
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/8766
SPARK-10614 SystemClock uses non-monotonic time in its wait logic. Thâ¦
This patch adds a new subclass, `MonotonicClock`, and switches
`ExecutorAllocationManager` to using it (it used to use
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/8744
SPARK-1537 publisher-side code and tests
This is the publisher side of the SPARK-1537 Yarn timeline server
integration: the event listener to asynchronously publish events to ATS; the
basic
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/7188#issuecomment-140119929
Superceded by
[https://github.com/apache/spark/pull/8748](https://github.com/apache/spark/pull/8748)
---
If your project is set up for it, you can reply
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-140114983
Failing test was
`org.apache.spark.network.sasl.SaslIntegrationSuite.testNoSaslClient` in
`spark-network-shuffle`; this is not downstream of the artifacts
901 - 1000 of 1115 matches
Mail list logo
