Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76864451
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76864438
[Test build #28201 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28201/consoleFull)
for PR 4821 at commit
[`8511141`](https://gith
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4821
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76860619
[Test build #28198 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28198/consoleFull)
for PR 4821 at commit
[`fdae14c`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76860628
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76855580
Okay I took a close look through this and it LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76850938
[Test build #28201 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28201/consoleFull)
for PR 4821 at commit
[`8511141`](https://githu
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25649521
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -574,6 +583,11 @@ private[spark] object JsonProtocol {
SparkListenerE
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76848440
LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled an
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25648636
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -574,6 +583,11 @@ private[spark] object JsonProtocol {
SparkListenerE
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25648384
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -574,6 +583,11 @@ private[spark] object JsonProtocol {
SparkListenerExe
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76846594
[Test build #28198 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28198/consoleFull)
for PR 4821 at commit
[`fdae14c`](https://githu
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25647830
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -89,16 +105,18 @@ class FsHistoryProviderSuite extends FunSu
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25647694
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -89,16 +105,18 @@ class FsHistoryProviderSuite extends F
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25647732
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -89,16 +105,18 @@ class FsHistoryProviderSuite extends F
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25641491
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -89,16 +105,18 @@ class FsHistoryProviderSuite extends FunSu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76831455
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76831439
[Test build #28182 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28182/consoleFull)
for PR 4821 at commit
[`654883d`](https://gith
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76821354
> flakiness of FsHistoryProviderSuite
Do you have any theories for why this would be related to the log's
modification time? The only place where it's used in FsH
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76815718
[Test build #28182 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28182/consoleFull)
for PR 4821 at commit
[`654883d`](https://githu
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76815711
By the way I gained first-hand experience of the flakiness of
`FsHistoryProviderSuite` from this patch. The tests passed locally but failed
here, and I suspect that it
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25628566
--- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala ---
@@ -71,6 +74,21 @@ private[spark] object CompressionCodec {
s"Consid
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25628525
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -111,19 +119,17 @@ private[spark] class EventLoggingListener(
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76787929
[Test build #28180 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28180/consoleFull)
for PR 4821 at commit
[`7f537cd`](https://githu
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76785685
@pwendell yes, that's what I'm trying to say. Even today, logs generated by
Spark 1.0, 1.1 and 1.2 (and soon 1.3) are not exactly the same - events have
different properti
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25628232
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -111,19 +119,17 @@ private[spark] class EventLoggingListener(
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76782633
@vanzin so just to be clear - you are anticipating a future case where we
need to read the version to correctly parse the logs? Is that the argument for
it? I am only ma
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25626361
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -45,18 +44,26 @@ class FsHistoryProviderSuite extends Fu
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76780546
@pwendell the header is needed because it contains potentially useful
information for the code parsing the logs. For example, now it contains the
Spark version, which migh
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25626186
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -111,19 +119,17 @@ private[spark] class EventLoggingListener(
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25625447
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -45,18 +44,26 @@ class FsHistoryProviderSuite extends FunSui
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25625518
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -111,19 +119,17 @@ private[spark] class EventLoggingListener(
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25625161
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -45,18 +44,26 @@ class FsHistoryProviderSuite extends FunSui
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25625032
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -279,52 +253,17 @@ private[spark] object EventLoggingListener extend
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25624653
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -111,19 +119,17 @@ private[spark] class EventLoggingListener(
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25624492
--- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala ---
@@ -71,6 +74,21 @@ private[spark] object CompressionCodec {
s"Consider s
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25624317
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -320,30 +320,24 @@ private[history] class FsHistoryProvider(conf:
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76673355
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76673346
[Test build #28159 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28159/consoleFull)
for PR 4821 at commit
[`7d6aa61`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76669704
[Test build #28158 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28158/consoleFull)
for PR 4821 at commit
[`59abee9`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76669712
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76668874
[Test build #28157 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28157/consoleFull)
for PR 4821 at commit
[`40e1ff6`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76668879
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-7392
[Test build #28159 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28159/consoleFull)
for PR 4821 at commit
[`7d6aa61`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76663956
[Test build #28158 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28158/consoleFull)
for PR 4821 at commit
[`59abee9`](https://githu
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76663334
As of the latest commit I have removed the Spark version and the metadata.
Please have a look.
---
If your project is set up for it, you can reply to this email and h
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25579983
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala ---
@@ -39,10 +39,9 @@ private[spark] class ReplayListenerBus extends
Spa
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25579955
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -322,28 +322,22 @@ private[history] class FsHistoryProvider(conf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25579899
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -322,28 +322,22 @@ private[history] class FsHistoryProvider(conf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25579888
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -322,28 +322,22 @@ private[history] class FsHistoryProvider(conf
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76663276
[Test build #28157 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28157/consoleFull)
for PR 4821 at commit
[`40e1ff6`](https://githu
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25579333
--- Diff:
core/src/main/scala/org/apache/spark/deploy/ApplicationDescription.scala ---
@@ -23,7 +23,9 @@ private[spark] class ApplicationDescription(
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25577300
--- Diff:
core/src/main/scala/org/apache/spark/deploy/ApplicationDescription.scala ---
@@ -23,7 +23,9 @@ private[spark] class ApplicationDescription(
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25577271
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,57 +223,67 @@ private[spark] object EventLoggingListener ex
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76641955
I took a pass on this with some feedback. Overall, it would be good to
really minimize the scope of the changes since this is so late in the game.
There is some clean-up
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25574126
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -99,6 +99,12 @@ case class SparkListenerExecutorRemoved(time: Long,
exec
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25573931
--- Diff:
core/src/main/scala/org/apache/spark/deploy/ApplicationDescription.scala ---
@@ -23,7 +23,9 @@ private[spark] class ApplicationDescription(
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25573801
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,57 +223,67 @@ private[spark] object EventLoggingListener exte
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25573776
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,57 +223,67 @@ private[spark] object EventLoggingListener exte
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76507612
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76507606
[Test build #28103 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28103/consoleFull)
for PR 4821 at commit
[`519e51a`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76505881
[Test build #28102 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28102/consoleFull)
for PR 4821 at commit
[`ef69276`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76505885
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76503962
[Test build #28103 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28103/consoleFull)
for PR 4821 at commit
[`519e51a`](https://githu
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25552458
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -110,17 +117,12 @@ private[spark] class EventLoggingListener(
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25552366
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,53 +219,60 @@ private[spark] object EventLoggingListener ex
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76502888
[Test build #28102 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28102/consoleFull)
for PR 4821 at commit
[`ef69276`](https://githu
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25552104
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,53 +219,60 @@ private[spark] object EventLoggingListener extend
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25552069
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -202,11 +204,11 @@ private[spark] object EventLoggingListener ex
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25552048
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,53 +219,60 @@ private[spark] object EventLoggingListener ex
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76497543
[Test build #28098 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28098/consoleFull)
for PR 4821 at commit
[`8db5a06`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76497550
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76496914
Looks reasonable, will wait for tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25549402
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -217,53 +219,60 @@ private[spark] object EventLoggingListener extend
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25549234
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -202,11 +204,11 @@ private[spark] object EventLoggingListener extend
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4821#discussion_r25549140
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -110,17 +117,12 @@ private[spark] class EventLoggingListener(
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76494032
Tests will fail to compile. I'm fixing this in the mean time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4821#issuecomment-76493745
[Test build #28098 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28098/consoleFull)
for PR 4821 at commit
[`8db5a06`](https://githu
78 matches
Mail list logo