[
https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922777#comment-16922777
]
Jungtaek Lim edited comment on SPARK-28770 at 9/4/19 7:21 PM:
--------------------------------------------------------------
As I commented in PR, I'm not sure EventMonster fits with the purpose of
testing - why we concern about such EventLoggingListener specific criteria in
ReplayListenerSuite? ReplayListener is designed to read and post all events to
its listeners. If EventMonster just buffers the events as it receives, it
shouldn't happen, as EventLoggingListener will buffer the events exactly same
as what it writes to the event log file.
It shows weird behavior or at least requires us to know about the details of
EventLoggingListener because EventMonster extends EventLoggingListener and it
doesn't buffer received events transparently.
was (Author: kabhwan):
As I commented in PR, I'm not sure EventMonster fits with the purpose of
testing - why we concern about the criteria in ReplayListenerSuite?
ReplayListener is designed to read and post all events to its listeners. If
EventMonster just buffers the events as it receives, it shouldn't happen, as
EventLoggingListener will buffer the events exactly same as what it writes to
the event log file.
It shows weird behavior or at least requires us to know about the details of
EventLoggingListener because EventMonster extends EventLoggingListener and it
doesn't buffer received events transparently.
> Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression
> failed
> -------------------------------------------------------------------------------
>
> Key: SPARK-28770
> URL: https://issues.apache.org/jira/browse/SPARK-28770
> Project: Spark
> Issue Type: Test
> Components: Spark Core
> Affects Versions: 2.4.3
> Environment: Community jenkins and our arm testing instance.
> Reporter: huangtianhua
> Priority: Major
>
> Test
> org.apache.spark.scheduler.ReplayListenerSuite.End-to-end replay with
> compression is failed see
> [https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-3.2/267/testReport/junit/org.apache.spark.scheduler/ReplayListenerSuite/End_to_end_replay_with_compression/]
>
> And also the test is failed on arm instance, I sent email to spark-dev
> before, and we suspect there is something related with the commit
> [https://github.com/apache/spark/pull/23767], we tried to revert it and the
> tests are passed:
> ReplayListenerSuite:
> - ...
> - End-to-end replay *** FAILED ***
> "[driver]" did not equal "[1]" (JsonProtocolSuite.scala:622)
> - End-to-end replay with compression *** FAILED ***
> "[driver]" did not equal "[1]" (JsonProtocolSuite.scala:622)
>
> Not sure what's wrong, hope someone can help to figure it out, thanks very
> much.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
