[ https://issues.apache.org/jira/browse/SPARK-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170006#comment-14170006 ]
Apache Spark commented on SPARK-3912: ------------------------------------- User 'tdas' has created a pull request for this issue: https://github.com/apache/spark/pull/2773 > FlumeStreamSuite is flaky, fails either with port binding issues or data not > being reliably sent > ------------------------------------------------------------------------------------------------ > > Key: SPARK-3912 > URL: https://issues.apache.org/jira/browse/SPARK-3912 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.1.0 > Reporter: Tathagata Das > Assignee: Tathagata Das > Priority: Critical > > Two problems. > 1. Attempts to start the service to start on different possible ports (to > avoid bind failures) was incorrect as the service is actually start lazily > (when receiver starts, not when the flume input stream is created). > 2. Lots of Thread.sleep was used to improve the probabilities that data sent > through avro to flume receiver was being sent. However, the sending may fail > for various unknown reasons, causing the test to fail. > 3. Thread.sleep was also used to send one record per batch and checks were > made on whether only one records was received in every batch. This was an > overkill because all we need to test in this unit test is whether data is > being sent and received or not, not about timings. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org