Sean Owen created SPARK-27032:
---------------------------------
Summary: Flaky test:
org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.HDFSMetadataLog:
metadata directory collision
Key: SPARK-27032
URL: https://issues.apache.org/jira/browse/SPARK-27032
Project: Spark
Issue Type: Test
Components: Spark Core, Tests
Affects Versions: 3.0.0
Reporter: Sean Owen
Assignee: Sean Owen
Locally and on Jenkins, I've frequently seen this test fail:
{code}
Error Message
The await method on Waiter timed out.
Stacktrace
org.scalatest.exceptions.TestFailedException: The await method on Waiter
timed out.
at org.scalatest.concurrent.Waiters$Waiter.awaitImpl(Waiters.scala:406)
at org.scalatest.concurrent.Waiters$Waiter.await(Waiters.scala:540)
at
org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19(HDFSMetadataLogSuite.scala:158)
at
org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19$adapted(HDFSMetadataLogSuite.scala:133)
...
{code}
See for example
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/6057/testReport/
There aren't obvious errors or problems with the test. Because it passes
sometimes, my guess is that the timeout is simply too short or the test too
long. I'd like to try reducing the number of threads/batches in the test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
