Sean Owen created SPARK-27032: --------------------------------- Summary: Flaky test: org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.HDFSMetadataLog: metadata directory collision Key: SPARK-27032 URL: https://issues.apache.org/jira/browse/SPARK-27032 Project: Spark Issue Type: Test Components: Spark Core, Tests Affects Versions: 3.0.0 Reporter: Sean Owen Assignee: Sean Owen
Locally and on Jenkins, I've frequently seen this test fail: {code} Error Message The await method on Waiter timed out. Stacktrace org.scalatest.exceptions.TestFailedException: The await method on Waiter timed out. at org.scalatest.concurrent.Waiters$Waiter.awaitImpl(Waiters.scala:406) at org.scalatest.concurrent.Waiters$Waiter.await(Waiters.scala:540) at org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19(HDFSMetadataLogSuite.scala:158) at org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19$adapted(HDFSMetadataLogSuite.scala:133) ... {code} See for example https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/6057/testReport/ There aren't obvious errors or problems with the test. Because it passes sometimes, my guess is that the timeout is simply too short or the test too long. I'd like to try reducing the number of threads/batches in the test. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org