My understanding is that it is good to have this, but a large fraction of flaky tests are actually also race conditions, etc.
On Thu, May 28, 2015 at 11:55 AM, Imran Rashid <iras...@cloudera.com> wrote: > Hi, > > I was just fixing a problem with too short a timeout on one of the unit > tests I added (https://issues.apache.org/jira/browse/SPARK-7919), and I > was wondering if this is a common problem w/ a lot of our flaky tests. > > Its really hard to know what to set the timeouts to -- you set the timeout > so it runs comfortably on your dev machine, but then discover its far too > short for the build machines. so up it some more, and then it turns out > there are some even slower build machines, and you need to make it longer > still ... > > Scalatest actually has a feature to handle this. You can wrap all > timeouts in scaled(...), and then you can pass in a flag with how much the > timeouts should get scaled by. > > > http://doc.scalatest.org/2.0/index.html#org.scalatest.concurrent.ScaledTimeSpans > > eg., I was looking at DriverSuite -- its been turned off because it was > too flaky, though the 60 second timeout already seems super long. ( > https://issues.apache.org/jira/browse/SPARK-7415) > > does anybody know if timeouts are a common source of flaky tests? Is it > worth trying to change existing timeouts in tests to use scaled() so keep > tests passing on slow build machines? Does it make sense for all future > tests to always use scaled() on timeouts? > > thanks, > Imran >
