I looked at the way that tests are run, an tthe only difference I see in the smoke tester jobs is that tests are run twice, once each for Java8 and Java9. Compared to non-smoke-tester jobs, this will double the likelihood of overall failure.
I looked at the suites that failed in the last ten runs on those two smoke tester jobs. Except for SearchHandlerTest, which I have since (hopefully) fixed, there are seven suites with failed tests - here they are along with their packages: TestExecutePlanAction o.a.s.cloud.autoscaling.sim TestComputePlanAction o.a.s.cloud.autoscaling.sim TestTriggerIntegration o.a.s.cloud.autoscaling.sim IndexSizeTriggerTest o.a.s.cloud.autoscaling CreateRoutedAliasTest o.a.s.cloud ReplaceNodeTest o.a.s.cloud MetricsHistoryHandlerTest o.a.s.handler.admin Some of those are pretty regular offenders AFAICT from http://fucit.org/solr-jenkins-reports/failure-report.html . Andrzej Białecki did some work on IndexSizeTriggerTest (SOLR-12392) and un-bad-apple’d its tests, but at least one of them is still failing since then - I’ll go add a comment on the issue. -- Steve www.lucidworks.com > On Jun 4, 2018, at 11:16 AM, Erick Erickson <erickerick...@gmail.com> wrote: > > Adrien: > > "Do you know whether there is something that makes these jobs any different?" > > Unfortunately no. I'm not really very well versed in the various test > environments, maybe Uwe or Steve Rowe or Hoss might have some insight? > > On Mon, Jun 4, 2018 at 3:34 AM, Adrien Grand <jpou...@gmail.com> wrote: >> Thanks for helping on this front Erick. I noticed a significant decrease in >> noise since you started badapple-ing bad tests, but I'm observing that our >> smoke-release builds still keep failing because of Solr tests (10 out of the >> last 10 builds) in spite of the fact that they disable bad apples: >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/ >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/ >> >> Do you know whether there is something that makes these jobs any different? >> >> Le mar. 29 mai 2018 à 18:12, Erick Erickson <erickerick...@gmail.com> a >> écrit : >>> >>> With the long weekend and the fact that the number of non-BadApple >>> tests is fairly small this week, I'll skip adding more BadApple tests >>> until next week. >>> >>> We're scarily close to the non-BaApple'd tests coming under control. >>> If we're lucky, we can draw a line in the sand soon then start working >>> on the backlog. I'll be encouraged if we can start shrinking the >>> BadApple'd tests. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > Begin forwarded message: > From: Erick Erickson <erickerick...@gmail.com> > Subject: Re: Not adding badapples this week. > Date: June 4, 2018 at 11:16:56 AM EDT > To: dev@lucene.apache.org > Reply-To: dev@lucene.apache.org > > Adrien: > > "Do you know whether there is something that makes these jobs any different?" > > Unfortunately no. I'm not really very well versed in the various test > environments, maybe Uwe or Steve Rowe or Hoss might have some insight? > > On Mon, Jun 4, 2018 at 3:34 AM, Adrien Grand <jpou...@gmail.com> wrote: >> Thanks for helping on this front Erick. I noticed a significant decrease in >> noise since you started badapple-ing bad tests, but I'm observing that our >> smoke-release builds still keep failing because of Solr tests (10 out of the >> last 10 builds) in spite of the fact that they disable bad apples: >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/ >> - https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/ >> >> Do you know whether there is something that makes these jobs any different? >> >> Le mar. 29 mai 2018 à 18:12, Erick Erickson <erickerick...@gmail.com> a >> écrit : >>> >>> With the long weekend and the fact that the number of non-BadApple >>> tests is fairly small this week, I'll skip adding more BadApple tests >>> until next week. >>> >>> We're scarily close to the non-BaApple'd tests coming under control. >>> If we're lucky, we can draw a line in the sand soon then start working >>> on the backlog. I'll be encouraged if we can start shrinking the >>> BadApple'd tests. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
