Sorry guys for this aweful email. Looks like Apache converted my nicely illustrated email into plain text. :(
Maybe I could attach the test reports as images, but I think you already got the idea. Andor On 07/18/2018 05:42 PM, Andor Molnar wrote: > Hi, > > *branch-3.4* > > I've taken a quick look at our Jenkins builds and in terms of flaky tests, > it looks like branch-3.4 is in a pretty good shape. The build hasn't failed > for 5-6 days on all JDKs which I think is pretty awesome. > > *branch-3.5* > > This branch is in very bad condition. Which is quite unfortunate given > we're in the middle of stabilising it. :) > Especially on JDK8, last successful build was 11 days ago. JDK9 (50% > failing) and JDK10 (30% failing) are looking better in the last 10 builds. > > Interestingly (apart from a few quite rare ones) it looks there's only 1 > test which is quite nasty on this branch: testManyChildWatchersAutoReset > > There's a Jira about fixing it and a fix has been merged by increasing the > timeout of the test, but having a bug on the branch is also possible > causing the test to fail even with 10 min timeout. > > I wasn't able to repro the failing test on my machine (Mac and CentOS7), it > always finished in 30-40 seconds maximum. On jenkins slaves it shows the > following: > > *JDK 8:* > > Report creation timed out. > > > *JDK 9:* > > New Failures > Chart > See children > Build Number ⇒ > Package-Class-Testmethod names ⇓ > 351 > 350 > 349 > 348 > 347 > 346 > 345 > 344 > 343 > 342 > 341 > 340 > 339 > 338 > 337 > 336 > 335 > 334 > testManyChildWatchersAutoReset > 45.604 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/351/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.337 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/350/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 21.904 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/349/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 583.063 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/348/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.325 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/347/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.383 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/346/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.362 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/345/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 21.139 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/344/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 24.031 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/343/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 584.200 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/342/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.327 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/341/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.323 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/340/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 23.737 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/339/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.406 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/338/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 547.004 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/337/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.393 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/336/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > N/A > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/test_results_analyzer/> > 373.955 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java9/334/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > > > *JDK 10:* > > > New Failures > Chart > See children > Build Number ⇒ > Package-Class-Testmethod names ⇓ > 110 > 109 > 108 > 107 > 106 > 105 > 104 > 103 > 102 > 101 > 100 > 99 > 98 > 97 > 96 > 95 > 94 > 93 > 92 > testManyChildWatchersAutoReset > 364.945 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/110/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 543.983 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/109/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 388.182 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/108/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.446 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/107/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.025 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/106/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 535.046 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/105/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.306 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/104/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 474.005 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/103/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 560.925 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/102/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.328 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/101/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 558.547 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/100/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.397 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/99/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.414 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/98/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 430.383 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/97/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 564.064 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/96/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 600.357 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/95/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 432.435 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/94/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 596.378 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/93/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > 39.242 > <https://builds.apache.org/view/S-Z/view/ZooKeeper/job/ZooKeeper_branch35_java10/92/testReport/org.apache.zookeeper.test/DisconnectedWatcherTest/testManyChildWatchersAutoReset> > > > It takes ages to complete on Jenkins for some reason and it looks like it > ends quite frequently close to the limit, so I suspect it should succeed > eventually if we were to increase the timeout even more. But is that > correct? > Bug or infrastructure issue? > > *master / 3.6* > > Pretty much the same as 3.5. I haven't seen testManyChildWatchersAutoReset > failing on this branch with JDK8 which is a bit confusing, but other then > that I see the same pattern on JDK9 and JDK10. Unable to generate the above > reports here, because Test Result Analyzer keep timeouting for me, but I'll > follow-up when I have them. > > Btw. Flaky Test report has been broken for 10 days, I'm going to raise a > ticket on that if somebody willing to fix it. (I'm planning to do so.) > It would be nice to see the report working again, because if my > observations are correct, we don't have too many annoying tests apart from > the one mentioned. > > Thanks, > Andor >
