Continued improvement I think. Or at least the improvements 3 weeks ago are working their way through the system. Note that the number of tests that _only_ failed three weeks ago is almost half the total. So I have some optimism that next week we’ll see a further large drop.
Here’s the synopsis, full report attached: Number of AwaitsFix: 38 Number of BadApples: 11 Raw fail count by week totals, most recent week first (corresponds to bits): Week: 0 had 28 failures Week: 1 had 21 failures Week: 2 had 47 failures Week: 3 had 142 failures **Annotated tests that didn't fail in the last 4 weeks. **Tests removed from the next two lists because they were specified in 'doNotEnable' in the properties file MoveReplicaHDFSTest.testNormalFailedMove **Annotations will be removed from the following tests because they haven't failed in the last 4 rollups. **Methods: 3 SolrZkClientTest.testSimpleUpdateACLs TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader TestDistributedStatsComponentCardinality.test ********Failures in Hoss' reports for the last 4 rollups. There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected These tests were NOT BadApple'd or AwaitsFix'd All tests that failed 4 weeks running will be BadApple'd unless there are objections Failures in the last 4 reports.. Report Pct runs fails test 0123 1.3 597 11 AliasIntegrationTest.testClusterStateProviderAPI 0123 26.7 65 17 HdfsAutoAddReplicasIntegrationTest.testSimple 0123 1.0 686 9 HttpPartitionWithTlogReplicasTest.test ************ Will BadApple all tests above this line except ones listed at the top**************
DO NOT ENABLE LIST: MoveReplicaHDFSTest.testFailedMove MoveReplicaHDFSTest.testNormalFailedMove TestControlledRealTimeReopenThread.testCRTReopen TestICUNormalizer2CharFilter.testRandomStrings TestICUTokenizerCJK TestImpersonationWithHadoopAuth.testForwarding TestLTRReRankingPipeline.testDifferentTopN TestRandomChains DO NOT ANNOTATE LIST CdcrBidirectionalTest.testBiDir IndexSizeTriggerTest.testMergeIntegration IndexSizeTriggerTest.testMixedBounds IndexSizeTriggerTest.testSplitIntegration IndexSizeTriggerTest.testTrigger InfixSuggestersTest.testShutdownDuringBuild ShardSplitTest.test ShardSplitTest.testSplitMixedReplicaTypes ShardSplitTest.testSplitWithChaosMonkey TestLatLonShapeQueries.testRandomBig TestRandomChains.testRandomChainsWithLargeStrings TestTriggerIntegration.testSearchRate Processing file (History bit 3): HOSS-2019-088-05.csv Processing file (History bit 2): HOSS-2019-08-12.csv Processing file (History bit 1): HOSS-2019-07-29.csv Processing file (History bit 0): HOSS-2019-07-08.csv Number of AwaitsFix: 38 Number of BadApples: 11 Raw fail count by week totals, most recent week first (corresponds to bits): Week: 0 had 28 failures Week: 1 had 21 failures Week: 2 had 47 failures Week: 3 had 142 failures **Annotated tests that didn't fail in the last 4 weeks. **Tests removed from the next two lists because they were specified in 'doNotEnable' in the properties file MoveReplicaHDFSTest.testNormalFailedMove **Annotations will be removed from the following tests because they haven't failed in the last 4 rollups. **Methods: 3 SolrZkClientTest.testSimpleUpdateACLs TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader TestDistributedStatsComponentCardinality.test ********Failures in Hoss' reports for the last 4 rollups. There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected These tests were NOT BadApple'd or AwaitsFix'd All tests that failed 4 weeks running will be BadApple'd unless there are objections Failures in the last 4 reports.. Report Pct runs fails test 0123 1.3 597 11 AliasIntegrationTest.testClusterStateProviderAPI 0123 26.7 65 17 HdfsAutoAddReplicasIntegrationTest.testSimple 0123 1.0 686 9 HttpPartitionWithTlogReplicasTest.test ************ Will BadApple all tests above this line except ones listed at the top************** 012 1.4 276 3 TestCloudJSONFacetJoinDomain.testRandom 0 23 6.2 685 176 BasicAuthIntegrationTest.testBasicAuth 0 23 16.7 49 5 CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates 0 23 1.4 496 3 RollingRestartTest.test 0 23 1.5 500 10 TestStressLiveNodes.testStress 0 23 1.4 525 8 TestUseDocValuesAsStored.testDuplicateMultiValued 0 2 4.0 252 7 TestRandomChains.testRandomChains 0 2 3.0 252 7 TestRandomChains.testRandomChainsWithLargeStrings 0 3 2.0 434 5 CollectionPropsTest.testWatcher 0 3 1.0 426 4 HttpPartitionTest.test 0 3 1.4 368 3 LeaderVoteWaitTimeoutTest.basicTest 0 3 2.7 389 22 RulesTest.doIntegrationTest 0 3 1.3 374 2 StatsReloadRaceTest.testParallelReloadAndStats 0 3 16.7 38 2 StressHdfsTest.test 0 3 1.0 427 2 TestLucene80DocValuesFormat.testTermsEnumVariableWidth 0 8.3 12 1 CdcrReplicationHandlerTest.testPartialReplicationAfterPeerSync 0 1.5 67 1 DimensionalRoutedAliasUpdateProcessorTest.testTimeCat 0 12.5 8 1 HdfsWriteToMultipleCollectionsTest.test 0 1.3 75 1 OverseerTest.testShardLeaderChange 0 1.0 96 1 TestRunWithRestrictedPermissions.testCompletelyForbidden1 0 1.0 96 1 TestRunWithRestrictedPermissions.testCompletelyForbidden2 0 1.0 96 1 TestRunWithRestrictedPermissions.testNormallyAllowedStuff 123 6.7 80 12 ShardSplitTest.testSplitWithChaosMonkey 123 3.7 518 8 SystemCollectionCompatTest.testBackCompat 12 6.3 216 9 TestCloudJSONFacetSKG.testRandom 1 1.0 100 1 AuditLoggerIntegrationTest.testQueuedTimeMetric 1 1.5 68 1 CollectionsAPIDistributedZkTest.testCollectionsAPI 1 53.3 15 8 ShardSplitTest.test 1 1.4 71 1 TestCollectionStateWatchers.testLiveNodeChangesTriggerWatches 1 1.0 97 1 TestDemoParallelLeafReader.testBasic 1 1.0 97 1 TestDemoParallelLeafReader.testRandomMultipleSchemaGens 1 1.3 79 1 TestExportWriter.testIndexOrder 1 1.3 76 1 TestLBHttp2SolrClient.testTwoServers 1 1.3 76 1 TestLBHttpSolrClient.testReliability 1 11.1 9 1 TestXYPolygonShapeQueries.testRandomBig 1 1.4 73 1 TestZKPropertiesWriter.testZKPropertiesWriter 23 0.7 484 3 CollectionPropsTest.testReadWriteCached 23 1.3 482 5 LeaderFailoverAfterPartitionTest.test 23 0.8 440 24 LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud 23 1.6 426 5 MetricsHistoryWithAuthIntegrationTest.testValuesAreCollected 23 9.1 43 2 Test2BPostings.test 23 0.8 398 2 TestDelegationWithHadoopAuth.testDelegationTokenRenew 23 2.0 435 4 TestDynamicLoadingUrl.testDynamicLoadingUrl 23 0.7 479 2 TestIndexingSequenceNumbers.testStressConcurrentCommit 23 0.8 400 3 TestJaegerConfigurator.testInjected 23 0.8 415 2 TestSolrCloudWithDelegationTokens.testDelegationTokenRenew 2 1.6 128 2 AutoScalingHandlerTest.testReadApi 2 0.8 122 1 LeaderElectionTest.testStressElection 2 2.3 87 2 SplitByPrefixTest.doTest 2 0.7 151 1 TestCachePurging.testBackgroundPurges 2 0.7 147 1 TestCloudSchemaless.test 2 7.6 172 13 TestContainerReqHandler.testPluginGlobalLoader 2 30.2 172 52 TestContainerReqHandler.testRuntimeLib 2 30.2 172 52 TestContainerReqHandler.testRuntimeLibWithSig2048 2 31.4 172 54 TestContainerReqHandler.testRuntimeLibWithSig512 2 0.9 117 1 TestInPlaceUpdatesDistrib.test 2 9.1 11 1 TestLucene80DocValuesFormat.testSortedVariableLengthManyVsStoredFields 2 0.7 153 1 TestSearcherManager.testSearcherManager 2 0.8 129 1 TestSimLargeCluster.testCreateLargeSimCollections 3 0.3 334 1 AuditLoggerIntegrationTest.testAsync 3 0.6 334 2 AuditLoggerIntegrationTest.testSynchronous 3 3.3 30 1 CdcrReplicationHandlerTest.testPartialReplicationWithTruncatedTlog 3 3.6 28 1 CdcrRequestHandlerTest.testLifeCycleActions 3 0.3 296 1 CloudExitableDirectoryReaderTest.test 3 0.2 658 1 DateRangePrefixTreeTest.testToStringISO8601 3 0.3 337 1 DeleteReplicaTest.deleteReplicaFromClusterState 3 6.5 337 22 DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica 3 7.1 337 24 DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplicaLegacy 3 0.3 299 1 DistributedQueueTest.testPeekElements 3 3.3 30 1 ForceLeaderTest.testReplicasInLowerTerms 3 3.0 33 1 ForceLeaderWithTlogReplicasTest.testReplicasInLowerTerms 3 3.7 27 1 HdfsBasicDistributedZk2Test.test 3 10.7 28 3 HdfsBasicDistributedZkTest.test 3 4.2 24 1 HdfsChaosMonkeySafeLeaderTest.test 3 3.4 29 1 HdfsTlogReplayBufferedWhileIndexingTest.test 3 0.3 286 1 IndexSizeTriggerTest.testEstimatedIndexSize 3 0.3 286 1 IndexSizeTriggerTest.testMixedBounds 3 0.3 286 1 IndexSizeTriggerTest.testSplitIntegration 3 0.3 329 1 JWTAuthPluginIntegrationTest.testMetrics 3 0.3 297 1 MathExpressionTest.testGammaDistribution 3 0.3 295 1 MissingSegmentRecoveryTest.testLeaderRecovery 3 0.3 302 1 MoveReplicaTest.test 3 0.3 298 1 NestedShardedAtomicUpdateTest.test 3 0.3 291 1 OverseerRolesTest.testOverseerRole 3 0.9 328 3 PeerSyncTest.test 3 0.3 293 1 ReindexCollectionTest.testLossySchema 3 8.9 369 33 ReplicationFactorTest.test 3 0.9 316 3 RulesTest.testHostFragmentRule 3 1.6 316 5 RulesTest.testInvokeApi 3 1.3 316 4 RulesTest.testModifyColl 3 1.6 316 5 RulesTest.testPortRule 3 0.9 316 3 RulesTest.testPortRuleInPresenceOfClusterPolicy 3 0.3 297 1 SearchHandlerTest.testRequireZkConnected 3 0.3 343 1 ShardSplitTest.testSplitShardWithRule 3 18.2 11 2 SharedFSAutoReplicaFailoverTest.test 3 0.3 296 1 SimpleMLTQParserTest.doTest 3 0.3 299 1 SolrRrdBackendFactoryTest.testBasic 3 0.3 290 1 SpellCheckCollatorTest.testEstimatedHitCounts 3 2.8 179 5 StreamDecoratorTest.testClassifyStream 3 0.3 295 1 TestBuiltInEvaluators.testDateFormatEvaluator 3 3.2 308 10 TestCloudSearcherWarming.testRepFactor1LeaderStartup 3 0.3 294 1 TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete 3 3.2 31 1 TestCompressingStoredFieldsFormat.testBigDocuments 3 4.9 308 15 TestConfigSetsAPI.testUserAndTestDefaultConfigsetsAreSame 3 0.7 294 2 TestDistribIDF.testSimpleQuery 3 0.3 331 1 TestDocValuesRewriteMethod.testRegexps 3 3.2 31 1 TestDuelingCodecsAtNight.testEquals 3 0.7 305 2 TestDynamicLoading.testDynamicLoading 3 1.3 297 4 TestExportWriter.testStringWithCase 3 0.6 316 2 TestFeatureSort.testFeatureMissingFeatureNameInSegment 3 0.9 316 3 TestFeatureSort.testFeatureMissingFieldInSegment 3 0.3 329 1 TestFieldCacheRewriteMethod.testRegexps 3 0.3 331 1 TestIndexWriterDelete.testUpdatesOnDiskFull 3 0.3 330 1 TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull 3 1.2 331 4 TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom 3 0.3 329 1 TestLucene80DocValuesFormat.testRandomAdvanceNumeric 3 0.3 330 1 TestLucene80DocValuesFormat.testSparseDocValuesVsStoredFields 3 0.3 329 1 TestLucene80DocValuesFormat.testTermsEnumFixedWidth 3 3.6 28 1 TestLucene80DocValuesFormat.testTermsEnumRandomMany 3 1.3 307 4 TestModelManagerPersistence.testFeaturePersistence 3 4.2 307 13 TestModelManagerPersistence.testFilePersistence 3 1.3 307 4 TestModelManagerPersistence.testInnerCustomClassesPersistence 3 2.6 302 8 TestModelManagerPersistence.testWrapperModelPersistence 3 0.3 286 1 TestNonDefinedSimilarityFactory.testCurrentBM25FromV8 3 0.6 328 2 TestPackedInts.testPackedLongValues 3 3.0 363 11 TestPolicyCloud.testCreateCollection 3 15.7 363 57 TestPolicyCloud.testCreateCollectionAddReplica 3 0.3 361 1 TestPolicyCloud.testCreateCollectionAddShardUsingPolicy 3 1.7 363 6 TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy 3 13.2 363 48 TestPolicyCloud.testCreateCollectionSplitShard 3 5.0 361 18 TestPolicyCloud.testDataProvider 3 1.1 361 4 TestPolicyCloud.testMetricsTag 3 10.2 324 33 TestRamUsageEstimator.testMap 3 0.3 346 1 TestReplicationHandler.doTestIndexAndConfigReplication 3 0.3 346 1 TestReplicationHandler.testEmptyCommits 3 0.3 329 1 TestReplicationHandlerBackup.doTestBackup 3 0.7 296 2 TestSimLargeCluster.testAddNode 3 0.3 291 1 TestSimTriggerIntegration.testNodeMarkersRegistration 3 0.3 291 1 TestSimTriggerIntegration.testTriggerThrottling 3 1.0 295 3 TestSkipOverseerOperations.testSkipLeaderOperations 3 0.7 287 2 TestSnapshotCloudManager.testSimulatorFromSnapshot 3 0.3 293 1 TestSolrDeletionPolicy1.testNumCommitsConfigured 3 0.3 328 1 TestSpanSearchEquivalence.testSpanBoostQuerySimplification 3 0.3 328 1 TestSpanSearchEquivalence.testSpanTermVersusTerm 3 0.9 323 3 TestUseDocValuesAsStored.testMultipleSearchResults 3 3.8 290 11 TestWrapperModel.testMethodOverridesAndDelegation 3 0.7 290 2 UninvertDocValuesMergePolicyTest.testIndexAndAddDocValues ***********************Annotations report Number of AwaitsFix: 38 Number of BadApples: 11 *****************AwaitsFix Annotations: ************Lucene AwaitsFix TestControlledRealTimeReopenThread.testCRTReopen @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737") TestICUNormalizer2CharFilter.testRandomStrings @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595") TestICUTokenizerCJK.TestICUTokenizerCJK suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222") TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161") ************Solr AwaitsFix CdcrBidirectionalTest.testBiDir @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12524") CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") CollectionsAPISolrJTest.testCreateAndDeleteShard @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13021") ComputePlanActionTest.testNodeLost @LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") ConcurrentCreateRoutedAliasTest.ConcurrentCreateRoutedAliasTest suite @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12386") // "Can't find resource" LeaderTragicEventTest.LeaderTragicEventTest suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13237") MoveReplicaHDFSTest.testFailedMove @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12080") // added 03-Oct-2018 NodeMarkersRegistrationTest.testNodeMarkersRegistration @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13376") ReplaceNodeNoTargetTest.ReplaceNodeNoTargetTest suite @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11067") ReplaceNodeNoTargetTest.test @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11067") SSLMigrationTest.SSLMigrationTest suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 17-Mar-2018 SaslZkACLProviderTest.testSaslZkACLProvider @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13075") ScheduledMaintenanceTriggerTest.testInactiveShardCleanup @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 17-Mar-2018 ScheduledTriggerTest.testTrigger @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Sep-2018 SearchRateTriggerIntegrationTest.testDeleteNode @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13163") SearchRateTriggerTest.testTrigger @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") Tagger2Test.testStopWords @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-8344") TestCollapseQParserPlugin.testStringCollapse @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974") TestHierarchicalDocBuilder.testThreeLevelHierarchy @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12801") // this test fails easily under beasting TestImpersonationWithHadoopAuth.testForwarding @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893") TestJsonFacetRefinement.testProcessEmptyRefinement @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556") TestJsonFacetRefinement.testSortedSubFacetRefinementWhenParentOnlyReturnedByOneShardProcessEmpty @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556") TestLTRReRankingPipeline.testDifferentTopN @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134") TestMinMaxOnMultiValuedField.testDoubleFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testFloatFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testIntFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testLongFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestPullReplica.TestPullReplica suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") TestSimComputePlanAction.testNodeLost @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") // if you beast this, eventually you will see TestSimExtremeIndexing.TestSimExtremeIndexing suite @org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12923") TestSimLargeCluster.testBasic @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // this test hits a timeout easily TestStressRecovery.TestStressRecovery suite @LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 TestTlogReplica.TestTlogReplica suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12313") TriLevelCompositeIdRoutingTest.test @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13369") *****************BadApple Annotations: ************Lucene BadApple ************Solr BadApple FullSolrCloudDistribCmdsTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 HdfsNNFailoverTest.HdfsNNFailoverTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 12-Jun-2018 HdfsRecoveryZkTest.HdfsRecoveryZkTest suite @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") MoveReplicaHDFSTest.testNormalFailedMove @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 MultiThreadedOCPTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Jul-2018 SolrZkClientTest.testSimpleUpdateACLs @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 TestDistributedStatsComponentCardinality.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TimeRoutedAliasUpdateProcessorTest.TimeRoutedAliasUpdateProcessorTest suite @LuceneTestCase.BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13059")
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org