MoveReplicaHDFSTest.test
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
TestModelManagerPersistence
all fail more than 10%, MoveReplicaHDFSTest 50%.
BasicAuthIntegrationTest.testBasicAuth comes in at just under 10%.
Short form:
There were 147 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are
objections
Failures in the last 4 reports..
Report Pct runs fails test
0123 8.3 913 44 BasicAuthIntegrationTest.testBasicAuth
0123 0.5 943 9
DimensionalRoutedAliasUpdateProcessorTest.testCatTime
0123 2.8 943 15
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
0123 10.4 954 84
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
0123 50.0 193 78 MoveReplicaHDFSTest.test
0123 3.2 911 13 RollingRestartTest.test
0123 3.6 103 5 ShardSplitTest.testSplitWithChaosMonkey
0123 6.8 929 45
TestCloudSearcherWarming.testRepFactor1LeaderStartup
0123 12.9 942 76
TestModelManagerPersistence.testFilePersistence
0123 11.4 938 71
TestModelManagerPersistence.testWrapperModelPersistence
0123 0.5 882 6
TestQueryingOnDownCollection.testQueryToDownCollectionShouldFailFast
0123 1.4 913 20
TestSkipOverseerOperations.testSkipLeaderOperations
0123 1.0 899 12 TestStressLiveNodes.testStress
************ Will BadApple all tests above this line except ones listed at the
top**************
Full results:
DO NOT ENABLE LIST:
MoveReplicaHDFSTest.testFailedMove
MoveReplicaHDFSTest.testNormalFailedMove
TestControlledRealTimeReopenThread.testCRTReopen
TestICUNormalizer2CharFilter.testRandomStrings
TestICUTokenizerCJK
TestImpersonationWithHadoopAuth.testForwarding
TestLTRReRankingPipeline.testDifferentTopN
TestRandomChains
DO NOT ANNOTATE LIST
CdcrBidirectionalTest.testBiDir
IndexSizeTriggerTest.testMergeIntegration
IndexSizeTriggerTest.testMixedBounds
IndexSizeTriggerTest.testSplitIntegration
IndexSizeTriggerTest.testTrigger
InfixSuggestersTest.testShutdownDuringBuild
ShardSplitTest.test
ShardSplitTest.testSplitMixedReplicaTypes
ShardSplitTest.testSplitWithChaosMonkey
TestLatLonShapeQueries.testRandomBig
TestRandomChains.testRandomChainsWithLargeStrings
TestTriggerIntegration.testSearchRate
Processing file (History bit 3): HOSS-2019-11-11.csv
Processing file (History bit 2): HOSS-2019-11-04.csv
Processing file (History bit 1): HOSS-2019-10-28.csv
Processing file (History bit 0): HOSS-2019-10-21.csv
Number of AwaitsFix: 38 Number of BadApples: 11
Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0 had 66 failures
Week: 1 had 83 failures
Week: 2 had 56 failures
Week: 3 had 49 failures
**Annotated tests that didn't fail in the last 4 weeks.
**Tests removed from the next two lists because they were specified in
'doNotEnable' in the properties file
no tests removed
**Annotations will be removed from the following tests because they haven't
failed in the last 4 rollups.
**Methods: 4
FullSolrCloudDistribCmdsTest.test
SolrZkClientTest.testSimpleUpdateACLs
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
TestDistributedStatsComponentCardinality.test
********Failures in Hoss' reports for the last 4 rollups.
There were 147 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are
objections
Failures in the last 4 reports..
Report Pct runs fails test
0123 8.3 913 44 BasicAuthIntegrationTest.testBasicAuth
0123 0.5 943 9
DimensionalRoutedAliasUpdateProcessorTest.testCatTime
0123 2.8 943 15
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
0123 10.4 954 84
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
0123 50.0 193 78 MoveReplicaHDFSTest.test
0123 3.2 911 13 RollingRestartTest.test
0123 3.6 103 5 ShardSplitTest.testSplitWithChaosMonkey
0123 6.8 929 45
TestCloudSearcherWarming.testRepFactor1LeaderStartup
0123 12.9 942 76
TestModelManagerPersistence.testFilePersistence
0123 11.4 938 71
TestModelManagerPersistence.testWrapperModelPersistence
0123 0.5 882 6
TestQueryingOnDownCollection.testQueryToDownCollectionShouldFailFast
0123 1.4 913 20
TestSkipOverseerOperations.testSkipLeaderOperations
0123 1.0 899 12 TestStressLiveNodes.testStress
************ Will BadApple all tests above this line except ones listed at the
top**************
012 0.5 721 4
ReindexCollectionTest.testSameTargetReindexing
012 1.0 742 13 SyncSliceTest.test
012 1.4 734 8 SystemCollectionCompatTest.testBackCompat
012 1.0 714 4
TestSolrCloudWithDelegationTokens.testDelegationTokenRenew
012 1.5 564 8 TestTlogReplica.testKillTlogReplica
012 3.4 88 10
TestXYMultiPolygonShapeQueries.testRandomBig
01 3 4.0 70 4 HdfsBasicDistributedZkTest.test
01 3 2.9 689 9 TestInPlaceUpdatesDistrib.test
01 1.1 468 3
TestDelegationWithHadoopAuth.testDelegationTokenRenew
01 0.5 518 3 TestPackedInts.testPackedLongValues
01 0.5 471 3 TestTlogReplica.testOnlyLeaderIndexes
01 6.1 67 3
TimeRoutedAliasUpdateProcessorTest.testDateMathInStart
01 37.6 453 201
ZkShardTermsTest.testCoreRemovalWhileRecovering
0 2 1.4 418 6
MissingSegmentRecoveryTest.testLeaderRecovery
0 2 23.3 51 8 StressHdfsTest.test
0 2 0.5 296 3
TestTlogReplica.testOutOfOrderDBQWithInPlaceUpdates
0 2 0.5 399 2 TestV2Request.testHttpSolrClient
0 3 3.8 48 2
CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
0 3 15.4 47 5
HdfsTlogReplayBufferedWhileIndexingTest.test
0 3 10.0 54 8 HdfsWriteToMultipleCollectionsTest.test
0 3 0.5 393 7 SplitShardTest.testLiveSplit
0 0.5 205 1
ChaosMonkeySafeLeaderWithPullReplicasTest.test
0 1.0 206 2
ConcurrentUpdateHttp2SolrClientTest.testConcurrentUpdate
0 3.4 29 1 HdfsBasicDistributedZk2Test.test
0 3.8 26 1
HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
0 0.5 202 1 LeaderVoteWaitTimeoutTest.basicTest
0 0.5 207 1 OverviewImplTest.testIsOptimized
0 0.5 207 1
TestConcurrentMergeScheduler.testFlushExceptions
0 1.0 105 1
TestDirectDocValuesFormat.testSortedNumericsMultipleValuesVsStoredFields
0 0.5 204 1
TestDocumentsWriterStallControl.testAccquireReleaseRace
0 0.5 206 1
TestPullReplicaErrorHandling.testPullReplicaDisconnectsFromZooKeeper
0 0.5 206 1
TestReplicationHandler.doTestIndexFetchWithMasterUrl
0 1.4 207 3
TestSearcherManager.testConcurrentIndexCloseSearchAndRefresh
0 0.5 199 1
TestSimTriggerIntegration.testNodeLostTriggerRestoreState
0 2.4 206 5 TestStressCloudBlindAtomicUpdates.test_dv
0 2.4 206 5
TestStressCloudBlindAtomicUpdates.test_dv_idx
0 2.4 206 5
TestStressCloudBlindAtomicUpdates.test_dv_stored
0 2.4 206 5
TestStressCloudBlindAtomicUpdates.test_dv_stored_idx
0 2.4 206 5
TestStressCloudBlindAtomicUpdates.test_stored_idx
0 1.0 203 2 TestTlogReplica.testDeleteById
0 3.4 29 1 TestXYPolygonShapeQueries.testRandomBig
123 0.6 707 5 MathExpressionTest.testGammaDistribution
123 3.4 72 4 Test2BPostings.test
12 0.3 505 2 AutoScalingHandlerTest.testReadApi
12 3.6 51 2 HdfsChaosMonkeyNothingIsSafeTest.test
12 0.6 519 3
HealthCheckHandlerTest.testHealthCheckHandler
12 0.4 412 3
TestDistribPackageStore.testPackageStoreManagement
12 0.3 515 2 TestLBHttpSolrClient.testReliability
12 0.3 504 2
TestPullReplicaErrorHandling.testCantConnectToLeader
1 3 0.3 495 3 HttpPartitionTest.test
1 3 0.9 496 4 HttpPartitionWithTlogReplicasTest.test
1 0.3 312 1 AssignBackwardCompatibilityTest.test
1 0.3 311 1
CdcrVersionReplicationTest.testCdcrDocVersions
1 1.0 314 3
CollectionsAPISolrJTest.testCreateCollWithDefaultClusterPropertiesOldFormat
1 0.3 321 1 HealthCheckHandlerTest.testHealthCheckV2Api
1 1.6 322 5 LeaderFailoverAfterPartitionTest.test
1 0.3 311 1 PeerSyncReplicationTest.test
1 0.3 317 1 ShardSplitTest.testSplitShardWithRule
1 0.3 317 1 ShardSplitTest.testSplitShardWithRuleLink
1 22.3 130 29
SolrPluginUtilsTest.testMinShouldMatchBadQueries
1 2.2 323 7
TestBadConfig.testSchemaMutableButNotManaged
1 0.3 314 1 TestIndexFileDeleter.testExcInDecRef
1 0.3 313 1 TestIndexWriterDelete.testDeletesOnDiskFull
1 0.3 314 1
TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit
1 0.3 313 1 TestLFUCache.testTimeDecayParams
1 0.3 312 1
TestLucene80DocValuesFormat.testSortedNumericBlocksOfVariousBitsPerValue
1 0.3 313 1
TestLucene80DocValuesFormat.testTermsEnumLongSharedPrefixes
1 0.3 312 1
TestLucene80DocValuesFormat.testTermsEnumVariableWidth
1 0.3 315 1 TestOfflineSorter.testThreadSafety
1 0.3 314 1
TestPerFieldDocValuesFormat.testSparseIntNumericsVsStoredFields
1 0.3 315 1 TestPrepRecovery.testLeaderNotResponding
1 0.3 315 1 TestPrepRecovery.testLeaderUnloaded
1 0.3 313 1
TestRandomChains.testRandomChainsWithLargeStrings
1 0.3 310 1 TestRecovery.stressLogReplay
1 0.7 267 2 TestSimScenario.testSuggestions
1 0.3 309 1
TestSimTriggerIntegration.testTriggerThrottling
1 0.7 268 2 TestTlogReplica.testRecovery
1 2.4 328 8
UpdateRequestProcessorFactoryTest.testConfiguration
23 2.0 385 15 DistributedTermsComponentTest.test
2 1.0 203 2 ChaosMonkeyNothingIsSafeTest.test
2 0.5 197 1
CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection
2 5.3 19 1 HdfsSyncSliceTest.test
2 4.2 24 1 HdfsUnloadDistributedZkTest.test
2 0.5 203 1 MathExpressionTest.testZipFDistribution
2 1.4 209 3 OverseerRolesTest.testOverseerRole
2 0.5 198 1 PeerSyncTest.test
2 2.6 196 5
SpellCheckCollatorTest.testEstimatedHitCounts
2 11.7 213 25 TestDistributedGrouping.test
2 0.5 200 1
TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
2 0.5 196 1
TestLucene70NormsFormat.testFewSparseLargeValues
2 0.5 197 1
TestPullReplicaErrorHandling.testCantConnectToPullReplica
2 0.5 198 1
TestReplicationHandler.doTestStressReplication
2 6.5 93 6 TestTlogReplica.testAddDocs
2 2.2 93 2 TestTlogReplica.testRemoveLeader
2 0.5 196 1 UnloadDistributedZkTest.test
3 1.6 186 3 BasicAuthOnSingleNodeTest.basicTest
3 0.6 177 1
CollectionsAPIAsyncDistributedZkTest.testAsyncRequests
3 0.6 176 1
ConfusionMatrixGeneratorTest.testGetConfusionMatrixWithBM25NB
3 0.6 179 1 LegacyNoFacetCloudTest.uniqueTest
3 0.1 1790 1
RandomGeoPolygonTest.testCompareSmallPolygons
3 0.6 178 1 ReplicationFactorTest.test
3 0.6 176 1 TestIndexWriterDelete.testUpdatesOnDiskFull
3 0.6 179 1
TestIndexingSequenceNumbers.testStressConcurrentCommit
3 0.6 177 1
TestLucene80DocValuesFormat.testRandomAdvanceNumeric
3 0.6 181 1 TestSQLHandler.testBasicGroupingFacets
3 0.6 177 1
TestSnapshotCloudManager.testSimulatorFromSnapshot
***********************Annotations report
Number of AwaitsFix: 38 Number of BadApples: 11
*****************AwaitsFix Annotations:
************Lucene AwaitsFix
TestControlledRealTimeReopenThread.testCRTReopen
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737";)
TestICUNormalizer2CharFilter.testRandomStrings
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595";)
TestICUTokenizerCJK.TestICUTokenizerCJK suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222";)
TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161";)
************Solr AwaitsFix
AutoAddReplicasIntegrationTest.testRapidStopStartStopWithPropChange
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13811";)
CdcrBidirectionalTest.testBiDir
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12524";)
CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";)
CollectionsAPISolrJTest.testCreateAndDeleteShard
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13021";)
ComputePlanActionTest.testNodeLost
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
ConcurrentCreateRoutedAliasTest.ConcurrentCreateRoutedAliasTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-12386";) // "Can't find resource"
LeaderTragicEventTest.LeaderTragicEventTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13237";)
MoveReplicaHDFSTest.testFailedMove
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12080";) //
added 03-Oct-2018
ReplaceNodeNoTargetTest.ReplaceNodeNoTargetTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067";)
ReplaceNodeNoTargetTest.test
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067";)
SSLMigrationTest.SSLMigrationTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
17-Mar-2018
SaslZkACLProviderTest.testSaslZkACLProvider
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13075";)
ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
17-Mar-2018
ScheduledTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
added 20-Sep-2018
SearchRateTriggerIntegrationTest.testDeleteNode
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13163";)
SearchRateTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
SolrExporterIntegrationTest.SolrExporterIntegrationTest suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13786";)
Tagger2Test.testStopWords
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-8344";)
TestCollapseQParserPlugin.testStringCollapse
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974";)
TestHierarchicalDocBuilder.testThreeLevelHierarchy
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12801";) //
this test fails easily under beasting
TestImpersonationWithHadoopAuth.testForwarding
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893";)
TestJsonFacetRefinement.testProcessEmptyRefinement
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556";)
TestJsonFacetRefinement.testSortedSubFacetRefinementWhenParentOnlyReturnedByOneShardProcessEmpty
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556";)
TestLTRReRankingPipeline.testDifferentTopN
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134";)
TestMinMaxOnMultiValuedField.testDoubleFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testFloatFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testIntFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testLongFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestPullReplica.TestPullReplica suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestSimComputePlanAction.testNodeLost
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";) //
if you beast this, eventually you will see
TestSimExtremeIndexing.TestSimExtremeIndexing suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12923";)
TestSimLargeCluster.testBasic
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
this test hits a timeout easily
TestStressRecovery.TestStressRecovery suite
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// 6-Sep-2018
TriLevelCompositeIdRoutingTest.test
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13369";)
*****************BadApple Annotations:
************Lucene BadApple
************Solr BadApple
FullSolrCloudDistribCmdsTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
HdfsNNFailoverTest.HdfsNNFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// 12-Jun-2018
HdfsRecoveryZkTest.HdfsRecoveryZkTest suite
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
MoveReplicaHDFSTest.testNormalFailedMove
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
14-Oct-2018
MultiThreadedOCPTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// added 20-Jul-2018
SolrZkClientTest.testSimpleUpdateACLs
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
6-Sep-2018
TestDistributedStatsComponentCardinality.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
14-Oct-2018
TimeRoutedAliasUpdateProcessorTest.TimeRoutedAliasUpdateProcessorTest suite
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-13059";)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
