It’s been a while. I think this is mostly informational. I was all excited when
the reports were getting soooo much better, but that was an artifact of some
test environments not being up and running.
When Mark’s test work hits, we’ll probably have to start over.
That said, people SHOULD LOOK HERE PERIODICALLY:
http://fucit.org/solr-jenkins-reports/failure-report.html
For instance, TestPackages has a 76% failure rate over the last week.
Here’s the top failures. I’m not going to annotate for a while.
There were 141 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
Failures in the last 4 reports..
Report Pct runs fails test
0123 5.0 849 47 BasicAuthIntegrationTest.testBasicAuth
0123 0.5 884 13
DimensionalRoutedAliasUpdateProcessorTest.testCatTime
0123 2.3 884 15
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
0123 18.0 873 69
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
0123 0.5 832 5 MathExpressionTest.testGammaDistribution
0123 3.9 852 37
TestCloudSearcherWarming.testRepFactor1LeaderStartup
0123 10.8 855 64
TestModelManagerPersistence.testFilePersistence
0123 11.2 857 66
TestModelManagerPersistence.testWrapperModelPersistence
**DO NOT ENABLE LIST:
MoveReplicaHDFSTest.testFailedMove
MoveReplicaHDFSTest.testNormalFailedMove
TestControlledRealTimeReopenThread.testCRTReopen
TestICUNormalizer2CharFilter.testRandomStrings
TestICUTokenizerCJK
TestImpersonationWithHadoopAuth.testForwarding
TestLTRReRankingPipeline.testDifferentTopN
TestRandomChains
DO NOT ANNOTATE LIST
CdcrBidirectionalTest.testBiDir
IndexSizeTriggerTest.testMergeIntegration
IndexSizeTriggerTest.testMixedBounds
IndexSizeTriggerTest.testSplitIntegration
IndexSizeTriggerTest.testTrigger
InfixSuggestersTest.testShutdownDuringBuild
ShardSplitTest.test
ShardSplitTest.testSplitMixedReplicaTypes
ShardSplitTest.testSplitWithChaosMonkey
TestLatLonShapeQueries.testRandomBig
TestRandomChains.testRandomChainsWithLargeStrings
TestTriggerIntegration.testSearchRate
Processing file (History bit 3): HOSS-2019-10-28.csv
Processing file (History bit 2): HOSS-2019-10-21.csv
Processing file (History bit 1): HOSS-2019-10-15.csv
Processing file (History bit 0): HOSS-2019-10-07.csv
Number of AwaitsFix: 40 Number of BadApples: 11
Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0 had 56 failures
Week: 1 had 49 failures
Week: 2 had 42 failures
Week: 3 had 69 failures
**Annotated tests that didn't fail in the last 4 weeks.
**Tests removed from the next two lists because they were specified in
'doNotEnable' in the properties file
no tests removed
**Annotations will be removed from the following tests because they haven't
failed in the last 4 rollups.
**Methods: 4
FullSolrCloudDistribCmdsTest.test
SolrZkClientTest.testSimpleUpdateACLs
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
TestDistributedStatsComponentCardinality.test
********Failures in Hoss' reports for the last 4 rollups.
There were 141 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are
objections
Failures in the last 4 reports..
Report Pct runs fails test
0123 5.0 849 47 BasicAuthIntegrationTest.testBasicAuth
0123 0.5 884 13
DimensionalRoutedAliasUpdateProcessorTest.testCatTime
0123 2.3 884 15
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
0123 18.0 873 69
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
0123 0.5 832 5 MathExpressionTest.testGammaDistribution
0123 3.9 852 37
TestCloudSearcherWarming.testRepFactor1LeaderStartup
0123 10.8 855 64
TestModelManagerPersistence.testFilePersistence
0123 11.2 857 66
TestModelManagerPersistence.testWrapperModelPersistence
************ Will BadApple all tests above this line except ones listed at the
top**************
012 3.7 58 3 ShardSplitTest.testSplitWithChaosMonkey
012 2.0 578 11
TestSkipOverseerOperations.testSkipLeaderOperations
01 3 4.3 49 4 Test2BPostings.test
01 3 0.5 619 5 TestStressLiveNodes.testStress
01 2.0 385 15 DistributedTermsComponentTest.test
01 34.2 77 27 MoveReplicaHDFSTest.test
01 1.0 376 3 RollingRestartTest.test
01 1.0 359 3
TestQueryingOnDownCollection.testQueryToDownCollectionShouldFailFast
0 23 1.5 650 5
MissingSegmentRecoveryTest.testLeaderRecovery
0 23 0.5 649 3
ReindexCollectionTest.testSameTargetReindexing
0 23 1.5 656 6 SystemCollectionCompatTest.testBackCompat
0 2 1.9 408 5 SyncSliceTest.test
0 2 1.3 391 59
TestDistribPackageStore.testPackageStoreManagement
0 2 0.5 394 2 UnloadDistributedZkTest.test
0 3 1.0 453 3 ChaosMonkeyNothingIsSafeTest.test
0 3 0.5 449 2 MathExpressionTest.testZipFDistribution
0 3 2.6 447 10
SpellCheckCollatorTest.testEstimatedHitCounts
0 0.5 194 1 AutoScalingHandlerTest.testReadApi
0 0.5 197 1
CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection
0 4.3 23 1 HdfsChaosMonkeyNothingIsSafeTest.test
0 5.3 19 1 HdfsSyncSliceTest.test
0 4.2 24 1 HdfsUnloadDistributedZkTest.test
0 0.5 198 1
HealthCheckHandlerTest.testHealthCheckHandler
0 1.4 209 3 OverseerRolesTest.testOverseerRole
0 0.5 198 1 PeerSyncTest.test
0 4.8 21 1 StressHdfsTest.test
0 11.7 213 25 TestDistributedGrouping.test
0 0.5 200 1
TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
0 0.5 199 1 TestLBHttpSolrClient.testReliability
0 0.5 196 1
TestLucene70NormsFormat.testFewSparseLargeValues
0 0.5 197 1
TestPullReplicaErrorHandling.testCantConnectToLeader
0 0.5 197 1
TestPullReplicaErrorHandling.testCantConnectToPullReplica
0 0.5 198 1
TestReplicationHandler.doTestStressReplication
0 0.5 196 1
TestSolrCloudWithDelegationTokens.testDelegationTokenRenew
0 6.5 93 6 TestTlogReplica.testAddDocs
0 3.2 93 3 TestTlogReplica.testKillTlogReplica
0 2.2 93 2
TestTlogReplica.testOutOfOrderDBQWithInPlaceUpdates
0 2.2 93 2 TestTlogReplica.testRemoveLeader
0 0.5 195 1 TestV2Request.testHttpSolrClient
0 23.1 26 6
TestXYMultiPolygonShapeQueries.testRandomBig
123 1.1 629 5 HttpPartitionTest.test
123 0.6 617 3 TestInPlaceUpdatesDistrib.test
12 0.6 396 3
ConfusionMatrixGeneratorTest.testGetConfusionMatrixWithBM25NB
12 4.8 29 2
HdfsTlogReplayBufferedWhileIndexingTest.test
12 0.6 383 4
TestSnapshotCloudManager.testSimulatorFromSnapshot
1 3 10.5 25 3 HdfsBasicDistributedZkTest.test
1 3 0.6 432 2 HttpPartitionWithTlogReplicasTest.test
1 3 0.6 433 2
TestIndexingSequenceNumbers.testStressConcurrentCommit
1 1.6 186 3 BasicAuthOnSingleNodeTest.basicTest
1 4.5 22 1
CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
1 0.6 177 1
CollectionsAPIAsyncDistributedZkTest.testAsyncRequests
1 20.8 24 5 HdfsWriteToMultipleCollectionsTest.test
1 0.6 179 1 LegacyNoFacetCloudTest.uniqueTest
1 0.1 1790 1
RandomGeoPolygonTest.testCompareSmallPolygons
1 0.6 178 1 ReplicationFactorTest.test
1 3.2 186 6 SplitShardTest.testLiveSplit
1 0.6 176 1 TestIndexWriterDelete.testUpdatesOnDiskFull
1 0.6 177 1
TestLucene80DocValuesFormat.testRandomAdvanceNumeric
1 0.6 181 1 TestSQLHandler.testBasicGroupingFacets
23 1.0 464 3 ShardSplitTest.testSplitShardWithRule
23 0.5 445 2
TestSimLargeCluster.testCreateLargeSimCollections
2 0.5 201 1
ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp
2 0.5 200 1 InfixSuggestersTest.testReloadDuringBuild
2 0.5 210 1 ShardSplitTest.testSplitLocking
2 1.0 199 2 TestCloudNestedDocsSort.test
2 0.5 199 1 TestLTROnSolrCloud.testSimpleQuery
2 0.5 216 1
TestMergeSchedulerExternal.testSubclassConcurrentMergeScheduler
2 25.0 4 1 TestPackages.testPackageAPI
2 50.0 4 2 TestPackages.testPluginFrompackage
2 25.0 4 1 TestPackages.testRuntimeLibWithSig2048
2 0.5 199 1
TestRequestForwarding.testMultiCollectionQuery
2 0.5 206 1
TestSnapshotCloudManager.testComplexSnapshot
2 0.5 200 1
TestSolrDeletionPolicy1.testNumCommitsConfigured
2 0.5 200 1 TestWithCollection.testAddReplicaWithPolicy
2 6.3 16 1
TimeRoutedAliasUpdateProcessorTest.testDateMathInStart
3 0.4 250 1 AliasIntegrationTest.testModifyPropertiesV2
3 0.4 249 1 AuditLoggerIntegrationTest.testAsync
3 0.4 249 1 AuditLoggerIntegrationTest.testSynchronous
3 0.4 249 1 BasicDistributedZkTest.test
3 0.4 248 1
CollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore
3 0.8 249 2 ExecutePlanActionTest.testExecute
3 0.4 252 1 MigrateRouteKeyTest.testMissingSplitKey
3 0.4 246 1 MoveReplicaTest.testFailedMove
3 0.0 2500 1 RandomizedTaggerTest.test
3 0.4 245 1 ShardRoutingTest.test
3 0.4 254 1 ShardSplitTest.testSplitAfterFailedSplit
3 0.4 254 1 ShardSplitTest.testSplitShardWithRuleLink
3 0.4 254 1
ShardSplitTest.testSplitStaticIndexReplicationLink
3 5.6 266 15 SolrInfoBeanTest.testCallMBeanInfo
3 0.8 119 1
StreamDecoratorTest.testParallelTerminatingDaemonUpdateStream
3 1.6 256 4 TestCloudJSONFacetJoinDomain.testRandom
3 1.6 255 4 TestCloudJSONFacetSKG.testRandom
3 0.4 253 1
TestConcurrentFacetedIndexing.testConcurrency
3 0.4 254 1
TestConcurrentMergeScheduler.testFlushExceptions
3 0.9 226 2
TestContainerReqHandler.testPluginFrompackage
3 0.4 243 1
TestDocCollectionWatcher.testWatchesWorkForStateFormat1
3 1.6 247 4
TestFieldCacheSortRandom.testRandomStringSort
3 1.6 252 4 TestGeo3DPoint.testGeo3DRelations
3 0.4 246 1
TestInPlaceUpdateWithRouteField.testUpdatingDocValuesWithRouteField
3 0.4 249 1 TestLFUCache.testSetLimits
3 0.8 252 2 TestLatLonDocValuesQueries.testAllLatEqual
3 0.4 251 1
TestLucene80DocValuesFormat.testSortedNumericBlocksOfVariousBitsPerValue
3 0.4 251 1
TestLucene80DocValuesFormat.testSparseSortedNumericBlocksOfVariousBitsPerValue
3 0.4 250 1 TestPrepRecovery.testLeaderUnloaded
3 0.8 251 2
TestRunWithRestrictedPermissions.testCompletelyForbidden1
3 0.8 251 2
TestRunWithRestrictedPermissions.testCompletelyForbidden2
3 0.8 251 2
TestRunWithRestrictedPermissions.testNormallyAllowedStuff
3 0.4 254 1
TestSloppyMath.testAgainstSlowVersionReasonable
***********************Annotations report
Number of AwaitsFix: 40 Number of BadApples: 11
*****************AwaitsFix Annotations:
************Lucene AwaitsFix
TestControlledRealTimeReopenThread.testCRTReopen
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737";)
TestICUNormalizer2CharFilter.testRandomStrings
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595";)
TestICUTokenizerCJK.TestICUTokenizerCJK suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222";)
TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161";)
************Solr AwaitsFix
AbstractAtomicUpdatesMultivalueTestBase.testMultivalueBinaryField
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13762";)
AutoAddReplicasIntegrationTest.testRapidStopStartStopWithPropChange
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13811";)
CdcrBidirectionalTest.testBiDir
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12524";)
CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";)
CollectionsAPISolrJTest.testCreateAndDeleteShard
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13021";)
ComputePlanActionTest.testNodeLost
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
ConcurrentCreateRoutedAliasTest.ConcurrentCreateRoutedAliasTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-12386";) // "Can't find resource"
LeaderTragicEventTest.LeaderTragicEventTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13237";)
MoveReplicaHDFSTest.testFailedMove
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12080";) //
added 03-Oct-2018
ReplaceNodeNoTargetTest.ReplaceNodeNoTargetTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067";)
ReplaceNodeNoTargetTest.test
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067";)
SSLMigrationTest.SSLMigrationTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
17-Mar-2018
SaslZkACLProviderTest.testSaslZkACLProvider
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13075";)
ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
17-Mar-2018
ScheduledTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
added 20-Sep-2018
SearchRateTriggerIntegrationTest.testDeleteNode
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13163";)
SearchRateTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
SolrExporterIntegrationTest.SolrExporterIntegrationTest suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13786";)
Tagger2Test.testStopWords
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-8344";)
TestCollapseQParserPlugin.testStringCollapse
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974";)
TestHierarchicalDocBuilder.testThreeLevelHierarchy
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12801";) //
this test fails easily under beasting
TestImpersonationWithHadoopAuth.testForwarding
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893";)
TestJsonFacetRefinement.testProcessEmptyRefinement
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556";)
TestJsonFacetRefinement.testSortedSubFacetRefinementWhenParentOnlyReturnedByOneShardProcessEmpty
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556";)
TestLTRReRankingPipeline.testDifferentTopN
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134";)
TestMinMaxOnMultiValuedField.testDoubleFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testFloatFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testIntFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.testLongFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestPackages.TestPackages suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13822";)
// leaks files
TestPullReplica.TestPullReplica suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestSimComputePlanAction.testNodeLost
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";) //
if you beast this, eventually you will see
TestSimExtremeIndexing.TestSimExtremeIndexing suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12923";)
TestSimLargeCluster.testBasic
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
this test hits a timeout easily
TestStressRecovery.TestStressRecovery suite
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// 6-Sep-2018
TriLevelCompositeIdRoutingTest.test
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13369";)
*****************BadApple Annotations:
************Lucene BadApple
************Solr BadApple
FullSolrCloudDistribCmdsTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
HdfsNNFailoverTest.HdfsNNFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// 12-Jun-2018
HdfsRecoveryZkTest.HdfsRecoveryZkTest suite
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
MoveReplicaHDFSTest.testNormalFailedMove
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
14-Oct-2018
MultiThreadedOCPTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
// added 20-Jul-2018
SolrZkClientTest.testSimpleUpdateACLs
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
annotated on: 24-Dec-2018
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
6-Sep-2018
TestDistributedStatsComponentCardinality.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";) //
14-Oct-2018
TimeRoutedAliasUpdateProcessorTest.TimeRoutedAliasUpdateProcessorTest suite
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-13059";)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
