Continued improvement I think. Or at least the improvements 3 weeks ago are
working their way through the system. Note that the number of tests that _only_
failed three weeks ago is almost half the total. So I have some optimism that
next week we’ll see a further large drop.
Here’s the synopsis, full report attached:
Number of AwaitsFix: 38 Number of BadApples: 11
Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0 had 28 failures
Week: 1 had 21 failures
Week: 2 had 47 failures
Week: 3 had 142 failures
**Annotated tests that didn't fail in the last 4 weeks.
**Tests removed from the next two lists because they were specified in
'doNotEnable' in the properties file
MoveReplicaHDFSTest.testNormalFailedMove
**Annotations will be removed from the following tests because they haven't
failed in the last 4 rollups.
**Methods: 3
SolrZkClientTest.testSimpleUpdateACLs
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
TestDistributedStatsComponentCardinality.test
********Failures in Hoss' reports for the last 4 rollups.
There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are
objections
Failures in the last 4 reports..
Report Pct runs fails test
0123 1.3 597 11
AliasIntegrationTest.testClusterStateProviderAPI
0123 26.7 65 17
HdfsAutoAddReplicasIntegrationTest.testSimple
0123 1.0 686 9 HttpPartitionWithTlogReplicasTest.test
************ Will BadApple all tests above this line except ones listed at the
top**************
DO NOT ENABLE LIST:
MoveReplicaHDFSTest.testFailedMove
MoveReplicaHDFSTest.testNormalFailedMove
TestControlledRealTimeReopenThread.testCRTReopen
TestICUNormalizer2CharFilter.testRandomStrings
TestICUTokenizerCJK
TestImpersonationWithHadoopAuth.testForwarding
TestLTRReRankingPipeline.testDifferentTopN
TestRandomChains
DO NOT ANNOTATE LIST
CdcrBidirectionalTest.testBiDir
IndexSizeTriggerTest.testMergeIntegration
IndexSizeTriggerTest.testMixedBounds
IndexSizeTriggerTest.testSplitIntegration
IndexSizeTriggerTest.testTrigger
InfixSuggestersTest.testShutdownDuringBuild
ShardSplitTest.test
ShardSplitTest.testSplitMixedReplicaTypes
ShardSplitTest.testSplitWithChaosMonkey
TestLatLonShapeQueries.testRandomBig
TestRandomChains.testRandomChainsWithLargeStrings
TestTriggerIntegration.testSearchRate
Processing file (History bit 3): HOSS-2019-088-05.csv
Processing file (History bit 2): HOSS-2019-08-12.csv
Processing file (History bit 1): HOSS-2019-07-29.csv
Processing file (History bit 0): HOSS-2019-07-08.csv
Number of AwaitsFix: 38 Number of BadApples: 11
Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0 had 28 failures
Week: 1 had 21 failures
Week: 2 had 47 failures
Week: 3 had 142 failures
**Annotated tests that didn't fail in the last 4 weeks.
**Tests removed from the next two lists because they were specified in
'doNotEnable' in the properties file
MoveReplicaHDFSTest.testNormalFailedMove
**Annotations will be removed from the following tests because they haven't
failed in the last 4 rollups.
**Methods: 3
SolrZkClientTest.testSimpleUpdateACLs
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
TestDistributedStatsComponentCardinality.test
********Failures in Hoss' reports for the last 4 rollups.
There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the
date I downloaded the rollup file, newest->oldest. See above for the dates the
files were collected
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are
objections
Failures in the last 4 reports..
Report Pct runs fails test
0123 1.3 597 11
AliasIntegrationTest.testClusterStateProviderAPI
0123 26.7 65 17
HdfsAutoAddReplicasIntegrationTest.testSimple
0123 1.0 686 9 HttpPartitionWithTlogReplicasTest.test
************ Will BadApple all tests above this line except ones listed at the
top**************
012 1.4 276 3 TestCloudJSONFacetJoinDomain.testRandom
0 23 6.2 685 176 BasicAuthIntegrationTest.testBasicAuth
0 23 16.7 49 5
CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
0 23 1.4 496 3 RollingRestartTest.test
0 23 1.5 500 10 TestStressLiveNodes.testStress
0 23 1.4 525 8
TestUseDocValuesAsStored.testDuplicateMultiValued
0 2 4.0 252 7 TestRandomChains.testRandomChains
0 2 3.0 252 7
TestRandomChains.testRandomChainsWithLargeStrings
0 3 2.0 434 5 CollectionPropsTest.testWatcher
0 3 1.0 426 4 HttpPartitionTest.test
0 3 1.4 368 3 LeaderVoteWaitTimeoutTest.basicTest
0 3 2.7 389 22 RulesTest.doIntegrationTest
0 3 1.3 374 2
StatsReloadRaceTest.testParallelReloadAndStats
0 3 16.7 38 2 StressHdfsTest.test
0 3 1.0 427 2
TestLucene80DocValuesFormat.testTermsEnumVariableWidth
0 8.3 12 1
CdcrReplicationHandlerTest.testPartialReplicationAfterPeerSync
0 1.5 67 1
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
0 12.5 8 1 HdfsWriteToMultipleCollectionsTest.test
0 1.3 75 1 OverseerTest.testShardLeaderChange
0 1.0 96 1
TestRunWithRestrictedPermissions.testCompletelyForbidden1
0 1.0 96 1
TestRunWithRestrictedPermissions.testCompletelyForbidden2
0 1.0 96 1
TestRunWithRestrictedPermissions.testNormallyAllowedStuff
123 6.7 80 12 ShardSplitTest.testSplitWithChaosMonkey
123 3.7 518 8 SystemCollectionCompatTest.testBackCompat
12 6.3 216 9 TestCloudJSONFacetSKG.testRandom
1 1.0 100 1
AuditLoggerIntegrationTest.testQueuedTimeMetric
1 1.5 68 1
CollectionsAPIDistributedZkTest.testCollectionsAPI
1 53.3 15 8 ShardSplitTest.test
1 1.4 71 1
TestCollectionStateWatchers.testLiveNodeChangesTriggerWatches
1 1.0 97 1 TestDemoParallelLeafReader.testBasic
1 1.0 97 1
TestDemoParallelLeafReader.testRandomMultipleSchemaGens
1 1.3 79 1 TestExportWriter.testIndexOrder
1 1.3 76 1 TestLBHttp2SolrClient.testTwoServers
1 1.3 76 1 TestLBHttpSolrClient.testReliability
1 11.1 9 1 TestXYPolygonShapeQueries.testRandomBig
1 1.4 73 1
TestZKPropertiesWriter.testZKPropertiesWriter
23 0.7 484 3 CollectionPropsTest.testReadWriteCached
23 1.3 482 5 LeaderFailoverAfterPartitionTest.test
23 0.8 440 24
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
23 1.6 426 5
MetricsHistoryWithAuthIntegrationTest.testValuesAreCollected
23 9.1 43 2 Test2BPostings.test
23 0.8 398 2
TestDelegationWithHadoopAuth.testDelegationTokenRenew
23 2.0 435 4 TestDynamicLoadingUrl.testDynamicLoadingUrl
23 0.7 479 2
TestIndexingSequenceNumbers.testStressConcurrentCommit
23 0.8 400 3 TestJaegerConfigurator.testInjected
23 0.8 415 2
TestSolrCloudWithDelegationTokens.testDelegationTokenRenew
2 1.6 128 2 AutoScalingHandlerTest.testReadApi
2 0.8 122 1 LeaderElectionTest.testStressElection
2 2.3 87 2 SplitByPrefixTest.doTest
2 0.7 151 1 TestCachePurging.testBackgroundPurges
2 0.7 147 1 TestCloudSchemaless.test
2 7.6 172 13
TestContainerReqHandler.testPluginGlobalLoader
2 30.2 172 52 TestContainerReqHandler.testRuntimeLib
2 30.2 172 52
TestContainerReqHandler.testRuntimeLibWithSig2048
2 31.4 172 54
TestContainerReqHandler.testRuntimeLibWithSig512
2 0.9 117 1 TestInPlaceUpdatesDistrib.test
2 9.1 11 1
TestLucene80DocValuesFormat.testSortedVariableLengthManyVsStoredFields
2 0.7 153 1 TestSearcherManager.testSearcherManager
2 0.8 129 1
TestSimLargeCluster.testCreateLargeSimCollections
3 0.3 334 1 AuditLoggerIntegrationTest.testAsync
3 0.6 334 2 AuditLoggerIntegrationTest.testSynchronous
3 3.3 30 1
CdcrReplicationHandlerTest.testPartialReplicationWithTruncatedTlog
3 3.6 28 1 CdcrRequestHandlerTest.testLifeCycleActions
3 0.3 296 1 CloudExitableDirectoryReaderTest.test
3 0.2 658 1 DateRangePrefixTreeTest.testToStringISO8601
3 0.3 337 1
DeleteReplicaTest.deleteReplicaFromClusterState
3 6.5 337 22
DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
3 7.1 337 24
DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplicaLegacy
3 0.3 299 1 DistributedQueueTest.testPeekElements
3 3.3 30 1 ForceLeaderTest.testReplicasInLowerTerms
3 3.0 33 1
ForceLeaderWithTlogReplicasTest.testReplicasInLowerTerms
3 3.7 27 1 HdfsBasicDistributedZk2Test.test
3 10.7 28 3 HdfsBasicDistributedZkTest.test
3 4.2 24 1 HdfsChaosMonkeySafeLeaderTest.test
3 3.4 29 1
HdfsTlogReplayBufferedWhileIndexingTest.test
3 0.3 286 1 IndexSizeTriggerTest.testEstimatedIndexSize
3 0.3 286 1 IndexSizeTriggerTest.testMixedBounds
3 0.3 286 1 IndexSizeTriggerTest.testSplitIntegration
3 0.3 329 1 JWTAuthPluginIntegrationTest.testMetrics
3 0.3 297 1 MathExpressionTest.testGammaDistribution
3 0.3 295 1
MissingSegmentRecoveryTest.testLeaderRecovery
3 0.3 302 1 MoveReplicaTest.test
3 0.3 298 1 NestedShardedAtomicUpdateTest.test
3 0.3 291 1 OverseerRolesTest.testOverseerRole
3 0.9 328 3 PeerSyncTest.test
3 0.3 293 1 ReindexCollectionTest.testLossySchema
3 8.9 369 33 ReplicationFactorTest.test
3 0.9 316 3 RulesTest.testHostFragmentRule
3 1.6 316 5 RulesTest.testInvokeApi
3 1.3 316 4 RulesTest.testModifyColl
3 1.6 316 5 RulesTest.testPortRule
3 0.9 316 3
RulesTest.testPortRuleInPresenceOfClusterPolicy
3 0.3 297 1 SearchHandlerTest.testRequireZkConnected
3 0.3 343 1 ShardSplitTest.testSplitShardWithRule
3 18.2 11 2 SharedFSAutoReplicaFailoverTest.test
3 0.3 296 1 SimpleMLTQParserTest.doTest
3 0.3 299 1 SolrRrdBackendFactoryTest.testBasic
3 0.3 290 1
SpellCheckCollatorTest.testEstimatedHitCounts
3 2.8 179 5 StreamDecoratorTest.testClassifyStream
3 0.3 295 1
TestBuiltInEvaluators.testDateFormatEvaluator
3 3.2 308 10
TestCloudSearcherWarming.testRepFactor1LeaderStartup
3 0.3 294 1
TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete
3 3.2 31 1
TestCompressingStoredFieldsFormat.testBigDocuments
3 4.9 308 15
TestConfigSetsAPI.testUserAndTestDefaultConfigsetsAreSame
3 0.7 294 2 TestDistribIDF.testSimpleQuery
3 0.3 331 1 TestDocValuesRewriteMethod.testRegexps
3 3.2 31 1 TestDuelingCodecsAtNight.testEquals
3 0.7 305 2 TestDynamicLoading.testDynamicLoading
3 1.3 297 4 TestExportWriter.testStringWithCase
3 0.6 316 2
TestFeatureSort.testFeatureMissingFeatureNameInSegment
3 0.9 316 3
TestFeatureSort.testFeatureMissingFieldInSegment
3 0.3 329 1 TestFieldCacheRewriteMethod.testRegexps
3 0.3 331 1 TestIndexWriterDelete.testUpdatesOnDiskFull
3 0.3 330 1
TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull
3 1.2 331 4
TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom
3 0.3 329 1
TestLucene80DocValuesFormat.testRandomAdvanceNumeric
3 0.3 330 1
TestLucene80DocValuesFormat.testSparseDocValuesVsStoredFields
3 0.3 329 1
TestLucene80DocValuesFormat.testTermsEnumFixedWidth
3 3.6 28 1
TestLucene80DocValuesFormat.testTermsEnumRandomMany
3 1.3 307 4
TestModelManagerPersistence.testFeaturePersistence
3 4.2 307 13
TestModelManagerPersistence.testFilePersistence
3 1.3 307 4
TestModelManagerPersistence.testInnerCustomClassesPersistence
3 2.6 302 8
TestModelManagerPersistence.testWrapperModelPersistence
3 0.3 286 1
TestNonDefinedSimilarityFactory.testCurrentBM25FromV8
3 0.6 328 2 TestPackedInts.testPackedLongValues
3 3.0 363 11 TestPolicyCloud.testCreateCollection
3 15.7 363 57
TestPolicyCloud.testCreateCollectionAddReplica
3 0.3 361 1
TestPolicyCloud.testCreateCollectionAddShardUsingPolicy
3 1.7 363 6
TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
3 13.2 363 48
TestPolicyCloud.testCreateCollectionSplitShard
3 5.0 361 18 TestPolicyCloud.testDataProvider
3 1.1 361 4 TestPolicyCloud.testMetricsTag
3 10.2 324 33 TestRamUsageEstimator.testMap
3 0.3 346 1
TestReplicationHandler.doTestIndexAndConfigReplication
3 0.3 346 1 TestReplicationHandler.testEmptyCommits
3 0.3 329 1 TestReplicationHandlerBackup.doTestBackup
3 0.7 296 2 TestSimLargeCluster.testAddNode
3 0.3 291 1
TestSimTriggerIntegration.testNodeMarkersRegistration
3 0.3 291 1
TestSimTriggerIntegration.testTriggerThrottling
3 1.0 295 3
TestSkipOverseerOperations.testSkipLeaderOperations
3 0.7 287 2
TestSnapshotCloudManager.testSimulatorFromSnapshot
3 0.3 293 1
TestSolrDeletionPolicy1.testNumCommitsConfigured
3 0.3 328 1
TestSpanSearchEquivalence.testSpanBoostQuerySimplification
3 0.3 328 1
TestSpanSearchEquivalence.testSpanTermVersusTerm
3 0.9 323 3
TestUseDocValuesAsStored.testMultipleSearchResults
3 3.8 290 11
TestWrapperModel.testMethodOverridesAndDelegation
3 0.7 290 2
UninvertDocValuesMergePolicyTest.testIndexAndAddDocValues
***********************Annotations report
Number of AwaitsFix: 38 Number of BadApples: 11
*****************AwaitsFix Annotations:
************Lucene AwaitsFix
TestControlledRealTimeReopenThread.testCRTReopen
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737")
TestICUNormalizer2CharFilter.testRandomStrings
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595")
TestICUTokenizerCJK.TestICUTokenizerCJK suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222")
TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161")
************Solr AwaitsFix
CdcrBidirectionalTest.testBiDir
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12524")
CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028")
CollectionsAPISolrJTest.testCreateAndDeleteShard
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13021")
ComputePlanActionTest.testNodeLost
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
ConcurrentCreateRoutedAliasTest.ConcurrentCreateRoutedAliasTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-12386") // "Can't find resource"
LeaderTragicEventTest.LeaderTragicEventTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13237")
MoveReplicaHDFSTest.testFailedMove
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12080") //
added 03-Oct-2018
NodeMarkersRegistrationTest.testNodeMarkersRegistration
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13376")
ReplaceNodeNoTargetTest.ReplaceNodeNoTargetTest suite
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067")
ReplaceNodeNoTargetTest.test
@LuceneTestCase.AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-11067")
SSLMigrationTest.SSLMigrationTest suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
17-Mar-2018
SaslZkACLProviderTest.testSaslZkACLProvider
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13075")
ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
17-Mar-2018
ScheduledTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
added 20-Sep-2018
SearchRateTriggerIntegrationTest.testDeleteNode
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13163")
SearchRateTriggerTest.testTrigger
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
Tagger2Test.testStopWords
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-8344")
TestCollapseQParserPlugin.testStringCollapse
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974")
TestHierarchicalDocBuilder.testThreeLevelHierarchy
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12801") //
this test fails easily under beasting
TestImpersonationWithHadoopAuth.testForwarding
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893")
TestJsonFacetRefinement.testProcessEmptyRefinement
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556")
TestJsonFacetRefinement.testSortedSubFacetRefinementWhenParentOnlyReturnedByOneShardProcessEmpty
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556")
TestLTRReRankingPipeline.testDifferentTopN
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134")
TestMinMaxOnMultiValuedField.testDoubleFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709")
TestMinMaxOnMultiValuedField.testFloatFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709")
TestMinMaxOnMultiValuedField.testIntFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709")
TestMinMaxOnMultiValuedField.testLongFieldCache
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709")
TestPullReplica.TestPullReplica suite
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
TestSimComputePlanAction.testNodeLost
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") //
if you beast this, eventually you will see
TestSimExtremeIndexing.TestSimExtremeIndexing suite
@org.apache.lucene.util.LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12923")
TestSimLargeCluster.testBasic
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
this test hits a timeout easily
TestStressRecovery.TestStressRecovery suite
@LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
// 6-Sep-2018
TestTlogReplica.TestTlogReplica suite
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12313")
TriLevelCompositeIdRoutingTest.test
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13369")
*****************BadApple Annotations:
************Lucene BadApple
************Solr BadApple
FullSolrCloudDistribCmdsTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
annotated on: 24-Dec-2018
HdfsNNFailoverTest.HdfsNNFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
// 12-Jun-2018
HdfsRecoveryZkTest.HdfsRecoveryZkTest suite
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
MoveReplicaHDFSTest.testNormalFailedMove
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
14-Oct-2018
MultiThreadedOCPTest.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
annotated on: 24-Dec-2018
SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")
// added 20-Jul-2018
SolrZkClientTest.testSimpleUpdateACLs
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
annotated on: 24-Dec-2018
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
6-Sep-2018
TestDistributedStatsComponentCardinality.test
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") //
14-Oct-2018
TimeRoutedAliasUpdateProcessorTest.TimeRoutedAliasUpdateProcessorTest suite
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-13059")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]