walterddr edited a comment on issue #7248:
URL: https://github.com/apache/pinot/issues/7248#issuecomment-931764431


   was able to consistently reproduce (by running exact test 100 times) here: 
https://github.com/apache/pinot/pull/7503/checks?check_run_id=3760273969. 
   
   Found the issue was a never ending Pinot PeriodicRebalance it seems. The 
following code log keeps repeating every 10 seconds until fork-timeout occurs. 
and it clearly seems to be helix internal code as none of the logs comes from 
pinot. 
   
   @Jackie-Jiang, @xiangfu0, @mayankshriv any thoughts? 
   
   <details>
     <summary>Log blocks</summary>
   
     ```txt
       2021-09-30T21:56:44.6124303Z 21:56:44.536 ConstraintItemBuilder - 
overwrite existing constraint-value. old-value: null, new-value: 100000
       2021-09-30T21:56:44.6126397Z 21:56:44.536 PropertyCache - Event 
HelixBrokerStarterTest::TASK::9071fd02 : Refreshed 1 property ClusterConstraint 
took 3 ms. Selective: false
       2021-09-30T21:56:54.3768577Z 21:56:44.545 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.3770881Z 21:56:44.545 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.3786915Z 21:56:44.546 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 1 ms. 
       2021-09-30T21:56:54.3789859Z 21:56:44.549 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 1 ms. 
       2021-09-30T21:56:54.3792399Z 21:56:44.549 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::9071fd02 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.3795790Z 21:56:44.549 CurrentStateCache - Event 
HelixBrokerStarterTest::DEFAULT::9071fd02 : END: CurrentStateCache.refresh() 
for cluster HelixBrokerStarterTest, started at : 1633039004546, took 3 ms
       2021-09-30T21:56:54.3800916Z 21:56:44.549 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.3807178Z 21:56:44.549 ResourceControllerDataProvider 
- Event 9071fd02 : END: ResourceControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039004507 took 42 for DEFAULT pipeline
       2021-09-30T21:56:54.3813130Z 21:56:44.549 Pipeline - END 
ReadClusterDataStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 42 ms for event 9071fd02
       2021-09-30T21:56:54.3816506Z 21:56:44.550 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.3819674Z 21:56:44.550 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.3822915Z 21:56:44.550 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.3826729Z 21:56:44.550 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event 9071fd02
       2021-09-30T21:56:54.3838663Z 21:56:44.550 Pipeline - END 
ResourceComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3841095Z 21:56:44.550 Pipeline - END 
ResourceValidationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3843452Z 21:56:44.550 Pipeline - END 
CurrentStateComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3846184Z 21:56:44.550 Pipeline - END 
TopStateHandoffReportStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3852967Z 21:56:44.550 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::ExternalViewComputeStage task to worker
       2021-09-30T21:56:54.3855028Z 21:56:44.550 Pipeline - END 
ExternalViewComputeStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3856791Z 21:56:44.550 CustomRebalancer - Computing 
BestPossibleMapping for testTable_OFFLINE
       2021-09-30T21:56:54.3858599Z 21:56:44.550 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::ExternalViewComputeStage
       2021-09-30T21:56:54.3860191Z 21:56:44.550 CustomRebalancer - Computing 
BestPossibleMapping for brokerResource
       2021-09-30T21:56:54.3861617Z 21:56:44.550 AutoRebalancer - Computing 
IdealState for leadControllerResource
       2021-09-30T21:56:54.3863328Z 21:56:44.550 AutoRebalancer - found the 
following participants with tag controller for leadControllerResource: 
[Controller_localhost_18998]
       2021-09-30T21:56:54.3865418Z 21:56:44.550 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::ExternalViewComputeStage, took 0 ms
       2021-09-30T21:56:54.3867030Z 21:56:44.550 CustomRebalancer - Computing 
BestPossibleMapping for newTable_OFFLINE
       2021-09-30T21:56:54.3868469Z 21:56:44.550 CustomRebalancer - Computing 
BestPossibleMapping for testTable_REALTIME
       2021-09-30T21:56:54.3870316Z 21:56:44.550 Pipeline - END 
BestPossibleStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3872140Z 21:56:44.550 
StateTransitionThrottleController - No throttle config is set!
       2021-09-30T21:56:54.3874044Z 21:56:44.550 Pipeline - END 
IntermediateStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3876531Z 21:56:44.550 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.3878490Z 21:56:44.550 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.3880546Z 21:56:44.550 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.3882563Z 21:56:44.550 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3884779Z 21:56:44.551 Pipeline - END 
ResourceMessageGenerationPhase for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event 9071fd02
       2021-09-30T21:56:54.3886913Z 21:56:44.551 Pipeline - END 
MessageSelectionStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3888923Z 21:56:44.551 Pipeline - END 
MessageThrottleStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3891042Z 21:56:44.551 Pipeline - END 
ResourceMessageDispatchStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3893158Z 21:56:44.551 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::PersistAssignmentStage
       2021-09-30T21:56:54.3895065Z 21:56:44.551 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::PersistAssignmentStage, took 0 ms
       2021-09-30T21:56:54.3897039Z 21:56:44.551 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::PersistAssignmentStage task to worker
       2021-09-30T21:56:54.3899328Z 21:56:44.551 Pipeline - END 
PersistAssignmentStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3901515Z 21:56:44.551 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::TargetExteralViewCalcStage task to worker
       2021-09-30T21:56:54.3903601Z 21:56:44.551 Pipeline - END 
TargetExteralViewCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3914163Z 21:56:44.551 GenericHelixController - END: 
Invoking DEFAULT controller pipeline for event PeriodicalRebalance::9071fd02 
for cluster HelixBrokerStarterTest, took 44 ms
       2021-09-30T21:56:54.3916801Z 21:56:44.551 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage
       2021-09-30T21:56:54.3918829Z 21:56:44.551 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage, took 0 ms
       2021-09-30T21:56:54.3920555Z 21:56:44.551 GenericHelixController - 
Callback time for event: PeriodicalRebalance took: 0 ms
       2021-09-30T21:56:54.3921692Z InQueue time for event: PeriodicalRebalance 
took: 5 ms
       2021-09-30T21:56:54.3922656Z TotalProcessed time for event: 
PeriodicalRebalance took: 44 ms
       2021-09-30T21:56:54.3924350Z 21:56:44.551 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::9071fd02 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.3926792Z 21:56:44.551 CurrentStateCache - Event 
HelixBrokerStarterTest::TASK::9071fd02 : END: CurrentStateCache.refresh() for 
cluster HelixBrokerStarterTest, started at : 1633039004549, took 2 ms
       2021-09-30T21:56:54.3928941Z 21:56:44.551 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.3931253Z 21:56:44.551 AssignableInstanceManager - 
AssignableInstanceManager built AssignableInstances from scratch based on 
contexts in TaskDataCache due to Controller switch or ClusterConfig change.
       2021-09-30T21:56:54.3933557Z 21:56:44.552 AssignableInstanceManager - 
Current quota capacity: 
{"Controller_localhost_18998":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}}}
       2021-09-30T21:56:54.3936136Z 21:56:44.552 WorkflowControllerDataProvider 
- Event 9071fd02 : END: WorkflowControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039004506 took 46 for TASK pipeline
       2021-09-30T21:56:54.3938758Z 21:56:44.552 Pipeline - END 
ReadClusterDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 46 ms for event 9071fd02
       2021-09-30T21:56:54.3940767Z 21:56:44.552 Pipeline - END 
ResourceComputationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3942809Z 21:56:44.552 Pipeline - END 
ResourceValidationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3944922Z 21:56:44.552 Pipeline - END 
CurrentStateComputationStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3946974Z 21:56:44.552 Pipeline - END 
TaskSchedulingStage for TASK pipeline for cluster HelixBrokerStarterTest. took: 
0 ms for event 9071fd02
       2021-09-30T21:56:54.3948686Z 21:56:44.552 TaskPersistDataStage - START 
TaskPersistDataStage.process()
       2021-09-30T21:56:54.3950555Z 21:56:44.552 TaskPersistDataStage - END 
TaskPersistDataStage.process() for cluster HelixBrokerStarterTest took 0 ms
       2021-09-30T21:56:54.3954010Z 21:56:44.552 Pipeline - END 
TaskPersistDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3957629Z 21:56:44.552 AbstractAsyncBaseStage - START 
AsyncProcess: TASK::TaskGarbageCollectionStage
       2021-09-30T21:56:54.3959665Z 21:56:44.552 AbstractAsyncBaseStage - END 
AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms
       2021-09-30T21:56:54.3961732Z 21:56:44.552 AbstractAsyncBaseStage - 
Submitted asynchronous TASK::TaskGarbageCollectionStage task to worker
       2021-09-30T21:56:54.3963785Z 21:56:44.552 Pipeline - END 
TaskGarbageCollectionStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3966186Z 21:56:44.552 Pipeline - END 
TaskMessageGenerationPhase for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3968281Z 21:56:44.552 Pipeline - END 
TaskMessageDispatchStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event 9071fd02
       2021-09-30T21:56:54.3970800Z 21:56:44.552 GenericHelixController - END: 
Invoking TASK controller pipeline for event PeriodicalRebalance::9071fd02 for 
cluster HelixBrokerStarterTest, took 46 ms
       2021-09-30T21:56:54.3973274Z 21:56:44.785 GenericHelixController - 
Controller rebalance pipeline triggered with event type: PeriodicalRebalance 
for cluster HelixBrokerStarterTest
       2021-09-30T21:56:54.3975242Z 21:56:44.787 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.3977275Z 21:56:44.787 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster DEFAULT 
event: PeriodicalRebalance  fb5f94d4
       2021-09-30T21:56:54.3979239Z 21:56:44.788 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.3981202Z 21:56:44.788 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster TASK 
event: PeriodicalRebalance  fb5f94d4
       2021-09-30T21:56:54.3983449Z 21:56:44.791 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.3985520Z 21:56:44.791 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 5 property IdealState 
took 2 ms. Selective: true
       2021-09-30T21:56:54.3987580Z 21:56:44.791 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.3989667Z 21:56:44.791 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 1 property LiveInstance 
took 0 ms. Selective: true
       2021-09-30T21:56:54.3991713Z 21:56:44.792 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.3993798Z 21:56:44.792 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.3996679Z 21:56:44.792 BaseControllerDataProvider - 
Event fb5f94d4 : Reloaded InstanceConfig for cluster HelixBrokerStarterTest, 
DEFAULT pipeline. Keys: [Broker_10.1.0.50_18099, Controller_localhost_18998, 
Broker_localhost_1, Broker_localhost_0, Server_localhost_0]
       2021-09-30T21:56:54.3999263Z 21:56:44.793 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4001296Z 21:56:44.793 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 5 property IdealState took 1 
ms. Selective: true
       2021-09-30T21:56:54.4003307Z 21:56:44.793 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4005923Z 21:56:44.793 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 1 property ResourceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4008181Z 21:56:44.793 BaseControllerDataProvider - 
Event fb5f94d4 : Reloaded ResourceConfig for cluster HelixBrokerStarterTest, 
DEFAULT pipeline. Cnt: 1
       2021-09-30T21:56:54.4010400Z 21:56:44.793 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4012441Z 21:56:44.794 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 1 property LiveInstance took 
1 ms. Selective: true
       2021-09-30T21:56:54.4014440Z 21:56:44.795 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4018540Z 21:56:44.795 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4021227Z 21:56:44.795 BaseControllerDataProvider - 
Event fb5f94d4 : Reloaded InstanceConfig for cluster HelixBrokerStarterTest, 
TASK pipeline. Keys: [Broker_10.1.0.50_18099, Controller_localhost_18998, 
Broker_localhost_1, Broker_localhost_0, Server_localhost_0]
       2021-09-30T21:56:54.4024049Z 21:56:44.795 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4026126Z 21:56:44.795 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 1 property ResourceConfig 
took 0 ms. Selective: true
       2021-09-30T21:56:54.4028350Z 21:56:44.795 BaseControllerDataProvider - 
Event fb5f94d4 : Reloaded ResourceConfig for cluster HelixBrokerStarterTest, 
TASK pipeline. Cnt: 1
       2021-09-30T21:56:54.4030790Z 21:56:44.809 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 8 property 
StateModelDefinition took 16 ms. Selective: false
       2021-09-30T21:56:54.4033140Z 21:56:44.811 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 8 property 
StateModelDefinition took 16 ms. Selective: false
       2021-09-30T21:56:54.4035324Z 21:56:44.812 ConstraintItemBuilder - 
overwrite existing constraint-value. old-value: null, new-value: 100000
       2021-09-30T21:56:54.4037449Z 21:56:44.812 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : Refreshed 1 property 
ClusterConstraint took 2 ms. Selective: false
       2021-09-30T21:56:54.4039331Z 21:56:44.814 ConstraintItemBuilder - 
overwrite existing constraint-value. old-value: null, new-value: 100000
       2021-09-30T21:56:54.4041422Z 21:56:44.814 PropertyCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : Refreshed 1 property ClusterConstraint 
took 3 ms. Selective: false
       2021-09-30T21:56:54.4043243Z 21:56:44.821 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.4045120Z 21:56:44.822 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 1 ms. 
       2021-09-30T21:56:54.4047184Z 21:56:44.822 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4049677Z 21:56:44.822 CurrentStateCache - Event 
HelixBrokerStarterTest::DEFAULT::fb5f94d4 : END: CurrentStateCache.refresh() 
for cluster HelixBrokerStarterTest, started at : 1633039004822, took 0 ms
       2021-09-30T21:56:54.4051857Z 21:56:44.822 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.4055012Z 21:56:44.822 ResourceControllerDataProvider 
- Event fb5f94d4 : END: ResourceControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039004787 took 35 for DEFAULT pipeline
       2021-09-30T21:56:54.4057710Z 21:56:44.822 Pipeline - END 
ReadClusterDataStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 35 ms for event fb5f94d4
       2021-09-30T21:56:54.4059857Z 21:56:44.823 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.4061869Z 21:56:44.823 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4063967Z 21:56:44.823 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.4065930Z 21:56:44.823 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.4068176Z 21:56:44.823 Pipeline - END 
ResourceComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4075786Z 21:56:44.823 Pipeline - END 
ResourceValidationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4078140Z 21:56:44.823 Pipeline - END 
CurrentStateComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4080370Z 21:56:44.823 Pipeline - END 
TopStateHandoffReportStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4082802Z 21:56:44.823 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::ExternalViewComputeStage
       2021-09-30T21:56:54.4084756Z 21:56:44.823 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::ExternalViewComputeStage, took 0 ms
       2021-09-30T21:56:54.4086816Z 21:56:44.823 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::ExternalViewComputeStage task to worker
       2021-09-30T21:56:54.4088846Z 21:56:44.823 Pipeline - END 
ExternalViewComputeStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4091208Z 21:56:44.823 CustomRebalancer - Computing 
BestPossibleMapping for testTable_OFFLINE
       2021-09-30T21:56:54.4092661Z 21:56:44.823 CustomRebalancer - Computing 
BestPossibleMapping for brokerResource
       2021-09-30T21:56:54.4094080Z 21:56:44.823 AutoRebalancer - Computing 
IdealState for leadControllerResource
       2021-09-30T21:56:54.4095796Z 21:56:44.823 AutoRebalancer - found the 
following participants with tag controller for leadControllerResource: 
[Controller_localhost_18998]
       2021-09-30T21:56:54.4097565Z 21:56:44.823 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.4099424Z 21:56:44.824 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 1 ms. 
       2021-09-30T21:56:54.4101121Z 21:56:44.824 CustomRebalancer - Computing 
BestPossibleMapping for newTable_OFFLINE
       2021-09-30T21:56:54.4102556Z 21:56:44.824 CustomRebalancer - Computing 
BestPossibleMapping for testTable_REALTIME
       2021-09-30T21:56:54.4104430Z 21:56:44.824 Pipeline - END 
BestPossibleStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event fb5f94d4
       2021-09-30T21:56:54.4108933Z 21:56:44.824 
StateTransitionThrottleController - No throttle config is set!
       2021-09-30T21:56:54.4113055Z 21:56:44.824 Pipeline - END 
IntermediateStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4115466Z 21:56:44.824 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.4117507Z 21:56:44.824 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.4119450Z 21:56:44.824 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4121599Z 21:56:44.824 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.4123722Z 21:56:44.824 Pipeline - END 
ResourceMessageGenerationPhase for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4125914Z 21:56:44.824 Pipeline - END 
MessageSelectionStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4127929Z 21:56:44.824 Pipeline - END 
MessageThrottleStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4129968Z 21:56:44.824 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4132003Z 21:56:44.824 Pipeline - END 
ResourceMessageDispatchStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4134134Z 21:56:44.824 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::PersistAssignmentStage
       2021-09-30T21:56:54.4136028Z 21:56:44.824 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::PersistAssignmentStage, took 0 ms
       2021-09-30T21:56:54.4138527Z 21:56:44.824 CurrentStateCache - Event 
HelixBrokerStarterTest::TASK::fb5f94d4 : END: CurrentStateCache.refresh() for 
cluster HelixBrokerStarterTest, started at : 1633039004824, took 0 ms
       2021-09-30T21:56:54.4142621Z 21:56:44.825 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.4144648Z 21:56:44.825 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::PersistAssignmentStage task to worker
       2021-09-30T21:56:54.4146843Z 21:56:44.825 Pipeline - END 
PersistAssignmentStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 1 ms for event fb5f94d4
       2021-09-30T21:56:54.4148920Z 21:56:44.825 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage
       2021-09-30T21:56:54.4150946Z 21:56:44.825 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage, took 0 ms
       2021-09-30T21:56:54.4153053Z 21:56:44.825 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::TargetExteralViewCalcStage task to worker
       2021-09-30T21:56:54.4155421Z 21:56:44.825 Pipeline - END 
TargetExteralViewCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4158033Z 21:56:44.825 GenericHelixController - END: 
Invoking DEFAULT controller pipeline for event PeriodicalRebalance::fb5f94d4 
for cluster HelixBrokerStarterTest, took 38 ms
       2021-09-30T21:56:54.4160025Z 21:56:44.825 GenericHelixController - 
Callback time for event: PeriodicalRebalance took: 0 ms
       2021-09-30T21:56:54.4161162Z InQueue time for event: PeriodicalRebalance 
took: 2 ms
       2021-09-30T21:56:54.4162146Z TotalProcessed time for event: 
PeriodicalRebalance took: 38 ms
       2021-09-30T21:56:54.4164248Z 21:56:44.825 AssignableInstanceManager - 
AssignableInstanceManager built AssignableInstances from scratch based on 
contexts in TaskDataCache due to Controller switch or ClusterConfig change.
       2021-09-30T21:56:54.4166546Z 21:56:44.825 AssignableInstanceManager - 
Current quota capacity: 
{"Controller_localhost_18998":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}}}
       2021-09-30T21:56:54.4169138Z 21:56:44.825 WorkflowControllerDataProvider 
- Event fb5f94d4 : END: WorkflowControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039004788 took 37 for TASK pipeline
       2021-09-30T21:56:54.4173605Z 21:56:44.825 Pipeline - END 
ReadClusterDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 37 ms for event fb5f94d4
       2021-09-30T21:56:54.4175770Z 21:56:44.825 Pipeline - END 
ResourceComputationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4177834Z 21:56:44.825 Pipeline - END 
ResourceValidationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4179968Z 21:56:44.825 Pipeline - END 
CurrentStateComputationStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4182030Z 21:56:44.825 Pipeline - END 
TaskSchedulingStage for TASK pipeline for cluster HelixBrokerStarterTest. took: 
0 ms for event fb5f94d4
       2021-09-30T21:56:54.4183761Z 21:56:44.825 TaskPersistDataStage - START 
TaskPersistDataStage.process()
       2021-09-30T21:56:54.4185640Z 21:56:44.825 TaskPersistDataStage - END 
TaskPersistDataStage.process() for cluster HelixBrokerStarterTest took 0 ms
       2021-09-30T21:56:54.4187716Z 21:56:44.825 Pipeline - END 
TaskPersistDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4189876Z 21:56:44.825 AbstractAsyncBaseStage - 
Submitted asynchronous TASK::TaskGarbageCollectionStage task to worker
       2021-09-30T21:56:54.4191933Z 21:56:44.825 Pipeline - END 
TaskGarbageCollectionStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4194012Z 21:56:44.825 AbstractAsyncBaseStage - START 
AsyncProcess: TASK::TaskGarbageCollectionStage
       2021-09-30T21:56:54.4196258Z 21:56:44.825 AbstractAsyncBaseStage - END 
AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms
       2021-09-30T21:56:54.4198293Z 21:56:44.825 Pipeline - END 
TaskMessageGenerationPhase for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4200416Z 21:56:44.825 Pipeline - END 
TaskMessageDispatchStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event fb5f94d4
       2021-09-30T21:56:54.4202947Z 21:56:44.825 GenericHelixController - END: 
Invoking TASK controller pipeline for event PeriodicalRebalance::fb5f94d4 for 
cluster HelixBrokerStarterTest, took 37 ms
       2021-09-30T21:56:54.4207866Z 21:56:54.101 GenericHelixController - 
Controller rebalance pipeline triggered with event type: PeriodicalRebalance 
for cluster HelixBrokerStarterTest
       2021-09-30T21:56:54.4209825Z 21:56:54.104 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.4211806Z 21:56:54.104 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster TASK 
event: PeriodicalRebalance  d06c6099
       2021-09-30T21:56:54.4213899Z 21:56:54.105 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.4215905Z 21:56:54.105 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster DEFAULT 
event: PeriodicalRebalance  d06c6099
       2021-09-30T21:56:54.4218178Z 21:56:54.109 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4220607Z 21:56:54.109 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 5 property IdealState 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4222632Z 21:56:54.110 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4224678Z 21:56:54.110 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 1 property LiveInstance 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4226697Z 21:56:54.110 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4228697Z 21:56:54.110 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 5 property IdealState took 1 
ms. Selective: true
       2021-09-30T21:56:54.4230695Z 21:56:54.111 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4232763Z 21:56:54.111 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4243705Z 21:56:54.111 BaseControllerDataProvider - 
Event d06c6099 : Reloaded InstanceConfig for cluster HelixBrokerStarterTest, 
DEFAULT pipeline. Keys: [Broker_10.1.0.50_18099, Controller_localhost_18998, 
Broker_localhost_1, Broker_localhost_0, Server_localhost_0]
       2021-09-30T21:56:54.4247820Z 21:56:54.111 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4249877Z 21:56:54.111 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 1 property LiveInstance took 
1 ms. Selective: true
       2021-09-30T21:56:54.4251921Z 21:56:54.111 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4254016Z 21:56:54.111 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 1 property ResourceConfig 
took 0 ms. Selective: true
       2021-09-30T21:56:54.4256170Z 21:56:54.111 BaseControllerDataProvider - 
Event d06c6099 : Reloaded ResourceConfig for cluster HelixBrokerStarterTest, 
DEFAULT pipeline. Cnt: 1
       2021-09-30T21:56:54.4266049Z 21:56:54.112 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4268113Z 21:56:54.112 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4270745Z 21:56:54.112 BaseControllerDataProvider - 
Event d06c6099 : Reloaded InstanceConfig for cluster HelixBrokerStarterTest, 
TASK pipeline. Keys: [Broker_10.1.0.50_18099, Controller_localhost_18998, 
Broker_localhost_1, Broker_localhost_0, Server_localhost_0]
       2021-09-30T21:56:54.4273274Z 21:56:54.119 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4275559Z 21:56:54.119 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 1 property ResourceConfig 
took 7 ms. Selective: true
       2021-09-30T21:56:54.4277934Z 21:56:54.119 BaseControllerDataProvider - 
Event d06c6099 : Reloaded ResourceConfig for cluster HelixBrokerStarterTest, 
TASK pipeline. Cnt: 1
       2021-09-30T21:56:54.4280354Z 21:56:54.126 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 8 property 
StateModelDefinition took 15 ms. Selective: false
       2021-09-30T21:56:54.4282241Z 21:56:54.128 ConstraintItemBuilder - 
overwrite existing constraint-value. old-value: null, new-value: 100000
       2021-09-30T21:56:54.4284432Z 21:56:54.128 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : Refreshed 1 property 
ClusterConstraint took 2 ms. Selective: false
       2021-09-30T21:56:54.4286747Z 21:56:54.135 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 8 property 
StateModelDefinition took 16 ms. Selective: false
       2021-09-30T21:56:54.4288778Z 21:56:54.136 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.4290654Z 21:56:54.137 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 0 ms. 
       2021-09-30T21:56:54.4292510Z 21:56:54.137 ConstraintItemBuilder - 
overwrite existing constraint-value. old-value: null, new-value: 100000
       2021-09-30T21:56:54.4294557Z 21:56:54.137 PropertyCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : Refreshed 1 property ClusterConstraint 
took 2 ms. Selective: false
       2021-09-30T21:56:54.4296623Z 21:56:54.137 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4299099Z 21:56:54.138 CurrentStateCache - Event 
HelixBrokerStarterTest::DEFAULT::d06c6099 : END: CurrentStateCache.refresh() 
for cluster HelixBrokerStarterTest, started at : 1633039014137, took 1 ms
       2021-09-30T21:56:54.4301242Z 21:56:54.138 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.4303799Z 21:56:54.138 ResourceControllerDataProvider 
- Event d06c6099 : END: ResourceControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039014105 took 33 for DEFAULT pipeline
       2021-09-30T21:56:54.4306434Z 21:56:54.138 Pipeline - END 
ReadClusterDataStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 33 ms for event d06c6099
       2021-09-30T21:56:54.4308558Z 21:56:54.138 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.4310572Z 21:56:54.138 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4312657Z 21:56:54.138 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.4314754Z 21:56:54.138 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.4316737Z 21:56:54.138 Pipeline - END 
ResourceComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4318811Z 21:56:54.138 Pipeline - END 
ResourceValidationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4320969Z 21:56:54.138 Pipeline - END 
CurrentStateComputationStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4323166Z 21:56:54.138 Pipeline - END 
TopStateHandoffReportStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4325259Z 21:56:54.138 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::ExternalViewComputeStage
       2021-09-30T21:56:54.4327232Z 21:56:54.138 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::ExternalViewComputeStage, took 0 ms
       2021-09-30T21:56:54.4329278Z 21:56:54.138 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::ExternalViewComputeStage task to worker
       2021-09-30T21:56:54.4331296Z 21:56:54.138 Pipeline - END 
ExternalViewComputeStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4333150Z 21:56:54.138 CustomRebalancer - Computing 
BestPossibleMapping for testTable_OFFLINE
       2021-09-30T21:56:54.4334594Z 21:56:54.138 CustomRebalancer - Computing 
BestPossibleMapping for brokerResource
       2021-09-30T21:56:54.4336012Z 21:56:54.138 AutoRebalancer - Computing 
IdealState for leadControllerResource
       2021-09-30T21:56:54.4337724Z 21:56:54.138 AutoRebalancer - found the 
following participants with tag controller for leadControllerResource: 
[Controller_localhost_18998]
       2021-09-30T21:56:54.4339398Z 21:56:54.139 CustomRebalancer - Computing 
BestPossibleMapping for newTable_OFFLINE
       2021-09-30T21:56:54.4340923Z 21:56:54.139 CustomRebalancer - Computing 
BestPossibleMapping for testTable_REALTIME
       2021-09-30T21:56:54.4342796Z 21:56:54.139 Pipeline - END 
BestPossibleStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event d06c6099
       2021-09-30T21:56:54.4344620Z 21:56:54.139 
StateTransitionThrottleController - No throttle config is set!
       2021-09-30T21:56:54.4346505Z 21:56:54.139 Pipeline - END 
IntermediateStateCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4348730Z 21:56:54.139 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::MaintenanceRecoveryStage task to worker
       2021-09-30T21:56:54.4350704Z 21:56:54.139 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage
       2021-09-30T21:56:54.4352657Z 21:56:54.139 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::MaintenanceRecoveryStage, took 0 ms
       2021-09-30T21:56:54.4354778Z 21:56:54.139 Pipeline - END 
MaintenanceRecoveryStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4357021Z 21:56:54.139 Pipeline - END 
ResourceMessageGenerationPhase for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4359184Z 21:56:54.139 Pipeline - END 
MessageSelectionStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4361187Z 21:56:54.139 Pipeline - END 
MessageThrottleStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4363299Z 21:56:54.139 Pipeline - END 
ResourceMessageDispatchStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4365535Z 21:56:54.139 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::PersistAssignmentStage
       2021-09-30T21:56:54.4367441Z 21:56:54.139 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::PersistAssignmentStage, took 0 ms
       2021-09-30T21:56:54.4369429Z 21:56:54.139 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::PersistAssignmentStage task to worker
       2021-09-30T21:56:54.4371390Z 21:56:54.139 Pipeline - END 
PersistAssignmentStage for DEFAULT pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4373449Z 21:56:54.140 AbstractAsyncBaseStage - START 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage
       2021-09-30T21:56:54.4375473Z 21:56:54.140 AbstractAsyncBaseStage - END 
AsyncProcess: DEFAULT::TargetExteralViewCalcStage, took 0 ms
       2021-09-30T21:56:54.4377580Z 21:56:54.140 AbstractAsyncBaseStage - 
Submitted asynchronous DEFAULT::TargetExteralViewCalcStage task to worker
       2021-09-30T21:56:54.4379658Z 21:56:54.140 Pipeline - END 
TargetExteralViewCalcStage for DEFAULT pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event d06c6099
       2021-09-30T21:56:54.4382223Z 21:56:54.140 GenericHelixController - END: 
Invoking DEFAULT controller pipeline for event PeriodicalRebalance::d06c6099 
for cluster HelixBrokerStarterTest, took 35 ms
       2021-09-30T21:56:54.4384188Z 21:56:54.140 GenericHelixController - 
Callback time for event: PeriodicalRebalance took: 0 ms
       2021-09-30T21:56:54.4385316Z InQueue time for event: PeriodicalRebalance 
took: 5 ms
       2021-09-30T21:56:54.4386294Z TotalProcessed time for event: 
PeriodicalRebalance took: 35 ms
       2021-09-30T21:56:54.4387727Z 21:56:54.144 InstanceMessagesCache - START: 
InstanceMessagesCache.refresh()
       2021-09-30T21:56:54.4389733Z 21:56:54.144 InstanceMessagesCache - END: 
InstanceMessagesCache.refresh(), 0 of Messages read from ZooKeeper. took 0 ms. 
       2021-09-30T21:56:54.4391789Z 21:56:54.145 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4394233Z 21:56:54.145 CurrentStateCache - Event 
HelixBrokerStarterTest::TASK::d06c6099 : END: CurrentStateCache.refresh() for 
cluster HelixBrokerStarterTest, started at : 1633039014144, took 1 ms
       2021-09-30T21:56:54.4396617Z 21:56:54.145 InstanceMessagesCache - END: 
updateRelayMessages(), 0 of valid relay messages in cache, took 0 ms. 
       2021-09-30T21:56:54.4398950Z 21:56:54.145 AssignableInstanceManager - 
AssignableInstanceManager built AssignableInstances from scratch based on 
contexts in TaskDataCache due to Controller switch or ClusterConfig change.
       2021-09-30T21:56:54.4401248Z 21:56:54.145 AssignableInstanceManager - 
Current quota capacity: 
{"Controller_localhost_18998":{"TASK_EXEC_THREAD":{"DEFAULT":"0/40"}}}
       2021-09-30T21:56:54.4403817Z 21:56:54.145 WorkflowControllerDataProvider 
- Event d06c6099 : END: WorkflowControllerDataProvider.refresh() for cluster 
HelixBrokerStarterTest, started at 1633039014104 took 41 for TASK pipeline
       2021-09-30T21:56:54.4406400Z 21:56:54.145 Pipeline - END 
ReadClusterDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 41 ms for event d06c6099
       2021-09-30T21:56:54.4408397Z 21:56:54.145 Pipeline - END 
ResourceComputationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4410448Z 21:56:54.145 Pipeline - END 
ResourceValidationStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4412550Z 21:56:54.146 Pipeline - END 
CurrentStateComputationStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 1 ms for event d06c6099
       2021-09-30T21:56:54.4414589Z 21:56:54.146 Pipeline - END 
TaskSchedulingStage for TASK pipeline for cluster HelixBrokerStarterTest. took: 
0 ms for event d06c6099
       2021-09-30T21:56:54.4416286Z 21:56:54.146 TaskPersistDataStage - START 
TaskPersistDataStage.process()
       2021-09-30T21:56:54.4418174Z 21:56:54.146 TaskPersistDataStage - END 
TaskPersistDataStage.process() for cluster HelixBrokerStarterTest took 0 ms
       2021-09-30T21:56:54.4420222Z 21:56:54.146 Pipeline - END 
TaskPersistDataStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4422339Z 21:56:54.146 AbstractAsyncBaseStage - 
Submitted asynchronous TASK::TaskGarbageCollectionStage task to worker
       2021-09-30T21:56:54.4424355Z 21:56:54.146 Pipeline - END 
TaskGarbageCollectionStage for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4426445Z 21:56:54.146 AbstractAsyncBaseStage - START 
AsyncProcess: TASK::TaskGarbageCollectionStage
       2021-09-30T21:56:54.4428417Z 21:56:54.146 AbstractAsyncBaseStage - END 
AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms
       2021-09-30T21:56:54.4430430Z 21:56:54.146 Pipeline - END 
TaskMessageGenerationPhase for TASK pipeline for cluster 
HelixBrokerStarterTest. took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4432517Z 21:56:54.146 Pipeline - END 
TaskMessageDispatchStage for TASK pipeline for cluster HelixBrokerStarterTest. 
took: 0 ms for event d06c6099
       2021-09-30T21:56:54.4435121Z 21:56:54.146 GenericHelixController - END: 
Invoking TASK controller pipeline for event PeriodicalRebalance::d06c6099 for 
cluster HelixBrokerStarterTest, took 42 ms
       2021-09-30T21:56:54.4437544Z 21:56:54.366 GenericHelixController - 
Controller rebalance pipeline triggered with event type: PeriodicalRebalance 
for cluster HelixBrokerStarterTest
       2021-09-30T21:56:54.4439693Z 21:56:54.369 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.4441706Z 21:56:54.369 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster DEFAULT 
event: PeriodicalRebalance  e66fe104
       2021-09-30T21:56:54.4443784Z 21:56:54.370 GenericHelixController - 
Controller already has periodical rebalance timer at period 10000
       2021-09-30T21:56:54.4445761Z 21:56:54.370 GenericHelixController - 
START: Invoking HelixBrokerStarterTest controller pipeline for cluster TASK 
event: PeriodicalRebalance  e66fe104
       2021-09-30T21:56:54.4447974Z 21:56:54.373 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4450011Z 21:56:54.373 PropertyCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : Refreshed 5 property IdealState took 1 
ms. Selective: true
       2021-09-30T21:56:54.4452252Z 21:56:54.374 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4454224Z 21:56:54.374 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4456273Z 21:56:54.374 PropertyCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : Refreshed 1 property LiveInstance took 
1 ms. Selective: true
       2021-09-30T21:56:54.4458412Z 21:56:54.374 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : Refreshed 5 property IdealState 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4460405Z 21:56:54.375 AbstractDataCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4462464Z 21:56:54.375 PropertyCache - Event 
HelixBrokerStarterTest::TASK::e66fe104 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4465098Z 21:56:54.375 BaseControllerDataProvider - 
Event e66fe104 : Reloaded InstanceConfig for cluster HelixBrokerStarterTest, 
TASK pipeline. Keys: [Broker_10.1.0.50_18099, Controller_localhost_18998, 
Broker_localhost_1, Broker_localhost_0, Server_localhost_0]
       2021-09-30T21:56:54.4467678Z 21:56:54.375 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:56:54.4469758Z 21:56:54.375 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : Refreshed 1 property LiveInstance 
took 1 ms. Selective: true
       2021-09-30T21:56:54.4471819Z 21:56:54.376 AbstractDataCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : 0 properties refreshed from ZK.
       2021-09-30T21:57:04.1087301Z 21:56:54.376 PropertyCache - Event 
HelixBrokerStarterTest::DEFAULT::e66fe104 : Refreshed 5 property InstanceConfig 
took 1 ms. Selective: true
     ```
   
   </detail>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to