surahman opened a new pull request #3735: URL: https://github.com/apache/incubator-heron/pull/3735
**_Issue #3733: CI Pipeline Failures Involving the <stmgr_unittest>_** There is an issue with running the `stmgr_unittest` on thread constrained systems. TravisCI also runs into this issue because the tier we run on provides us with 2 cores. The initial part of this PR is supposed to test to see if the offending tests have all been isolated. On a local system with reduced cores the following tests have are stuck waiting in the following busy loop: ```java while (dummy_stmgr->NumStartBPMsgs() == 0) sleep(1); ``` In these tests: https://github.com/apache/incubator-heron/blob/c32c09a0f09caca74dbf5715aa0632f0a50356c9/heron/stmgr/tests/cpp/server/stmgr_unittest.cpp#L1030-L1031 https://github.com/apache/incubator-heron/blob/c32c09a0f09caca74dbf5715aa0632f0a50356c9/heron/stmgr/tests/cpp/server/stmgr_unittest.cpp#L1140-L1141 https://github.com/apache/incubator-heron/blob/c32c09a0f09caca74dbf5715aa0632f0a50356c9/heron/stmgr/tests/cpp/server/stmgr_unittest.cpp#L1397-L1398 Here are the tests on a 4-core Ubuntu system now passing where they were failing before: <details><summary>Test results</summary> ```bash INFO: Elapsed time: 881.986s, Critical Path: 157.75s INFO: 2697 processes: 1222 internal, 1475 local. INFO: Build completed successfully, 2697 total actions //heron/api/tests/cpp:serialization_unittest PASSED in 0.0s //heron/api/tests/java:BaseWindowedBoltTest PASSED in 0.5s //heron/api/tests/java:ConfigTest PASSED in 0.5s //heron/api/tests/java:CountStatAndMetricTest PASSED in 0.3s //heron/api/tests/java:GeneralReduceByKeyAndWindowOperatorTest PASSED in 0.7s //heron/api/tests/java:HeronSubmitterTest PASSED in 3.0s //heron/api/tests/java:JoinOperatorTest PASSED in 0.9s //heron/api/tests/java:KVStreamletShadowTest PASSED in 0.5s //heron/api/tests/java:KeyByOperatorTest PASSED in 0.7s //heron/api/tests/java:LatencyStatAndMetricTest PASSED in 0.4s //heron/api/tests/java:ReduceByKeyAndWindowOperatorTest PASSED in 0.9s //heron/api/tests/java:StreamletImplTest PASSED in 0.6s //heron/api/tests/java:StreamletShadowTest PASSED in 0.7s //heron/api/tests/java:StreamletUtilsTest PASSED in 0.3s //heron/api/tests/java:UtilsTest PASSED in 0.4s //heron/api/tests/java:WaterMarkEventGeneratorTest PASSED in 0.5s //heron/api/tests/java:WindowManagerTest PASSED in 0.5s //heron/api/tests/java:WindowedBoltExecutorTest PASSED in 0.8s //heron/api/tests/scala:api-scala-test PASSED in 1.2s WARNING: //heron/api/tests/scala:api-scala-test: Test execution time (1.2s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small". //heron/ckptmgr/tests/java:CheckpointManagerServerTest PASSED in 1.0s //heron/common/tests/cpp/basics:fileutils_unittest PASSED in 0.0s //heron/common/tests/cpp/basics:rid_unittest PASSED in 0.0s //heron/common/tests/cpp/basics:strutils_unittest PASSED in 0.0s //heron/common/tests/cpp/basics:utils_unittest PASSED in 0.0s //heron/common/tests/cpp/config:topology-config-helper_unittest PASSED in 0.0s //heron/common/tests/cpp/errors:errors_unittest PASSED in 0.0s //heron/common/tests/cpp/errors:module_unittest PASSED in 0.1s //heron/common/tests/cpp/errors:syserrs_unittest PASSED in 0.0s //heron/common/tests/cpp/metrics:count-metric_unittest PASSED in 0.0s //heron/common/tests/cpp/metrics:mean-metric_unittest PASSED in 0.0s //heron/common/tests/cpp/metrics:multi-count-metric_unittest PASSED in 0.0s //heron/common/tests/cpp/metrics:multi-mean-metric_unittest PASSED in 0.1s //heron/common/tests/cpp/metrics:time-spent-metric_unittest PASSED in 1.3s //heron/common/tests/cpp/network:http_unittest PASSED in 0.1s //heron/common/tests/cpp/network:order_unittest PASSED in 0.1s //heron/common/tests/cpp/network:packet_unittest PASSED in 0.1s //heron/common/tests/cpp/network:piper_unittest PASSED in 2.0s //heron/common/tests/cpp/network:rate_limit_unittest PASSED in 4.1s //heron/common/tests/cpp/network:switch_unittest PASSED in 0.3s //heron/common/tests/cpp/threads:spcountdownlatch_unittest PASSED in 2.1s //heron/common/tests/java:ByteAmountTest PASSED in 0.3s //heron/common/tests/java:CommunicatorTest PASSED in 0.3s //heron/common/tests/java:ConfigReaderTest PASSED in 0.4s //heron/common/tests/java:EchoTest PASSED in 1.0s //heron/common/tests/java:FileUtilsTest PASSED in 1.3s //heron/common/tests/java:HeronServerTest PASSED in 1.6s //heron/common/tests/java:PackageTypeTest PASSED in 0.3s //heron/common/tests/java:SysUtilsTest PASSED in 5.5s //heron/common/tests/java:SystemConfigTest PASSED in 0.4s //heron/common/tests/java:TopologyUtilsTest PASSED in 0.4s //heron/common/tests/java:WakeableLooperTest PASSED in 1.4s //heron/common/tests/python/pex_loader:pex_loader_unittest PASSED in 1.1s //heron/downloaders/tests/java:DLDownloaderTest PASSED in 1.1s //heron/downloaders/tests/java:ExtractorTests PASSED in 0.4s //heron/downloaders/tests/java:RegistryTest PASSED in 0.4s //heron/executor/tests/python:executor_unittest PASSED in 1.5s //heron/healthmgr/tests/java:BackPressureDetectorTest PASSED in 0.7s //heron/healthmgr/tests/java:BackPressureSensorTest PASSED in 0.8s //heron/healthmgr/tests/java:BufferSizeSensorTest PASSED in 0.7s //heron/healthmgr/tests/java:DataSkewDiagnoserTest PASSED in 0.7s //heron/healthmgr/tests/java:ExecuteCountSensorTest PASSED in 0.7s //heron/healthmgr/tests/java:GrowingWaitQueueDetectorTest PASSED in 0.7s //heron/healthmgr/tests/java:HealthManagerTest PASSED in 0.9s //heron/healthmgr/tests/java:HealthPolicyConfigReaderTest PASSED in 0.5s //heron/healthmgr/tests/java:LargeWaitQueueDetectorTest PASSED in 0.7s //heron/healthmgr/tests/java:MetricsCacheMetricsProviderTest PASSED in 0.8s //heron/healthmgr/tests/java:PackingPlanProviderTest PASSED in 0.9s //heron/healthmgr/tests/java:ProcessingRateSkewDetectorTest PASSED in 0.7s //heron/healthmgr/tests/java:ScaleUpResolverTest PASSED in 0.8s //heron/healthmgr/tests/java:SlowInstanceDiagnoserTest PASSED in 0.8s //heron/healthmgr/tests/java:UnderProvisioningDiagnoserTest PASSED in 0.8s //heron/healthmgr/tests/java:WaitQueueSkewDetectorTest PASSED in 0.9s //heron/instance/tests/java:ActivateDeactivateTest PASSED in 0.7s //heron/instance/tests/java:BoltInstanceTest PASSED in 0.5s //heron/instance/tests/java:BoltStatefulInstanceTest PASSED in 2.7s //heron/instance/tests/java:ConnectTest PASSED in 0.8s //heron/instance/tests/java:CustomGroupingTest PASSED in 0.7s //heron/instance/tests/java:EmitDirectBoltTest PASSED in 0.9s //heron/instance/tests/java:EmitDirectSpoutTest PASSED in 0.6s //heron/instance/tests/java:GlobalMetricsTest PASSED in 0.4s //heron/instance/tests/java:HandleReadTest PASSED in 0.7s //heron/instance/tests/java:HandleWriteTest PASSED in 5.7s //heron/instance/tests/java:MultiAssignableMetricTest PASSED in 0.3s //heron/instance/tests/java:SpoutInstanceTest PASSED in 2.7s //heron/instance/tests/java:SpoutStatefulInstanceTest PASSED in 2.6s //heron/instance/tests/python/network:event_looper_unittest PASSED in 3.1s //heron/instance/tests/python/network:gateway_looper_unittest PASSED in 11.0s //heron/instance/tests/python/network:heron_client_unittest PASSED in 1.2s //heron/instance/tests/python/network:metricsmgr_client_unittest PASSED in 1.2s //heron/instance/tests/python/network:protocol_unittest PASSED in 1.1s //heron/instance/tests/python/network:st_stmgrcli_unittest PASSED in 1.3s //heron/instance/tests/python/utils:communicator_unittest PASSED in 1.1s //heron/instance/tests/python/utils:custom_grouping_unittest PASSED in 1.1s //heron/instance/tests/python/utils:global_metrics_unittest PASSED in 1.4s //heron/instance/tests/python/utils:log_unittest PASSED in 1.0s //heron/instance/tests/python/utils:metrics_helper_unittest PASSED in 1.1s //heron/instance/tests/python/utils:outgoing_tuple_helper_unittest PASSED in 1.1s //heron/instance/tests/python/utils:pplan_helper_unittest PASSED in 1.2s //heron/instance/tests/python/utils:py_metrics_unittest PASSED in 1.1s //heron/instance/tests/python/utils:topology_context_impl_unittest PASSED in 1.1s //heron/instance/tests/python/utils:tuple_helper_unittest PASSED in 1.1s //heron/io/dlog/tests/java:DLInputStreamTest PASSED in 0.6s //heron/io/dlog/tests/java:DLOutputStreamTest PASSED in 0.5s //heron/metricscachemgr/tests/java:CacheCoreTest PASSED in 0.5s //heron/metricscachemgr/tests/java:MetricsCacheQueryUtilsTest PASSED in 0.4s //heron/metricscachemgr/tests/java:MetricsCacheTest PASSED in 0.4s //heron/metricsmgr/tests/java:FileSinkTest PASSED in 0.5s //heron/metricsmgr/tests/java:HandleTManagerLocationTest PASSED in 0.6s //heron/metricsmgr/tests/java:MetricsCacheSinkTest PASSED in 9.5s //heron/metricsmgr/tests/java:MetricsManagerServerTest PASSED in 0.8s //heron/metricsmgr/tests/java:MetricsUtilTests PASSED in 0.4s //heron/metricsmgr/tests/java:PrometheusSinkTests PASSED in 0.4s //heron/metricsmgr/tests/java:SinkExecutorTest PASSED in 0.5s //heron/metricsmgr/tests/java:TManagerSinkTest PASSED in 9.5s //heron/metricsmgr/tests/java:WebSinkTest PASSED in 0.5s //heron/packing/tests/java:FirstFitDecreasingPackingTest PASSED in 1.2s //heron/packing/tests/java:PackingPlanBuilderTest PASSED in 0.4s //heron/packing/tests/java:PackingUtilsTest PASSED in 0.4s //heron/packing/tests/java:ResourceCompliantRRPackingTest PASSED in 0.8s //heron/packing/tests/java:RoundRobinPackingTest PASSED in 0.7s //heron/packing/tests/java:ScorerTest PASSED in 0.3s //heron/scheduler-core/tests/java:HttpServiceSchedulerClientTest PASSED in 1.7s //heron/scheduler-core/tests/java:JsonFormatterUtilsTest PASSED in 0.5s //heron/scheduler-core/tests/java:LaunchRunnerTest PASSED in 1.4s //heron/scheduler-core/tests/java:LauncherUtilsTest PASSED in 2.4s //heron/scheduler-core/tests/java:LibrarySchedulerClientTest PASSED in 0.5s //heron/scheduler-core/tests/java:RuntimeManagerMainTest PASSED in 3.8s //heron/scheduler-core/tests/java:RuntimeManagerRunnerTest PASSED in 3.4s //heron/scheduler-core/tests/java:SchedulerClientFactoryTest PASSED in 2.1s //heron/scheduler-core/tests/java:SchedulerMainTest PASSED in 4.9s //heron/scheduler-core/tests/java:SchedulerServerTest PASSED in 0.5s //heron/scheduler-core/tests/java:SchedulerUtilsTest PASSED in 1.3s //heron/scheduler-core/tests/java:SubmitDryRunRenderTest PASSED in 2.3s //heron/scheduler-core/tests/java:SubmitterMainTest PASSED in 1.7s //heron/scheduler-core/tests/java:UpdateDryRunRenderTest PASSED in 2.4s //heron/scheduler-core/tests/java:UpdateTopologyManagerTest PASSED in 12.2s //heron/schedulers/tests/java:AuroraCLIControllerTest PASSED in 0.5s //heron/schedulers/tests/java:AuroraContextTest PASSED in 0.3s //heron/schedulers/tests/java:AuroraLauncherTest PASSED in 0.9s //heron/schedulers/tests/java:AuroraSchedulerTest PASSED in 3.9s //heron/schedulers/tests/java:HeronExecutorTaskTest PASSED in 1.6s //heron/schedulers/tests/java:HeronMasterDriverTest PASSED in 2.6s //heron/schedulers/tests/java:KubernetesContextTest PASSED in 0.5s //heron/schedulers/tests/java:KubernetesControllerTest PASSED in 0.4s //heron/schedulers/tests/java:KubernetesLauncherTest PASSED in 1.0s //heron/schedulers/tests/java:KubernetesSchedulerTest PASSED in 0.9s //heron/schedulers/tests/java:KubernetesUtilsTest PASSED in 0.4s //heron/schedulers/tests/java:LaunchableTaskTest PASSED in 0.5s //heron/schedulers/tests/java:LocalLauncherTest PASSED in 1.2s //heron/schedulers/tests/java:LocalSchedulerTest PASSED in 0.6s //heron/schedulers/tests/java:MarathonControllerTest PASSED in 1.6s //heron/schedulers/tests/java:MarathonLauncherTest PASSED in 0.9s //heron/schedulers/tests/java:MarathonSchedulerTest PASSED in 0.5s //heron/schedulers/tests/java:MesosFrameworkTest PASSED in 0.6s //heron/schedulers/tests/java:MesosLauncherTest PASSED in 0.6s //heron/schedulers/tests/java:MesosSchedulerTest PASSED in 0.8s //heron/schedulers/tests/java:NomadSchedulerTest PASSED in 3.2s //heron/schedulers/tests/java:SlurmControllerTest PASSED in 1.5s //heron/schedulers/tests/java:SlurmLauncherTest PASSED in 1.1s //heron/schedulers/tests/java:SlurmSchedulerTest PASSED in 1.8s //heron/schedulers/tests/java:TaskResourcesTest PASSED in 0.4s //heron/schedulers/tests/java:TaskUtilsTest PASSED in 0.3s //heron/schedulers/tests/java:V1ControllerTest PASSED in 2.8s //heron/schedulers/tests/java:VolumesTests PASSED in 0.3s //heron/schedulers/tests/java:YarnLauncherTest PASSED in 0.7s //heron/schedulers/tests/java:YarnSchedulerTest PASSED in 0.5s //heron/simulator/tests/java:AllGroupingTest PASSED in 0.3s //heron/simulator/tests/java:CustomGroupingTest PASSED in 0.4s //heron/simulator/tests/java:FieldsGroupingTest PASSED in 1.0s //heron/simulator/tests/java:InstanceExecutorTest PASSED in 0.5s //heron/simulator/tests/java:LowestGroupingTest PASSED in 0.4s //heron/simulator/tests/java:RotatingMapTest PASSED in 0.3s //heron/simulator/tests/java:ShuffleGroupingTest PASSED in 0.3s //heron/simulator/tests/java:SimulatorTest PASSED in 0.4s //heron/simulator/tests/java:TopologyManagerTest PASSED in 0.4s //heron/simulator/tests/java:TupleCacheTest PASSED in 0.3s //heron/simulator/tests/java:XORManagerTest PASSED in 0.5s //heron/spi/tests/java:ConfigLoaderTest PASSED in 2.2s //heron/spi/tests/java:ConfigTest PASSED in 1.2s //heron/spi/tests/java:ContextTest PASSED in 0.3s //heron/spi/tests/java:ExceptionInfoTest PASSED in 0.3s //heron/spi/tests/java:KeysTest PASSED in 0.3s //heron/spi/tests/java:MetricsInfoTest PASSED in 0.3s //heron/spi/tests/java:MetricsRecordTest PASSED in 0.3s //heron/spi/tests/java:NetworkUtilsTest PASSED in 2.1s //heron/spi/tests/java:PackingPlanTest PASSED in 0.3s //heron/spi/tests/java:ResourceTest PASSED in 0.3s //heron/spi/tests/java:ShellUtilsTest PASSED in 1.6s //heron/spi/tests/java:TokenSubTest PASSED in 0.3s //heron/spi/tests/java:UploaderUtilsTest PASSED in 0.7s //heron/statefulstorages/tests/java:DlogStorageTest PASSED in 2.5s //heron/statefulstorages/tests/java:HDFSStorageTest PASSED in 2.5s //heron/statefulstorages/tests/java:LocalFileSystemStorageTest PASSED in 1.4s //heron/statemgrs/tests/cpp:zk-statemgr_unittest PASSED in 0.0s //heron/statemgrs/tests/java:CuratorStateManagerTest PASSED in 0.8s //heron/statemgrs/tests/java:LocalFileSystemStateManagerTest PASSED in 1.2s //heron/statemgrs/tests/java:ZkUtilsTest PASSED in 1.6s //heron/statemgrs/tests/python:configloader_unittest PASSED in 1.0s //heron/statemgrs/tests/python:statemanagerfactory_unittest PASSED in 1.2s //heron/statemgrs/tests/python:zkstatemanager_unittest PASSED in 1.1s //heron/stmgr/tests/cpp/grouping:all-grouping_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/grouping:custom-grouping_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/grouping:fields-grouping_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/grouping:lowest-grouping_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/grouping:shuffle-grouping_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/server:checkpoint-gateway_unittest PASSED in 1.4s WARNING: //heron/stmgr/tests/cpp/server:checkpoint-gateway_unittest: Test execution time (1.4s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small". //heron/stmgr/tests/cpp/server:stateful-restorer_unittest PASSED in 0.1s WARNING: //heron/stmgr/tests/cpp/server:stateful-restorer_unittest: Test execution time (0.1s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small". //heron/stmgr/tests/cpp/server:stmgr_unittest PASSED in 20.1s WARNING: //heron/stmgr/tests/cpp/server:stmgr_unittest: Test execution time (20.1s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small". //heron/stmgr/tests/cpp/util:neighbour_calculator_unittest PASSED in 0.0s WARNING: //heron/stmgr/tests/cpp/util:neighbour_calculator_unittest: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small". //heron/stmgr/tests/cpp/util:rotating-map_unittest PASSED in 0.0s //heron/stmgr/tests/cpp/util:tuple-cache_unittest PASSED in 3.6s //heron/stmgr/tests/cpp/util:xor-manager_unittest PASSED in 4.0s //heron/tmanager/tests/cpp/server:stateful_checkpointer_unittest PASSED in 0.1s //heron/tmanager/tests/cpp/server:stateful_restorer_unittest PASSED in 5.0s //heron/tmanager/tests/cpp/server:tcontroller_unittest PASSED in 0.0s //heron/tmanager/tests/cpp/server:tmanager_unittest PASSED in 36.2s //heron/tools/apiserver/tests/java:ConfigUtilsTests PASSED in 0.5s //heron/tools/apiserver/tests/java:TopologyResourceTests PASSED in 0.7s //heron/tools/cli/tests/python:client_command_unittest PASSED in 1.4s //heron/tools/cli/tests/python:opts_unittest PASSED in 1.1s //heron/tools/explorer/tests/python:explorer_unittest PASSED in 1.3s //heron/tools/tracker/tests/python:query_operator_unittest PASSED in 1.6s //heron/tools/tracker/tests/python:query_unittest PASSED in 1.6s //heron/tools/tracker/tests/python:topology_unittest PASSED in 1.5s //heron/tools/tracker/tests/python:tracker_unittest PASSED in 1.5s //heron/uploaders/tests/java:DlogUploaderTest PASSED in 0.7s //heron/uploaders/tests/java:GcsUploaderTests PASSED in 0.5s //heron/uploaders/tests/java:HdfsUploaderTest PASSED in 0.5s //heron/uploaders/tests/java:HttpUploaderTest PASSED in 0.8s //heron/uploaders/tests/java:LocalFileSystemConfigTest PASSED in 0.3s //heron/uploaders/tests/java:LocalFileSystemContextTest PASSED in 0.3s //heron/uploaders/tests/java:LocalFileSystemUploaderTest PASSED in 0.4s //heron/uploaders/tests/java:S3UploaderTest PASSED in 1.1s //heron/uploaders/tests/java:ScpUploaderTest PASSED in 0.5s INFO: Build completed successfully, 2697 total actions ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
