GSharayu commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r967315539
########## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/SegmentAssignment.java: ########## @@ -31,20 +31,17 @@ /** * Interface for segment assignment and table rebalance. - * <p> - * TODO: Add SegmentAssignmentStrategy interface and support custom segment assignment strategy (e.g. cost based segment - * assignment). SegmentAssignmentStrategy should not be coupled with SegmentAssignment, and SegmentAssignment - * should be able to choose the segment assignment strategy based on the configuration. */ public interface SegmentAssignment { /** * Initializes the segment assignment. - * * @param helixManager Helix manager * @param tableConfig Table config + * @param instancePartitionsMap Map from type (OFFLINE|CONSUMING|COMPLETED) to instance partitions */ - void init(HelixManager helixManager, TableConfig tableConfig); + void init(HelixManager helixManager, TableConfig tableConfig, + Map<InstancePartitionsType, InstancePartitions> instancePartitionsMap); Review Comment: > ``` > 2022-09-08T23:47:33.2814091Z org.apache.pinot.controller.api.exception.TableAlreadyExistsException: Table config for testTable_OFFLINE already exists. If this is unexpected, try deleting the table to remove all metadata associated with it. > 2022-09-08T23:47:33.2815139Z at org.apache.pinot.controller.helix.core.PinotHelixResourceManager.addTable(PinotHelixResourceManager.java:1412) > 2022-09-08T23:47:33.2816081Z at org.apache.pinot.controller.helix.core.rebalance.TableRebalancerClusterStatelessTest.testRebalanceWithTiers(TableRebalancerClusterStatelessTest.java:343) > 2022-09-08T23:47:33.2816919Z at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2022-09-08T23:47:33.2817485Z at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2022-09-08T23:47:33.2818120Z at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2022-09-08T23:47:33.2818640Z at java.base/java.lang.reflect.Method.invoke(Method.java:566) > 2022-09-08T23:47:33.2819129Z at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108) > 2022-09-08T23:47:33.2819624Z at org.testng.internal.Invoker.invokeMethod(Invoker.java:661) > 2022-09-08T23:47:33.2820029Z at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869) > 2022-09-08T23:47:33.2820469Z at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193) > 2022-09-08T23:47:33.2820970Z at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126) > 2022-09-08T23:47:33.2835876Z at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) > 2022-09-08T23:47:33.2836443Z at org.testng.TestRunner.privateRun(TestRunner.java:744) > 2022-09-08T23:47:33.2836810Z at org.testng.TestRunner.run(TestRunner.java:602) > 2022-09-08T23:47:33.2837169Z at org.testng.SuiteRunner.runTest(SuiteRunner.java:380) > 2022-09-08T23:47:33.2837552Z at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375) > 2022-09-08T23:47:33.2837963Z at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340) > 2022-09-08T23:47:33.2838328Z at org.testng.SuiteRunner.run(SuiteRunner.java:289) > 2022-09-08T23:47:33.2838718Z at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) > 2022-09-08T23:47:33.2839140Z at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) > 2022-09-08T23:47:33.2839547Z at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301) > 2022-09-08T23:47:33.2839920Z at org.testng.TestNG.runSuitesLocally(TestNG.java:1226) > 2022-09-08T23:47:33.2840271Z at org.testng.TestNG.runSuites(TestNG.java:1144) > 2022-09-08T23:47:33.2840588Z at org.testng.TestNG.run(TestNG.java:1115) > 2022-09-08T23:47:33.2841002Z at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:284) > 2022-09-08T23:47:33.2841558Z at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75) > 2022-09-08T23:47:33.2842129Z at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:119) > 2022-09-08T23:47:33.2842887Z at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428) > 2022-09-08T23:47:33.2843451Z at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) > 2022-09-08T23:47:33.2843953Z at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562) > 2022-09-08T23:47:33.2844449Z at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548) > 2022-09-08T23:47:33.2844718Z > 2022-09-08T23:47:33.2845206Z [ERROR] org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testUpdateBrokerResource Time elapsed: 0.216 s > 2022-09-08T23:47:33.2846308Z [ERROR] org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testValidateDimTableTenantConfig Time elapsed: 0.072 s > 2022-09-08T23:47:33.2847330Z [ERROR] org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testValidateTenantConfig Time elapsed: 0.052 s > 2022-09-08T23:47:33.2848188Z [ERROR] org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testBrokerTenant Time elapsed: 0.953 s > 2022-09-08T23:47:33.2848940Z [ERROR] org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testEmptyServerTenant Time elapsed: 0.02 s > 2022-09-08T23:47:33.2849714Z [ERROR] org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testServerTenant Time elapsed: 0.388 s > 2022-09-08T23:47:33.2850500Z [ERROR] org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testDualModeController Time elapsed: 9.822 s > 2022-09-08T23:47:33.2851344Z [ERROR] org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testHelixOnlyController Time elapsed: 1.718 s > 2022-09-08T23:47:33.2852184Z [ERROR] org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testPinotOnlyController Time elapsed: 5.248 s > 2022-09-08T23:47:33.2853009Z [ERROR] org.apache.pinot.controller.api.PinotBrokerRestletResourceStatelessTest.testGetBrokers Time elapsed: 14.044 s > 2022-09-08T23:47:33.2853853Z [ERROR] org.apache.pinot.controller.api.PinotIngestionRestletResourceStatelessTest.testIngestEndpoint Time elapsed: 0.492 s > 2022-09-08T23:47:33.2855076Z [ERROR] org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testDefaultPinotTaskManagerNoScheduler Time elapsed: 3.905 s > 2022-09-08T23:47:33.2856187Z [ERROR] org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testPinotTaskManagerSchedulerWithRestart Time elapsed: 8.857 s > 2022-09-08T23:47:33.2857313Z [ERROR] org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testPinotTaskManagerSchedulerWithUpdate Time elapsed: 2.65 s > 2022-09-08T23:47:33.2858407Z [ERROR] org.apache.pinot.controller.validation.ValidationManagerStatelessTest.testRebuildBrokerResourceWhenBrokerAdded Time elapsed: 0.04 s > 2022-09-08T23:47:33.3510290Z [INFO] > 2022-09-08T23:47:33.3511030Z [INFO] Results: > 2022-09-08T23:47:33.3512144Z [INFO] > 2022-09-08T23:47:33.3512504Z [ERROR] Failures: > 2022-09-08T23:47:33.3513121Z [ERROR] TableRebalancerClusterStatelessTest.testRebalance:210 expected [DONE] but found [FAILED] > 2022-09-08T23:47:33.3534306Z [ERROR] TableRebalancerClusterStatelessTest.testRebalanceWithTiers:343 ยป TableAlreadyExists > ``` > > It looks that rebalancing is failing. Fixed this as well.I think this was not showing up because of other flaky test. In the existing code, we set BalanceNumSegmentAssignmentStrategy as default and update table config replication which did not match the assignment strategy which caused this test to fail. Handled change in strategyAssignmentFactory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org