[jira] [Updated] (KYLIN-4397) Use newLinkedHashMap in AssignmentUtil.java
[ https://issues.apache.org/jira/browse/KYLIN-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cpugputpu updated KYLIN-4397: - Description: The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. was: The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. > Use newLinkedHashMap in AssignmentUtil.java > --- > > Key: KYLIN-4397 > URL: https://issues.apache.org/jira/browse/KYLIN-4397 > Project: Kylin > Issue Type: Bug >Reporter: cpugputpu >Priority: Minor > > The tests in > _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ > can fail and the failure is presented as follow. > _java.lang.AssertionError: expected:<2> but was:<1>_ > _at > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ > when executing _assertEquals(2, rsAssignment.size());_ > > The root cause of this failure is when executing _Map List>> result = assigner.reBalancePlan(rsList, cubes, > existingAssignments);_ it will iterate over a HashMap. The stack trace is > here: > java.util.HashMap$EntrySet.iterator(HashMap.java:1014) > > org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) > > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ > The specification about HashMap says that "this class makes no guarantees as > to the order of the map; in particular, it does not guarantee that the order > will remain constant over time". The documentation is here for your > reference: > [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] > The fix is to use LinkedHashMap instead of HashMap when initializing a Map. > In this way, the non-deterministic behaviour is eliminated and the test will > not suffer from the failure again. The
[jira] [Updated] (KYLIN-4397) Use newLinkedHashMap in AssignmentUtil.java
[ https://issues.apache.org/jira/browse/KYLIN-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cpugputpu updated KYLIN-4397: - Summary: Use newLinkedHashMap in AssignmentUtil.java (was: Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java) > Use newLinkedHashMap in AssignmentUtil.java > --- > > Key: KYLIN-4397 > URL: https://issues.apache.org/jira/browse/KYLIN-4397 > Project: Kylin > Issue Type: Bug >Reporter: cpugputpu >Priority: Minor > > The tests in > _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ > can fail and the failure is presented as follow. > _java.lang.AssertionError: expected:<2> but was:<1>_ > _at > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ > when executing _assertEquals(2, rsAssignment.size());_ > > The root cause of this failure is when executing _Map List>> result = assigner.reBalancePlan(rsList, cubes, > existingAssignments);_ it will iterate over a HashMap. The stack trace is > here: > _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) > org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ > The specification about HashMap says that "this class makes no guarantees as > to the order of the map; in particular, it does not guarantee that the order > will remain constant over time". The documentation is here for your > reference: > [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] > The fix is to use LinkedHashMap instead of HashMap when initializing a Map. > In this way, the non-deterministic behaviour is eliminated and the test will > not suffer from the failure again. The code will be more stable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4397) Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java
cpugputpu created KYLIN-4397: Summary: Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java Key: KYLIN-4397 URL: https://issues.apache.org/jira/browse/KYLIN-4397 Project: Kylin Issue Type: Bug Reporter: cpugputpu The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4386) Use LinkedHashMap for a deterministic order in test
cpugputpu created KYLIN-4386: Summary: Use LinkedHashMap for a deterministic order in test Key: KYLIN-4386 URL: https://issues.apache.org/jira/browse/KYLIN-4386 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Reporter: cpugputpu The tests in _org.apache.kylin.stream.coordinator.CoordinatorTest#testReassignFailOnStartNew_ and _org.apache.kylin.stream.coordinator.CoordinatorTest#testReassignFailOnStopAndSync_ will fail when making the following assertions: _assertSame(ClusterStateException.ClusterState.ROLLBACK_FAILED, rune.getClusterState());_ _assertSame(ClusterStateException.TransactionStep.STOP_AND_SNYC, rune.getTransactionStep());_ The failure is reported as follows: java.lang.Exception: Unexpected exception, expected but was at org.apache.kylin.stream.coordinator.CoordinatorTest.testReassignFailOnStartNew(CoordinatorTest.java:271) After my analysis, I find that the root cause is that the variable _newAssignMap_ is a HashMap, which makes no guarantee about the order when iterating in _org.apache.kylin.stream.coordinator.Coordinator.doReassign(Coordinator.java:516)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap so that the non-deterministic behaviour is eliminated. It will not affect the performance and can make the tests more stable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
