Yingyi Bu has posted comments on this change. Change subject: ASTERIXDB-1566, ASTERIXDB-1733: Hash Group By and Hash Join conform to the memory budget ......................................................................
Patch Set 54: (10 comments) https://asterix-gerrit.ics.uci.edu/#/c/1056/54/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml File asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml: Line 62: <value>256KB</value> keep the number unchanged. Line 66: <value>256KB</value> keep the number unchanged. https://asterix-gerrit.ics.uci.edu/#/c/1056/54/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm File asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.adm: Line 16: "compiler.groupmemory": 262144, Ideally, those two numbers should be unchanged. https://asterix-gerrit.ics.uci.edu/#/c/1056/54/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java File hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java: Line 298: public static int calculateGroupByTableCardinality(int memoryBudgetInBytes, int numberOfGroupByColumns, public -> private Line 313: long possibleNumberOfHashEntries = (long) 2 << numberOfBits; (long) 2 << numberOfBits -> 2L << numberOfBits Line 317: long groupByTableByteSize = SerializableHashTable.getExpectedTableSizeInByte((int) groupByTableSize, frameSize); data loss? https://asterix-gerrit.ics.uci.edu/#/c/1056/54/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SerializableHashTable.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SerializableHashTable.java: Line 44: private static final Logger LOGGER = Logger.getLogger(SerializableHashTable.class.getName()); > MAJOR SonarQube violation: address this. https://asterix-gerrit.ics.uci.edu/#/c/1056/54/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java: Line 444: public static int getExpectedTableSizeInFrame(int tableSize, int frameSize) { this should be a long? Line 447: .ceil(((double) getNumberOfEntryInSlot() * 2 * getUnitSize() * tableSize) / (double) frameSize)); it seems that it's not necessary to go through the double detour. Using long/integers and ignoring things <1 would be fine? Line 490: bytes[offset++] = (byte) (value); > MAJOR SonarQube violation: address this. -- To view, visit https://asterix-gerrit.ics.uci.edu/1056 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2b323e9a2141b4c1dd1652a360d2d9354d3bc3f5 Gerrit-PatchSet: 54 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Taewoo Kim <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Steven Jacobs <[email protected]> Gerrit-Reviewer: Taewoo Kim <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Wenhai Li <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-HasComments: Yes
