Yingyi Bu has posted comments on this change. Change subject: ASTERIXDB-1566, ASTERIXDB-1733: Hash Group By and Hash Join conform to the memory budget ......................................................................
Patch Set 50: (15 comments) https://asterix-gerrit.ics.uci.edu/#/c/1056/50/asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml File asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml: Line 58: <value>640KB</value> Why do you change the sort memory? Does that change the coverage of the external algorithm? Line 63: </property> Is the group memory too large? It used to be 5 pages so as to be able to cover the external algorithm code path. Line 66: <value>640KB</value> Is the join memory too large? It used to be 5 pages so as to be able to cover the external algorithm code path. https://asterix-gerrit.ics.uci.edu/#/c/1056/50/asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java File asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java: Line 1055: // Temp: to be deleted Should you delete the system.out.println? https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java File hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java: Line 298: public static int calculateGroupByTableEntrySize(int memoryBudgetInBytes, int numberOfGroupByColumns, calculateGroupByTableEntrySize --> calculateGroupByTableCardinality EntrySize is a bit confusing. Should the code be shared between join and group-by? https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java: Line 48: private final IHyracksTaskContext ctx; > MAJOR SonarQube violation: address this? https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java: Line 234: // Temp: Remove temporary code Line 649: // Temp: debug purpose only. Remove the temporary code? Line 651: StringBuffer buf = new StringBuffer(); > MAJOR SonarQube violation: address this. https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java: Line 482: + ((bytes[offset + 2] & 0xff) << 8) + ((bytes[offset + 3] & 0xff) << 0); > CRITICAL SonarQube violation: address this. Line 490: bytes[offset++] = (byte) (value); > MAJOR SonarQube violation: address this. Line 517: @Override remove temporary code. https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AggregationTest.java File hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AggregationTest.java: Line 163: int frameLimits = 5; why does it have to be 5? 4 should still work? Similar for other methods. https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java File hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java: Line 166: OptimizedHybridHashJoinOperatorDescriptor join = new OptimizedHybridHashJoinOperatorDescriptor(spec, 24, 122, Why does it have to be larger? https://asterix-gerrit.ics.uci.edu/#/c/1056/50/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/AbstractExternalGroupbyTest.java File hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/AbstractExternalGroupbyTest.java: Line 190: int numFrames = 50; why does it have to be 50? Would that decrease the coverage for the code path that does the spilling? -- To view, visit https://asterix-gerrit.ics.uci.edu/1056 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2b323e9a2141b4c1dd1652a360d2d9354d3bc3f5 Gerrit-PatchSet: 50 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Taewoo Kim <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Steven Jacobs <[email protected]> Gerrit-Reviewer: Taewoo Kim <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Wenhai Li <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-HasComments: Yes
