Hello Jenkins, I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/1238 to look at the new patch set (#13). Change subject: Add the forward for ParallelSort. ...................................................................... Add the forward for ParallelSort. Based on the aggFuncs, enable the parallel sort by: 1. Maintaining the rangeMap within the HyracksJobletConext 2. Setup forward operator to connect the partitioning and sorting operators. 3. Add the unit test for parallel histogram based on construction-merge. 4. Add four partitioners as well as the schema-driven assigner in asterixTest. Change-Id: Ifc3ca935f569d33bda81e4c3bcf7daad8749d84c --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java A asterixdb/asterix-app/data/skew/zipfan1.tbl A asterixdb/asterix-app/data/skew/zipfan2.tbl M asterixdb/asterix-app/pom.xml A asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/AsterixAlgebricksHyracksIntegrationUtil.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ParallelSortRuntimeTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_double/sum_double.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_float/sum_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int16/sum_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int64/sum_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int8/sum_int8.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/global-rg/global-avg.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/local-rg/local-rg.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double-null/rg-double-null.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double/rg-double.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_double/rg_double.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16/rg_int16.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int32/rg_int32.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int64/rg_int64.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_string/rg_string.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/AsterixBuiltinFunctions.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/GlobalRangeGeneratorTypeComputer.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/LocalRangeGeneratorTypeComputer.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RangeGeneratorTypeComputer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/ScalarVersionOfAggregateResultType.java M asterixdb/asterix-om/src/test/java/org/apache/asterix/om/typecomputer/TypeComputerTest.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/histogram/AsterixStreamingHistogram.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateFunction.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceCombinerRule.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksTaskContext.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePartitionComputerFactory.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Task.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/FieldHashPartitionComputerFactory.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/RandomPartitionComputerFactory.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/RepartitionComputerFactory.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/FieldRangePartitionComputerFactory.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangeMap.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/connectors/LocalityAwareMToNPartitioningConnectorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/connectors/MToNPartitioningConnectorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/connectors/MToNPartitioningMergingConnectorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/connectors/MToNPartitioningWithMessageConnectorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/GraceHashJoinOperatorNodePushable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/GraceHashJoinPartitionBuildOperatorNodePushable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoinOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/HistogramType.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHeapList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHistogram.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/repartition/FieldRangeRepartitionComputerFactory.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/repartition/ForwardOperatorNodePushable.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/repartition/MaterializingForwardOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/repartition/MaterializingSampleTaskState.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractStreamingHistogram.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/DominantQuantile.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantile.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantileComparator.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinDifferenceQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinSumCountQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/RepartitioningState.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/util/HistogramUtils.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/range/TypeTranslateTest.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfan1.tbl A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfan2.tbl M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/pom.xml A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramAlgorithmTests.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramFieldType.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HyracksStreamingHistogram.java M hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestTaskContext.java 113 files changed, 15,872 insertions(+), 49 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/38/1238/13 -- To view, visit https://asterix-gerrit.ics.uci.edu/1238 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifc3ca935f569d33bda81e4c3bcf7daad8749d84c Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Wenhai Li <lwhaym...@yahoo.com> Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu>