Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24271 )
Change subject: IMPALA-14904: Calcite planner: Enable width_bucket function ...................................................................... IMPALA-14904: Calcite planner: Enable width_bucket function This commit enables the width_bucket function. The code for resolving/coercing decimal operand types had to be refactored and modified. The following fixes were needed to enable this: - The config.fmpp now treats width_bucket as a non-reserved word so that it can be parsed as a function name. - After IMPALA-14903, the return type is no longer modified, so no check is needed within CoerceNodes when resolving the function. - If the operand for the aggregation function in CoerceNodes is a decimal, it will simply derive the operand from the input if the function has a decimal wildcard as an operand. If the decimal type is explicitly specified for the aggregate function, a cast may be needed, so the type returned by the UDF function is used. - CoerceNodes now checks the precision and scale for casting in addition to just the datatype - CoerceOperandShuttle coerces the decimal operands - If it is a binary arithmetic or binary comparison, there is no need for coercing since the backend can handle different decimal types for these functions - If the return type is a decimal, then the common type has already been determined through inferReturnType in the validation stage and this can be used. - If the return type is something other than a decimal, the common type is derived, and an exception is thrown if there is no common type. This is tested within the test_decimal_fuzz.py e2e test. - A small fix was made to the FunctionResolver which now uses the actual datatypes instead of a 'normalized' wildcard type for resolving. - There are several instances where the TypeCompatibility.DEFAULT is used, but this is incorrect for Decimal V2, which is the only mode supported by the Calcite planner. The TypeCompatibility.STRICT_DECIMAL is used instead. - The ImpalaTypeFactoryImpl was created to handle compatible types across rows in a values clause. This acts well as a singleton object, so there are multiple places that have been changed to handle this. - The ImpalaTypeCoercionImpl.binaryComparison method has been added to prevent casting on binary comparisons for decimals with different precisions and/or scales. Change-Id: I2944fde1301e48b3fb0c29be7290b1ad9533e5b6 Reviewed-on: http://gerrit.cloudera.org:8080/24271 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Michael Smith <[email protected]> --- M java/calcite-planner/src/main/codegen/config.fmpp M java/calcite-planner/src/main/java/org/apache/impala/calcite/coercenodes/CoerceNodes.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/coercenodes/CoerceOperandShuttle.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/CommonOperatorFunctions.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaCastFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaCoalesceFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaDecodeFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionImpl.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeFactoryImpl.java M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q32.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q47.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q53.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q57.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q63.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q65.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q89.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q92.test 35 files changed, 408 insertions(+), 270 deletions(-) Approvals: Impala Public Jenkins: Verified Michael Smith: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/24271 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2944fde1301e48b3fb0c29be7290b1ad9533e5b6 Gerrit-Change-Number: 24271 Gerrit-PatchSet: 14 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]>
