Attila Jeges has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type ...................................................................... IMPALA-7368: Add initial support for DATE type DATE values describe a particular year/month/day in the form yyyy-MM-dd. DATE values do not have a time of day component. The range of values supported for the DATE type is 0000-01-01 to 9999-12-31. This initial DATE type support covers TEXT fileformat only. 'DateValue' is used as the internal type to represent DATE values. The changes are as follows: - Explicit casting between DATE and other types: - from STRING to DATE. The string value must be formatted as yyyy-MM-dd. - from DATE to STRING. The resulting string value is formatted as yyyy-MM-dd. - from BIGINT, INT, SMALLINT, TINYINT to DATE. The source value is interpreted as a number of days since the epoch. - from DATE to BIGINT, INT, SMALLINT, TINYINT. The resulting integer is the number of days since epoch. - from DOUBLE, FLOAT, DECIMAL to DATE. The source value's fractional part is ignored, the integer part is interpreted as a number of days since epoch. - from DATE to DOUBLE, FLOAT, DECIMAL. The resulting value is the number of days since epoch. - from TIMESTAMP to DATE. The source timestamp's time of day component is ignored. - from DATE to TIMESTAMP. The target timestamp's time of day component is set to 00:00:00. - Implicit casting between DATE and other types: - from STRING to DATE if the source string value is used in a context where a DATE value is expected. - from DATE to TIMESTAMP if the source date value is used in a context where a TIMESTAMP value is expected. - Since both STRING -> DATE and STRING -> TIMESTAMP implicit conversions are possible, the function resolution logic was changed to select the right version of overloaded functions. - Codegen infrastructure changes for expression evaluation. - 'IS [NOT] NULL' and '[NOT] IN' predicates. - Common comparison operators (including the 'BETWEEN' operator). - Infrastructure changes for built-in functions. - Some built-in functions: conditional, aggregate, analytical and math functions. - C++ UDF/UDA support. - Support partitioning and grouping by DATE. - Beeswax, HiveServer2 support. These items are tightly coupled and it makes sense to implement them in one change-set. Testing: - A new partitioned TEXT table 'functional.date_tbl' was introduced for DATE-related tests. - BE and FE tests were extended to cover DATE type. - E2E tests: - since DATE type is supported for TEXT fileformat only, most DATE tests were implemented separately in tests/query_test/test_date_queries.py. Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f --- M be/src/codegen/codegen-anyval.cc M be/src/codegen/codegen-anyval.h M be/src/codegen/gen_ir_descriptions.py M be/src/codegen/llvm-codegen.cc M be/src/exec/aggregator.cc M be/src/exec/data-source-scan-node.cc M be/src/exec/hash-table.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scanner-ir.cc M be/src/exec/hdfs-table-sink.cc M be/src/exec/text-converter.cc M be/src/exec/text-converter.inline.h M be/src/exprs/agg-fn-evaluator.cc M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/aggregate-functions.h M be/src/exprs/anyval-util.cc M be/src/exprs/anyval-util.h M be/src/exprs/case-expr.cc M be/src/exprs/case-expr.h M be/src/exprs/cast-functions-ir.cc M be/src/exprs/cast-functions.h M be/src/exprs/conditional-functions-ir.cc M be/src/exprs/conditional-functions.h M be/src/exprs/decimal-operators-ir.cc M be/src/exprs/decimal-operators.h M be/src/exprs/expr-test.cc M be/src/exprs/expr-value.h M be/src/exprs/hive-udf-call.cc M be/src/exprs/hive-udf-call.h M be/src/exprs/in-predicate-ir.cc M be/src/exprs/in-predicate.h M be/src/exprs/is-null-predicate-ir.cc M be/src/exprs/literal.cc M be/src/exprs/literal.h M be/src/exprs/math-functions-ir.cc M be/src/exprs/math-functions.h M be/src/exprs/null-literal.cc M be/src/exprs/null-literal.h M be/src/exprs/operators-ir.cc M be/src/exprs/operators.h M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/exprs/scalar-expr-ir.cc M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/exprs/scalar-fn-call.cc M be/src/exprs/scalar-fn-call.h M be/src/exprs/slot-ref.cc M be/src/exprs/slot-ref.h M be/src/exprs/timestamp-functions.h M be/src/exprs/utility-functions-ir.cc M be/src/exprs/utility-functions.h M be/src/rpc/thrift-util.cc M be/src/runtime/date-test.cc M be/src/runtime/date-value.cc M be/src/runtime/date-value.h M be/src/runtime/raw-value-ir.cc M be/src/runtime/raw-value-test.cc M be/src/runtime/raw-value.cc M be/src/runtime/raw-value.inline.h M be/src/runtime/types.cc M be/src/runtime/types.h M be/src/service/fe-support.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M be/src/testutil/test-udas.cc M be/src/testutil/test-udfs.cc M be/src/udf/udf-test.cc M be/src/udf/udf.h M be/src/util/CMakeLists.txt M be/src/util/static-asserts.cc M be/src/util/string-parser-test.cc M be/src/util/string-parser.h M be/src/util/symbols-util-test.cc M be/src/util/symbols-util.cc M bin/rat_exclude_files.txt M common/function-registry/impala_functions.py M common/thrift/Exprs.thrift M common/thrift/Types.thrift M ext-data-source/test/src/main/java/org/apache/impala/extdatasource/AllTypesDataSource.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java A fe/src/main/java/org/apache/impala/analysis/DateLiteral.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java M fe/src/main/java/org/apache/impala/catalog/ColumnStats.java M fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java M fe/src/main/java/org/apache/impala/catalog/Function.java M fe/src/main/java/org/apache/impala/catalog/PrimitiveType.java M fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java M fe/src/main/java/org/apache/impala/catalog/ScalarType.java M fe/src/main/java/org/apache/impala/catalog/Type.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/util/FunctionUtils.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java M fe/src/test/java/org/apache/impala/analysis/LiteralExprTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/hive/executor/TestUdf.java M fe/src/test/java/org/apache/impala/service/JdbcTest.java M testdata/bin/compute-table-stats.sh M testdata/bin/create-data-source-table.sql A testdata/data/date_tbl/0000.txt A testdata/data/date_tbl/0001.txt A testdata/data/date_tbl/0002.txt A testdata/data/date_tbl/0003.txt M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/data-source-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test A testdata/workloads/functional-query/queries/QueryTest/compute-stats-date.test M testdata/workloads/functional-query/queries/QueryTest/data-source-tables.test A testdata/workloads/functional-query/queries/QueryTest/date.test M testdata/workloads/functional-query/queries/QueryTest/decimal-exprs.test M testdata/workloads/functional-query/queries/QueryTest/functions-ddl.test M testdata/workloads/functional-query/queries/QueryTest/joins.test M testdata/workloads/functional-query/queries/QueryTest/misc.test M testdata/workloads/functional-query/queries/QueryTest/subquery.test M testdata/workloads/functional-query/queries/QueryTest/uda.test M testdata/workloads/functional-query/queries/QueryTest/udf.test M testdata/workloads/functional-query/queries/QueryTest/views-ddl.test M tests/custom_cluster/test_permanent_udfs.py M tests/hs2/test_fetch.py M tests/metadata/test_compute_stats.py A tests/query_test/test_date_queries.py M tests/query_test/test_udfs.py M tests/test-hive-udfs/src/main/java/org/apache/impala/TestUdf.java 141 files changed, 2,910 insertions(+), 215 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/12481/3 -- To view, visit http://gerrit.cloudera.org:8080/12481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f Gerrit-Change-Number: 12481 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Jeges <atti...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>