Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/16188 )
Change subject: IMPALA-9478: Profiles should indicate if custom UDFs are being
used
......................................................................
IMPALA-9478: Profiles should indicate if custom UDFs are being used
Adds a marker to runtime profiles and explain plans indicating if custom
(e.g. non-built in) user-defined functions are being used. For explain
plans, a SQL-style comment is added after any function call. For runtime
profiles, a new Frontend entry called "User Defined Functions (UDFs)"
lists out all UDFs analyzed during planning.
Take the following example:
create function hive_lower(string) returns string location
'/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFLower';
set explain_level=3;
explain select * from functional.alltypes order by hive_lower(string_col);
...
01:SORT
order by: default.hive_lower(string_col) /* JAVA UDF */ ASC
materialized: default.hive_lower(string_col) /* JAVA UDF */
...
This shows up in the runtime profile as well.
When the above query is actually run, the runtime profile includes the
following entry:
Frontend
User Defined Functions (UDFs): default.hive_lower
Error messages will also include SQL-style comments about any UDFs used.
For example:
select aggfn(int_col) over (partition by int_col) from
functional.alltypesagg
Throws:
Aggregate function 'default.aggfn(int_col) /* NATIVE UDF */' not
supported with OVER clause.
Testing:
* Added tests to test_udfs.py
* Ran core tests
Change-Id: I79122e6cc74fd5a62c76962289a1615fbac2f345
Reviewed-on: http://gerrit.cloudera.org:8080/16188
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
M fe/src/main/java/org/apache/impala/service/FrontendProfile.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java
M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java
M
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M tests/query_test/test_udfs.py
7 files changed, 88 insertions(+), 21 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16188
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I79122e6cc74fd5a62c76962289a1615fbac2f345
Gerrit-Change-Number: 16188
Gerrit-PatchSet: 9
Gerrit-Owner: Sahil Takiar <stak...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
