Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14458 )
Change subject: IMPALA-9053: DDLs should generate lineage graphs. ...................................................................... IMPALA-9053: DDLs should generate lineage graphs. DDLs like 'create table' should generate minimal lineage graphs so that consumers like Atlas can use information like 'queryText' to establish lineages. This change adds a call to the computeLineageGraph() method during analysis phase of createTable which populates the graph with basic information like queryText. If it is a CTAS, this graph is enhanced in the "insert" phase with dependencies. Testing: Add an EE test to verify lineage information and also to check it is flushed to disk properly. Change-Id: Ia6c7ed9fe3265fd777fe93590cf4eb2d9ba0dd1e Reviewed-on: http://gerrit.cloudera.org:8080/14458 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M testdata/workloads/functional-query/queries/QueryTest/lineage.test M tests/custom_cluster/test_lineage.py 4 files changed, 61 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14458 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia6c7ed9fe3265fd777fe93590cf4eb2d9ba0dd1e Gerrit-Change-Number: 14458 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: radford nguyen <radford.ngu...@gmail.com>
