Fredy Wijaya has uploaded a new patch set (#17). ( 
http://gerrit.cloudera.org:8080/13399 )

Change subject: IMPALA-8564: Add table/view create time in the lineage graph
......................................................................

IMPALA-8564: Add table/view create time in the lineage graph

This patch adds table/view create time in the lineage graph. This is
needed for Impala/Atlas integration. See ATLAS-3080.

Below is an example of the updated lineage graph.
{
    "queryText":"create table lineage_test_tbl as select int_col, tinyint_col 
from functional.alltypes",
    "queryId":"0:0",
    "hash":"407f23b24758ffcb2ac445b9703f5c44",
    "user":"dummy_user",
    "timestamp":1547867921,
    "edges":[
        {
            "sources":[
                1
            ],
            "targets":[
                0
            ],
            "edgeType":"PROJECTION"
        },
        {
            "sources":[
                3
            ],
            "targets":[
                2
            ],
            "edgeType":"PROJECTION"
        }
    ],
    "vertices":[
        {
            "id":0,
            "vertexType":"COLUMN",
            "vertexId":"int_col",
            "metadata":{
                "tableName":"default.lineage_test_tbl",
                "tableCreateTime":1559151337
            }
        },
        {
            "id":1,
            "vertexType":"COLUMN",
            "vertexId":"functional.alltypes.int_col",
            "metadata":{
                "tableName":"functional.alltypes",
                "tableCreateTime":1559151317
            }
        },
        {
            "id":2,
            "vertexType":"COLUMN",
            "vertexId":"tinyint_col",
            "metadata":{
                "tableName":"default.lineage_test_tbl",
                "tableCreateTime":1559151337
            }
        },
        {
            "id":3,
            "vertexType":"COLUMN",
            "vertexId":"functional.alltypes.tinyint_col",
            "metadata":{
                "tableName":"functional.alltypes",
                "tableCreateTime":1559151317
            }
        }
    ]
}

Testing:
- Updated lineage tests in PlannerTest
- Updated test_lineage.py
- Ran all FE tests

Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
---
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/impala-server.cc
M be/src/util/lineage-util.h
M common/thrift/CatalogService.thrift
M common/thrift/LineageGraph.thrift
M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java
M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
M testdata/workloads/functional-planner/queries/PlannerTest/lineage.test
M tests/custom_cluster/test_lineage.py
15 files changed, 2,136 insertions(+), 623 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13399/17
--
To view, visit http://gerrit.cloudera.org:8080/13399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
Gerrit-Change-Number: 13399
Gerrit-PatchSet: 17
Gerrit-Owner: Fredy Wijaya <fwij...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Fredy Wijaya <fwij...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>

Reply via email to