Hello Anonymous Coward (498), Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13352 to look at the new patch set (#11). Change subject: [WIP] IMPALA-8473: publish lineage info via hook ...................................................................... [WIP] IMPALA-8473: publish lineage info via hook This commit introduces a hook mechanism for publishing, lineage data specifically, but query information more generally, from Impala. Hooks can be consumed by downstream consumers (i.e. runtime dependencies) at supported places during Impala execution: - impalad startup - post-query execution The consumers are to be frontend Java dependencies intiated at runtime. The hadoop property `impala.post.exec.hooks` specifies a comma-separated list of hook consumer implementation classes that are instantiated and registered at impala start up. Lineage information is passed from the backend after a query completes (but before it returns) and given to every hook to execute asynchronously. IOW, a query may complete and return to the user before any or all hooks have completed executing. Tests: TBD Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1 --- M be/src/service/frontend.cc M be/src/service/frontend.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/Frontend.thrift A fe/src/main/java/org/apache/impala/hooks/PostQueryHookContext.java A fe/src/main/java/org/apache/impala/hooks/QueryExecHook.java A fe/src/main/java/org/apache/impala/hooks/QueryExecHookManager.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/test/java/org/apache/impala/hooks/QueryExecHookManagerTest.java A fe/src/test/java/org/apache/impala/testutil/AlwaysErrorQueryExecHook.java A fe/src/test/java/org/apache/impala/testutil/DummyQueryExecHook.java 16 files changed, 579 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/13352/11 -- To view, visit http://gerrit.cloudera.org:8080/13352 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1 Gerrit-Change-Number: 13352 Gerrit-PatchSet: 11 Gerrit-Owner: radford nguyen <radford.ngu...@gmail.com> Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya <fwij...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: radford nguyen <radford.ngu...@gmail.com>