radford nguyen has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13352 )
Change subject: [WIP] IMPALA-8473: publish lineage info via hook ...................................................................... [WIP] IMPALA-8473: publish lineage info via hook This commit introduces a hook mechanism for publishing, lineage data specifically, but query information more generally, from Impala. Hooks can be consumed by downstream consumers (i.e. runtime dependencies) at supported places during Impala execution: - impalad startup - post-query execution The consumers are to be frontend Java dependencies intiated at runtime. The hadoop property `impala.post.exec.hooks` specifies a comma-separated list of hook consumer implementation classes that are instantiated and registered at impala start up. Lineage information is passed from the backend after a query completes (but before it returns) and given to every hook to execute asynchronously. IOW, a query may complete and return to the user before any or all hooks have completed executing. Tests: TBD Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1 --- M be/src/service/frontend.cc M be/src/service/frontend.h M be/src/service/impala-server.cc A fe/src/main/java/org/apache/impala/hooks/ImpalaExecHookManager.java A fe/src/main/java/org/apache/impala/hooks/ImpalaHookContext.java A fe/src/main/java/org/apache/impala/hooks/ImpalaQueryExecHook.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java 7 files changed, 361 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/13352/5 -- To view, visit http://gerrit.cloudera.org:8080/13352 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1 Gerrit-Change-Number: 13352 Gerrit-PatchSet: 5 Gerrit-Owner: radford nguyen <radford.ngu...@gmail.com> Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya <fwij...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
