kasakrisz commented on code in PR #5370:
URL: https://github.com/apache/hive/pull/5370#discussion_r1694718749
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/Generator.java:
##########
@@ -72,18 +133,11 @@ public Generator(Set<String> hooks) {
@Override
public ParseContext transform(ParseContext pctx) throws SemanticException {
- if (hooks != null && hooks.contains(ATLAS_HOOK_CLASSNAME)) {
- // Atlas would be interested in lineage information for
insert,load,create etc.
- if (!pctx.getQueryProperties().isCTAS()
- && !pctx.getQueryProperties().isMaterializedView()
- && pctx.getQueryProperties().isQuery()
- && pctx.getCreateTable() == null
- && pctx.getCreateViewDesc() == null
- && (pctx.getLoadTableWork() == null ||
pctx.getLoadTableWork().isEmpty())) {
- LOG.debug("Not evaluating lineage");
- return pctx;
- }
+ if (!statementFilter.test(pctx)) {
Review Comment:
Yes, it breaks existing configs but previously Atlas HiveHook-related
filters were hardcoded and impossible to test. Now
`hive.lineage.statement.filter` is independent of hooks and hence applies for
any hook making it possible to test using `LineageLogger`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
