----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62457/ -----------------------------------------------------------
(Updated Sept. 21, 2017, 7:47 p.m.) Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, and Suma Shivaprasad. Changes ------- Addressed review comments. Bugs: ATLAS-2157 https://issues.apache.org/jira/browse/ATLAS-2157 Repository: atlas Description ------- As reported in user list, executing Hive SQL that use temprorary tables (for example insert statement) causes Atlas hook to report the following failure. The fix is to ignore this error, as it doesn't result in loss of functionality. 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table values__tmp__table__1 not found: default.values__tmp__table__1 table not found 17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due to error java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884) at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() failed. at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701) at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268) at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83) at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) ... 6 more Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.processHiveEntity() failed. at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731) at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668) ... 12 more Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities() failed. at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597) at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711) ... 13 more Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities() failed. at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589) at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found values__tmp__table__1 at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186) at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568) ... 15 more Diffs (updated) ----- addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 7dc2e2f1 Diff: https://reviews.apache.org/r/62457/diff/2/ Changes: https://reviews.apache.org/r/62457/diff/1-2/ Testing ------- Verified that the error is not reported in the following cases: - CREATE TEMPORARY TABLE tmp1(c1 string); - insert into tmp1 values('1'); Also verified that lineage details are recorded correctly - for both table and column level lineages: - CREATE TABLE t1(c1 STRING, c2 STRING); - CREATE TABLE s1(c1 STRING, c2 STRING); - CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t, s1 s; Thanks, Madhan Neethiraj