hi,all:
I want to change LineageLogger code to output more log info .
The hive version I am using is 2.3.6 I create a new mvn project and cp
LineageLogger code to my own class named MyLineage. Then I add my code to
output more log info ,last I build a new jar. but when I config it to
<hive.exec.post.hooks> ,the log file didn't contain any edges info :
"edges":[],"vertices":[] .This is the result:
lineage.MyLineage:
{"version":"1.0","user":”man","timestamp":1593499925,"duration":9707,"jobIds":["job_1586773491164_0151"],"engine":"mr","database":"temp","hash":"c7722f236ce2fc5197ce71f7da4922dc","queryText":"insert
into tmp_test_c select * from tmp_test_b","edges":[],"vertices":[]}
I've tried to use hive's own LineageLogger with the same sql , and all is
ok.This is the result:
hooks.LineageLogger:
{"version":"1.0","user":"maintain","timestamp":1593501887,"duration":9923,"jobIds":["job_1586773491164_0152"],"engine":"mr","database":"temp","hash":"c7722f236ce2fc5197ce71f7da4922dc","queryText":"insert
into tmp_test_c select * from
tmp_test_b","edges":[{"sources":[2],"targets":[0],"edgeType":"PROJECTION"},{"sources":[3],"targets":[1],"edgeType":"PROJECTION"}],"vertices":[{"id":0,"vertexType":"COLUMN","vertexId":"temp.tmp_test_c.s1"},{"id":1,"vertexType":"COLUMN","vertexId":"temp.tmp_test_c.s2"},{"id":2,"vertexType":"COLUMN","vertexId":"temp.tmp_test_b.s1"},{"id":3,"vertexType":"COLUMN","vertexId":"temp.tmp_test_b.s2"}]}
I note that ,if I only config MyLineageLogger , LinkedHashMap<String,
ObjectPair<SelectOperator,org.apache.hadoop.hive.ql.metadata.Table>>
finalSelOps = index.getFinalSelectOps();
will return 0
can anybody give me some clue , why this happen and what should I do ? thanks...
