Varun, Thanks for reaching out about this issue. With no experience direct experience with NiFi, my help may not be fully accurate. However, here is what could be going on. Hive queries can launch Tez in session mode and non-session mode. Non-session mode execution is simple 1) launch an AM (application master controlling process) 2) submit one query (as a DAG) 3) after completion shutdown the AM. In this, one can see that a query (and hence one DAG) is executed. In session mode, The AM stays up to allow for submission of multiple queries (DAGs). So the session execution is more complex 1) launch an AM 2) 2) Submit one query 3) Repeat step until session timeout is expired.
It could be that NiFi is launching the Hive query in session mode with a long timeout and not shutting down the AM. In that case the below steps may help. The configuration to control session mode timeout is below. The default is "300" meaning 300 seconds or 5 minutes. tez.session.am.dag.submit.timeout.secs It could also be that NiFi is unable to accurately track the status of Tez jobs' completion, in that case I would reach out to NiFi team. As we get closer to solving this. I may ask you at some point for logs if the above solution doesn't work. Regards, jeagles Apache Tez PMC Chair On Thu, Apr 25, 2019 at 8:54 AM Varun Rao <[email protected]> wrote: > > Hello, > > We are using Nifi (1.6.0.3.2.0.8-3) with Hive (1.2), hive is running on tez > (2.6.5.0-292). The query on tez-view shows it is completed but on Nifi we > still see tez queries queued up 10 minutes after the query is finished on > tez-view. We have considered lowering the parameter > tez.am.container.idle.release-timeout-min.millis however this hasn't made any > differences. > > Do you know what can be causing this? > > Thanks
