Hi Amit, Thank you for the follow up. What you describe sounds like a bug but I am not able to reproduce it. Can you open an issue in Jira with an outline of your code and how you submit the job?
> Could you also recommend us the best practice in FLIP6, should we use YARN session or submit jobs in non-detached mode? This should be independent of FLIP-6. We are aiming at backwards compatibility but apparently there are still things that require polishing. Best, Gary On Thu, Apr 19, 2018 at 12:49 PM, Amit Jain <aj201...@gmail.com> wrote: > > Hi Gary, > > We found the underlying issue with the following problem. > Few of our jobs are stuck with logs [1], these jobs are only able to allocate JM and couldn't get any TM, however, there are ample resource on our cluster. > > We are running ETL merge job here. In this job, we first find new deltas and if there is no delta detected then we make exit without actually executing the job. I think this is the reason we see no TM allocation is happening. > > I believe in above case (non-detached mode) we should mark the submitted application as complete compare to running. Please share your thoughts on this. > Should I log this improvement in JIRA? > > Could you also recommend us the best practice in FLIP6, should we use YARN session or submit jobs in non-detached mode? > > -- > Thanks, > Amit >
