Without scheduler debug logs it would be hard to say. However, there is a well-known answer to these race conditions. YARN allocation protocol is prone to race conditions by design. And it is expected that a few extra allocations can occur. This is orthogonal to Tez.
Bikas From: Fabio C. [mailto:[email protected]] Sent: Tuesday, February 10, 2015 4:03 AM To: [email protected] Subject: Unexpected containers allocated Hi everyone, I was running the tez wordcount example on a 6 nodes cluster. The input file is made up by 11 splits (so I expect 11 containers to be allocated for the first vertex). I notice that the Capacity Scheduler always allocates 12 container, 11 will start (they find a pending task), while one is immediately released by tez, since there is no other pending task. I'm sorry I cannot enable DEBUG logging level (to see the actual request content) since the cluster is shared among several users, but I was wondering if it is Tez asking for an extra container (and why) or if it's a RM policy (do you know what it is?). This was not happening while I was playing with 2 VM on my local computer, nor it happens on the cluster when running the MR wordcunt without Tez. Thanks a lot Fabio
