On 9/16/14, 8:17 PM, Thaddeus Diamond wrote:
Got the swimlanes tool working. It revealed a problem. In the
left-hand side of the graph (container ID) it shows a job (1TB data
scan) kicking off 200 containers. Cool, right?
That means container reuse isn't enabled/not working.
It is actually spinning up 200 containers, but only 100 at any given time.
The green vertical marker, is where the container life begins and the
red line is where it ends (the number there is actually the exit code of
the container).
This usually happens if someone runs different size Map/Reduce vertices.
Posting the image will give me a better idea of what's broken.
Cheers,
Gopal
My cluster configuration:
10 datanodes/nodemanagers
20CPU allocated / node to NMs
20GB RAM / node to the NMs
2GB min, 8GB max allocation / container
When I spin up a full MapReduce job doing the same 1TB data scan it
shows 100 containers in use in the ResourceManager web UI. Tez had the
exact same showing, only swimlanes is different.
Thoughts?
Thad
