Hi, I have submitted spark job in yarn client mode. The executor and cores were dynamically allocated. In the job i have 20 partitions, so 5 container each with 4 core has been submitted. It almost processed all the records but it never exit the job and in the application master container i am seeing the below error message.
INFO yarn.YarnAllocator: Canceling requests for 0 executor containers WARN yarn.YarnAllocator: Expected to find pending requests, but found none. The same job i ran it for only 1000 records which successfully finished. Can anyone help me to sort out this issue. Spark version:2.0( AWS EMR). -- Selvam Raman "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
