Hi All,
I have started learning Hadoop and its related components. I am following a tutorial on Hadoop Administration on Udemy. As part of the learning process, i ran the following command: $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jarrandomtextwriter -Ddfs.replication=1 /user/bigdata/randomtextwriter Above command created 30 files each of size 1 GB. Then i ran the below reduce command: $ yarn jar/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar \ wordcount \ -Dmapreduce.input.fileinputformat.split.minsize=268435456\ -Dmapreduce.job.reduces=8 \ /user/bigdata/randomtext \ /user/bigdata/wordcount After executing the above command, I just thought of killing the application after some time so i ran 'yarn application -list' first which listed a lot many applications out of which one was *wordc**ount*. I killed that particular application using 'yarn application -kill application-id'. However, when i checked the scheduler, i could see that several applications were still showing in Pending state so i ran the following command: $ for x in $(yarn application -list -appStates ACCEPTED | awk 'NR > 2 { print $1 }'); do yarn application -kill $x; done It was killing the applications as I could see the 'Apps Completed' count was going up but as soon as all the apps got killed, I saw those applications again getting created. Even if I stop the whole cluster and start again, the scheduler shows that there are submitted applications in Pending state. Here's the content of fair-scheduler.xml: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <allocations> <queue name="root"> <schedulingPolicy>drf</schedulingPolicy> <queue name="default"> <schedulingPolicy>drf</schedulingPolicy> </queue> </queue> <queuePlacementPolicy> <rule name="specified" create="false"/> <rule name="default" create="true"/> </queuePlacementPolicy> </allocations> This is just a test cluster. I just want to kill the applications/clear the application queue. Any help will really be appreciated as I am struggling with it for the last few days. Regards
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org For additional commands, e-mail: user-h...@hadoop.apache.org