Understanding Fair Schedulers better. Can we create mulitple pools in Fair Schedulers. I guess Yes. Please correct me.
Suppose I have 2 pools in my fair-scheduler.xml 1. Hadoop-users : Min map : 10, Max map : 50, Min Reduce : 10, Max Reduce : 50 2. Admin-users: Min map : 20, Max map : 80, Min Reduce : 20, Max Reduce : 80 I have 5 users, who will be using these pools. How will I allocate specific pools to specific users ? Suppose I want user1,user2 to use "Hadoop-users" pool and user3,user4,user5 to use "Admin users" In http://hadoop.apache.org/common/docs/r0.20.205.0/fair_scheduler.html they have mentioned allocations something like this. <?xml version="1.0"?> <allocations> <pool name="sample_pool"> <minMaps>5</minMaps> <minReduces>5</minReduces> <maxMaps>25</maxMaps> <maxReduces>25</maxReduces> <minSharePreemptionTimeout>300</minSharePreemptionTimeout> </pool> <user name="sample_user"> <maxRunningJobs>6</maxRunningJobs> </user> <userMaxJobsDefault>3</userMaxJobsDefault> <fairSharePreemptionTimeout>600</fairSharePreemptionTimeout> </allocations> I tried creating more pools, its happening, but how to allocate users to use specific pools ? Thanks, Praveenesh