BTW, if anybody can help commit YARN-1301, which helps debugging the blacklist issues. Thanks! - Zhijie
On Mon, Nov 25, 2013 at 11:08 AM, Zhijie Shen <zs...@hortonworks.com> wrote: > Hi Grandl, > > Have you checked the following log message "Update the blacklist for ..."? > What did it tell? > > BTW, YarnScheduler#allocate takes the blacklist additions/removes. You may > also want to verify YarnScheduler#allocate gets the empty list or not. > > Thanks, > Zhijie > > > On Mon, Nov 25, 2013 at 10:38 AM, Grandl Robert <rgra...@yahoo.com> wrote: > >> I am not sure I posted this question in the right thread. >> >> >> Do you have some thoughts with the below mail ? Is there any >> configuration parameter for RM to enable blacklisting ? Anyway, it seems my >> blacklist is sent empty to RM, so I think is an AM problem though. >> >> >> Please let me know about your thoughts, >> >> robert >> >> >> >> >> On Saturday, November 23, 2013 2:31 PM, Grandl Robert <rgra...@yahoo.com> >> wrote: >> >> Hi, >> >> I am not 100% sure it's a bug in AM, but it seems so. I am blacklisting >> some hosts, which are send through an AllocateRequest in makeRemoteRequest >> method in RMContainerRequest.java. >> >> However, the blacklist is empty at RM. I traced all over the places where >> this is lost, and I found that in >> ApplicationMasterProtocolPBClientImpl.java, in allocate(AllocateRequest >> request) method, if I print the >> request.getResourceBlacklistRequest().getBlacklistAdditiions() I see the >> list, but after it does AllocateRequestProto requestProto = (..).getProto(), >> and I print the >> requestProto.getBlacklistRequest().getBlacklistAdditions() now is EMPTY. >> >> So basically, AM send an empty black list to RM. I went even further down >> and traced till YarnServiceProtos.java(in hadoop-yarn-api), but at this >> point all the logs I have, are removed. it seems this code is regenerated >> every time I am compiling yarn-api .... >> >> Is there something I am missing guys, or is a bug ? Interesting is that >> release containers list is propagated correctly but I think it comes on >> other path. I am running hadoop-yarn 2.3. >> >> Thanks, >> robert >> > > > > -- > Zhijie Shen > Hortonworks Inc. > http://hortonworks.com/ > -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
