Hi Robert Hadoop RPC has typically been blocking. There were some changes made to make the client side asynchronous - https://issues.apache.org/jira/browse/HADOOP-12909. I believe this is used in some HDFS interfaces. https://issues.apache.org/jira/browse/HADOOP-11552 added an option for server side handoff. For this one, I'd look at the example. I don't know if anyone uses this yet. There were some plans of using it for YARN-1503. Not sure if that has happened yet though. Hopefully for a lot more. Planning on using this in Tez as well at some point, for task to AM communication. Otherwise there's alternates like grpc, thrift.
HTH On Thu, Mar 2, 2017 at 1:33 PM, Robert Grandl <[email protected]> wrote: > Hi all, > This might be a bit out of topic, but I don't know where to ask. > > I am trying to create a non blocking RPC service in Yarn and Tez. However, > I observed that all the communication is blocking. > > I was trying to implement a non blocking service inheriting from > .Interface instead of .BlockingInterface, however I have a really hard time > to make it work. > > Is there any example in Tez or Yarn codebase I can look at. Or do you have > any other good pointers on how I can build such a service atop? > Otherwise, how can I do a non blocking service using protobuf in any of > the codebases? Is any such mechanism in Tez or Yarn? > > Thanks in advance for any advice,Robert > >
