Sounds like it could be a SPOF. On Thu, Jun 10, 2010 at 7:47 AM, <[email protected]> wrote:
> Hey, > > This is a really neat idea.... if anyone has a way to do this, could you > share? > I'll bet this could be very interesting! Thanks... > > Best, > HAL > > > > Hi, > > I wanted to ask if it is possible to intercept every communication > > that > > takes place between hadoop's map reduce task i.e between JobTracker and > > TaskTracker and make it pass through my own communication library. > > So, if JobTracker and TaskTracker talk through http or rpc, i would like > > to > > intercept the call and let it pass through my communication library. If > it > > is possible can anyone tell me that which set of classes i need to look > at > > hadoop's distribution. > > > > Similarly, for the hdfs, is it possible to let all the communication that > > is > > happening between namenode and datanode to pass through my communication > > library. > > > > Reason for doing that is that i want all the communication to happen > > through > > a communication library that resolves every communication problem that we > > can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By > > using that library all the headache of communication will be gone. So, we > > will be able to use hadoop quite easily and there will be no > communication > > problems. > > > > Thats my master's project. So, i want to know how to start and where to > > look > > for. > > > > I would really appreciate a reply. > > > > Regards, > > Ahmad Shahzad > > > > >
