Hi,
I wanted to ask if it is possible to intercept every communication that
takes place between hadoop's map reduce task i.e between JobTracker and
TaskTracker and make it pass through my own communication library.
So, if JobTracker and TaskTracker talk through http or rpc, i would like to
intercept the call and let it pass through my communication library. If it
is possible can anyone tell me that which set of classes i need to look at
hadoop's distribution.
Similarly, for the hdfs, is it possible to let all the communication that is
happening between namenode and datanode to pass through my communication
library.
Reason for doing that is that i want all the communication to happen through
a communication library that resolves every communication problem that we
can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By
using that library all the headache of communication will be gone. So, we
will be able to use hadoop quite easily and there will be no communication
problems.
Thats my master's project. So, i want to know how to start and where to look
for.
I would really appreciate a reply.
Regards,
Ahmad Shahzad
