I agree with this approach, although munge is kinda hacky. It is easy
though. =)
Avery
On 3/20/12 5:52 PM, Eugene Koontz wrote:
Hi Giraphers,
I think it might be good to look at how we can add support for new
hadoop versions. Currently we have hadoop_facebook
(https://issues.apache.org/jira/browse/GIRAPH-14). I am considering
adding new ones such as hadoop_0.24. Looking at the code, it seems
that the main hadoop variation between the stock hadoop used (0.203.0)
versus facebook has to do with the new security-related APIs in the
latter that is, fortunately, also available in hadoop 0.23 and 0.24.
So, hopefully we can make use of the existing work that Avery has done
for hadoop_facebook and apply it to other hadoop versions. Therefore I
would propose that:
1. a new munge flag HADOOP_SECURE to be used in RPCCommunication.java
and a few other places, where we are currently checking for
HADOOP_FACEBOOK and HADOOP.
2. we make a new profile called hadoop_secure, which, as with
hadoop_facebook, will use the above munge flag.
3. we make a new profile "hadoop_0.20.203" for the existing default
hadoop and make it the default profile (activeByDefault=true). This
will makes it easier to handle the differences in the hadoop library
dependency set that have happened between 0.20.203 and hadoop trunk.
Please see https://github.com/ekoontz/giraph/tree/security-profile for
my branch that implements the above.
Thanks,
-Eugene
