[ https://issues.apache.org/jira/browse/HADOOP-9639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751400#comment-13751400 ]
Koji Noguchi commented on HADOOP-9639: -------------------------------------- I'm not comfortable with the proposed design where it almost blindly trust other users to do the right thing and upload the right file. (Not that I don't trust my users :) For example, it's easy for me to corrupt the /sharedcache directories by creating directories with permission 700. But what worries me the most is, entire security is based on a checksum. Qutoing from [wikipedia|http://en.wikipedia.org/wiki/Checksum] "It is important to not use a checksum in a security related application, as a checksum does not have the properties required to protect data from intentional tampering." > truly shared cache for jars (jobjar/libjar) > ------------------------------------------- > > Key: HADOOP-9639 > URL: https://issues.apache.org/jira/browse/HADOOP-9639 > Project: Hadoop Common > Issue Type: New Feature > Components: filecache > Affects Versions: 2.0.4-alpha > Reporter: Sangjin Lee > Assignee: Sangjin Lee > Attachments: shared_cache_design.pdf, shared_cache_design_v2.pdf > > > Currently there is the distributed cache that enables you to cache jars and > files so that attempts from the same job can reuse them. However, sharing is > limited with the distributed cache because it is normally on a per-job basis. > On a large cluster, sometimes copying of jobjars and libjars becomes so > prevalent that it consumes a large portion of the network bandwidth, not to > speak of defeating the purpose of "bringing compute to where data is". This > is wasteful because in most cases code doesn't change much across many jobs. > I'd like to propose and discuss feasibility of introducing a truly shared > cache so that multiple jobs from multiple users can share and cache jars. > This JIRA is to open the discussion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira