[jira] [Commented] (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016892#comment-13016892 ] Hudson commented on MAPREDUCE-2096: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.2.txt, > mapreduce-2096.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12993917#comment-12993917 ] Hudson commented on MAPREDUCE-2096: --- Integrated in Hadoop-Mapreduce-22-branch #33 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/]) > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.2.txt, > mapreduce-2096.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978526#action_12978526 ] Hudson commented on MAPREDUCE-2096: --- Integrated in Hadoop-Mapreduce-trunk-Commit #572 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/572/]) > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.2.txt, > mapreduce-2096.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978094#action_12978094 ] Devaraj Das commented on MAPREDUCE-2096: +1 > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.2.txt, > mapreduce-2096.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975957#action_12975957 ] Todd Lipcon commented on MAPREDUCE-2096: Results on mapreduce-2096.2.txt: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. Unit tests pass except for the known timeouts from trunk. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.2.txt, > mapreduce-2096.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970376#action_12970376 ] Devaraj Das commented on MAPREDUCE-2096: Patch looks fine. Todd, could you please get back with the results from running the full test suite + test-patch. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.22.0 > > Attachments: mapreduce-2096-index-oob.txt, mapreduce-2096.txt, > secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966780#action_12966780 ] Todd Lipcon commented on MAPREDUCE-2096: We don't need it absolutely immediately, except that it's very tough to verify this change even manually in the current state of the project :( I've got the patch forward ported but still need to fix up one or two things and run through the non-secure unit tests. I should post a final version this weekend. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966737#action_12966737 ] Devaraj Das commented on MAPREDUCE-2096: bq. Does anyone have a suggestion on how to get common's native library build onto mapreduce's library path post-split? Do we need it immediately? I believe we don't have any MR tests with security ON. If that is the case, we could go ahead with a MR patch that is not dependent on the native libs (HADOOP-6978). > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[
https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931645#action_12931645
]
Todd Lipcon commented on MAPREDUCE-2096:
Unfortunately O_NOFOLLOW is not sufficient. The man page for {{open(2)}} says:
bq. If pathname is a symbolic link, then the open fails... Symbolic links in
earlier components of the pathname will still be followed.
... meaning that a user can still exploit this by substituting a symlink for
some intermediate path component, and read someone else's stderr/stdout file.
> Secure local filesystem IO from symlink vulnerabilities
> ---
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
>Affects Versions: 0.22.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Blocker
> Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt,
> secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing
> list.
> The vulnerability is that MR daemons occasionally open files that are located
> in a path where the user has write access. A malicious user may place a
> symlink in place of the expected file in order to cause the daemon to instead
> read another file on the system -- one which the attacker may not naturally
> be able to access. This includes delegation tokens belong to other users, log
> files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931610#action_12931610 ] Devaraj Das commented on MAPREDUCE-2096: One thought - could we get away with opening the files in C using open(...O_RDONLY|O_NOFOLLOW), and have the JNI return a fdobject that is then used to get the fileinputstreams ? I am wondering whether this is sufficient for preventing the symlink attacks.. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929892#action_12929892 ] Todd Lipcon commented on MAPREDUCE-2096: Does anyone have a suggestion on how to get common's native library build onto mapreduce's library path post-split? It seems we should be publishing a tarball of common/build/native into maven, and then retrieving it with ivy from mapreduce, perhaps? Does anyone have a better idea or should I open a JIRA to publish the native build as an artifact? > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929884#action_12929884 ] Todd Lipcon commented on MAPREDUCE-2096: Thanks for reminding me to upload the trunk work. I put a patch up on HADOOP-6978 which blocks this. Working on the MR trunk patch as well while that one gets reviewed/committed. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929844#action_12929844 ] Owen O'Malley commented on MAPREDUCE-2096: -- How is the trunk patch going for this one? > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, > secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918656#action_12918656 ] Devaraj Das commented on MAPREDUCE-2096: bq. This patch fixes the above issue by having TaskRunner call to a version of reportDiagonsticInfo that doesn't authorize the caller. The same should be done for fsError. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: secure-files-9.txt, secure-files-authorized-jvm-fix.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915919#action_12915919 ] Owen O'Malley commented on MAPREDUCE-2096: -- I think we need to move away from checking in any of the autoconf/automake generated files. Let's run autoreconf everytime and make the ignored by subversion and git. It should also be noted that Devaraj contributed significantly to this patch. > Secure local filesystem IO from symlink vulnerabilities > --- > > Key: MAPREDUCE-2096 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker, security, tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: secure-files-9.txt > > > This JIRA is to contribute a patch developed on the private security@ mailing > list. > The vulnerability is that MR daemons occasionally open files that are located > in a path where the user has write access. A malicious user may place a > symlink in place of the expected file in order to cause the daemon to instead > read another file on the system -- one which the attacker may not naturally > be able to access. This includes delegation tokens belong to other users, log > files, keytabs, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
