[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HDFS-245: -- Resolution: Fixed Fix Version/s: 0.22.0 Release Note: HDFS-245. Adds a symlink implementation to HDFS. This complements the new symlink feature added in HADOOP-6421 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Fix For: 0.22.0 Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symlink41-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink41-hdfs.patch Hey Sanjay, The attached patch addresses your feedback and merges with trunk. I had to resolve a bunch of diffs with and add new code for HDFS-946 which introduced HdfsFileStatus so would be good to check out those changes in particular. Details: * Added missing throws of UnresolvedLinkException to DFSClient * Removed UnresolvedLinkException from NameNode * Fixed Lease#findPath per your suggestion, did the same to getCompleteBlocksTotal since it should also not throw an UnresolvedLinkException * Filed HADOOP-6585 and HDFS-995 to do the new FsStatus APIs and convert existing code to the new interfaces. * FSNameSystem#verifyParentDir now passes true for resolveLink, tests are in the base class so will be in an upcoming common change. * Replaced the numBlocks -2 check with == -1 * When through whitespace changes, they should just remove trailing whitespaces now. * Updated HDFS-995 with your suggestions for checking #isDirectory() * I noticed INode#constructPath was using StringBuffer, changed it to use StringBuilder * FSNamesystem#metaSave no longers throws UnresolvedSymlinkException. It just use java.io.File for the local file so symlinks should be handled transparently by the local file system. * Also, for those following along, we should not have to worry about UnresolvedLinkExceptions in loadFSImage and friends because intermediate symlinks are resolved when objects are created, so the paths that get persisted are the fully resolved ones, ie creating /link/file will persist /dir/file so when we load the image we will call addToParent(../dir/file..). This should be more clear when volume relative symlinks are resolved internally (HDFS-932). The code asserts on these cases. * getLinkTarget never throws an UnresolvedLinkException, it always resolves the first symlink in the path and ignores the remainder, so I left the throws clause as is Thanks, Eli Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symlink41-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Status: Open (was: Patch Available) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symlink41-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Status: Patch Available (was: Open) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symlink41-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Status: Open (was: Patch Available) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Status: Patch Available (was: Open) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink40-hdfs.patch Patch attached. Removes testSetTimes since it was moved to common but otherwise the same. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Status: Patch Available (was: Open) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symlink40-hdfs.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink39-hdfs.patch Attached patch merges with trunk. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symlink39-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink38-hdfs.patch Attached patch merges with trunk to resolve with the DFSClient refactoring. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symlink38-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink37-hdfs.patch Hey Sanjay, Thanks for posting, these are all addressed in patches 34-36, modulo using FileContextTestHelper which I'll defer to when HADOOP-5916 is resolved. Attached a patch that's a minor update to the last one to fix a bug in ImageLoaderCurrent. Thanks, Eli Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: design-doc-v4.txt Attached the latest design doc that captures the final path resolution and API changes, as well as incorporates feedback from Hairong. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, design-doc-v4.txt, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch, symlink37-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[
https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HDFS-245:
-
Attachment: symlink35-hdfs.patch
Hey Sanjay,
Thanks for the review. Latest patch attached, addresses your feedback and
merges in trunk.
bq. File Jira to resolve dot relative and volume-root relative names locally in
HDFS (this Jira can be completed later)
Filed HDFS-932.
bq. explicitly declare the unresolvedLinkException in methods that throw them-
we are moving towards declaring all exceptions (see HDFS-717). ClientProtocol,
Namenode, FSNamesystem, etc
Done. I had to modify quite a few functions both in hdfs and common since the
UnresolvedLinkException flows all the way up to FileContext. I left the tests
as is, I also didn't modify FileSystem.
bq. UnresolvedPathException, suggestion: adding a method called
getUnresolvedPath for completeness.
Done.
bq. createSymlinkXXX -don't pass in permission object as parameter - it
suggests that permissions matter for symlinks
Done. I left the permissions as default since INodes do not support null
PermissionStatus or PermissionStatus' created with a null FsPermission.
Currently links are rwx everyone so access is checked on the link target. And
like files accessed to the links themselves should be subject to the containing
directory's permissions. I started to modify TestDFSPermission to cover this,
which requires converting the test to use FileContext, and just converting it
to use FileContext caused some of the tests to fail, which I'm looking into, if
you don't mind will address in a follow-on jira. I filed HDFS-876 for porting
tests over to FileContext a while back and HDFS-934 to extend those for
additional coverage after they've been ported.
bq. FSDirectory#addToParent - if not a symlink, then pass null instead of a
null string.
The symlink value needs to be a string since we serialize it to image/edit log,
ie readString and writeString don't support nulls.
bq. getExistingPathINodes() typo: javadoc for param resolveLink is missing the
param name
Fixed.
bq. Add comment that exception is thrown for symlinks occurs in the middle of
the path.
Done.
bq. The multiple if statements with same or negation of condition - can be
confusing and error prone in future
I rewrote this to use just two if that read like the logic: If the current
node is a symlink then throw an UnresolvedPathException, unconditionally if we
are not on the last path component, and conditionally if we are on the last
path component and we want to resolve it. Think this is more clear.
{code}
if (curNode.isLink() (!lastComp || (lastComp resolveLink))) {
throw new UnresolvedPathException(...)
}
if (lastComp || !curNode.isDirectory()) {
break;
}
{code}
bq. FSDirectory#getFileBlocks add: if (symlink) return null.
Fixed. Shouldn't be executed in practice since we always complete a resolved
path.
bq. FSDirectory#getListing() calls getNode(srcs, false) // shouldn't
resolveLink be true
Fixed. FileContext#listStatus was not resolving links, fixed that. I added
FileContextSymlinkBaseTest#testListStatusUsingLink to test this case.
bq. FSDirectory#getPreferredBlockSize calls getNode(filename, false ) //
shouldn't resolveLink be true
Fixed. Looks like DFSClient#getBlockSize is the only caller and isn't covered
by any unit tests (you can remove the method and compile hdfs). Filed HDFS-929
to either remove it or add a test.
I replaced the throw of UnresolvedPathException in FSNameSystem, it wasn't
necessary as the call to getFileINode resolves the links. There's now only a
single place we throw UnresolvedPathException which was my original intent.
Latest patch attached here and to HADOOP-6421.
Thanks,
Eli
Create symbolic links in HDFS
-
Key: HDFS-245
URL: https://issues.apache.org/jira/browse/HDFS-245
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: dhruba borthakur
Assignee: Eli Collins
Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt,
designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch,
symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch,
symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch,
symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt,
symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch,
symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt,
symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt,
symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch,
symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch,
symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch,
symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch,
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink33-hdfs.patch Patch attached. Contains the hdfs side of the changes for HADOOP-6421, small reworking of permissions for createSymlink. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink31-hdfs.patch Latest patch attached. It addresses feedback from Sanjay's review of the common patch on HADOOP-6421, mostly just pulls the bulk of the symlink tests into a common class. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symlink31-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink30-hdfs.patch Attached a minor update to the last patch to address javadoc and checkstyle. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symlink30-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.pat ch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink29-hdfs.patch Here's the latest patch that implements the resolution behavior posted to HDFS-245 that Doug and Sanjay and I discussed. Rebased against trunk and passes test-core. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink29-hdfs.patch Minor update to the last patch to address Cos's comment in HDFS-254. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink28-hdfs.patch Latest patch. * Implements path resolution as discussed in the above comment and HADOOP-6427 * Additional tests in TestLink to cover the above * Resolved against trunk * See HADOOP-6421 for the common changes Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink27-hdfs.patch Latest patch. Updated the implementation to reflect semantics discussed in HADOOP-6427 and requisite tests. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: designdocv3.txt Updated design doc. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symlink27-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink-26-hdfs.patch Latest patch. Jives with the one posted to HADOOP-6421. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink-26-hdfs.patch Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch, symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink-25-hdfs.patch Attached latest patch. Reflects changes necessary for adding symlinks to LocalFs and more tests, eg symlinks across file systems. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: Eli Collins Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch, symlink24-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-245: - Attachment: symlink22-hdfs.patch symlink22-common.patch Uploaded patches per the last comment that incorporate Doug's review and minor fixes like getting the offline image viewer test to work with symlinks. Also verified that hdfs oiv using a fsimage with symlinks look OK. Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch, symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt, symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch, symlink22-common.patch, symlink22-hdfs.patch, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[
https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HDFS-245:
-
Attachment: symlink19-hdfs-delta.patch
symlink19-common-delta.patch
Here are patches that add more tests and implement the above FileContext API.
They are deltas against symlink19 so it's easier to review what's changed. They
also contain some resolutions from merging with trunk. I'm writing more tests
but wanted to post what I have so far for feedback.
Note that they change the default behavior to fully resolve symlinks and add an
API equivalent to {{lstat}} for clients that want to operate on symlinks. The
FileContext API could have been implemented without adding a new ClientProtocol
function by creating a function {{resolveFully}} that calls {{getLinkTarget}}
on the return value of {{resolve}}, but that would require an additional round
trip for all FileContext functions that need to full resolve links.
Create symbolic links in HDFS
-
Key: HDFS-245
URL: https://issues.apache.org/jira/browse/HDFS-245
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: dhruba borthakur
Assignee: dhruba borthakur
Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt,
HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symLink1.patch,
symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch,
symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch,
symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt,
symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch,
symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch,
symlink19-hdfs.txt, symLink4.patch, symLink5.patch, symLink6.patch,
symLink8.patch, symLink9.patch
HDFS should support symbolic links. A symbolic link is a special type of file
that contains a reference to another file or directory in the form of an
absolute or relative path and that affects pathname resolution. Programs
which read or write to files named by a symbolic link will behave as if
operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-245: -- Attachment: symlink17-hdfs.txt symlink17-common.txt Here is a patch that conforms to the modified consensus arrived earlier in the week. I would like everybody to review the symlink17-common.txt patch that has changes to the FileSystem API. This code is not yet tested in great detail and is uploaded only for review purposes. (Also, this patch does not yet modify the FileContext API.) Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: 4044_20081030spi.java, designdocv1.txt, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch, symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-245) Create symbolic links in HDFS
[ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuweimin updated HDFS-245: --- Attachment: symlink-0.20.0.patch this a copy of symLink14.patch for hadoop released version 0.20.0 usage: put symlink-0.20.0.patch into $HADOOP_HOME cd $HADOOP_HOME patch -p0 symlink-0.20.0.patch ant jar mv hadoop-0.20.0-core.jar hadoop-0.20.0-core.jar.bak cp build/hadoop-0.20.1-dev-core.jar restart the hadoop system at last Create symbolic links in HDFS - Key: HDFS-245 URL: https://issues.apache.org/jira/browse/HDFS-245 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: 4044_20081030spi.java, HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch, symLink14.patch, symLink15.txt, symLink15.txt, symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch HDFS should support symbolic links. A symbolic link is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution. Programs which read or write to files named by a symbolic link will behave as if operating directly on the target file. However, archiving utilities can handle symbolic links specially and manipulate them directly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
