[jira] [Commented] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609095#comment-13609095 ] Srikanth Sundarrajan commented on MAPREDUCE-5014: - A revised patch is now uploaded. With this patch, it should be possible to pass -Ddistcp.copy.listing.class argument to the DistCp command (or on the passed configuration in the DistCp() constructor if using API) to use the new CopyListing. The custom copy listing if extending SimpleCopyListing would use the list of source paths as input by default. If the custom copy listing, requires input to be passed any other way, the same would have to be passed through OPTS (-D params) during invocation. The custom copy listing class can be included in the classpath by including the same via export HADOOP_CLASSPATH > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch, MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Patch Available (was: Open) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch, MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: (was: MAPREDUCE-5014.patch) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Open (was: Patch Available) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch, MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Patch Available (was: Open) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch, MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Open (was: Patch Available) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Open (was: Patch Available) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: (was: MAPREDUCE-5014.patch) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Patch Available (was: Open) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Open (was: Patch Available) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Patch Available (was: Open) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: (was: MAPREDUCE-5014.patch) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Target Version/s: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk (was: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5) Status: Patch Available (was: Open) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: (was: MAPREDUCE-5014.patch) > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
[ https://issues.apache.org/jira/browse/MAPREDUCE-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Sundarrajan updated MAPREDUCE-5014: Attachment: MAPREDUCE-5014.patch > Extending DistCp through a custom CopyListing is not possible > - > > Key: MAPREDUCE-5014 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp >Affects Versions: 0.23.0, 0.23.1, 0.23.3, trunk, 0.23.4, 0.23.5 >Reporter: Srikanth Sundarrajan >Assignee: Srikanth Sundarrajan > Attachments: MAPREDUCE-5014.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > * While it is possible to implement a custom CopyListing in DistCp, DistCp > driver class doesn't allow for using this custom CopyListing. > * Allow SimpleCopyListing to provide an option to exclude files (For instance > it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as > premature copy can indicate that the entire data is available at the > destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5014) Extending DistCp through a custom CopyListing is not possible
Srikanth Sundarrajan created MAPREDUCE-5014: --- Summary: Extending DistCp through a custom CopyListing is not possible Key: MAPREDUCE-5014 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5014 Project: Hadoop Map/Reduce Issue Type: Improvement Components: distcp Affects Versions: 0.23.5, 0.23.4, 0.23.3, 0.23.1, 0.23.0, trunk Reporter: Srikanth Sundarrajan Assignee: Srikanth Sundarrajan * While it is possible to implement a custom CopyListing in DistCp, DistCp driver class doesn't allow for using this custom CopyListing. * Allow SimpleCopyListing to provide an option to exclude files (For instance it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as premature copy can indicate that the entire data is available at the destination) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1941) Need a servlet in JobTracker to stream contents of the job history file
[
https://issues.apache.org/jira/browse/MAPREDUCE-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888710#action_12888710
]
Srikanth Sundarrajan commented on MAPREDUCE-1941:
-
{quote}
This can be done in Job client itself, no? History url is already available in
JobStatus.
{quote}
While the history file name may be available through JobStatus, the history
file is owned by user who runs the job tracker. However access to history file
should be governed by JobACL.VIEW_JOB. Hence the request to have a separate
servlet to provide job history file contents.
> Need a servlet in JobTracker to stream contents of the job history file
> ---
>
> Key: MAPREDUCE-1941
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1941
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
>Affects Versions: 0.22.0
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>
> There is no convenient mechanism to retrieve the contents of the job history
> file. Need a way to retrieve the job history file contents from Job Tracker.
> This can perhaps be implemented as a servlet on the Job tracker.
> * Create a jsp/servlet that accepts job id as a request parameter
> * Stream the contents of the history file corresponding to the job id, if
> user has permissions to view the job details.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-1941) Need a servlet in JobTracker to stream contents of the job history file
Need a servlet in JobTracker to stream contents of the job history file --- Key: MAPREDUCE-1941 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1941 Project: Hadoop Map/Reduce Issue Type: New Feature Components: jobtracker Affects Versions: 0.22.0 Reporter: Srikanth Sundarrajan Assignee: Srikanth Sundarrajan There is no convenient mechanism to retrieve the contents of the job history file. Need a way to retrieve the job history file contents from Job Tracker. This can perhaps be implemented as a servlet on the Job tracker. * Create a jsp/servlet that accepts job id as a request parameter * Stream the contents of the history file corresponding to the job id, if user has permissions to view the job details. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
