[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-17 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653574#comment-16653574 ] Ted Yu commented on HADOOP-15850: - I haven't found how to pass the per chunk informatio

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653546#comment-16653546 ] Steve Loughran commented on HADOOP-15850: - Distcp code is pretty complex these

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652876#comment-16652876 ] Ted Yu commented on HADOOP-15850: - I tried to add '-blocksperchunk 0' option when invok

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652765#comment-16652765 ] Ted Yu commented on HADOOP-15850: - The DistCpOptions instance for the DistCp session is

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652736#comment-16652736 ] Ted Yu commented on HADOOP-15850: - [~jojochuang]: See the link to MapReduceBackupCopyJo

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652726#comment-16652726 ] Ted Yu commented on HADOOP-15850: - Running the backup test against hadoop 3.0.x / 3.1.y

[jira] [Commented] (HADOOP-15850) CopyCommitter#concatFileChunks should check that the source file to be merged is a split

2018-10-16 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652723#comment-16652723 ] Wei-Chiu Chuang commented on HADOOP-15850: -- Ted, if I understand it correctly