Re: s3n exceptions
Are you using 0.18? I know that the copy HDFS to s3n isn't supported there yet. I think there's a fix in 0.19. / Per On Mon, Nov 24, 2008 at 2:11 AM, Alexander Aristov [EMAIL PROTECTED] wrote: Hi all I am testing s3n file system facilities and try to copy from hdfs to S3 in original format And I get next errors 08/11/24 05:04:49 INFO mapred.JobClient: Running job: job_200811240437_0004 08/11/24 05:04:50 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:00 INFO mapred.JobClient: map 44% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) 08/11/24 05:05:15 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) -- Best Regards Alexander Aristov
Re: s3n exceptions
yes, I am trying 0.18.2 But according to hadoop wiki it is supported http://wiki.apache.org/hadoop/AmazonS3 and http://issues.apache.org/jira/browse/HADOOP-930 Alexander 2008/11/26 Per Jacobsson [EMAIL PROTECTED] Are you using 0.18? I know that the copy HDFS to s3n isn't supported there yet. I think there's a fix in 0.19. / Per On Mon, Nov 24, 2008 at 2:11 AM, Alexander Aristov [EMAIL PROTECTED] wrote: Hi all I am testing s3n file system facilities and try to copy from hdfs to S3 in original format And I get next errors 08/11/24 05:04:49 INFO mapred.JobClient: Running job: job_200811240437_0004 08/11/24 05:04:50 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:00 INFO mapred.JobClient: map 44% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) 08/11/24 05:05:15 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) -- Best Regards Alexander Aristov -- Best Regards Alexander Aristov
Re: s3n exceptions
I'm thinking of this bug: http://issues.apache.org/jira/browse/HADOOP-3361 It was the cause of problems I hit trying to copy from HDFS to S3N with 0.18.1. / Per On Wed, Nov 26, 2008 at 12:57 PM, Alexander Aristov [EMAIL PROTECTED] wrote: yes, I am trying 0.18.2 But according to hadoop wiki it is supported http://wiki.apache.org/hadoop/AmazonS3 and http://issues.apache.org/jira/browse/HADOOP-930 Alexander 2008/11/26 Per Jacobsson [EMAIL PROTECTED] Are you using 0.18? I know that the copy HDFS to s3n isn't supported there yet. I think there's a fix in 0.19. / Per On Mon, Nov 24, 2008 at 2:11 AM, Alexander Aristov [EMAIL PROTECTED] wrote: Hi all I am testing s3n file system facilities and try to copy from hdfs to S3 in original format And I get next errors 08/11/24 05:04:49 INFO mapred.JobClient: Running job: job_200811240437_0004 08/11/24 05:04:50 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:00 INFO mapred.JobClient: map 44% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) 08/11/24 05:05:15 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) -- Best Regards Alexander Aristov -- Best Regards Alexander Aristov
s3n exceptions
Hi all I am testing s3n file system facilities and try to copy from hdfs to S3 in original format And I get next errors 08/11/24 05:04:49 INFO mapred.JobClient: Running job: job_200811240437_0004 08/11/24 05:04:50 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:00 INFO mapred.JobClient: map 44% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: map 0% reduce 0% 08/11/24 05:05:03 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) 08/11/24 05:05:15 INFO mapred.JobClient: Task Id : attempt_200811240437_0004_m_00_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 7 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:542) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) -- Best Regards Alexander Aristov
