Re: S3 Hadoop FileSystems

2016-04-30 Thread Chris Nauroth
I have some more information regarding MD5 verification with s3a. It turns out that s3a does have the MD5 verification. It's just not visible from reading the s3a code, because the MD5 verification is performed entirely within the AWS SDK library dependency. If you're interested in more

Completed Tasks in YARN will not release resources

2016-04-30 Thread Benjamin Kim
Has anyone encountered this problem with YARN. It all started after an attempt to upgrade from CDH 5.4.8 to CDH 5.5.2. I ran jobs overnight, and they never completed. But, it did take down the YARN ResourceManager and multiple NodeManagers after 5 or 6 hours. There was one job that out of 450