catalin-luca commented on pull request #2002: URL: https://github.com/apache/hbase/pull/2002#issuecomment-726070015
> > > > After inspecting thread dumps, I saw lots of time spent in re-creating HTTP connections. It seemed that the connections were not being re-used because the `HFile.Reader` was not reading all the bytes after seeking to read the trailer. In turn this causes the connection to get aborted and it can't be pooled. > > > > > > > > > So the issue is bound to JVM instance, since the better performance with multiple tasks only, but not with multiple threads on same JVM instance? > > > > > > Yup. > > Hum, It seems there's room for improvement in the reader code. Also, have you tried to increase fs.s3a.connection.maximum and fs.s3a.threads.max properties in hbase config? We are actually running over EMRFS in amazon. Increasing the equivalent max connections property did not yield any improvements. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
