[
https://issues.apache.org/jira/browse/HBASE-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276684#comment-14276684
]
Matteo Bertozzi commented on HBASE-12773:
-----------------------------------------
ok, what about avoiding the extra NN call to get the file len and do something
like?
{code}
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
@@ -206,8 +206,11 @@ public class LoadIncrementalHFiles extends Configured
implements Tool {
}
Path familyDir = stat.getPath();
byte[] family = familyDir.getName().getBytes();
- Path[] hfiles = FileUtil.stat2Paths(fs.listStatus(familyDir));
- for (Path hfile : hfiles) {
+ FileStatus[] hfiles = fs.listStatus(familyDir);
+ for (FileStatus hfileStatus : hfiles) {
+ // Skip "_", reference, HFileLink
+ Path hfile = hfileStatus.getPath();
+ long length = hfileStatus.getLen();
// Skip "_", reference, HFileLink
String fileName = hfile.getName();
if (fileName.startsWith("_")) continue;
{code}
we already have the FileStatus for the hfiles, but we were throwing them away
with stat2Paths()
> Add warning message when user is trying to bulkload a large HFile.
> ------------------------------------------------------------------
>
> Key: HBASE-12773
> URL: https://issues.apache.org/jira/browse/HBASE-12773
> Project: HBase
> Issue Type: Improvement
> Reporter: Srikanth Srungarapu
> Assignee: Srikanth Srungarapu
> Priority: Minor
> Attachments: HBASE-12773.patch, HBASE-12773_v2.patch,
> HBASE-12773_v3.patch
>
>
> As per the description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
