bhasudha commented on a change in pull request #689: [HUDI-25] Optimize
HoodieInputFormat.listStatus for faster Hive Incremental queries
URL: https://github.com/apache/incubator-hudi/pull/689#discussion_r287533328
##########
File path:
hoodie-hadoop-mr/src/main/java/com/uber/hoodie/hadoop/HoodieInputFormat.java
##########
@@ -218,4 +206,174 @@ protected static HoodieTableMetaClient
getTableMetaClient(FileSystem fs, Path da
LOG.info("Reading hoodie metadata from path " + baseDir.toString());
return new HoodieTableMetaClient(fs.getConf(), baseDir.toString());
}
+
+ /**
+ * InputPathHandler takes in a set of input paths and incremental tables
list. Then, classifies the
+ * input paths to incremental, non incremental paths and non hoodie paths.
This is then accessed
+ * later to mutate the JobConf before processing incremental mode queries
and snapshot queries.
+ */
+ private class InputPathHandler {
Review comment:
Sure. Done!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
