vinothchandar commented on a change in pull request #600: Timeline Service
with Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r276511122
##########
File path:
hoodie-client/src/main/java/com/uber/hoodie/io/HoodieMergeHandle.java
##########
@@ -69,17 +67,14 @@
public HoodieMergeHandle(HoodieWriteConfig config, String commitTime,
HoodieTable<T> hoodieTable,
Iterator<HoodieRecord<T>> recordItr, String fileId) {
super(config, commitTime, hoodieTable);
- this.fileSystemView = hoodieTable.getROFileSystemView();
String partitionPath = init(fileId, recordItr);
init(fileId, partitionPath,
- fileSystemView.getLatestDataFiles(partitionPath)
- .filter(dataFile ->
dataFile.getFileId().equals(fileId)).findFirst());
+ hoodieTable.getROFileSystemView().getLatestDataFile(partitionPath,
fileId).get());
}
public HoodieMergeHandle(HoodieWriteConfig config, String commitTime,
HoodieTable<T> hoodieTable,
- Map<String, HoodieRecord<T>> keyToNewRecords, String fileId,
Optional<HoodieDataFile> dataFileToBeMerged) {
+ Map<String, HoodieRecord<T>> keyToNewRecords, String fileId,
HoodieDataFile dataFileToBeMerged) {
Review comment:
To clarify, the reason the dataFile could be optional, is the case when we
write logs first and then later compaction creates the parquet file.. i.e the
first slice wont have data files.. we should support this at the data structure
level, even if we don't do this today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services