yihua commented on code in PR #12594:
URL: https://github.com/apache/hudi/pull/12594#discussion_r1933427995
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java:
##########
@@ -674,26 +690,31 @@ private static Map<HeaderMetadataType, String>
getUpdatedHeader(Map<HeaderMetada
updatedHeader.put(
HeaderMetadataType.IS_PARTIAL, Boolean.toString(true));
}
+ if (shouldWriteRecordPositions && baseInstantTimeForPositions.isPresent())
{
Review Comment:
Fixed.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java:
##########
@@ -79,12 +78,11 @@ public abstract class HoodieDataBlock extends
HoodieLogBlock {
* NOTE: This ctor is used on the write-path (ie when records ought to be
written into the log)
*/
public HoodieDataBlock(List<HoodieRecord> records,
- boolean shouldWriteRecordPositions,
Map<HeaderMetadataType, String> header,
Map<FooterMetadataType, String> footer,
String keyFieldName) {
super(header, footer, Option.empty(), Option.empty(), null, false);
- if (shouldWriteRecordPositions) {
+ if (containsBaseFileInstantTimeOfPositions()) {
Review Comment:
Fixed. Also added `HoodieLogBlock#addRecordPositionsIfRequired` to avoid
code duplication.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java:
##########
@@ -674,26 +690,31 @@ private static Map<HeaderMetadataType, String>
getUpdatedHeader(Map<HeaderMetada
updatedHeader.put(
HeaderMetadataType.IS_PARTIAL, Boolean.toString(true));
}
+ if (shouldWriteRecordPositions && baseInstantTimeForPositions.isPresent())
{
+ updatedHeader.put(
+ HeaderMetadataType.BASE_FILE_INSTANT_TIME_OF_RECORD_POSITIONS,
+ baseInstantTimeForPositions.get());
+ }
return updatedHeader;
}
- private static HoodieLogBlock getBlock(HoodieWriteConfig writeConfig,
- HoodieLogBlock.HoodieLogBlockType
logDataBlockFormat,
- List<HoodieRecord> records,
- boolean shouldWriteRecordPositions,
- Map<HeaderMetadataType, String>
header,
- String keyField) {
+ private static HoodieLogBlock getDataBlock(HoodieWriteConfig writeConfig,
+ HoodieLogBlock.HoodieLogBlockType
logDataBlockFormat,
+ List<HoodieRecord> records,
+ Map<HeaderMetadataType, String>
header,
+ String keyField) {
switch (logDataBlockFormat) {
case AVRO_DATA_BLOCK:
- return new HoodieAvroDataBlock(records, shouldWriteRecordPositions,
header, keyField);
+ return new HoodieAvroDataBlock(records, header, keyField);
case HFILE_DATA_BLOCK:
+ // Not supporting positions in HFile data blocks
+
header.remove(HeaderMetadataType.BASE_FILE_INSTANT_TIME_OF_RECORD_POSITIONS);
Review Comment:
Disabled positions for MDT. Still keeping this logic here as a safe-guard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
