manojpec commented on a change in pull request #4352:
URL: https://github.com/apache/hudi/pull/4352#discussion_r784432391
##########
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java
##########
@@ -146,6 +155,97 @@ protected BaseTableMetadata(HoodieEngineContext
engineContext, HoodieMetadataCon
.getAllFilesInPartitions(partitions);
}
+ @Override
+ public Option<ByteBuffer> getBloomFilter(final PartitionIndexID
partitionIndexID, final FileIndexID fileIndexID)
+ throws HoodieMetadataException {
+ if (!isMetaIndexBloomFilterEnabled) {
+ LOG.error("Meta bloom filter index is disabled!");
+ return Option.empty();
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ final String bloomIndexKey =
partitionIndexID.asBase64EncodedString().concat(fileIndexID.asBase64EncodedString());
+ Option<HoodieRecord<HoodieMetadataPayload>> hoodieRecord =
getRecordByKey(bloomIndexKey,
+ MetadataPartitionType.BLOOM_FILTERS.partitionPath());
+ metrics.ifPresent(m ->
m.updateMetrics(HoodieMetadataMetrics.LOOKUP_BLOOM_FILTERS_METADATA_STR,
timer.endTimer()));
+
+ if (!hoodieRecord.isPresent()) {
+ LOG.error("Meta bloom filter index: lookup failed for partition: " +
partitionIndexID.getName() + ", file: "
+ + fileIndexID.getName());
+ return Option.empty();
+ }
+
+ final Option<HoodieMetadataBloomFilter> fileBloomFilter =
hoodieRecord.get().getData().getBloomFilterMetadata();
+ if (!fileBloomFilter.isPresent()) {
+ LOG.error("Meta bloom filter index: bloom filter missing for partition:
" + partitionIndexID.getName()
Review comment:
Today meta indices are turned on by configs. Not all records written out
might carry the payload optional fields like bloom or column. Want to handle it
here if the writer wrote this record with wrong configs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
