nsivabalan commented on a change in pull request #4243:
URL: https://github.com/apache/hudi/pull/4243#discussion_r766111679
##########
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java
##########
@@ -149,7 +149,7 @@
private static final Logger LOG =
LogManager.getLogger(TestHoodieBackedMetadata.class);
- public static List<Arguments> bootstrapAndTableOperationTestArgs() {
+ public static List<Arguments> tableTypeAndBooleanArgs() {
Review comment:
this is used for other tests as well
```
@MethodSource("tableTypeAndBooleanArgs")
public void testMetadataTableBootstrap(HoodieTableType tableType, boolean
addRollback) throws Exception {
```
##########
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java
##########
@@ -133,7 +133,8 @@ public static void deleteMetadataTable(String basePath,
HoodieEngineContext cont
public static List<HoodieRecord>
convertMetadataToRecords(HoodieCleanMetadata cleanMetadata, String instantTime)
{
List<HoodieRecord> records = new LinkedList<>();
int[] fileDeleteCount = {0};
- cleanMetadata.getPartitionMetadata().forEach((partition,
partitionMetadata) -> {
+ cleanMetadata.getPartitionMetadata().forEach((partitionName,
partitionMetadata) -> {
+ final String partition = partitionName.equals("") ? NON_PARTITIONED_NAME
: partitionName;
Review comment:
unfortunately I could not find any other good place to fit it in.
we will have a handle to partition name only by going through the commit
metadata (getPartitionMetadata), and we have to fix it before creating
HoodieRecord. Let me know if you can find a better way to go about this. happy
to incorporate.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java
##########
@@ -282,20 +283,22 @@ private static void
processRollbackMetadata(HoodieActiveTimeline metadataTableTi
List<HoodieRecord> records = new LinkedList<>();
int[] fileChangeCount = {0, 0}; // deletes, appends
- partitionToDeletedFiles.forEach((partition, deletedFiles) -> {
+ partitionToDeletedFiles.forEach((partitionName, deletedFiles) -> {
fileChangeCount[0] += deletedFiles.size();
+ final String partition = partitionName.equals("") ? NON_PARTITIONED_NAME
: partitionName;
Option<Map<String, Long>> filesAdded = Option.empty();
- if (partitionToAppendedFiles.containsKey(partition)) {
- filesAdded = Option.of(partitionToAppendedFiles.remove(partition));
+ if (partitionToAppendedFiles.containsKey(partitionName)) {
Review comment:
again, we create HoodieRecords for metadata table within forEach, and so
couldn't do it outside.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
