SEPURI-SAI-KRISHNA commented on code in PR #19706:
URL: https://github.com/apache/hudi/pull/19706#discussion_r3835909495
##########
hudi-common/src/main/java/org/apache/hudi/common/table/PartitionPathParser.java:
##########
@@ -70,15 +89,35 @@ private static Object[] getPartitionValues(String[]
partitionFields,
partitionValues[i] = inferDateValue(partitionPath, parts, pathSegment,
numDateDirs, fieldSchema);
pathSegment += numDateDirs;
} else {
- String segment = parts[pathSegment];
+ // A slash-separated value occupies every segment this field is
entitled to, mirroring the
+ // way [[#inferDateValue]] consumes them for a time-based column
+ int numDirs = valueSpansSegments ? parts.length -
partitionFields.length + 1 : 1;
+ String segment = rejoinDashSeparatedValue(parts, pathSegment, numDirs);
String[] segmentParts = segment.split(EQUALS_SIGN);
partitionValues[i] = parseValue(segmentParts[segmentParts.length - 1],
fieldSchema);
- pathSegment++;
+ pathSegment += numDirs;
}
}
return partitionValues;
}
+ /**
+ * Undoes the {@code -} -> {@code /} substitution the writer performs for
+ * {@code hoodie.datasource.write.slash.separated.date.partitioning},
rejoining the {@code numDirs}
+ * path segments starting at {@code pathSegment} back into the single value
they were written from.
+ * For a value that was not slash-separated {@code numDirs} is 1 and the
segment is returned as-is.
+ */
+ private static String rejoinDashSeparatedValue(String[] parts, int
pathSegment, int numDirs) {
+ if (numDirs == 1) {
+ return parts[pathSegment];
+ }
+ StringBuilder value = new StringBuilder(parts[pathSegment]);
+ for (int i = 1; i < numDirs; i++) {
+ value.append(DASH).append(parts[pathSegment + i]);
+ }
+ return value.toString();
+ }
+
@VisibleForTesting
static Object parseValue(String partitionValue, HoodieSchema fieldSchema) {
if (partitionValue.equals(DEFAULT_PARTITION_PATH) ||
partitionValue.equals(DEPRECATED_DEFAULT_PARTITION_PATH)) {
Review Comment:
Renamed to `joinSegmentsWithDash`. Agreed the old name described the input's
prior state rather than what the method does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
