[GitHub] [hudi] codope commented on a diff in pull request #7542: [HUDI-5469] Hive doesn't respect the space at the end of partition path, so remove it to avoid dupl…

[GitBox]

codope commented on code in PR #7542:
URL: https://github.com/apache/hudi/pull/7542#discussion_r1058573755


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##
@@ -131,6 +131,9 @@ public static String getRecordPartitionPath(GenericRecord 
record, List p
   } else {
 if (encodePartitionPath) {
   fieldVal = PartitionPathEncodeUtils.escapePathName(fieldVal);
+} else {
+  // Hive doesn't respect the space at the end, so remove it to avoid 
duplicate keys error
+  fieldVal = fieldVal.trim();

Review Comment:
   I'm not sure if I follow the fix. So, if the value is trimmed then isn't it 
same as untrimmed value and hence duplicate partition key?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hudi] codope commented on a diff in pull request #7542: [HUDI-5469] Hive doesn't respect the space at the end of partition path, so remove it to avoid dupl…

[GitBox]

codope commented on code in PR #7542:
URL: https://github.com/apache/hudi/pull/7542#discussion_r1058572914


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##
@@ -131,6 +131,9 @@ public static String getRecordPartitionPath(GenericRecord 
record, List p
   } else {
 if (encodePartitionPath) {
   fieldVal = PartitionPathEncodeUtils.escapePathName(fieldVal);
+} else {
+  // Hive doesn't respect the space at the end, so remove it to avoid 
duplicate keys error
+  fieldVal = fieldVal.trim();
 }

Review Comment:
   I think it's ok to error out if other engines don't support trailing spaces. 
We can check with Presto/Trino.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org