hudi-agent commented on code in PR #19753:
URL: https://github.com/apache/hudi/pull/19753#discussion_r3863803833
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestHoodieSparkSqlWriter.scala:
##########
@@ -1420,6 +1420,57 @@ def testBulkInsertForDropPartitionColumn(): Unit = {
val tableMetaClient = createMetaClient(spark, tempBasePath)
new TableSchemaResolver(tableMetaClient).getTableSchema(false)
}
+
+ /**
+ * Test that upsert works correctly when partition path contains Unicode
characters.
+ * Reproduces a bug where UTF-8 bytes for characters like "ΓΌ" (U+00FC) get
misinterpreted
+ * as Latin-1 during the String-to-Path round-trip, causing file-not-found
errors on the
+ * second write.
+ */
+ @Test
+ def testUpsertWithUnicodePartitionPath(): Unit = {
+ val options = Map(
+ DataSourceWriteOptions.TABLE_TYPE.key ->
HoodieTableType.COPY_ON_WRITE.name(),
+ DataSourceWriteOptions.PRECOMBINE_FIELD.key -> "ts",
+ DataSourceWriteOptions.RECORDKEY_FIELD.key -> "uuid",
+ DataSourceWriteOptions.PARTITIONPATH_FIELD.key -> "company",
+ DataSourceWriteOptions.KEYGENERATOR_CLASS_NAME.key ->
"org.apache.hudi.keygen.SimpleKeyGenerator",
+ HoodieWriteConfig.TBL_NAME.key -> "hoodie_test",
+ "hoodie.insert.shuffle.parallelism" -> "1",
+ "hoodie.upsert.shuffle.parallelism" -> "1",
Review Comment:
π€ nit: could you use
`FileSystemViewStorageConfig.REMOTE_RESPONSE_CHARSET.key()` here instead of the
raw string literal? The constant was just introduced in this PR, so referencing
it directly keeps the test in sync automatically if the key ever changes.
<sub><i>β οΈ AI-generated; verify before applying. React π/π to flag
quality.</i></sub>
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineServiceClient.java:
##########
@@ -39,12 +41,14 @@ public class TimelineServiceClient extends
TimelineServiceClientBase {
protected final String timelineServerHost;
protected final int timelineServerPort;
protected final int timeoutMs;
+ protected final String responseCharsetName;
Review Comment:
π€ nit: `HoodieTableServiceManagerClient` stores a `Charset` field resolved
once in the constructor, but here `responseCharsetName` stays as a `String` and
`Charset.forName()` is called twice inside `executeRequest`. It might be worth
storing a `Charset` field here too for consistency and to avoid the repeated
per-request lookup.
<sub><i>β οΈ AI-generated; verify before applying. React π/π to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
