github-actions[bot] commented on code in PR #70118:
URL: https://github.com/apache/doris/pull/70118#discussion_r4462762991
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java:
##########
@@ -1856,13 +1858,37 @@ public String getShowCreateInfo() {
if (isPartialUpdate) {
appendProperties(sb, CreateRoutineLoadInfo.PARTIAL_COLUMNS,
isPartialUpdate, false);
}
+ // UPDATE_FLEXIBLE_COLUMNS leaves isPartialUpdate false, so without
the mode the job would be recreated
+ // as an UPSERT one. CREATE only accepts the new key policy together
with a partial update mode.
+ if (uniqueKeyUpdateMode != TUniqueKeyUpdateMode.UPSERT) {
+ appendProperties(sb, CreateRoutineLoadInfo.UNIQUE_KEY_UPDATE_MODE,
uniqueKeyUpdateMode.name(), false);
+ if (partialUpdateNewKeyPolicy == TPartialUpdateNewRowPolicy.ERROR)
{
+ appendProperties(sb,
CreateRoutineLoadInfo.PARTIAL_UPDATE_NEW_KEY_POLICY, "ERROR", false);
+ }
+ }
appendProperties(sb, JsonFileFormatProperties.PROP_JSON_PATHS,
getJsonPaths(), false);
appendProperties(sb, JsonFileFormatProperties.PROP_STRIP_OUTER_ARRAY,
isStripOuterArray(), false);
appendProperties(sb, JsonFileFormatProperties.PROP_NUM_AS_STRING,
isNumAsString(), false);
appendProperties(sb, JsonFileFormatProperties.PROP_FUZZY_PARSE,
isFuzzyParse(), false);
appendProperties(sb, JsonFileFormatProperties.PROP_JSON_ROOT,
getJsonRoot(), false);
+ if (getEnclose() != 0) {
+ appendProperties(sb, CsvFileFormatProperties.PROP_ENCLOSE, new
String(new byte[]{getEnclose()}), false);
Review Comment:
[P2] Preserve high-bit CSV delimiter bytes when rendering them
On a UTF-8 FE (the default in `conf/fe.conf`), decoding an isolated high-bit
byte with `new String(byte[])` produces U+FFFD. CREATE accepts `enclose = 'é'`
and stores byte `0xE9` because `CsvFileFormatProperties` casts the single
character to a byte. This new output prints the replacement character, and
recreating it stores `0xFD`. The adjacent `escape` conversion has the same
problem: `escape = 'é'` changes from `0xC3` to `0xEF`. The recreated job
therefore uses a different CSV parsing byte. Use a reversible representation
matching each property's CREATE decoder and add non-ASCII round-trip cases;
explicitly selecting UTF-8 alone would preserve the loss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
