[
https://issues.apache.org/jira/browse/HUDI-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375657#comment-17375657
]
ASF GitHub Bot commented on HUDI-2087:
--------------------------------------
yanghua commented on a change in pull request #3174:
URL: https://github.com/apache/hudi/pull/3174#discussion_r662858365
##########
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##########
@@ -408,6 +408,12 @@ public void cleanHandlesGracefully() {
final HoodieRecordLocation loc = record.getCurrentLocation();
final String fileID = loc.getFileId();
final String partitionPath = record.getPartitionPath();
+ // append only mode always use FlinkCreateHandle
+ if (loc.getInstantTime().equals("A")) {
Review comment:
Can we extract the hard code flags e.g. `A` or `I` to be constants?
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java
##########
@@ -186,6 +186,12 @@ private FlinkOptions() {
.defaultValue(TABLE_TYPE_COPY_ON_WRITE)
.withDescription("Type of table to write. COPY_ON_WRITE (or)
MERGE_ON_READ");
+ public static final ConfigOption<Boolean> APPEND_ONLY_ENABLE = ConfigOptions
+ .key("append.only.enable")
Review comment:
`append_only.enable` looks better?
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java
##########
@@ -106,11 +107,17 @@
*/
private final boolean globalIndex;
+ private final boolean appendOnly;
+
public BucketAssignFunction(Configuration conf) {
this.conf = conf;
this.isChangingRecords = WriteOperationType.isChangingRecords(
WriteOperationType.fromValue(conf.getString(FlinkOptions.OPERATION)));
this.globalIndex = conf.getBoolean(FlinkOptions.INDEX_GLOBAL_ENABLED);
+ this.appendOnly = conf.getBoolean(FlinkOptions.APPEND_ONLY_ENABLE);
+ if (appendOnly) {
+
ValidationUtils.checkArgument(conf.getString(FlinkOptions.TABLE_TYPE).equals(HoodieTableType.COPY_ON_WRITE.name()),
"Append only only support in COPY_ON_WRITE.");
Review comment:
too long...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support Append only in Flink stream
> -----------------------------------
>
> Key: HUDI-2087
> URL: https://issues.apache.org/jira/browse/HUDI-2087
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Flink Integration
> Reporter: yuzhaojing
> Assignee: yuzhaojing
> Priority: Major
> Labels: pull-request-available
>
> It is necessary to support append mode in flink stream, as the data lake
> should be able to write log type data as parquet high performance without
> merge.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
