ahmedabu98 commented on code in PR #38039:
URL: https://github.com/apache/beam/pull/38039#discussion_r3032701082
##########
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFilesSchemaTransformProvider.java:
##########
@@ -71,14 +71,11 @@ public static Builder builder() {
public abstract @Nullable Map<String, String> getConfigProperties();
@SchemaFieldDescription(
- "For a streaming pipeline, sets the frequency at which incoming files
are appended. Defaults to 600 (10 minutes). "
- + "A commit is triggered when either this or append batch size is
reached.")
+ "For a streaming pipeline, sets the frequency at which incoming files
are appended (default 600, or 10min).")
public abstract @Nullable Integer getTriggeringFrequencySeconds();
- @SchemaFieldDescription(
- "For a streaming pipeline, sets the desired number of appended files
per commit. Defaults to 100,000 files. "
- + "A commit is triggered when either this or append triggering
interval is reached.")
- public abstract @Nullable Integer getAppendBatchSize();
+ @SchemaFieldDescription("The number of data files per manifest (default
10,000 files).")
Review Comment:
To tweak the performance of AddFiles, and the performance of table queries
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
