markap14 commented on code in PR #10964:
URL: https://github.com/apache/nifi/pull/10964#discussion_r2907005098
##########
nifi-extension-bundles/nifi-aws-bundle/nifi-aws-kinesis/src/main/java/org/apache/nifi/processors/aws/kinesis/ConsumeKinesis.java:
##########
@@ -272,47 +238,49 @@ Specifies the string (interpreted as UTF-8) to use for
demarcating multiple Kine
static final PropertyDescriptor STREAM_POSITION_TIMESTAMP = new
PropertyDescriptor.Builder()
.name("Stream Position Timestamp")
- .description("Timestamp position in stream from which to start
reading Kinesis Records. The timestamp must be in ISO 8601 format.")
+ .description("Timestamp position in stream from which to start
reading Kinesis Records. Must be in ISO 8601 format.")
.required(true)
.addValidator(StandardValidators.ISO8601_INSTANT_VALIDATOR)
.dependsOn(INITIAL_STREAM_POSITION, InitialPosition.AT_TIMESTAMP)
.build();
- static final PropertyDescriptor MAX_BYTES_TO_BUFFER = new
PropertyDescriptor.Builder()
- .name("Max Bytes to Buffer")
- .description("""
- The maximum size of Kinesis Records that can be buffered
in memory before being processed by NiFi.
- If the buffer size exceeds the limit, the processor will
stop consuming new records until free space is available.
-
- Using a larger value may increase the throughput, but will
do so at the expense of using more memory.
- """)
+ static final PropertyDescriptor MAX_RECORDS_PER_REQUEST = new
PropertyDescriptor.Builder()
+ .name("Max Records Per Request")
+ .description("The maximum number of records to retrieve per
GetRecords call. Maximum is 10,000.")
.required(true)
- .addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
- .defaultValue("100 MB")
+ .defaultValue("1000")
+ .addValidator(StandardValidators.createLongValidator(1, 10000,
true))
.build();
- static final PropertyDescriptor CHECKPOINT_INTERVAL = new
PropertyDescriptor.Builder()
- .name("Checkpoint Interval")
+ static final PropertyDescriptor MAX_BATCH_DURATION = new
PropertyDescriptor.Builder()
+ .name("Max Batch Duration")
.description("""
- Interval between checkpointing consumed Kinesis records.
To checkpoint records each time the Processor is run, set this value to 0
seconds.
-
- More frequent checkpoint may reduce performance and
increase DynamoDB costs,
- but less frequent checkpointing may result in duplicates
when a Shard lease is lost or NiFi is restarted.
- """)
+ The maximum amount of time to spend consuming records in a
single invocation before \
+ committing the session and checkpointing.""")
.required(true)
- .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
.defaultValue("5 sec")
+ .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
.build();
- static final PropertyDescriptor METRICS_PUBLISHING = new
PropertyDescriptor.Builder()
- .name("Metrics Publishing")
- .description("Specifies where Kinesis usage metrics are published
to.")
+ static final PropertyDescriptor MAX_BATCH_SIZE = new
PropertyDescriptor.Builder()
+ .name("Max Batch Size")
+ .description("""
+ The maximum amount of data to consume in a single
invocation before committing the \
+ session and checkpointing.""")
.required(true)
- .allowableValues(MetricsPublishing.class)
- .defaultValue(MetricsPublishing.DISABLED)
+ .defaultValue("10 MB")
+ .addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
.build();
- static final PropertyDescriptor PROXY_CONFIGURATION_SERVICE =
ProxyConfiguration.createProxyConfigPropertyDescriptor(ProxySpec.HTTP,
ProxySpec.HTTP_AUTH);
+ static final PropertyDescriptor ENDPOINT_OVERRIDE = new
PropertyDescriptor.Builder()
Review Comment:
No, we want to expose Endpoint Override. Many users use Endpoint Overrides
for a lot of AWS services.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
