KKcorps commented on code in PR #13649:
URL: https://github.com/apache/pinot/pull/13649#discussion_r1690763727
##########
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConfig.java:
##########
@@ -70,7 +70,12 @@ public class KinesisConfig {
public static final String DEFAULT_IAM_ROLE_BASED_ACCESS_ENABLED = "false";
public static final String DEFAULT_SESSION_DURATION_SECONDS = "900";
public static final String DEFAULT_ASYNC_SESSION_UPDATED_ENABLED = "true";
- public static final String DEFAULT_RPS_LIMIT = "5";
+
+ // Kinesis has a default limit of 5 getRecord requests per second per shard.
Review Comment:
done
##########
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumer.java:
##########
@@ -63,7 +63,16 @@ public KinesisConsumer(KinesisConfig config, KinesisClient
kinesisClient) {
@Override
public synchronized KinesisMessageBatch
fetchMessages(StreamPartitionMsgOffset startMsgOffset, int timeoutMs) {
- KinesisPartitionGroupOffset startOffset = (KinesisPartitionGroupOffset)
startMsgOffset;
+ try {
+ return getKinesisMessageBatch((KinesisPartitionGroupOffset)
startMsgOffset);
+ } catch (ProvisionedThroughputExceededException pte) {
+ LOGGER.debug("Provisioned throughput exceeded while fetching messages
from Kinesis stream: {}", pte.getMessage());
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
