Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/2131#discussion_r68402254
--- Diff:
flink-streaming-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxy.java
---
@@ -159,42 +217,65 @@ public String getShardIterator(KinesisStreamShard
shard, String shardIteratorTyp
return kinesisClient.getShardIterator(shard.getStreamName(),
shard.getShardId(), shardIteratorType, startingSeqNum).getShardIterator();
}
+ private List<KinesisStreamShard> getShardsOfStream(String streamName,
String lastSeenShardId) throws InterruptedException {
+ List<KinesisStreamShard> shardsOfStream = new ArrayList<>();
+
+ DescribeStreamResult describeStreamResult;
+ do {
+ describeStreamResult = describeStream(streamName,
lastSeenShardId);
+
+ List<Shard> shards =
describeStreamResult.getStreamDescription().getShards();
+ for (Shard shard : shards) {
+ shardsOfStream.add(new
KinesisStreamShard(streamName, shard));
+ }
+
+ if (shards.size() != 0) {
+ lastSeenShardId = shards.get(shards.size() -
1).getShardId();
+ }
+ } while
(describeStreamResult.getStreamDescription().isHasMoreShards());
+
+ return shardsOfStream;
+ }
+
/**
* Get metainfo for a Kinesis stream, which contains information about
which shards this Kinesis stream possess.
*
+ * This method is using a "full jitter" approach described in
+ * <a
href="http://google.com";>https://www.awsarchitectureblog.com/2015/03/backoff.html</a>.
This is necessary
--- End diff --
Why is this linking to google.com?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
