GitHub user yssharma opened a pull request:
https://github.com/apache/spark/pull/18029
[SPARK-20168][WIP][DStream] Add changes to use kinesis fetches from
specified timestamp
## What changes were proposed in this pull request?
Kinesis client can resume from a specified time
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723184
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -112,4 +112,38 @@ class Kinesis
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723865
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -148,17 +149,31 @@ private[kinesis] class Kines
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119724691
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -38,6 +40,7 @@ private[kinesis] class Kines
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723361
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -57,6 +58,26 @@ public void t
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119724818
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -84,6 +84,7 @@ private[kinesis] class KinesisRe
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119725220
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala
---
@@ -73,7 +73,7 @@ object KinesisUtils {
// S
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119984045
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -38,6 +40,7 @@ private[kinesis] class Ki
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119986280
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -111,5 +110,28 @@ class Kine
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119986035
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -45,7 +46,7 @@ public void
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120181290
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -193,6 +197,21 @@ object KinesisInputDStre
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120180756
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -100,6 +103,7 @@ object KinesisInputDStrea
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120180952
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -111,5 +111,29 @@ class Kinesi
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120234200
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -100,6 +103,7 @@ object KinesisInputDStr
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120234538
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -100,6 +103,7 @@ object KinesisInputDStrea
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120235619
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -193,6 +197,21 @@ object KinesisInputDSt
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120235938
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -193,6 +197,21 @@ object KinesisInputDStre
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120236059
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -100,6 +103,7 @@ object KinesisInputDStr
Github user yssharma commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r120236128
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -193,6 +197,21 @@ object KinesisInputDSt
19 matches
Mail list logo
