Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/19552#discussion_r146416338
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -388,7 +388,7 @@ object SQLConf {
.stringConf
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120545
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -104,12 +103,61 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120489
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120477
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120566
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -104,12 +103,61 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120535
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -72,7 +70,8 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120475
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134120473
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed
Github user budde commented on the issue:
https://github.com/apache/spark/pull/18029
@yssharma Let me know what you think of my review suggestions. I should be
able to review any updates from here on in a timely manner but you will still
need @brkyvz or another Spark committer to do
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134068101
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134063427
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067474
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -148,18 +149,28 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134063380
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067523
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067110
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -148,18 +149,28 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067791
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -308,7 +308,6 @@ object
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067693
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134068186
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r134067537
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user budde commented on the issue:
https://github.com/apache/spark/pull/18029
@yssharma Apologies for the slow reply-- very little of my professional dev
work happens on GitHub so I can be prone to miss things :-/
You would introduce a ```InitialPosition``` interface
Github user budde commented on the issue:
https://github.com/apache/spark/pull/18029
My understanding of @brkyvz's suggestion is that you remove the two
```initialPositionInStream``` and ```initialPositionInStreamTimestamp```
arguments in favor of the single ```InitialPosition
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119725220
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala
---
@@ -73,7 +73,7 @@ object KinesisUtils
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119724818
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -84,6 +84,7 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723361
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -57,6 +58,26 @@ public void
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723865
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -148,17 +149,31 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119724691
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -38,6 +40,7 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r119723184
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -112,4 +112,38 @@ class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/18029
@yssharma I'll try to take a look later today or tomorrow
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r114922000
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -60,12 +61,19 @@ private[kinesis] class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17467
Fair enough. I took another look and I think I may have been thinking of
the way things worked in an earlier revision of this code. I think the case
class is reasonable.
---
If your project is set
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r114850001
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReadConfigurations.scala
---
@@ -0,0 +1,71
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r114850114
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReadConfigurations.scala
---
@@ -0,0 +1,71
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r114851170
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReadConfigurations.scala
---
@@ -0,0 +1,71
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r114851396
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReadConfigurations.scala
---
@@ -0,0 +1,71
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112839963
--- Diff: docs/streaming-kinesis-integration.md ---
@@ -216,3 +216,7 @@ de-aggregate records during consumption.
- If no Kinesis checkpoint info exists
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112816898
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -295,6 +306,23 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112816922
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -295,6 +306,23 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112817123
--- Diff: docs/streaming-kinesis-integration.md ---
@@ -216,3 +216,7 @@ de-aggregate records during consumption.
- If no Kinesis checkpoint info exists
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112816822
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -295,6 +306,23 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112816810
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -295,6 +306,23 @@ class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17467
@yssharma Fair enough. I'll try to get your update reviewed later today
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112764788
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -147,6 +153,17 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112764350
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -112,7 +116,8 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112765111
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -17,21 +17,24 @@
package
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112765206
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -17,21 +17,24 @@
package
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112766374
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -147,6 +153,17 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112766633
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDDSuite.scala
---
@@ -101,6 +101,37 @@ abstract class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112764808
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -147,6 +153,17 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112764344
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -83,7 +86,8 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112566462
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -147,6 +152,14 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112565900
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -249,6 +252,17 @@ object
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112344999
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -249,6 +252,17 @@ object
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112344746
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -249,6 +252,17 @@ object
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17467
Not a Spark committer, but I've contributed to this component in the past.
I would strongly prefer an approach that avoids adding an additional parameter
to all of the Kinesis classes
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112343978
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -147,6 +152,14 @@ class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17467#discussion_r112343115
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala
---
@@ -83,7 +83,8 @@ class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Awesome, thanks for reviewing this!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107977628
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentials.scala
---
@@ -0,0 +1,179 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107975990
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentials.scala
---
@@ -0,0 +1,179 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107971313
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentials.scala
---
@@ -0,0 +1,175 @@
+/*
+ * Licensed
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Sure, want me to add it to this PR or open a new one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Updated per your feedback, thanks for taking a thorough look. I
also renamed the ```longLivedCredsProvider``` of ```STSCredentials``` to just
```longLivedCreds``` to match the updated naming
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107768752
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -0,0 +1,63
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107764949
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -0,0 +1,63
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107762239
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -22,24 +22,28 @@ import
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107761342
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,238 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759778
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentialsBuilderSuite.scala
---
@@ -0,0 +1,112
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107760106
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -22,24 +22,28 @@ import
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759746
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentialsBuilderSuite.scala
---
@@ -0,0 +1,112
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759722
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/SparkAWSCredentialsBuilderSuite.scala
---
@@ -0,0 +1,112
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759395
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -0,0 +1,112
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759339
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -0,0 +1,112
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759287
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -0,0 +1,63
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107759010
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,238 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107758905
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,238 @@ private[kinesis] class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Updated per your feedback. Most significant change is renaming
```SerializableCredentialsProvider``` to ```SparkAWSCredentials``` (as well as
renaming its subclasses) and refactoring its
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107319004
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SerializableCredentialsProvider.scala
---
@@ -83,3 +84,146 @@ private
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318994
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SerializableCredentialsProvider.scala
---
@@ -83,3 +84,146 @@ private
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318916
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SerializableCredentialsProvider.scala
---
@@ -83,3 +84,146 @@ private
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318178
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,256 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318157
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,256 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318150
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,256 @@ private[kinesis] class
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107318164
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/SerializableCredentialsProvider.scala
---
@@ -83,3 +84,146 @@ private
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Updated the PR to remove ```defaultMessageHandler()``` from
```KinesisUtils``` in favor of keeping this method in
```KinesisInputDStream```. My thought here was that this would be a better
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r107025906
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -71,7 +75,256 @@ private[kinesis] class
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz PR has been updated, apologies for the delay. I've added
```SerializableCredentialsProvider.Builder```, which I'm willing to hear
suggestions for a better name on. I wanted to stay away from
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz A conference took up a lot of my time last week but I should have
it updated later today
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz I think if we're eliminating the constructor arguments then the
second approach you've proposed might make more sense. I can't think of
anything cleaner.
---
If your project is set up
Github user budde commented on the issue:
https://github.com/apache/spark/pull/17250
@brkyvz Actually, now that I think about it, do we need to make
```messageHandler``` a constructor arg since ```Builder``` is a generic class?
There's probably a way we could get around this but I'd
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806326
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806315
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806378
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806348
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806322
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806375
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806316
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806339
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806333
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806353
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/17250#discussion_r105806364
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisDStream.scala
---
@@ -0,0 +1,559 @@
+/*
+ * Licensed
1 - 100 of 302 matches
Mail list logo
