Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20554
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168626487
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -112,14 +112,18 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168625863
--- Diff:
external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-future-version.bin
---
@@ -0,0 +1,2 @@
+0v9
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168625742
--- Diff:
external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-version-2.1.0.bin
---
@@ -1 +1 @@
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168558972
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -112,14 +112,18 @@ abstract
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168559060
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
---
@@ -303,94 +302,75 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168591005
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ *
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168558562
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -306,7 +307,7 @@ private[kafka010] class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167127585
--- Diff:
external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-version-2.1.0.bin
---
@@ -1 +1 @@
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168366098
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
---
@@ -303,94 +302,75 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r168120568
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167815276
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167811474
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala
---
@@ -28,50 +28,40 @@ import
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167807584
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167809278
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
---
@@ -303,94 +302,75 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167124768
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
---
@@ -303,94 +302,75 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167124564
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
---
@@ -303,94 +302,75 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167124346
--- Diff:
external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-future-version.bin
---
@@ -0,0 +1,2 @@
+0v9
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167124308
--- Diff:
external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-version-2.1.0.bin
---
@@ -1 +1 @@
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123917
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123837
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123713
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123614
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123580
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123513
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -0,0 +1,410 @@
+/*
+ * Licensed to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20554#discussion_r167123199
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala
---
@@ -408,8 +401,27 @@ private[kafka010] object
GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/20554
[SPARK-23362][SS] Migrate Kafka Microbatch source to v2
## What changes were proposed in this pull request?
Migrating KafkaSource (with data source v1) to KafkaMicroBatchReader (with
data source
27 matches
Mail list logo
