Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r214231328
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/CustomMetrics.java ---
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r214227481
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/CustomMetrics.java ---
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r208109148
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -196,6 +237,18 @@ trait ProgressReporter
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r208106031
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -196,6 +237,18 @@ trait ProgressReporter
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r208089043
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -196,6 +237,18 @@ trait ProgressReporter
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21721
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r207232187
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,27 @@ class SourceProgress protected[sql](
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r207090934
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,27 @@ class SourceProgress protected[sql](
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r207042893
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,8 @@ class SourceProgress protected[sql](
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206392487
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,8 @@ class SourceProgress protected[sql](
val
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206374910
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,8 @@ class SourceProgress protected[sql](
val
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206374120
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/CustomMetrics.java ---
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206373655
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -143,18 +151,46 @@ trait ProgressReporter
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206373303
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -163,7 +163,8 @@ class SourceProgress protected[sql](
val
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206349942
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/SupportsCustomWriterMetrics.java
---
@@ -0,0 +1,45 @@
+/*
+
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206347147
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/SupportsCustomReaderMetrics.java
---
@@ -0,0 +1,45 @@
+/*
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206325719
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/SupportsCustomReaderMetrics.java
---
@@ -0,0 +1,45 @@
+/*
+
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206322694
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/SupportsCustomReaderMetrics.java
---
@@ -0,0 +1,45 @@
+/*
+
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206241038
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/SupportsCustomWriterMetrics.java
---
@@ -0,0 +1,45 @@
+/*
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206237610
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -143,18 +150,50 @@ trait ProgressReporter
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206010782
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/SupportsCustomWriterMetrics.java
---
@@ -0,0 +1,45 @@
+/*
+
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206010370
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -143,18 +150,50 @@ trait ProgressReporter
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r206009928
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -143,18 +150,50 @@ trait ProgressReporter
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r201402523
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/JsonUtils.scala
---
@@ -95,4 +95,25 @@ private object JsonUtils {
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200730270
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/JsonUtils.scala
---
@@ -95,4 +95,25 @@ private object JsonUtils {
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200730240
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -178,12 +180,18 @@ class SourceProgress protected[sql](
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200538008
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/JsonUtils.scala
---
@@ -95,4 +95,25 @@ private object JsonUtils {
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200537533
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -379,3 +384,16 @@ private[kafka010]
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200537276
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/SupportsCustomMetrics.java
---
@@ -0,0 +1,30 @@
+/*
--- End diff --
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21721#discussion_r200537454
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -178,12 +180,18 @@ class SourceProgress protected[sql](
GitHub user arunmahadevan opened a pull request:
https://github.com/apache/spark/pull/21721
[SPARK-24748][SS] Support for reporting custom metrics via StreamingQuery
Progress
## What changes were proposed in this pull request?
Currently the Structured Streaming sources and
31 matches
Mail list logo
