Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r210708107
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -254,3 +259,10 @@ class SinkProgress protected[sql](
}
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r210707152
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -254,3 +259,10 @@ class SinkProgress protected[sql](
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r209621895
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -198,11 +198,14 @@ class SourceProgress protected[sql](
*
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208902692
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -213,6 +216,12 @@ class SinkProgress protected[sql](
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208902157
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -198,11 +198,14 @@ class SourceProgress protected[sql](
*
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208901343
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -58,6 +61,7 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208750925
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -198,11 +198,14 @@ class SourceProgress protected[sql](
*
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208749439
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -58,6 +61,7 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208751032
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -213,6 +216,12 @@ class SinkProgress protected[sql](
override
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208107058
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -46,6 +46,9 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r208100226
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -46,6 +46,9 @@ case class
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r207991171
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -179,3 +192,24 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r207961663
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -179,3 +192,24 @@ class
Github user vackosar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r207716505
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriterCommitProgress.java
---
@@ -0,0 +1,31 @@
+/*
+ *
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21919#discussion_r207662087
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriterCommitProgress.java
---
@@ -0,0 +1,31 @@
+/*
+ *
GitHub user vackosar opened a pull request:
https://github.com/apache/spark/pull/21919
[SPARK-24933][SS] Report numOutputRows in SinkProgress via WrittenComâ¦
## What changes were proposed in this pull request?
SinkProgress should report similar properties like
16 matches
Mail list logo
