Github user efimpoberezkin closed the pull request at:
https://github.com/apache/spark/pull/21063
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user efimpoberezkin closed the pull request at:
https://github.com/apache/spark/pull/21392
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user efimpoberezkin closed the pull request at:
https://github.com/apache/spark/pull/20832
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21392
Sure, no problem.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21392
@HyukjinKwon Hi, I've stopped working on it for some time now
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21392#discussion_r190473131
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala
---
@@ -45,6 +45,11 @@ private[sql
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21392#discussion_r190160821
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala
---
@@ -45,6 +45,11 @@ private[sql
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21392#discussion_r190158513
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala
---
@@ -153,9 +161,13 @@ private
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21392
@jose-torres Hi Jose, could you take a look at this pr please? I had doubts
how to properly implement error reporting logic we were discussing and this is
what I came up with.
Also
GitHub user efimpoberezkin opened a pull request:
https://github.com/apache/spark/pull/21392
[SPARK-24063][SS] Control maximum epoch backlog for ContinuousExecution
## What changes were proposed in this pull request?
This pull request adds maxEpochBacklog SQL configuration
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@tdas can't start tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@tdas Hi, seems like it's good to go. Could you merge this when you have
time?
---
-
To unsubscribe, e-mail: reviews
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@tdas Hi, could you give me some advice on how to resolve this error? I'm
not sure how this PR could cause this, especially considering it was passing
tests before and no changes were made
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20936#discussion_r184361843
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala
---
@@ -137,30 +137,65 @@ private
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21063#discussion_r184342675
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -111,7 +112,12 @@ trait ProgressReporter
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@felixcheung hi, could you merge this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21063
@jose-torres Hi Jose, could you review and tell me if we need more tests or
a single one will do? Also is it fine to assert status after every stream
action
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@tdas @jose-torres Hi, I tested this with
https://github.com/apache/spark/pull/20983, can we run jenkins tests now
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21063
Hi @jose-torres, I made some changes to this PR according to your comment,
could you review it please
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20983
@tdas @jose-torres should we merge this now that it passes tests and test
SPARK-23503 changes?
---
-
To unsubscribe, e
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/21063
@jose-torres Hi Jose, I've added a couple of update messages to places you
suggested, query stop indeed seems to be reported in StreamExecution.
I have a question regarding updating
GitHub user efimpoberezkin opened a pull request:
https://github.com/apache/spark/pull/21063
[SPARK-23886][Structured Streaming][WIP] Update query status for
ContinuousExecution
## What changes were proposed in this pull request?
Added query status updates
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20983#discussion_r180679855
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/EpochCoordinatorSuite.scala
---
@@ -0,0 +1,225
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20983
@jose-torres
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20983#discussion_r180203421
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/EpochCoordinatorSuite.scala
---
@@ -0,0 +1,202
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20983
@jose-torres this PR is ready for a review
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user efimpoberezkin commented on the issue:
https://github.com/apache/spark/pull/20936
@jose-torres Could you review latest changes for this PR please?
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user efimpoberezkin opened a pull request:
https://github.com/apache/spark/pull/20983
[SPARK-23747][Structured Streaming] Add EpochCoordinator unit tests
## What changes were proposed in this pull request?
Unit tests for EpochCoordinator that test correct sequencing
GitHub user efimpoberezkin opened a pull request:
https://github.com/apache/spark/pull/20936
[SPARK-23503][Structured Streaming] Enforcing sequencing of committed
epochs for Continuous Execution
## What changes were proposed in this pull request?
Made changes
Github user efimpoberezkin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20832#discussion_r174706413
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ---
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends
Column(name
GitHub user efimpoberezkin opened a pull request:
https://github.com/apache/spark/pull/20832
[SPARK-20536][SQL] Extend ColumnName to create StructFields with explicit
nullable
## What changes were proposed in this pull request?
Extended ColumnName with methods that create
32 matches
Mail list logo
