anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446918879
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
Review Comment:
Same here - will fix in a separate PR
--
T
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446918772
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -452,6 +452,7 @@ class RocksDB(
* Drop uncommitted changes, and
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446858015
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -452,6 +452,7 @@ class RocksDB(
* Drop uncommitted changes, and
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446857416
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
Review Comment:
Actually looked at other places - it does seem
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446857416
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
Review Comment:
Actually looked at other places - it does seem
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446853378
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
Review Comment:
If you look at the indent pattern - the braces
HeartSaVioR commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1446779347
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -452,6 +452,7 @@ class RocksDB(
* Drop uncommitted changes, and r
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1445834995
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala:
##
@@ -84,10 +84,11 @@ trait ReadStateStore {
/**
* Clean up t
HeartSaVioR commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1445805415
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala:
##
@@ -84,10 +84,11 @@ trait ReadStateStore {
/**
* Clean up th
HeartSaVioR commented on PR #44542:
URL: https://github.com/apache/spark/pull/44542#issuecomment-1882649328
That said, what we really need to fix is the behavior when maintenance task
fails. A single streaming query using faulty state store provider
implementation can lead every other state
HeartSaVioR commented on PR #44542:
URL: https://github.com/apache/spark/pull/44542#issuecomment-1882621196
I believe the real thing here is that the failure of maintenance task is
hammering all active state store providers, effectively impacting to all
stateful tasks on the executor.
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1438964350
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -434,22 +434,26 @@ case class StateStoreRestoreExec(
numC
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1438964350
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -434,22 +434,26 @@ case class StateStoreRestoreExec(
numC
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1438963938
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -434,22 +434,26 @@ case class StateStoreRestoreExec(
numC
rangadi commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1438942186
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -434,22 +434,26 @@ case class StateStoreRestoreExec(
numColsPr
anishshri-db commented on code in PR #44542:
URL: https://github.com/apache/spark/pull/44542#discussion_r1438619998
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -434,22 +434,26 @@ case class StateStoreRestoreExec(
numC
anishshri-db commented on PR #44542:
URL: https://github.com/apache/spark/pull/44542#issuecomment-1872492772
cc - @HeartSaVioR - PTAL whenever you get a chance, thx !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
17 matches
Mail list logo
