Re: [PR] [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed [flink]
masteryhx closed pull request #24768: [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed URL: https://github.com/apache/flink/pull/24768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed [flink]
masteryhx commented on code in PR #24768:
URL: https://github.com/apache/flink/pull/24768#discussion_r1607490839
##
flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/ForStKeyedStateBackend.java:
##
@@ -147,15 +160,30 @@ public S createState(@Nonnull
StateDescriptor stateDes
@Override
@Nonnull
public StateExecutor createStateExecutor() {
-// TODO: Make io parallelism configurable
-return new ForStStateExecutor(4, db,
optionsContainer.getWriteOptions());
+synchronized (lock) {
Review Comment:
Thanks for the update.
It makes sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed [flink]
ljz2051 commented on code in PR #24768:
URL: https://github.com/apache/flink/pull/24768#discussion_r1604803348
##
flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/ForStKeyedStateBackend.java:
##
@@ -147,15 +160,30 @@ public S createState(@Nonnull
StateDescriptor stateDes
@Override
@Nonnull
public StateExecutor createStateExecutor() {
-// TODO: Make io parallelism configurable
-return new ForStStateExecutor(4, db,
optionsContainer.getWriteOptions());
+synchronized (lock) {
Review Comment:
1. I think `StateExecutor#shutdown` should be invoked **exactly once**. (The
`CloseableRegistry` in `RocksDBKeyedStateBackend` has the similar semantics).
2. As for the semantics of `close` and `dispose`, StateBackend should call
`close` method to close some dynamic threads and input/output streams first,
then it can dispose some resources.
So I think the more reasonable way is to shutdown the stateExecutors in
`close` method, and force to close the `ForStKeyedStateBackend` in `dispose`
method if the `ForStKeyedStateBackend` has not been closed yet.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed [flink]
masteryhx commented on code in PR #24768:
URL: https://github.com/apache/flink/pull/24768#discussion_r1604579199
##
flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/ForStKeyedStateBackend.java:
##
@@ -147,15 +160,30 @@ public S createState(@Nonnull
StateDescriptor stateDes
@Override
@Nonnull
public StateExecutor createStateExecutor() {
-// TODO: Make io parallelism configurable
-return new ForStStateExecutor(4, db,
optionsContainer.getWriteOptions());
+synchronized (lock) {
Review Comment:
I mean that it should be invoked in both `close` and `dispose` method which
is similar to `RocksDBKeyedStateBackend`.
The semantic of `close` and `dispose` described above is also from it.
Otherwise the `close` method seems not so necessary.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-35047][state] Shutdown StateExecutors when ForStKeyedStateBackend is closed [flink]
ljz2051 commented on PR #24768: URL: https://github.com/apache/flink/pull/24768#issuecomment-2116858619 > I think we'd better note the `shutdown` contract down into the description/Javadoc of `StateExecutor`. The `StateExecutor` is managed by `KeyedStateBackend` and the `AEC` or operators won't take care of the lifecycle of it. @Zakelly Thanks for your suggestion. I have refined the JavaDoc in `AsyncKeyedStateBackend#createStateExecutor` and `StateExecutor`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
