[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443752#comment-16443752 ] Sihua Zhou commented on FLINK-9022: --- [~srichter] Nice catch! > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Stefan Richter >Priority: Blocker > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407697#comment-16407697 ] ASF GitHub Bot commented on FLINK-9022: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5716 > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Sihua Zhou >Priority: Blocker > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407387#comment-16407387 ] ASF GitHub Bot commented on FLINK-9022: --- Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/5716 Strange...the Travis failed and the failed is related to `YarnClusterDescriptorTest`, but it seems not related to this PR... > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Sihua Zhou >Priority: Blocker > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406416#comment-16406416 ] ASF GitHub Bot commented on FLINK-9022: --- Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/5716 Hi @tillrohrmann I think maybe stefan is too busy with the testing work for 1.5 currently, could you please have a look at this? This is obvious bug that need to be addressed. > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Sihua Zhou >Priority: Blocker > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404683#comment-16404683 ] ASF GitHub Bot commented on FLINK-9022: --- Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/5716 @StefanRRichter Could you please have a look at this? I think this should be fixed for 1.5. > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Sihua Zhou >Priority: Major > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404681#comment-16404681 ] ASF GitHub Bot commented on FLINK-9022: --- GitHub user sihuazhou opened a pull request: https://github.com/apache/flink/pull/5716 [FLINK-9022][state] fix resource release in StreamTaskStateInitializerImpl.streamOperatorStateContext() ## What is the purpose of the change This PR fixes the bug in `StreamTaskStateInitializerImpl.streamOperatorStateContext()` related to resource releasing. ## Brief change log - fix resource release in StreamTaskStateInitializerImpl.streamOperatorStateContext() ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) You can merge this pull request into a Git repository by running: $ git pull https://github.com/sihuazhou/flink fix_resource_close_in_StreamTaskStateInitializerImpl Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5716.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5716 commit e5591005f8470f24bae30670c2c9094d13b9f35c Author: sihuazhouDate: 2018-03-19T11:48:32Z fix resource release in StreamTaskStateInitializerImpl.streamOperatorStateContext() > fix resource close in > `StreamTaskStateInitializerImpl.streamOperatorStateContext()` > --- > > Key: FLINK-9022 > URL: https://issues.apache.org/jira/browse/FLINK-9022 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing >Affects Versions: 1.5.0 >Reporter: Sihua Zhou >Assignee: Sihua Zhou >Priority: Major > Fix For: 1.5.0 > > > We have the following code in > {{StreamTaskStateInitializerImpl.streamOperatorStateContext()}} which is > incorrect: > {code} > } catch (Exception ex) { > // cleanup if something went wrong before results got published. > if > (streamTaskCloseableRegistry.unregisterCloseable(keyedStatedBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(operatorStateBackend)) { > IOUtils.closeQuietly(keyedStatedBackend); // this should close > operatorStateBackend > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawKeyedStateInputs)) { > IOUtils.closeQuietly(rawKeyedStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > if > (streamTaskCloseableRegistry.unregisterCloseable(rawOperatorStateInputs)) { > IOUtils.closeQuietly(rawOperatorStateInputs); > } > throw new Exception("Exception while creating > StreamOperatorStateContext.", ex); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)