[jira] [Closed] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

2018-04-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8699. - Resolution: Fixed Release Note: (was: Richter has fixed this in his hotfix.) > Fix concurrency pro

[jira] [Reopened] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

2018-04-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reopened FLINK-8699: --- Reopen to fix the release note(The mistake that I made a long time ago...) according to Aljoscha's commen

[jira] [Closed] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-04-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9102. - Resolution: Invalid Release Note: (was: Impossible for flip6) > Make the JobGraph disable queued s

[jira] [Reopened] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-04-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reopened FLINK-9102: --- reopen for fixing release note. > Make the JobGraph disable queued scheduling for cluster with fixed TMs

[jira] [Commented] (FLINK-9055) WebUI shows job as Running although not enough resources are available

2018-04-05 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427032#comment-16427032 ] Sihua Zhou commented on FLINK-9055: --- Hi [~fhueske], thanks for your reply, I was thinkin

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-04 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425258#comment-16425258 ] Sihua Zhou commented on FLINK-9120: --- Hi [~dhirajpraj] I think maybe we should leave this

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424091#comment-16424091 ] Sihua Zhou commented on FLINK-9120: --- Hi [~dhirajpraj] sorry for the delay reply, I was b

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423739#comment-16423739 ] Sihua Zhou commented on FLINK-9120: --- Hi [~dhirajpraj] happy to know that it working fine

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423662#comment-16423662 ] Sihua Zhou commented on FLINK-9120: --- Hmm...It looks like the JobManager still try to dep

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423614#comment-16423614 ] Sihua Zhou commented on FLINK-9120: --- Hi [~dhirajpraj] I found some exceptions in your up

[jira] [Commented] (FLINK-9120) Task Manager Fault Tolerance issue

2018-04-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423593#comment-16423593 ] Sihua Zhou commented on FLINK-9120: --- Hi [~dhirajpraj] from the log I found that the oper

[jira] [Assigned] (FLINK-8205) Multi key get

2018-04-02 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-8205: - Assignee: Sihua Zhou > Multi key get > - > > Key: FLINK-8205 >

[jira] [Commented] (FLINK-8205) Multi key get

2018-04-02 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423536#comment-16423536 ] Sihua Zhou commented on FLINK-8205: --- Ah, no response for a few days, I'm taking this tic

[jira] [Assigned] (FLINK-9116) Introduce getAll and removeAll for MapState

2018-03-30 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9116: - Assignee: Sihua Zhou > Introduce getAll and removeAll for MapState >

[jira] [Created] (FLINK-9116) Introduce getAll and removeAll for MapState

2018-03-30 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-9116: - Summary: Introduce getAll and removeAll for MapState Key: FLINK-9116 URL: https://issues.apache.org/jira/browse/FLINK-9116 Project: Flink Issue Type: New Feature

[jira] [Commented] (FLINK-8205) Multi key get

2018-03-29 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420071#comment-16420071 ] Sihua Zhou commented on FLINK-8205: --- Hi guys, this issue seems to be inactive for some t

[jira] [Commented] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2018-03-29 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419188#comment-16419188 ] Sihua Zhou commented on FLINK-7219: --- [~aljoscha] Got it and sorry... > Current allocate

[jira] [Closed] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7219. - Resolution: Fixed Release Note: fixed by till in 1.4 > Current allocate strategy cann‘t achieve the

[jira] [Commented] (FLINK-9081) ResourceManagerTaskExecutorTest is unstable

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417720#comment-16417720 ] Sihua Zhou commented on FLINK-9081: --- This can be reproduced by insert a sleep: {code}

[jira] [Assigned] (FLINK-9081) ResourceManagerTaskExecutorTest is unstable

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9081: - Assignee: Sihua Zhou > ResourceManagerTaskExecutorTest is unstable >

[jira] [Commented] (FLINK-9070) Improve performance of RocksDBMapState.clear()

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417588#comment-16417588 ] Sihua Zhou commented on FLINK-9070: --- Hi [~kien_truong] Thanks for sharing the code and n

[jira] [Closed] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9102. - Resolution: Invalid > Make the JobGraph disable queued scheduling for cluster with fixed TMs > ---

[jira] [Reopened] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reopened FLINK-9102: --- > Make the JobGraph disable queued scheduling for cluster with fixed TMs > ---

[jira] [Closed] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9102. - Resolution: Fixed Release Note: Impossible for flip6 > Make the JobGraph disable queued scheduling f

[jira] [Updated] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9102: -- Description: When we start cluster locally with fixed TMS and we should disable queued scheduling for Jo

[jira] [Updated] (FLINK-9102) Make the JobGraph disable queued scheduling for cluster with fixed TMs

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9102: -- Summary: Make the JobGraph disable queued scheduling for cluster with fixed TMs (was: Make the JobGraph

[jira] [Created] (FLINK-9102) Make the JobGraph disable queued scheduling for Flip6LocalStreamEnvironment

2018-03-28 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-9102: - Summary: Make the JobGraph disable queued scheduling for Flip6LocalStreamEnvironment Key: FLINK-9102 URL: https://issues.apache.org/jira/browse/FLINK-9102 Project: Flink

[jira] [Commented] (FLINK-9055) WebUI shows job as Running although not enough resources are available

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417248#comment-16417248 ] Sihua Zhou commented on FLINK-9055: --- Hi [~fhueske] shall we fail the job immediately whe

[jira] [Assigned] (FLINK-9040) JobVertex#setMaxParallelism does not validate argument

2018-03-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9040: - Assignee: (was: Sihua Zhou) > JobVertex#setMaxParallelism does not validate argument > --

[jira] [Assigned] (FLINK-9082) Submission with higher parallelism than task slots fails with TimeoutException

2018-03-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9082: - Assignee: Sihua Zhou > Submission with higher parallelism than task slots fails with TimeoutExcep

[jira] [Updated] (FLINK-8968) Fix native resource leak caused by ReadOptions

2018-03-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8968: -- Priority: Blocker (was: Major) > Fix native resource leak caused by ReadOptions >

[jira] [Assigned] (FLINK-9087) Return value of broadcastEvent should be closed in StreamTask#performCheckpoint

2018-03-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9087: - Assignee: (was: Sihua Zhou) > Return value of broadcastEvent should be closed in > StreamTas

[jira] [Assigned] (FLINK-9087) Return value of broadcastEvent should be closed in StreamTask#performCheckpoint

2018-03-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9087: - Assignee: Sihua Zhou > Return value of broadcastEvent should be closed in > StreamTask#performCh

[jira] [Commented] (FLINK-9070) Improve performance of RocksDBMapState.clear()

2018-03-25 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413431#comment-16413431 ] Sihua Zhou commented on FLINK-9070: --- Hi [~kien_truong], this is interesting. I agree wit

[jira] [Commented] (FLINK-9060) Deleting state using KeyedStateBackend.getKeys() throws Exception

2018-03-23 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411238#comment-16411238 ] Sihua Zhou commented on FLINK-9060: --- [~kkl0u] Thanks for pointing out this! After readin

[jira] [Commented] (FLINK-9058) Relax ListState.addAll() and ListState.update() to take Iterable

2018-03-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410823#comment-16410823 ] Sihua Zhou commented on FLINK-9058: --- Shall we also relax the {{MapState.putAll()}} to ta

[jira] [Commented] (FLINK-9060) Deleting state using KeyedStateBackend.getKeys() throws Exception

2018-03-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410669#comment-16410669 ] Sihua Zhou commented on FLINK-9060: --- 1. For {{MemoryStateBackendTest}} this is because a

[jira] [Assigned] (FLINK-9060) Deleting state using KeyedStateBackend.getKeys() throws Exception

2018-03-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9060: - Assignee: Sihua Zhou > Deleting state using KeyedStateBackend.getKeys() throws Exception > --

[jira] [Assigned] (FLINK-9055) WebUI shows job as Running although not enough resources are available

2018-03-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9055: - Assignee: Sihua Zhou > WebUI shows job as Running although not enough resources are available > -

[jira] [Commented] (FLINK-9047) SlotPool can fail to release slots

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408249#comment-16408249 ] Sihua Zhou commented on FLINK-9047: --- Hi [~till.rohrmann] have you already work on this?

[jira] [Commented] (FLINK-9040) JobVertex#setMaxParallelism does not validate argument

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407920#comment-16407920 ] Sihua Zhou commented on FLINK-9040: --- Hi [~Zentol] there's one thing I want to confirm wi

[jira] [Resolved] (FLINK-9018) Unclosed snapshotCloseableRegistry in RocksDBKeyedStateBackend#FullSnapshotStrategy#performSnapshot

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou resolved FLINK-9018. --- Resolution: Fixed > Unclosed snapshotCloseableRegistry in > RocksDBKeyedStateBackend#FullSnapshotStra

[jira] [Reopened] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reopened FLINK-8699: --- Reopen because the problem still exists. > Fix concurrency problem in rocksdb full checkpoint > -

[jira] [Assigned] (FLINK-9041) Refactor StreamTaskTest to not use scala and akka

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9041: - Assignee: Sihua Zhou > Refactor StreamTaskTest to not use scala and akka > --

[jira] [Assigned] (FLINK-9040) JobVertex#setMaxParallelism does not valid argument

2018-03-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9040: - Assignee: Sihua Zhou > JobVertex#setMaxParallelism does not valid argument >

[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it

2018-03-20 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407403#comment-16407403 ] Sihua Zhou commented on FLINK-9026: --- Hi [~till.rohrmann] I'm a bit confused about "We sh

[jira] [Assigned] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it

2018-03-20 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9026: - Assignee: Sihua Zhou > Unregister finished tasks from TaskManagerMetricGroup and close it > -

[jira] [Created] (FLINK-9028) flip6 should check config before starting cluster

2018-03-20 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-9028: - Summary: flip6 should check config before starting cluster Key: FLINK-9028 URL: https://issues.apache.org/jira/browse/FLINK-9028 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-9018) Unclosed snapshotCloseableRegistry in RocksDBKeyedStateBackend#FullSnapshotStrategy#performSnapshot

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405781#comment-16405781 ] Sihua Zhou commented on FLINK-9018: --- Since this is a minor change, I covered it in [571

[jira] [Assigned] (FLINK-9018) Unclosed snapshotCloseableRegistry in RocksDBKeyedStateBackend#FullSnapshotStrategy#performSnapshot

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9018: - Assignee: Sihua Zhou > Unclosed snapshotCloseableRegistry in > RocksDBKeyedStateBackend#FullSnap

[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405130#comment-16405130 ] Sihua Zhou commented on FLINK-9026: --- Hi [~till.rohrmann] have you already work on this?

[jira] [Commented] (FLINK-9019) Unclosed closeableRegistry in StreamTaskStateInitializerImpl#rawOperatorStateInputs

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405049#comment-16405049 ] Sihua Zhou commented on FLINK-9019: --- I think we can't use try-with-resources here, cause

[jira] [Updated] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9022: -- Priority: Blocker (was: Major) > fix resource close in > `StreamTaskStateInitializerImpl.streamOperato

[jira] [Created] (FLINK-9022) fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()`

2018-03-19 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-9022: - Summary: fix resource close in `StreamTaskStateInitializerImpl.streamOperatorStateContext()` Key: FLINK-9022 URL: https://issues.apache.org/jira/browse/FLINK-9022 Project:

[jira] [Commented] (FLINK-9019) Unclosed closeableRegistry in StreamTaskStateInitializerImpl#rawOperatorStateInputs

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404580#comment-16404580 ] Sihua Zhou commented on FLINK-9019: --- Hi [~yuzhih...@gmail.com], why this should be a bug

[jira] [Commented] (FLINK-9018) Unclosed snapshotCloseableRegistry in RocksDBKeyedStateBackend#FullSnapshotStrategy#performSnapshot

2018-03-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404577#comment-16404577 ] Sihua Zhou commented on FLINK-9018: --- Hi [~yuzhih...@gmail.com], why this should be a bug

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403893#comment-16403893 ] Sihua Zhou commented on FLINK-8922: --- ...Sorry to tell you guys that [~srichter] was righ

[jira] [Commented] (FLINK-8976) End-to-end test: Resume with different parallelism

2018-03-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402023#comment-16402023 ] Sihua Zhou commented on FLINK-8976: --- Hi [~till.rohrmann] could I ask why this issue won'

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401945#comment-16401945 ] Sihua Zhou commented on FLINK-8922: --- [~srichter] Yes, the downside is obvious and I also

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401925#comment-16401925 ] Sihua Zhou commented on FLINK-8922: --- Ah, happy to tell you guys that finally I find the

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401522#comment-16401522 ] Sihua Zhou commented on FLINK-8922: --- [~StephanEwen] I tried out your suggestion, still s

[jira] [Commented] (FLINK-8969) Move TimerService into state backend

2018-03-15 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401485#comment-16401485 ] Sihua Zhou commented on FLINK-8969: --- Hi [~phoenixjiangnan], Where is the previous discus

[jira] [Created] (FLINK-8968) Fix native resource leak caused by ReadOptions

2018-03-15 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8968: - Summary: Fix native resource leak caused by ReadOptions Key: FLINK-8968 URL: https://issues.apache.org/jira/browse/FLINK-8968 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-15 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400595#comment-16400595 ] Sihua Zhou commented on FLINK-8922: --- Hmm... I will try it out, I'm looping the code of

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396836#comment-16396836 ] Sihua Zhou commented on FLINK-8922: --- After having some tried (bumping {{rocksdbjni}} to

[jira] [Created] (FLINK-8927) Eagerly release the checkpoint object created from RocksDB

2018-03-12 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8927: - Summary: Eagerly release the checkpoint object created from RocksDB Key: FLINK-8927 URL: https://issues.apache.org/jira/browse/FLINK-8927 Project: Flink Issue Type

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395214#comment-16395214 ] Sihua Zhou commented on FLINK-8922: --- Agreed with [~srichter], and the statistics I poste

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395198#comment-16395198 ] Sihua Zhou commented on FLINK-8922: --- [~StephanEwen] About the costs with (or without) WA

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395128#comment-16395128 ] Sihua Zhou commented on FLINK-8922: --- [~StephanEwen] I am not sure whether the failed tes

[jira] [Commented] (FLINK-8922) Revert FLINK-8859 because it causes segfaults in testing

2018-03-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395050#comment-16395050 ] Sihua Zhou commented on FLINK-8922: --- [~srichter] Thanks for let me know, I will have a d

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

2018-03-11 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394817#comment-16394817 ] Sihua Zhou commented on FLINK-8918: --- Hi [~fhueske] could you please have a look at this

[jira] [Created] (FLINK-8918) Introduce Runtime Filter Join

2018-03-11 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8918: - Summary: Introduce Runtime Filter Join Key: FLINK-8918 URL: https://issues.apache.org/jira/browse/FLINK-8918 Project: Flink Issue Type: Bug Components: T

[jira] [Commented] (FLINK-4811) Checkpoint Overview should list failed checkpoints

2018-03-07 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390689#comment-16390689 ] Sihua Zhou commented on FLINK-4811: --- This jira's state is still Unresolved, I think this

[jira] [Updated] (FLINK-8790) Improve performance for recovery from incremental checkpoint

2018-03-07 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8790: -- Fix Version/s: (was: 1.5.0) 1.6.0 > Improve performance for recovery from increme

[jira] [Closed] (FLINK-8846) Introduce `parallel recovery` mode for incremental checkpoint

2018-03-07 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8846. - Resolution: Staged Fix Version/s: (was: 1.6.0) Not the right time to do this, we should wait un

[jira] [Updated] (FLINK-8845) Use WriteBatch to improve performance for recovery in RocksDB backend

2018-03-07 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8845: -- Description: Base on {{WriteBatch}} we could get 30% ~ 50% performance lift when loading data into Rocks

[jira] [Updated] (FLINK-8845) Use WriteBatch to improve performance for recovery in RocksDB backend

2018-03-07 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8845: -- Summary: Use WriteBatch to improve performance for recovery in RocksDB backend (was: Introduce `parall

[jira] [Commented] (FLINK-8845) Introduce `parallel recovery` mode for full checkpoint (savepoint)

2018-03-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389179#comment-16389179 ] Sihua Zhou commented on FLINK-8845: --- Event though, {{SstFileWriter}} could not help us t

[jira] [Commented] (FLINK-8845) Introduce `parallel recovery` mode for full checkpoint (savepoint)

2018-03-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389123#comment-16389123 ] Sihua Zhou commented on FLINK-8845: --- Unfortunately, even though according to RocksDB [w

[jira] [Commented] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

2018-03-06 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387820#comment-16387820 ] Sihua Zhou commented on FLINK-8871: --- [~srichter] That is fine, looking forward the PR. ;

[jira] [Commented] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

2018-03-05 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387335#comment-16387335 ] Sihua Zhou commented on FLINK-8871: --- [~srichter] Have you already work on this or decide

[jira] [Created] (FLINK-8859) RocksDB backend should pass WriteOption to Rocks.put() when restoring

2018-03-05 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8859: - Summary: RocksDB backend should pass WriteOption to Rocks.put() when restoring Key: FLINK-8859 URL: https://issues.apache.org/jira/browse/FLINK-8859 Project: Flink

[jira] [Updated] (FLINK-8845) Introduce `parallel recovery` mode for full checkpoint (savepoint)

2018-03-05 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8845: -- Summary: Introduce `parallel recovery` mode for full checkpoint (savepoint) (was: Introduce `parallel

[jira] [Updated] (FLINK-8845) Introduce `parallel recovery` mode for fully checkpoint (savepoint)

2018-03-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8845: -- Summary: Introduce `parallel recovery` mode for fully checkpoint (savepoint) (was: Introducing `paral

[jira] [Updated] (FLINK-8846) Introduce `parallel recovery` mode for incremental checkpoint

2018-03-03 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8846: -- Summary: Introduce `parallel recovery` mode for incremental checkpoint (was: ntroducing `parallel recov

[jira] [Created] (FLINK-8846) ntroducing `parallel recovery` mode for incremental checkpoint

2018-03-03 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8846: - Summary: ntroducing `parallel recovery` mode for incremental checkpoint Key: FLINK-8846 URL: https://issues.apache.org/jira/browse/FLINK-8846 Project: Flink Issue

[jira] [Created] (FLINK-8845) Introducing `parallel recovery` mode for fully checkpoint (savepoint)

2018-03-03 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8845: - Summary: Introducing `parallel recovery` mode for fully checkpoint (savepoint) Key: FLINK-8845 URL: https://issues.apache.org/jira/browse/FLINK-8845 Project: Flink

[jira] [Closed] (FLINK-8816) Remove the oldWorker only after starting newWorker successfully in registerTaskExecutorInternal()

2018-03-01 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8816. - Resolution: Won't Fix Release Note: The bug case cannot happen in flink. > Remove the oldWorker only

[jira] [Closed] (FLINK-8817) Decrement numPendingContainerRequests only when request container successfully

2018-03-01 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8817. - Resolution: Won't Fix > Decrement numPendingContainerRequests only when request container successfully > -

[jira] [Created] (FLINK-8817) Decrement numPendingContainerRequests only when request container successfully

2018-02-28 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8817: - Summary: Decrement numPendingContainerRequests only when request container successfully Key: FLINK-8817 URL: https://issues.apache.org/jira/browse/FLINK-8817 Project: Flink

[jira] [Commented] (FLINK-8753) Introduce savepoint that go though the incremental checkpoint path

2018-02-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381476#comment-16381476 ] Sihua Zhou commented on FLINK-8753: --- Sorry for the interruption, but after have a look a

[jira] [Created] (FLINK-8816) Remove the oldWorker only after starting newWorker successfully in registerTaskExecutorInternal()

2018-02-28 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8816: - Summary: Remove the oldWorker only after starting newWorker successfully in registerTaskExecutorInternal() Key: FLINK-8816 URL: https://issues.apache.org/jira/browse/FLINK-8816

[jira] [Commented] (FLINK-7866) Weigh list of preferred locations for scheduling

2018-02-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380288#comment-16380288 ] Sihua Zhou commented on FLINK-7866: --- [~till.rohrmann] Got it, since the branch release 1

[jira] [Closed] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2018-02-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7873. - Resolution: Duplicate Release Note: fixed by stefan. > Introduce CheckpointCacheManager for reading

[jira] [Closed] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2018-02-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8044. - Resolution: Duplicate Release Note: Already fixed by Stefan Richter. > Introduce scheduling mechanis

[jira] [Updated] (FLINK-8753) Introduce savepoint that go though the incremental checkpoint path

2018-02-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8753: -- Summary: Introduce savepoint that go though the incremental checkpoint path (was: Introduce Incremental

[jira] [Comment Edited] (FLINK-8753) Introduce Incremental savepoint

2018-02-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377908#comment-16377908 ] Sihua Zhou edited comment on FLINK-8753 at 2/27/18 2:26 AM: [~

[jira] [Commented] (FLINK-8753) Introduce Incremental savepoint

2018-02-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377908#comment-16377908 ] Sihua Zhou commented on FLINK-8753: --- [~StephanEwen] Thanks for your reply. Indeed, what

[jira] [Created] (FLINK-8790) Improve performance for recovery from incremental checkpoint

2018-02-26 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8790: - Summary: Improve performance for recovery from incremental checkpoint Key: FLINK-8790 URL: https://issues.apache.org/jira/browse/FLINK-8790 Project: Flink Issue T

[jira] [Created] (FLINK-8777) improve resource release when recovery from failover

2018-02-26 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8777: - Summary: improve resource release when recovery from failover Key: FLINK-8777 URL: https://issues.apache.org/jira/browse/FLINK-8777 Project: Flink Issue Type: Impr

<    1   2   3   4   5   >