[jira] [Commented] (FLINK-9364) Add doc of the memory usage in flink

[ https://issues.apache.org/jira/browse/FLINK-9364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571015#comment-16571015 ] Sihua Zhou commented on FLINK-9364: --- Hi [~till.rohrmann] I was catch up by some terrible work, so I

[jira] [Updated] (FLINK-9364) Add doc of the memory usage in flink

[ https://issues.apache.org/jira/browse/FLINK-9364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9364: -- Fix Version/s: (was: 1.6.0) 1.7.0 > Add doc of the memory usage in flink >

[jira] [Closed] (FLINK-9804) KeyedStateBackend.getKeys() does not work on RocksDB MapState

[ https://issues.apache.org/jira/browse/FLINK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9804. - Resolution: Fixed Merged in: 1.6.0: def2aed5c75b5a00815186d3343e66cb1dc01ac0 1.5.2:

[jira] [Commented] (FLINK-9804) KeyedStateBackend.getKeys() does not work on RocksDB MapState

[ https://issues.apache.org/jira/browse/FLINK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540077#comment-16540077 ] Sihua Zhou commented on FLINK-9804: --- [~aljoscha] Yes, I'm writing the PR description... >

[jira] [Assigned] (FLINK-9804) KeyedStateBackend.getKeys() does not work on RocksDB MapState

[ https://issues.apache.org/jira/browse/FLINK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9804: - Assignee: Sihua Zhou (was: vinoyang) > KeyedStateBackend.getKeys() does not work on RocksDB

[jira] [Commented] (FLINK-9804) KeyedStateBackend.getKeys() does not work on RocksDB MapState

[ https://issues.apache.org/jira/browse/FLINK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540034#comment-16540034 ] Sihua Zhou commented on FLINK-9804: --- Hi [~yanghua] do you start working on this already? I just

[jira] [Commented] (FLINK-9736) Potential null reference in KeyGroupPartitionedPriorityQueue#poll()

[ https://issues.apache.org/jira/browse/FLINK-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533506#comment-16533506 ] Sihua Zhou commented on FLINK-9736: --- Hi [~yuzhih...@gmail.com] AFAIK, {{heapOfKeyGroupHeaps}} will

[jira] [Commented] (FLINK-9735) Potential resource leak in RocksDBStateBackend#getDbOptions

[ https://issues.apache.org/jira/browse/FLINK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532982#comment-16532982 ] Sihua Zhou commented on FLINK-9735: --- I think this might be a trade-off of the design, - 1) let the use

[jira] [Commented] (FLINK-9735) Potential resource leak in RocksDBStateBackend#getDbOptions

[ https://issues.apache.org/jira/browse/FLINK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532956#comment-16532956 ] Sihua Zhou commented on FLINK-9735: --- Oh [~yuzhih...@gmail.com] you are right! The case in

[jira] [Commented] (FLINK-9735) Potential resource leak in RocksDBStateBackend#getDbOptions

[ https://issues.apache.org/jira/browse/FLINK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532728#comment-16532728 ] Sihua Zhou commented on FLINK-9735: --- Hi [~yuzhih...@gmail.com], I think this is a non issue, because

[jira] [Commented] (FLINK-9736) Potential null reference in KeyGroupPartitionedPriorityQueue#poll()

[ https://issues.apache.org/jira/browse/FLINK-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532726#comment-16532726 ] Sihua Zhou commented on FLINK-9736: --- Hi [~yuzhih...@gmail.com] I think this might look like a non

[jira] [Closed] (FLINK-9351) RM stop assigning slot to Job because the TM killed before connecting to JM successfully

[ https://issues.apache.org/jira/browse/FLINK-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9351. - Resolution: Duplicate This issue have been fixed by the way in the PR of

[jira] [Updated] (FLINK-9474) Add runtime support of distinct filter using ElasticBloomFilter

[ https://issues.apache.org/jira/browse/FLINK-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9474: -- Description: We can implement an approximate version of "distinct filter" base on the "Elastic Bloom

[jira] [Updated] (FLINK-9474) Add runtime support of distinct filter using ElasticBloomFilter

[ https://issues.apache.org/jira/browse/FLINK-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9474: -- Summary: Add runtime support of distinct filter using ElasticBloomFilter (was: Introduce an

[jira] [Updated] (FLINK-9619) Always close the task manager connection when the container is completed in YarnResourceManager

[ https://issues.apache.org/jira/browse/FLINK-9619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9619: -- Issue Type: Improvement (was: Bug) > Always close the task manager connection when the container is

[jira] [Updated] (FLINK-9619) Always close the task manager connection when the container is completed in YarnResourceManager

[ https://issues.apache.org/jira/browse/FLINK-9619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9619: -- Priority: Major (was: Critical) > Always close the task manager connection when the container is

[jira] [Created] (FLINK-9661) TTL state should support to do time shift after restoring from checkpoint( savepoint).

Sihua Zhou created FLINK-9661: - Summary: TTL state should support to do time shift after restoring from checkpoint( savepoint). Key: FLINK-9661 URL: https://issues.apache.org/jira/browse/FLINK-9661

[jira] [Commented] (FLINK-9585) Logger in ZooKeeperStateHandleStore is public and non-final

[ https://issues.apache.org/jira/browse/FLINK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522981#comment-16522981 ] Sihua Zhou commented on FLINK-9585: --- [~Zentol] thanks for pointing out this issue for me, I got it. >

[jira] [Closed] (FLINK-9585) Logger in ZooKeeperStateHandleStore is public and non-final

[ https://issues.apache.org/jira/browse/FLINK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9585. - Resolution: Fixed Merged in: master: 5fa61d8ceac8f865002dc0ef84dc1a3c65753d0b > Logger in

[jira] [Created] (FLINK-9633) Flink doesn't use the Savepoint path's filesystem to create the OuptutStream on Task.

Sihua Zhou created FLINK-9633: - Summary: Flink doesn't use the Savepoint path's filesystem to create the OuptutStream on Task. Key: FLINK-9633 URL: https://issues.apache.org/jira/browse/FLINK-9633

[jira] [Commented] (FLINK-9622) DistributedCacheDfsTest failed on travis

[ https://issues.apache.org/jira/browse/FLINK-9622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517878#comment-16517878 ] Sihua Zhou commented on FLINK-9622: --- one more instances:

[jira] [Commented] (FLINK-9622) DistributedCacheDfsTest failed on travis

[ https://issues.apache.org/jira/browse/FLINK-9622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517874#comment-16517874 ] Sihua Zhou commented on FLINK-9622: --- cc [~dawidwys] > DistributedCacheDfsTest failed on travis >

[jira] [Created] (FLINK-9622) DistributedCacheDfsTest failed on travis

Sihua Zhou created FLINK-9622: - Summary: DistributedCacheDfsTest failed on travis Key: FLINK-9622 URL: https://issues.apache.org/jira/browse/FLINK-9622 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-9619) Always close the task manager connection when the container is completed in YarnResourceManager

Sihua Zhou created FLINK-9619: - Summary: Always close the task manager connection when the container is completed in YarnResourceManager Key: FLINK-9619 URL: https://issues.apache.org/jira/browse/FLINK-9619

[jira] [Commented] (FLINK-9417) Send heartbeat requests from RPC endpoint's main thread

[ https://issues.apache.org/jira/browse/FLINK-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517790#comment-16517790 ] Sihua Zhou commented on FLINK-9417: --- Hi [~till.rohrmann] One thing come to my mind, If we send

[jira] [Created] (FLINK-9613) YARNSessionCapacitySchedulerITCase failed because YarnTestBase.checkClusterEmpty()

Sihua Zhou created FLINK-9613: - Summary: YARNSessionCapacitySchedulerITCase failed because YarnTestBase.checkClusterEmpty() Key: FLINK-9613 URL: https://issues.apache.org/jira/browse/FLINK-9613 Project:

[jira] [Updated] (FLINK-9601) Snapshot of CopyOnWriteStateTable will failed when the amount of record is more than MAXIMUM_CAPACITY

[ https://issues.apache.org/jira/browse/FLINK-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9601: -- Summary: Snapshot of CopyOnWriteStateTable will failed when the amount of record is more than

[jira] [Created] (FLINK-9601) Snapshot of CopyOnWriteStateTable will failed, when the amount of record is more than MAXIMUM_CAPACITY

Sihua Zhou created FLINK-9601: - Summary: Snapshot of CopyOnWriteStateTable will failed, when the amount of record is more than MAXIMUM_CAPACITY Key: FLINK-9601 URL: https://issues.apache.org/jira/browse/FLINK-9601

[jira] [Assigned] (FLINK-9455) Make SlotManager aware of multi slot TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9455: - Assignee: Till Rohrmann (was: Sihua Zhou) > Make SlotManager aware of multi slot TaskManagers

[jira] [Commented] (FLINK-9455) Make SlotManager aware of multi slot TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513336#comment-16513336 ] Sihua Zhou commented on FLINK-9455: --- Hi [~till.rohrmann] the more I thought about this , the more I

[jira] [Assigned] (FLINK-9584) Unclosed streams in Bucketing-/RollingSink

[ https://issues.apache.org/jira/browse/FLINK-9584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9584: - Assignee: Sihua Zhou > Unclosed streams in Bucketing-/RollingSink >

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510828#comment-16510828 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow], I didn't see the email you sent yet, but I just had a

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510757#comment-16510757 ] Sihua Zhou commented on FLINK-9506: --- Hi, I think you could send your question to the

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510750#comment-16510750 ] Sihua Zhou commented on FLINK-9506: --- I'm + 1 to close this ticket and move to ML, we can definitely

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510716#comment-16510716 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] "If we further comment out recordStore.add() then

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510649#comment-16510649 ] Sihua Zhou commented on FLINK-9506: --- Additional, do you make sure that you are using the RocksDb

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510647#comment-16510647 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] in the input_stop_when_timer_run.png, does the yellow

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510580#comment-16510580 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] I don't think this a limitation in Flink, we have more

[jira] [Comment Edited] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509818#comment-16509818 ] Sihua Zhou edited comment on FLINK-9506 at 6/12/18 4:34 PM: Hi [~yow] What do

[jira] [Comment Edited] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509818#comment-16509818 ] Sihua Zhou edited comment on FLINK-9506 at 6/12/18 4:26 PM: Hi [~yow] What do

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509818#comment-16509818 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] What do you means by the empty ProcessAggregation? Did

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507815#comment-16507815 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] I think we can close this ticket now, do you agree?

[jira] [Commented] (FLINK-9561) Lack of the api to set RocksDB Option by flink config

[ https://issues.apache.org/jira/browse/FLINK-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507249#comment-16507249 ] Sihua Zhou commented on FLINK-9561: --- Hi [~aitozi] I think flink already supported this, you can

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504729#comment-16504729 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] From the top of my head, I list answers here: - >> 1.

[jira] [Comment Edited] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503505#comment-16503505 ] Sihua Zhou edited comment on FLINK-9506 at 6/7/18 9:25 AM: --- [~yow] Maybe there

[jira] [Created] (FLINK-9546) The heartbeatTimeoutIntervalMs of HeartbeatMonitor should be larger than 0

Sihua Zhou created FLINK-9546: - Summary: The heartbeatTimeoutIntervalMs of HeartbeatMonitor should be larger than 0 Key: FLINK-9546 URL: https://issues.apache.org/jira/browse/FLINK-9546 Project: Flink

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503505#comment-16503505 ] Sihua Zhou commented on FLINK-9506: --- [~yow] Maybe there is one more optimization that could have a try,

[jira] [Commented] (FLINK-9455) Make SlotManager aware of multi slot TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502935#comment-16502935 ] Sihua Zhou commented on FLINK-9455: --- [~till.rohrmann] Thanks for your reply, I agreed. I'll give a

[jira] [Commented] (FLINK-8845) Use WriteBatch to improve performance for recovery in RocksDB backend

[ https://issues.apache.org/jira/browse/FLINK-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502809#comment-16502809 ] Sihua Zhou commented on FLINK-8845: --- Hi [~noliran] Yes, this is on purpose. In fact, we

[jira] [Closed] (FLINK-8753) Introduce savepoint that go though the incremental checkpoint path

[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8753. - Resolution: Invalid > Introduce savepoint that go though the incremental checkpoint path >

[jira] [Commented] (FLINK-9455) Make SlotManager aware of multi slot TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502064#comment-16502064 ] Sihua Zhou commented on FLINK-9455: --- Hi [~till.rohrmann], I think this is a bit tricky than it looks

[jira] [Commented] (FLINK-8601) Introduce ElasticBloomFilter for Approximate calculation and other situations of performance optimization

[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502037#comment-16502037 ] Sihua Zhou commented on FLINK-8601: --- Hi [~aljoscha] [~StephanEwen] would be really appreciate if anyone

[jira] [Closed] (FLINK-8602) Improve recovery performance for rocksdb backend

[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8602. - Resolution: Done All sub-tasks are done. > Improve recovery performance for rocksdb backend >

[jira] [Commented] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically

[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501312#comment-16501312 ] Sihua Zhou commented on FLINK-9043: --- Okay, I see it is quite tricky to provide a completely automatic

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499873#comment-16499873 ] Sihua Zhou commented on FLINK-9506: --- [~yow] I think RocksDB backend could give a more stable

[jira] [Comment Edited] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499867#comment-16499867 ] Sihua Zhou edited comment on FLINK-9506 at 6/4/18 8:05 AM: --- Thanks for trying

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499867#comment-16499867 ] Sihua Zhou commented on FLINK-9506: --- Thanks for trying it out. Then I think the performance drop and

[jira] [Comment Edited] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499672#comment-16499672 ] Sihua Zhou edited comment on FLINK-9506 at 6/4/18 2:22 AM: --- Hi [~yow] could you

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499672#comment-16499672 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] could you please just replace the getKey() as follow and

[jira] [Commented] (FLINK-9506) Flink ReducingState.add causing more than 100% performance drop

[ https://issues.apache.org/jira/browse/FLINK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499395#comment-16499395 ] Sihua Zhou commented on FLINK-9506: --- Hi [~yow] could you please give some information of the `keyBy()`?

[jira] [Commented] (FLINK-9486) Introduce TimerState in keyed state backend

[ https://issues.apache.org/jira/browse/FLINK-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497956#comment-16497956 ] Sihua Zhou commented on FLINK-9486: --- [~srichter] Thanks for your reply, looking forward your PR! ;) >

[jira] [Commented] (FLINK-9486) Introduce TimerState in keyed state backend

[ https://issues.apache.org/jira/browse/FLINK-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497940#comment-16497940 ] Sihua Zhou commented on FLINK-9486: --- Hi [~srichter] Do you already work on this? If not, I'd like to

[jira] [Updated] (FLINK-9468) get outputLimit of LimitedConnectionsFileSystem incorrectly

[ https://issues.apache.org/jira/browse/FLINK-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9468: -- Priority: Critical (was: Blocker) > get outputLimit of LimitedConnectionsFileSystem incorrectly >

[jira] [Commented] (FLINK-9480) Let local recovery support rescaling

[ https://issues.apache.org/jira/browse/FLINK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494908#comment-16494908 ] Sihua Zhou commented on FLINK-9480: --- [~srichter] Thanks for your reply, the reason and the use case I

[jira] [Updated] (FLINK-9480) Let local recovery support rescaling

[ https://issues.apache.org/jira/browse/FLINK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9480: -- Affects Version/s: 1.5.0 > Let local recovery support rescaling >

[jira] [Updated] (FLINK-9480) Let local recovery support rescaling

[ https://issues.apache.org/jira/browse/FLINK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9480: -- Component/s: State Backends, Checkpointing > Let local recovery support rescaling >

[jira] [Commented] (FLINK-9480) Let local recovery support rescaling

[ https://issues.apache.org/jira/browse/FLINK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494880#comment-16494880 ] Sihua Zhou commented on FLINK-9480: --- [~stefanrichte...@gmail.com] What do you think of this? > Let

[jira] [Created] (FLINK-9480) Let local recovery support rescaling

Sihua Zhou created FLINK-9480: - Summary: Let local recovery support rescaling Key: FLINK-9480 URL: https://issues.apache.org/jira/browse/FLINK-9480 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9479) Let the rescale API to use local recovery

Sihua Zhou created FLINK-9479: - Summary: Let the rescale API to use local recovery Key: FLINK-9479 URL: https://issues.apache.org/jira/browse/FLINK-9479 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9475) introduce an approximate version of "select distinct"

Sihua Zhou created FLINK-9475: - Summary: introduce an approximate version of "select distinct" Key: FLINK-9475 URL: https://issues.apache.org/jira/browse/FLINK-9475 Project: Flink Issue Type:

[jira] [Created] (FLINK-9474) Introduce an approximate version of "count distinct"

Sihua Zhou created FLINK-9474: - Summary: Introduce an approximate version of "count distinct" Key: FLINK-9474 URL: https://issues.apache.org/jira/browse/FLINK-9474 Project: Flink Issue Type: New

[jira] [Updated] (FLINK-9364) Add doc of the memory usage in flink

[ https://issues.apache.org/jira/browse/FLINK-9364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9364: -- Summary: Add doc of the memory usage in flink (was: Add doc for the memory usage in flink) > Add doc

[jira] [Created] (FLINK-9468) get outputLimit of LimitedConnectionsFileSystem incorrectly

Sihua Zhou created FLINK-9468: - Summary: get outputLimit of LimitedConnectionsFileSystem incorrectly Key: FLINK-9468 URL: https://issues.apache.org/jira/browse/FLINK-9468 Project: Flink Issue

[jira] [Assigned] (FLINK-9455) Make SlotManager aware of multi slot TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9455: - Assignee: Sihua Zhou > Make SlotManager aware of multi slot TaskManagers >

[jira] [Assigned] (FLINK-9456) Let ResourceManager notify JobManager about failed/killed TaskManagers

[ https://issues.apache.org/jira/browse/FLINK-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9456: - Assignee: Sihua Zhou > Let ResourceManager notify JobManager about failed/killed TaskManagers >

[jira] [Commented] (FLINK-8205) Multi key get

[ https://issues.apache.org/jira/browse/FLINK-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492731#comment-16492731 ] Sihua Zhou commented on FLINK-8205: --- Hi [~kkl0u] are you still willing to help with the design and the

[jira] [Commented] (FLINK-8601) Introduce ElasticBloomFilter for Approximate calculation and other situations of performance optimization

[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492600#comment-16492600 ] Sihua Zhou commented on FLINK-8601: --- Hi [~aljoscha] could you please have a look at this? I updated the

[jira] [Assigned] (FLINK-9410) Replace NMClient with NMClientAsync in YarnResourceManager

[ https://issues.apache.org/jira/browse/FLINK-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9410: - Assignee: mingleizhang (was: Sihua Zhou) > Replace NMClient with NMClientAsync in

[jira] [Assigned] (FLINK-9351) RM stop assigning slot to Job because the TM killed before connecting to JM successfully

[ https://issues.apache.org/jira/browse/FLINK-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9351: - Assignee: Sihua Zhou > RM stop assigning slot to Job because the TM killed before connecting to

[jira] [Commented] (FLINK-9426) Harden RocksDBWriteBatchPerformanceTest.benchMark()

[ https://issues.apache.org/jira/browse/FLINK-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487276#comment-16487276 ] Sihua Zhou commented on FLINK-9426: --- one instance: https://travis-ci.org/apache/flink/jobs/382584476 >

[jira] [Created] (FLINK-9426) Harden RocksDBWriteBatchPerformanceTest.benchMark()

Sihua Zhou created FLINK-9426: - Summary: Harden RocksDBWriteBatchPerformanceTest.benchMark() Key: FLINK-9426 URL: https://issues.apache.org/jira/browse/FLINK-9426 Project: Flink Issue Type: Bug

[jira] [Assigned] (FLINK-9417) Send heartbeat requests from RPC endpoint's main thread

[ https://issues.apache.org/jira/browse/FLINK-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9417: - Assignee: Sihua Zhou > Send heartbeat requests from RPC endpoint's main thread >

[jira] [Updated] (FLINK-8601) Introduce ElasticBloomFilter for Approximate calculation and other situations of performance optimization

[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h2. *Motivation* There are some scenarios drive us to introduce this ElasticBloomFilter,

[jira] [Updated] (FLINK-8601) Introduce ElasticBloomFilter for Approximate calculation and other situations of performance optimization

[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h2. *Motivation* There are some scenarios drive us to introduce this ElasticBloomFilter,

[jira] [Updated] (FLINK-9070) Improve performance of RocksDBMapState.clear()

[ https://issues.apache.org/jira/browse/FLINK-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9070: -- Affects Version/s: (was: 1.6.0) 1.5.0 > Improve performance of

[jira] [Updated] (FLINK-9070) Improve performance of RocksDBMapState.clear()

[ https://issues.apache.org/jira/browse/FLINK-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9070: -- Fix Version/s: (was: 1.6.0) 1.5.0 > Improve performance of

[jira] [Assigned] (FLINK-9410) Replace NMClient with NMClientAsync in YarnResourceManager

[ https://issues.apache.org/jira/browse/FLINK-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-9410: - Assignee: Sihua Zhou > Replace NMClient with NMClientAsync in YarnResourceManager >

[jira] [Closed] (FLINK-9401) Data lost when rescaling the job from incremental checkpoint

[ https://issues.apache.org/jira/browse/FLINK-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-9401. - Resolution: Invalid > Data lost when rescaling the job from incremental checkpoint >

[jira] [Created] (FLINK-9401) Data lost when rescaling the job from incremental checkpoint

Sihua Zhou created FLINK-9401: - Summary: Data lost when rescaling the job from incremental checkpoint Key: FLINK-9401 URL: https://issues.apache.org/jira/browse/FLINK-9401 Project: Flink Issue

[jira] [Commented] (FLINK-9375) Introduce AbortCheckpoint message from JM to TMs

[ https://issues.apache.org/jira/browse/FLINK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480041#comment-16480041 ] Sihua Zhou commented on FLINK-9375: --- Hi [~srichter][~yanghua]I think this is a bit looks like a

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479462#comment-16479462 ] Sihua Zhou commented on FLINK-9373: --- [~srichter] FYI

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478865#comment-16478865 ] Sihua Zhou commented on FLINK-9373: --- I will updated the PR quickly. > Fix potential data losing for

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478848#comment-16478848 ] Sihua Zhou commented on FLINK-9373: --- I think that makes sense. > Fix potential data losing for

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477433#comment-16477433 ] Sihua Zhou commented on FLINK-8918: --- [~fhueske] related to the saturated problem, I want to add some

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477429#comment-16477429 ] Sihua Zhou commented on FLINK-8918: --- Hi [~fhueske], sorry that I'm definitely not asked you to promise

[jira] [Comment Edited] (FLINK-8918) Introduce Runtime Filter Join

[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477373#comment-16477373 ] Sihua Zhou edited comment on FLINK-8918 at 5/16/18 12:49 PM: - Hi [~fhueske]

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477373#comment-16477373 ] Sihua Zhou commented on FLINK-8918: --- Hi [~fhueske] thank you very much for your reply, you are right

[jira] [Updated] (FLINK-9373) Fix potential data losing for RocksDBBackend

[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9373: -- Fix Version/s: 1.5.0 > Fix potential data losing for RocksDBBackend >

[jira] [Updated] (FLINK-9373) Fix potential data losing for RocksDBBackend

[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-9373: -- Summary: Fix potential data losing for RocksDBBackend (was: Always call RocksIterator.status() to

[jira] [Updated] (FLINK-8918) Introduce Runtime Filter Join

[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8918: -- Description: In general, stream join is one of the most performance cost task. For every record from

  1   2   3   4   5   >