[jira] [Updated] (FLINK-8602) Improve recovery performance for rocksdb backend

2018-02-25 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Description: Right now, RocksDB backend supports {{fully checkpoint}} & {{incremental checkpoint}}.

[jira] [Updated] (FLINK-8602) Improve recovery performance for rocksdb backend

2018-02-25 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Description: Right now, RocksDB backend supports {{fully checkpoint}} & {{incremental checkpoint}}.

[jira] [Updated] (FLINK-8602) Improve recovery performance for rocksdb backend

2018-02-25 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Summary: Improve recovery performance for rocksdb backend (was: Accelerate recover from failover when

[jira] [Commented] (FLINK-8753) Introduce Incremental savepoint

2018-02-23 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374425#comment-16374425 ] Sihua Zhou commented on FLINK-8753: --- [~gyfora] For your proposal, I think there is a problem that

[jira] [Commented] (FLINK-8753) Introduce Incremental savepoint

2018-02-23 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374414#comment-16374414 ] Sihua Zhou commented on FLINK-8753: --- [~aljoscha] Thanks for your reply. I agree that if there is a rough

[jira] [Assigned] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-7219: - Assignee: (was: Sihua Zhou) > Current allocate strategy cann‘t achieve the optimal effect

[jira] [Commented] (FLINK-8601) Introduce PartitionedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373978#comment-16373978 ] Sihua Zhou commented on FLINK-8601: --- [~aljoscha] Do you have any advice on this? How about I submit a PR

[jira] [Commented] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373970#comment-16373970 ] Sihua Zhou commented on FLINK-8602: --- [~stefanrichte...@gmail.com] Could you please have a look at this?

[jira] [Updated] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Description: Currently, when enable incremental checkpoint, if user change the parallelism then

[jira] [Commented] (FLINK-8753) Introduce Incremental savepoint

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373962#comment-16373962 ] Sihua Zhou commented on FLINK-8753: --- [~stefanrichte...@gmail.com] What do you think? > Introduce

[jira] [Updated] (FLINK-8753) Introduce Incremental savepoint

2018-02-22 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8753: -- Description: Right now, savepoint goes through the full checkpoint path, take a savepoint could be

[jira] [Created] (FLINK-8753) Introduce Incremental savepoint

2018-02-22 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8753: - Summary: Introduce Incremental savepoint Key: FLINK-8753 URL: https://issues.apache.org/jira/browse/FLINK-8753 Project: Flink Issue Type: New Feature

[jira] [Updated] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Affects Version/s: (was: 1.4.0) 1.5.0 > Accelerate recover from failover

[jira] [Updated] (FLINK-8601) Introduce PartitionedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Affects Version/s: (was: 1.4.0) 1.5.0 > Introduce PartitionedBloomFilter for

[jira] [Closed] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

2018-02-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-8699. - Resolution: Fixed Release Note: Richter has fixed this in his hotfix. > Fix concurrency problem in

[jira] [Created] (FLINK-8699) Fix concurrency problem in rocksdb full checkpoint

2018-02-19 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8699: - Summary: Fix concurrency problem in rocksdb full checkpoint Key: FLINK-8699 URL: https://issues.apache.org/jira/browse/FLINK-8699 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-8679) RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with namespace

2018-02-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8679: -- Affects Version/s: 1.5.0 > RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with

[jira] [Updated] (FLINK-8679) RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with namespace

2018-02-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8679: -- Priority: Blocker (was: Critical) > RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter

[jira] [Updated] (FLINK-8679) RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with namespace

2018-02-17 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8679: -- Priority: Critical (was: Major) > RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter

[jira] [Created] (FLINK-8679) RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with namespace

2018-02-16 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8679: - Summary: RocksDBKeyedBackend.getKeys(stateName, namespace) doesn't filter data with namespace Key: FLINK-8679 URL: https://issues.apache.org/jira/browse/FLINK-8679

[jira] [Created] (FLINK-8676) Memory Leak in AbstractKeyedStateBackend.applyToAllKeys() when backend is base on RocksDB

2018-02-16 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8676: - Summary: Memory Leak in AbstractKeyedStateBackend.applyToAllKeys() when backend is base on RocksDB Key: FLINK-8676 URL: https://issues.apache.org/jira/browse/FLINK-8676

[jira] [Closed] (FLINK-6980) TypeExtractor.getForObject can't get typeinfo correctly.

2018-02-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-6980. - Resolution: Won't Fix > TypeExtractor.getForObject can't get typeinfo correctly. >

[jira] [Created] (FLINK-8657) Fix incorrect description for external checkpoint vs savepoint

2018-02-14 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8657: - Summary: Fix incorrect description for external checkpoint vs savepoint Key: FLINK-8657 URL: https://issues.apache.org/jira/browse/FLINK-8657 Project: Flink

[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2018-02-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364215#comment-16364215 ] Sihua Zhou commented on FLINK-8044: --- Sorry [~till.rohrmann], finally I can't come up with a perfect

[jira] [Created] (FLINK-8639) Fix always need to seek multiple times when iterator RocksDBMapState

2018-02-12 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8639: - Summary: Fix always need to seek multiple times when iterator RocksDBMapState Key: FLINK-8639 URL: https://issues.apache.org/jira/browse/FLINK-8639 Project: Flink

[jira] [Updated] (FLINK-8601) Introduce PartitionedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h3. Backgroud Bloom filter is useful in many situation, for example: * 1. Approximate

[jira] [Updated] (FLINK-8601) Introduce PartitionedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Summary: Introduce PartitionedBloomFilter for Approximate calculation and other situations of

[jira] [Commented] (FLINK-8601) Introduce LinkedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358051#comment-16358051 ] Sihua Zhou commented on FLINK-8601: --- [~fhueske] I have update the document, could you please have a loot

[jira] [Updated] (FLINK-8601) Introduce LinkedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Component/s: (was: Core) State Backends, Checkpointing > Introduce

[jira] [Updated] (FLINK-8601) Introduce LinkedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h3. Backgroud Bloom filter is useful in many situation, for example: * 1. Approximate

[jira] [Commented] (FLINK-8601) Introduce LinkedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356796#comment-16356796 ] Sihua Zhou commented on FLINK-8601: --- [~jark] When creating a BloomFilter user have to special the

[jira] [Updated] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Description: Currently, when enable incremental checkpoint, if user change the parallelism then

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356715#comment-16356715 ] Sihua Zhou commented on FLINK-3089: --- [~phoenixjiangnan] Sorry for I did not reply for so long, because I

[jira] [Updated] (FLINK-8601) Introduce LinkedBloomFilter for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h3. Backgroud Bloom filter is useful in many situation, for example: * 1. Approximate

[jira] [Commented] (FLINK-8601) Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356703#comment-16356703 ] Sihua Zhou commented on FLINK-8601: --- [~fhueske] Ok, I will modify the description and also add an

[jira] [Commented] (FLINK-8601) Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356663#comment-16356663 ] Sihua Zhou commented on FLINK-8601: --- [~fhueske], thanks for you reply. I think this is not a new state

[jira] [Updated] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8602: -- Description: Currently, when enable incremental checkpoint, if user change the parallelism then

[jira] [Created] (FLINK-8602) Accelerate recover from failover when use incremental checkpoint

2018-02-08 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8602: - Summary: Accelerate recover from failover when use incremental checkpoint Key: FLINK-8602 URL: https://issues.apache.org/jira/browse/FLINK-8602 Project: Flink

[jira] [Updated] (FLINK-8601) Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization

2018-02-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8601: -- Description: h3. Backgroud Bloom filter is useful in many situation, for example: * 1. Approximate

[jira] [Created] (FLINK-8601) Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization

2018-02-07 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8601: - Summary: Introduce LinkedBloomFilterState for Approximate calculation and other situations of performance optimization Key: FLINK-8601 URL:

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2018-01-08 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317829#comment-16317829 ] Sihua Zhou commented on FLINK-3089: --- [~phoenixjiangnan] Maybe we shouldn't enforce TtlDB delete all

[jira] [Reopened] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-26 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reopened FLINK-7873: --- reopen for PR. > Introduce CheckpointCacheManager for reading checkpoint data locally when > performing

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255081#comment-16255081 ] Sihua Zhou commented on FLINK-7873: --- [~srichter] Happy to do that, will open a PR and attach the doc

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255065#comment-16255065 ] Sihua Zhou commented on FLINK-7873: --- closed. > Introduce CheckpointCacheManager for reading checkpoint

[jira] [Closed] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7873. - Resolution: Duplicate Community plan to introduce this in Flink 1.5 and this issue seem to interfered

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255055#comment-16255055 ] Sihua Zhou commented on FLINK-7873: --- [~srichter] As you will introduce this feature in Flink 1.5

[jira] [Updated] (FLINK-8018) RMQ does not support disabling queueDeclare, when the user has no declaration permissions, it cannot connect

2017-11-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-8018: -- Fix Version/s: 1.5.0 > RMQ does not support disabling queueDeclare, when the user has no declaration >

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16250747#comment-16250747 ] Sihua Zhou commented on FLINK-7873: --- [~srichter] Thanks for your reply and the introduction of your

[jira] [Commented] (FLINK-4256) Fine-grained recovery

2017-11-10 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247272#comment-16247272 ] Sihua Zhou commented on FLINK-4256: --- Partial recovery seem to more useful for Batch job, for stream job

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-11-10 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247268#comment-16247268 ] Sihua Zhou commented on FLINK-7873: --- After a deeper thought, I found a way that can simply support all

[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2017-11-10 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247263#comment-16247263 ] Sihua Zhou commented on FLINK-8044: --- Hi [~till.rohrmann] I have addressed

[jira] [Created] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2017-11-10 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8044: - Summary: Introduce scheduling mechanism to satisfy both state locality and input Key: FLINK-8044 URL: https://issues.apache.org/jira/browse/FLINK-8044 Project: Flink

[jira] [Created] (FLINK-8018) RMQ does not support disabling queueDeclare, when the user has no declaration permissions, it cannot connect

2017-11-07 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8018: - Summary: RMQ does not support disabling queueDeclare, when the user has no declaration permissions, it cannot connect Key: FLINK-8018 URL:

[jira] [Commented] (FLINK-7866) Weigh list of preferred locations for scheduling

2017-11-02 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16237111#comment-16237111 ] Sihua Zhou commented on FLINK-7866: --- [~till.rohrmann] thanks and so happy for thinking of me when you

[jira] [Assigned] (FLINK-7866) Weigh list of preferred locations for scheduling

2017-11-02 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou reassigned FLINK-7866: - Assignee: Sihua Zhou > Weigh list of preferred locations for scheduling >

[jira] [Commented] (FLINK-7866) Weigh list of preferred locations for scheduling

2017-11-01 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235215#comment-16235215 ] Sihua Zhou commented on FLINK-7866: --- Hi [~till.rohrmann], i saw you have made a PR for

[jira] [Commented] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-10-31 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233643#comment-16233643 ] Sihua Zhou commented on FLINK-7873: --- [~stefanrichte...@gmail.com] Thanks very much for your reply, you

[jira] [Updated] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-10-30 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Description: Why i introduce this: Current recover strategy will always read checkpoint data from

[jira] [Updated] (FLINK-7873) Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

2017-10-30 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Summary: Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

[jira] [Updated] (FLINK-7873) Introduce HybridStreamHandle to optimize the recovery mechanism and try to read the checkpoint data locally

2017-10-28 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Description: Current recover strategy will always read checkpoint data from remote FileStream (HDFS).

[jira] [Updated] (FLINK-7873) Introduce HybridStreamHandle to optimize the recovery mechanism and try to read the checkpoint data locally

2017-10-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Summary: Introduce HybridStreamHandle to optimize the recovery mechanism and try to read the checkpoint

[jira] [Updated] (FLINK-7873) Introduce HybridStreamStateHandle for quick recovery from checkpoint.

2017-10-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Description: Current recover strategy will always read checkpoint data from remote FileStream (HDFS).

[jira] [Updated] (FLINK-7873) Introduce HybridStreamStateHandle for quick recovery from checkpoint.

2017-10-27 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7873: -- Description: Current recover strategy will always read checkpoint data from remote FileStream (HDFS).

[jira] [Created] (FLINK-7873) Introduce HybridStreamStateHandle for quick recovery from checkpoint.

2017-10-19 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-7873: - Summary: Introduce HybridStreamStateHandle for quick recovery from checkpoint. Key: FLINK-7873 URL: https://issues.apache.org/jira/browse/FLINK-7873 Project: Flink

[jira] [Commented] (FLINK-7866) Weigh list of preferred locations for scheduling

2017-10-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210618#comment-16210618 ] Sihua Zhou commented on FLINK-7866: --- [~till.rohrmann] have you already work on this? If not, can i take

[jira] [Closed] (FLINK-7218) ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-10-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7218. - Resolution: Duplicate Release Note: Can be resolved in FLINK-7866, closing the PR for this too. >

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-10-17 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207212#comment-16207212 ] Sihua Zhou commented on FLINK-7153: --- I prefer [2], it also fix FLINK-7218 and do a better job than the

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2017-08-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131864#comment-16131864 ] Sihua Zhou commented on FLINK-3089: --- [~richtesn] I'm not sure if I understand the problem you said

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2017-08-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131845#comment-16131845 ] Sihua Zhou commented on FLINK-3089: --- [~stefanrichte...@gmail.com] i think ttl is also fine for

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2017-08-17 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129974#comment-16129974 ] Sihua Zhou commented on FLINK-3089: --- [~ymost],all we have done it to modify the

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2017-08-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129896#comment-16129896 ] Sihua Zhou commented on FLINK-3089: --- We use TtlDB(provided by rocksdb that extends Rocksdbs) to support

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-21 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096249#comment-16096249 ] Sihua Zhou commented on FLINK-7153: --- [~aljoscha] I issues contains two subproblems, for the first one

[jira] [Comment Edited] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2017-07-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094193#comment-16094193 ] Sihua Zhou edited comment on FLINK-7219 at 7/20/17 5:25 AM: Thanks for your

[jira] [Commented] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2017-07-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094193#comment-16094193 ] Sihua Zhou commented on FLINK-7219: --- Thanks for your reply. This question really needs to be further

[jira] [Comment Edited] (FLINK-7218) ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-07-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093373#comment-16093373 ] Sihua Zhou edited comment on FLINK-7218 at 7/19/17 4:32 PM: I made a PR for

[jira] [Comment Edited] (FLINK-7218) ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-07-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093373#comment-16093373 ] Sihua Zhou edited comment on FLINK-7218 at 7/19/17 4:31 PM: I made a PR for

[jira] [Commented] (FLINK-7218) ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-07-19 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093373#comment-16093373 ] Sihua Zhou commented on FLINK-7218: --- I made a PR for this issue, can anybody review for me?

[jira] [Commented] (FLINK-6980) TypeExtractor.getForObject can't get typeinfo correctly.

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091689#comment-16091689 ] Sihua Zhou commented on FLINK-6980: --- [~twalthr] Actually, we are building a stream sql framework base on

[jira] [Issue Comment Deleted] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7153: -- Comment: was deleted (was: Current allocate strategy cann‘t achieve the optimal effect with input's

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091579#comment-16091579 ] Sihua Zhou commented on FLINK-7153: --- Current allocate strategy cann‘t achieve the optimal effect with

[jira] [Issue Comment Deleted] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7153: -- Comment: was deleted (was: ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091577#comment-16091577 ] Sihua Zhou commented on FLINK-7153: --- ExecutionVertex.getPreferredLocationsBasedOnInputs() will always

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-18 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091575#comment-16091575 ] Sihua Zhou commented on FLINK-7153: --- After thinking about is, i found that this issue contains two

[jira] [Created] (FLINK-7219) Current allocate strategy cann‘t achieve the optimal effect with input's location

2017-07-18 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-7219: - Summary: Current allocate strategy cann‘t achieve the optimal effect with input's location Key: FLINK-7219 URL: https://issues.apache.org/jira/browse/FLINK-7219 Project:

[jira] [Created] (FLINK-7218) ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-07-18 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-7218: - Summary: ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty Key: FLINK-7218 URL: https://issues.apache.org/jira/browse/FLINK-7218 Project: Flink

[jira] [Comment Edited] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086915#comment-16086915 ] Sihua Zhou edited comment on FLINK-7153 at 7/16/17 3:37 PM: Hi [~StephanEwen],

[jira] [Closed] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7180. - Resolution: Resolved > CoGroupStream perform checkpoint failed > ---

[jira] [Commented] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087554#comment-16087554 ] Sihua Zhou commented on FLINK-7180: --- Checked my pom.xml, i'm using Flink 1.3.0 After upgrading to

[jira] [Comment Edited] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087397#comment-16087397 ] Sihua Zhou edited comment on FLINK-7180 at 7/14/17 2:41 PM: [~aljoscha] Sorry

[jira] [Commented] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087397#comment-16087397 ] Sihua Zhou commented on FLINK-7180: --- [~aljoscha] Sorry to reply late reply, i was on the train all the

[jira] [Created] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-7180: - Summary: CoGroupStream perform checkpoint failed Key: FLINK-7180 URL: https://issues.apache.org/jira/browse/FLINK-7180 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-7180) CoGroupStream perform checkpoint failed

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7180: -- Description: When using the CoGroup api and enable the checkpoint, Job will failed when performing

[jira] [Comment Edited] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086915#comment-16086915 ] Sihua Zhou edited comment on FLINK-7153 at 7/14/17 6:08 AM: Hi [~StephanEwen],

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-14 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086915#comment-16086915 ] Sihua Zhou commented on FLINK-7153: --- Hi [~StephanEwen], Here is my solution. For problem 1, We need

[jira] [Commented] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085706#comment-16085706 ] Sihua Zhou commented on FLINK-7153: --- [~StephanEwen] Thanks for you reply, can i contribute to this

[jira] [Commented] (FLINK-5747) Eager Scheduling should deploy all Tasks together

2017-07-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1608#comment-1608 ] Sihua Zhou commented on FLINK-5747: --- Hi,[~StephanEwen], there's some problems i found with Eager

[jira] [Updated] (FLINK-7153) Eager Scheduling can't allocate source for ExecutionGraph correctly

2017-07-13 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou updated FLINK-7153: -- Summary: Eager Scheduling can't allocate source for ExecutionGraph correctly (was: JobManager can't

[jira] [Commented] (FLINK-7153) JobManager can't allocate source for ExecutionGraph correctly

2017-07-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085038#comment-16085038 ] Sihua Zhou commented on FLINK-7153: --- Hi, [~fhueske] can you help me to verify the problem i posted

[jira] [Closed] (FLINK-7160) Support hive like udtf

2017-07-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sihua Zhou closed FLINK-7160. - Resolution: Duplicate > Support hive like udtf > -- > > Key:

[jira] [Commented] (FLINK-7160) Support hive like udtf

2017-07-12 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083796#comment-16083796 ] Sihua Zhou commented on FLINK-7160: --- Hi, Fabian Hueske, you are right this is a duplicate issue. >

<    1   2   3   4   5   >