Re: [PROPOSAL] Introduce Elastic Bloom Filter For Flink

2018-06-12 Thread sihua zhou
Hi, no more feedbacks these days...I guess it's because you guys are too busy and since I didn't receive any negative feedbacks and there're already some positive feedbacks. So I want to implement this *Elastic Bloom Filter* based on the current design doc(because I have time to do it currentl

Re: [PROPOSAL] Introduce Elastic Bloom Filter For Flink

2018-06-12 Thread Fabian Hueske
Hi Sihua, Sorry for not replying earlier. I have one question left. If I understood the design of the linked Bloomfilter nodes right, users would need to configure a TTL to be able to remove a node. When nodes are removed, we would need to insert every key into the current node which would not be

Re: [PROPOSAL] Introduce Elastic Bloom Filter For Flink

2018-06-12 Thread sihua zhou
Hi Fabian, Thanks a lot for your reply, you are right that users would need to configure a TTL for the Elastic Filter to recycle the memory resource. For every Linked BloomFilter Nodes, only the head node is writable, the other nodes are all full, they are only immutable(only readable, we imp

[jira] [Created] (FLINK-9569) Confusing construction of AvroSerializers for generic records

2018-06-12 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-9569: -- Summary: Confusing construction of AvroSerializers for generic records Key: FLINK-9569 URL: https://issues.apache.org/jira/browse/FLINK-9569 Project: Flin

[jira] [Created] (FLINK-9570) SQL Client merging environments uses AbstractMap

2018-06-12 Thread JIRA
Dominik Wosiński created FLINK-9570: --- Summary: SQL Client merging environments uses AbstractMap Key: FLINK-9570 URL: https://issues.apache.org/jira/browse/FLINK-9570 Project: Flink Issue Ty

Clarity on Flink 1.5 Rescale mechanism

2018-06-12 Thread Sampath Bhat
Hello In flink 1.5 release notes - https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes Various Other Features and Improvements: Applications can be rescaled without manually triggering a savepoint. Under the hood, Flink will still take a savepoint, stop the application, and r

[jira] [Created] (FLINK-9571) Switch to internal states in StateBinder

2018-06-12 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9571: -- Summary: Switch to internal states in StateBinder Key: FLINK-9571 URL: https://issues.apache.org/jira/browse/FLINK-9571 Project: Flink Issue Type: Sub-ta

[jira] [Created] (FLINK-9572) Extend InternalAppendingState with internal stored state access

2018-06-12 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9572: -- Summary: Extend InternalAppendingState with internal stored state access Key: FLINK-9572 URL: https://issues.apache.org/jira/browse/FLINK-9572 Project: Flink

[jira] [Created] (FLINK-9573) Check for leadership with leader session id

2018-06-12 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-9573: Summary: Check for leadership with leader session id Key: FLINK-9573 URL: https://issues.apache.org/jira/browse/FLINK-9573 Project: Flink Issue Type: Improve

Re: [PROPOSAL] Introduce Elastic Bloom Filter For Flink

2018-06-12 Thread sihua zhou
Hi, Maybe I would like to add more information concerning to the Linked Filter Nodes on each key group. The reason that we need to maintance a Linked Filter Nodes is that we need to handle data skew, data skew is also the most challenging problem that we need to overcome. Because we don't know

Re: Clarity on Flink 1.5 Rescale mechanism

2018-06-12 Thread Stefan Richter
Hi, it means that you can now modify the parallelism of a running job with a new „modify“ command in the CLI, see [1]. Adding task manager will add their offered slots to the pool of available slots, it will not automatically change the parallelism. Best, Stefan [1] https://ci.apache.org/proj

Static code analysis for Flink project

2018-06-12 Thread simeon.arkhipov
Hello Flink community. I am new in Flink project and probably don't understand it a lot. Could you please clarify one question to me? I download Flink sources and build it from scratch. I found checkstyle guidelines that every Flink developer should follow which is very useful. However, I didn

Re: Static code analysis for Flink project

2018-06-12 Thread Ted Yu
I took a look at some of the blocker defects. e.g. https://sonarcloud.io/project/issues?id=org.apache.flink%3Aflink-parent&open=AWPxETxA3e-qcckj1Sl1&resolved=false&severities=BLOCKER&types=BUG For ./flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/s

how to build the connectors and examples from the source

2018-06-12 Thread Chris Kellogg
How can one build a connectors jar from the source? Also, is there a quick way to build the examples from the source without having to do a mvn clean package -DskipTests? Thanks. Chris

Re: how to build the connectors and examples from the source

2018-06-12 Thread Ted Yu
Which connector from the following list are you trying to build ? https://flink.apache.org/ecosystem.html#connectors The available connectors from 1.5.0 are quite recent. Is there any functionality missing in the 1.5.0 release ? Thanks On Tue, Jun 12, 2018 at 5:17 PM, Chris Kellogg wrote: > H

Re: WELCOME to dev@flink.apache.org

2018-06-12 Thread Sandish Kumar HN
Can someone add me as a contributor Mail:sanysand...@gmail.com FullName: Sandish Kumar HN On 12 June 2018 at 23:14, wrote: > Hi! This is the ezmlm program. I'm managing the > dev@flink.apache.org mailing list. > > I'm working for my owner, who can be reached > at dev-ow...@flink.apache.org. >