Re: Unbalanced job scheduling

2017-10-17 Thread AndreaKinn
I'm in contact with the founder of the library to deal with the problem. I'm trying also to understand how implement myself slotSharingGroups -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Unbalanced job scheduling

2017-10-17 Thread Fabian Hueske
Setting the slot sharing group is Flink's mechanism to solve this issue. I'd consider this a limitation of the library that provides LEARN and SELECT. Did you consider to open an issue at (or contributing to) the library to support setting the slotSharing group? 2017-10-17 9:38 GMT+02:00 AndreaKi

Re: Unbalanced job scheduling

2017-10-17 Thread AndreaKinn
Yes, I considered them but unfortunately I can't call setSlotSharingGroup method on LEARN and SELECT operators. I can call it on the other operators but this means that the two LEARN method will be constrained in the same "unnamed" slot. -- Sent from: http://apache-flink-user-mailing-list-archi

Re: Unbalanced job scheduling

2017-10-17 Thread Fabian Hueske
Hi Andrea, have you looked into assigning slot sharing groups [1]? Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#task-chaining-and-resource-groups 2017-10-16 18:01 GMT+02:00 AndreaKinn : > Hi all, > I want to expose you my program flow. >

Unbalanced job scheduling

2017-10-16 Thread AndreaKinn
Hi all, I want to expose you my program flow. I have the following operators: kafka-source -> timestamp-extractor -> map -> keyBy -> window -> apply -> LEARN -> SELECT -> process -> cassandra-sink the LEARN and SELECT operators belong to an external library supported by flink. LEARN is a very he