Re: fan out parallel-able operator sub-task beyond total slots number

2016-04-18 Thread Till Rohrmann
Hi Chen, two subtasks of the same operator can never be executed within the same slot/pipeline. The `slotSharingGroup` allows you to only control which subtasks of different operators can be executed along side in the same slot. It basically allows you to break pipelines into smaller ones.

fan out parallel-able operator sub-task beyond total slots number

2016-04-17 Thread Chen Qin
Hi there, I try run large number of subtasks within a task slot using slot sharing group. The usage scenario tried to adress operator that makes a network call with high latency yet less memory or cpu footprint. (sample code below) >From doc provided, slotsharinggroup seems the place to look