Re: kafka consumer parallelism

2017-10-05 Thread r. r.
Thanks a lot, Carst! I hadn't realized that Best regards > Оригинално писмо >От: Carst Tankink ctank...@bol.com >Относно: Re: kafka consumer parallelism >До: "r. r." >Изпратено на: 05.10.2017 09:04 > Hi, > > Th

Re: kafka consumer parallelism

2017-10-04 Thread Carst Tankink
is the map() spread out only if I do as you suggest messageStream.rebalance().map(..) ? Best regards Rob > Оригинално писмо >От: Carst Tankink ctank...@bol.com >Относно: Re: kafka consumer

Re: kafka consumer parallelism

2017-10-04 Thread r. r.
as you suggest messageStream.rebalance().map(..) ? Best regards Rob > Оригинално писмо >От: Carst Tankink ctank...@bol.com >Относно: Re: kafka consumer parallelism >До: "user@flink.apache.org" >Изпратено на: 03.10.2017 11:30 > (Acciden

Re: kafka consumer parallelism

2017-10-03 Thread Carst Tankink
localhost:2181 Regards, Tovi -Original Message- From: Timo Walther [mailto:twal...@apache.org] Sent: יום ב 02 אוקטובר 2017 20:59 To: user@flink.apache.org Subject: Re: kafka consumer parallelism Hi, I'm not a Kafka expert but I th

RE: kafka consumer parallelism

2017-10-03 Thread Sofer, Tovi
- From: Timo Walther [mailto:twal...@apache.org] Sent: יום ב 02 אוקטובר 2017 20:59 To: user@flink.apache.org Subject: Re: kafka consumer parallelism Hi, I'm not a Kafka expert but I think you need to have more than 1 Kafka partition to process multiple documents at the same time. Make

Re: kafka consumer parallelism

2017-10-02 Thread Timo Walther
Hi, I'm not a Kafka expert but I think you need to have more than 1 Kafka partition to process multiple documents at the same time. Make also sure to send the documents to different partitions. Regards, Timo Am 10/2/17 um 6:46 PM schrieb r. r.: Hello I'm running a job with "flink run -p5"

kafka consumer parallelism

2017-10-02 Thread r. r.
Hello I'm running a job with "flink run -p5" and additionally set env.setParallelism(5). The source of the stream is Kafka, the job uses FlinkKafkaConsumer010. In Flink UI though I notice that if I send 3 documents to Kafka, only one 'instance' of the consumer seems to receive Kafka's record and