Yes, in the end the requests to HBase are the bottle neck and the latency will manifest in different places of the job depending on where there is a queue. If there is a queue between map and flatMap elements will sit there and wait and you’ll see latency there. If map and flatMap are chained you will see the latency in the form of Kafka consumer lag (Kafka itself is the queue here).
Best, Aljoscha > On 29. Jun 2017, at 18:30, sohimankotia <sohimanko...@gmail.com> wrote: > > Few last doubts : > > 1. So If I increase parallelism latency will decrease because load will get > distributed ? > 2. But if load will increase latency will also increase if parallelism is > more ? > 3. Let's say If I remove partitioner , and Hbase Op is still there in Flat > map . Then also this latency would be there ? > 4. If yes in point 3 , then latency would from reading elements from kafka ? > > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Partitioner-is-spending-around-2-to-4-minutes-while-pushing-data-to-next-operator-tp13913p14076.html > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.
