Re: Submitting Flink application on YARN parameter

2018-04-28 Thread Kien Truong
Hi, You have to enable CPU scheduling in YARN, otherwise it always shows that only 1 CPU is allocated for each container, regardless of how many Flink try to allocated. TaskManager memory is 1400MB, but Flink reserves some amount for for off-heap memory, so the actual heap size is smaller.

Submitting Flink application on YARN parameter

2018-04-28 Thread Soheil Pourbafrani
Hi, I have an Flink .jar file and I submit it on yarn cluster using the command: flink run -m yarn-cluster -yn 5 -yjm 768 -ytm 1400 -ys 2 -yqu streamQ my_program.jar According to the submitting command I expect: // It will create 5 containers > satisfied // each container should use 2 core

Re: A trivial update on README

2018-04-28 Thread Stephan Ewen
Hi! Thank you for pinging us on this one - I left a comment, after that I would merge the PR. The Flink community had a bit of a slow phase for PRs, with many committers being at the Flink Forward Conference, being involved in the 1.5 release testing. We plan to speed up reviews again now.

Re: Setting the parallelism in a cluster of machines properly

2018-04-28 Thread m@xi
The TaskManager cannot reach the JobManager. I get this error. Any ideas? Caused by: org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost connection to the JobManager. Best, Max -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Raspberry Pi Memory Configuration

2018-04-28 Thread Nicholas Walton
Hi, Hope this is the right place to ask, but I have a 5 Raspberry Pi 3 cluster running Flink on which I’m hitting memory issues. Each Pi has 1Gb and is running with a 256Gb USB drive, however I don’t seem to be able to configure Java to use more than 256M of heap. The memory problem, I’m

Re: RateLimit for Kinesis Producer

2018-04-28 Thread Lasse Nedergaard
Hi I encounter the same problem for the Kinesis producer and will try to play around with the config setting and look into the code base. If I figure it out l let you know and please do the same if you figure it out before me Med venlig hilsen / Best regards Lasse Nedergaard > Den 27. apr.