Re: Error running Flink job in Yarn-cluster mode

2017-06-13 Thread Aljoscha Krettek
You’re welcome. 😃 > On 13. Jun 2017, at 10:24, Biplob Biswas wrote: > > Thanks a lot Aljoscha (again) > > I created my project from scratch and used the flink-maven-archetype and now > it works on the yarn-cluster mode. I was creating a fat jar initially as > well with my old project setup so n

Re: Error running Flink job in Yarn-cluster mode

2017-06-13 Thread Biplob Biswas
Thanks a lot Aljoscha (again) I created my project from scratch and used the flink-maven-archetype and now it works on the yarn-cluster mode. I was creating a fat jar initially as well with my old project setup so not really sure what went wrong there as it was working on my local test environment

Re: Error running Flink job in Yarn-cluster mode

2017-06-12 Thread Aljoscha Krettek
Hi, What version of Flink are you using? Also, how are you building your job? Is it a fat-jar, maybe based in the Flink Quickstart project? Best, Aljoscha > On 9. Jun 2017, at 11:49, Biplob Biswas wrote: > > One more thing i noticed is that the streaming wordcount from the flink > package wor

Re: Error running Flink job in Yarn-cluster mode

2017-06-09 Thread Biplob Biswas
One more thing i noticed is that the streaming wordcount from the flink package works when i run it but when i used the same github code, packaged it and uploaded the fat jar with the word count example to the cluster, i get the same error. I am wondering, How can making my uber jar produce such e

Re: Error running Flink job in Yarn-cluster mode

2017-06-09 Thread Biplob Biswas
Hi Nico, I tried running my job with 3 and even 2 yarn containers and the result is the same. Then I tried running the example wordcount(streaming and batch both) and they seem to find the task and job managers and run succesfully. ./bin/flink run -m yarn-cluster -yn 3 -yt ./lib ./examples/stream

Re: Error running Flink job in Yarn-cluster mode

2017-06-08 Thread Nico Kruber
I'm no expert here, but are 4 yarn containers/task managers (-yn 4) not too many for 3 data nodes (=3 dn?)? also, isn't the YARN UI reflecting its own jobs, i.e. running flink, as opposed to running the actual flink job? or did you mean that the flink web ui (through yarn) showed the submitted