I’m trying to submit a stand-alone Flink job to a YARN cluster running on EMR
(Elastic MapReduce) nodes in AWS. When it tries to start a container for the
Job Manager, it fails. The error message from the container is below. The
command I’m using is:
$ flink run -m yarn-cluster -yn 1 -ynm test1
I know this is really basic but have you verified that you're Flink lib
folder contains log4j-1.2.17.jar? I imagine that's fine given the
yarn-session.sh approach is working fine. What version of EMR are you
running? What version of Flink?
-Jamie
On Thu, Jun 30, 2016 at 11:28 AM, Hanson, Bruc
Hi Bruce,
I just spun up an EMR cluster and tried this out. Hadoop 2.7.2 and Flink
1.0.3. I ran the exact same command as you and everything works just fine.
Please verify one thing, though. In your command you do not specify the
path to the Flink executable, which means it's just getting pick
You can also have a look at the YARN client logs, which should print which
JARs are uploaded. The container logs should also log the class path.
On Sun, Jul 3, 2016 at 6:04 PM, Jamie Grier wrote:
> Hi Bruce,
>
> I just spun up an EMR cluster and tried this out. Hadoop 2.7.2 and Flink
> 1.0.3.