After upgrading to flink 1.4.0 using the hadoop-free build option, I’m seeing the following error on startup in the app master:
2017-12-12 18:23:15.473 [main] ERROR o.a.f.m.r.clusterframework.MesosApplicationMasterRunner - Mesos JobManager initialization failed <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34208>java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34315> at org.apache.flink.runtime.clusterframework.overlays.HadoopUserOverlay$Builder.fromEnvironment(HadoopUserOverlay.java:74) <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34439> at org.apache.flink.mesos.entrypoint.MesosEntrypointUtils.applyOverlays(MesosEntrypointUtils.java:145) <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34543> Looking at the code, it appears that the HadoopUserOverlay always tries to init the UserGroupInformation class, and is failing. Same error with or without the flink-shaded-hadoop2 library included. This is my lib dir: flink-appmaster-1.0-SNAPSHOT.jar flink-s3-fs-presto-1.4.0.jar jul-to-slf4j-1.7.25.jar sentry-1.5.3.jar flink-dist_2.11-1.4.0.jar flink-shaded-hadoop2-1.4.0.jar log4j-over-slf4j-1.7.25.jar sentry-logback-1.5.3.jar flink-metrics-prometheus-1.4.0.jar jackson-core-2.8.10.jar logback-classic-1.1.11.jar flink-python_2.11-1.4.0.jar jcl-over-slf4j-1.7.25.jar logback-core-1.1.11.jar -- Jared Stehler Chief Architect - Intellify Learning o: 617.701.6330 x703
