Hi Alexandru, This sounds reasonable that it might be because of this minikube command failed, but I am not a kubernetes expert. I cc Till who knows more on this.
Best, Dawid On 19/12/2018 14:16, Alexandru Gutan wrote: > Thanks! > I'm using now the *flink:1.7.0-hadoop24-scala_2.12* image. > The Hadoop related error is gone, but I have a new error: > > Starting Task Manager > config file: > jobmanager.rpc.address: flink-jobmanager > jobmanager.rpc.port: 6123 > jobmanager.heap.size: 1024m > taskmanager.heap.size: 1024m > taskmanager.numberOfTaskSlots: 2 > parallelism.default: 1 > rest.port: 8081 > blob.server.port: 6124 > query.server.port: 6125 > Starting taskexecutor as a console application on host > flink-taskmanager-54b679f8bb-22b4r. > 2018-12-19 13:09:38,469 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -------------------------------------------------------------------------------- > 2018-12-19 13:09:38,470 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Starting TaskManager (Version: 1.7.0, Rev:49da9f9, Date:28.11.2018 @ > 17:59:06 UTC) > 2018-12-19 13:09:38,470 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - OS > current user: flink > 2018-12-19 13:09:38,921 WARN > org.apache.hadoop.util.NativeCodeLoader - Unable > to load native-hadoop library for your platform... using builtin-java > classes where applicable > 2018-12-19 13:09:39,307 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Current Hadoop/Kerberos user: flink > 2018-12-19 13:09:39,307 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - JVM: > OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.181-b13 > 2018-12-19 13:09:39,307 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Maximum heap size: 922 MiBytes > 2018-12-19 13:09:39,307 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > JAVA_HOME: /docker-java-home/jre > 2018-12-19 13:09:39,318 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Hadoop version: 2.4.1 > 2018-12-19 13:09:39,318 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - JVM > Options: > 2018-12-19 13:09:39,319 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -XX:+UseG1GC > 2018-12-19 13:09:39,319 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -Xms922M > 2018-12-19 13:09:39,320 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -Xmx922M > 2018-12-19 13:09:39,320 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -XX:MaxDirectMemorySize=8388607T > 2018-12-19 13:09:39,320 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties > 2018-12-19 13:09:39,320 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml > 2018-12-19 13:09:39,320 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Program Arguments: > 2018-12-19 13:09:39,321 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > --configDir > 2018-12-19 13:09:39,321 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > /opt/flink/conf > 2018-12-19 13:09:39,321 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Classpath: > /opt/flink/lib/flink-python_2.12-1.7.0.jar:/opt/flink/lib/flink-shaded-hadoop2-uber-1.7.0.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.15.jar:/opt/flink/lib/flink-dist_2.12-1.7.0.jar::: > 2018-12-19 13:09:39,321 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > -------------------------------------------------------------------------------- > 2018-12-19 13:09:39,323 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Registered UNIX signal handlers for [TERM, HUP, INT] > 2018-12-19 13:09:39,329 INFO > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > Maximum number of open file descriptors is 1048576. > 2018-12-19 13:09:39,366 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: jobmanager.rpc.address, flink-jobmanager > 2018-12-19 13:09:39,367 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: jobmanager.rpc.port, 6123 > 2018-12-19 13:09:39,367 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: jobmanager.heap.size, 1024m > 2018-12-19 13:09:39,367 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: taskmanager.heap.size, 1024m > 2018-12-19 13:09:39,368 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: taskmanager.numberOfTaskSlots, 2 > 2018-12-19 13:09:39,369 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: parallelism.default, 1 > 2018-12-19 13:09:39,370 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: rest.port, 8081 > 2018-12-19 13:09:39,372 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: blob.server.port, 6124 > 2018-12-19 13:09:39,374 INFO > org.apache.flink.configuration.GlobalConfiguration - > Loading configuration property: query.server.port, 6125 > 2018-12-19 13:09:39,511 INFO > org.apache.flink.runtime.security.modules.HadoopModule - Hadoop > user set to flink (auth:SIMPLE) > 2018-12-19 13:10:00,708 ERROR > org.apache.flink.runtime.taskexecutor.TaskManagerRunner - > TaskManager initialization failed. > *java.net.UnknownHostException: flink-jobmanager: Temporary failure in > name resolution > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) > at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929) > at > java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324) > at java.net.InetAddress.getAllByName0(InetAddress.java:1277) > at java.net.InetAddress.getAllByName(InetAddress.java:1193) > at java.net.InetAddress.getAllByName(InetAddress.java:1127) > at java.net.InetAddress.getByName(InetAddress.java:1077) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:167) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:133) > at > org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:89) > at > org.apache.flink.runtime.taskexecutor.TaskManagerRunner.<init>(TaskManagerRunner.java:127) > at > org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManager(TaskManagerRunner.java:330) > at > org.apache.flink.runtime.taskexecutor.TaskManagerRunner$1.call(TaskManagerRunner.java:301) > at > org.apache.flink.runtime.taskexecutor.TaskManagerRunner$1.call(TaskManagerRunner.java:298) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556) > at > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > at > org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:298)* > * > * > Is the minikube ssh... command supposed to mitigate this? (*minikube > ssh 'sudo ip link set docker0 promisc on')* > Maybe the minikube ssh.. command didn't get executed properly? Is > there a way to check if it had executed correctly?* > * > Or is it another type of issue?* > * > * > * > *Thank you! > * > > On Wed, 19 Dec 2018 at 12:12, Dawid Wysakowicz <dwysakow...@apache.org > <mailto:dwysakow...@apache.org>> wrote: > > Hi, > > You used a hadoopless docker image, therefore it cannot find > hadoop dependencies. It is ok if you don't need to use any, the > bolded messages are just INFO, those are not errors. > > Best, > > Dawid > > On 19/12/2018 12:58, Alexandru Gutan wrote: >> Dear all, >> >> I followed the instructions found here: >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.7/ops/deployment/kubernetes.html >> Minikube version 0.31-01 >> Kubernetes version 1.10 >> Flink Docker image: flink:latest (1.7.0-scala_2.12) >> >> I ran the following commands: >> >> minikube start >> minikube ssh 'sudo ip link set docker0 promisc on' >> kubectl create -f jobmanager-deployment.yaml >> kubectl create -f taskmanager-deployment.yaml >> kubectl create -f jobmanager-service.yaml >> >> The 2 taskmanagers fail. >> Output: >> >> Starting Task Manager >> config file: >> jobmanager.rpc.address: flink-jobmanager >> jobmanager.rpc.port: 6123 >> jobmanager.heap.size: 1024m >> taskmanager.heap.size: 1024m >> taskmanager.numberOfTaskSlots: 2 >> parallelism.default: 1 >> rest.port: 8081 >> blob.server.port: 6124 >> query.server.port: 6125 >> Starting taskexecutor as a console application on host >> flink-taskmanager-7679c9d55d-n2trk. >> 2018-12-19 11:42:45,216 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> >> -------------------------------------------------------------------------------- >> 2018-12-19 11:42:45,218 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Starting TaskManager (Version: 1.7.0, Rev:49da9f9, >> Date:28.11.2018 @ 17:59:06 UTC) >> 2018-12-19 11:42:45,218 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> OS current user: flink >> 2018-12-19 11:42:45,219 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> *Current Hadoop/Kerberos user: <no hadoop dependency found>* >> 2018-12-19 11:42:45,219 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.181-b13 >> 2018-12-19 11:42:45,219 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Maximum heap size: 922 MiBytes >> 2018-12-19 11:42:45,220 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> JAVA_HOME: /docker-java-home/jre >> 2018-12-19 11:42:45,220 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> No Hadoop Dependency available >> 2018-12-19 11:42:45,221 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> JVM Options: >> 2018-12-19 11:42:45,221 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - -XX:+UseG1GC >> 2018-12-19 11:42:45,221 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - -Xms922M >> 2018-12-19 11:42:45,221 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - -Xmx922M >> 2018-12-19 11:42:45,221 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - -XX:MaxDirectMemorySize=8388607T >> 2018-12-19 11:42:45,223 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - >> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties >> 2018-12-19 11:42:45,223 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - >> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml >> 2018-12-19 11:42:45,223 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Program Arguments: >> 2018-12-19 11:42:45,223 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - --configDir >> 2018-12-19 11:42:45,224 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner >> - /opt/flink/conf >> 2018-12-19 11:42:45,224 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Classpath: >> >> /opt/flink/lib/flink-python_2.12-1.7.0.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.15.jar:/opt/flink/lib/flink-dist_2.12-1.7.0.jar::: >> 2018-12-19 11:42:45,224 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> >> -------------------------------------------------------------------------------- >> 2018-12-19 11:42:45,228 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Registered UNIX signal handlers for [TERM, HUP, INT] >> 2018-12-19 11:42:45,233 INFO >> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - >> Maximum number of open file descriptors is 1048576. >> 2018-12-19 11:42:45,249 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: jobmanager.rpc.address, >> flink-jobmanager >> 2018-12-19 11:42:45,250 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: jobmanager.rpc.port, 6123 >> 2018-12-19 11:42:45,251 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: jobmanager.heap.size, 1024m >> 2018-12-19 11:42:45,251 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: taskmanager.heap.size, 1024m >> 2018-12-19 11:42:45,251 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: taskmanager.numberOfTaskSlots, 2 >> 2018-12-19 11:42:45,252 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: parallelism.default, 1 >> 2018-12-19 11:42:45,252 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: rest.port, 8081 >> 2018-12-19 11:42:45,254 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: blob.server.port, 6124 >> 2018-12-19 11:42:45,254 INFO >> org.apache.flink.configuration.GlobalConfiguration - >> Loading configuration property: query.server.port, 6125 >> *2018-12-19 11:42:45,261 INFO >> org.apache.flink.core.fs.FileSystem - >> Hadoop is not in the classpath/dependencies. The extended set of >> supported File Systems via Hadoop is not available. >> 2018-12-19 11:42:45,282 INFO >> org.apache.flink.runtime.security.modules.HadoopModuleFactory - >> Cannot create Hadoop Security Module because Hadoop cannot be >> found in the Classpath. >> 2018-12-19 11:42:45,311 INFO >> org.apache.flink.runtime.security.SecurityUtils - >> Cannot install HadoopSecurityContext because Hadoop cannot be >> found in the Classpath.* >> >> Any suggestions? Should I try maybe the Hadoop images? (I'm not >> planning to integrate with Hadoop) >> >> Thank you! >> || >
signature.asc
Description: OpenPGP digital signature
