Hi, Harshith As a supplementary note to Yang, the issue seems to be that something went wrong when trying to connect to the ResourceManager. There may be two possibilities, the leader of ResourceManager does not write the znode or the TaskExecutor fails to connect to it. If you turn on the DEBUG log, it will help a lot. Also, you could watch the content znode "/leader/resource_manager_lock" of ZooKeeper.
Best, Yangze Guo On Fri, Jan 17, 2020 at 5:11 PM Yang Wang <danrtsey...@gmail.com> wrote: > > Hi Kumar Bolar, Harshith, > > Could you please check the jobmanager log to find out what address the akka > is listening? > Also the address could be used to connected to the jobmanager on the > taskmanger machine. > > BTW, if you could share the debug level logs of jobmanger and taskmanger. It > will help a lot to find > the root cause. > > > Best, > Yang > > Kumar Bolar, Harshith <hk...@arity.com> 于2020年1月16日周四 下午7:10写道： >> >> Hi all, >> >> >> >> We were previously using RHEL for our Flink machines. I'm currently working >> on moving them over to Ubuntu. When I start the task manager, it fails to >> connect to the job manager with the following message - >> >> >> >> 2020-01-16 10:54:42,777 INFO >> org.apache.flink.runtime.util.LeaderRetrievalUtils - Trying to >> select the network interface and address to use by connecting to the leading >> JobManager. >> >> 2020-01-16 10:54:42,778 INFO >> org.apache.flink.runtime.util.LeaderRetrievalUtils - TaskManager >> will try to connect for 10000 milliseconds before falling back to heuristics >> >> 2020-01-16 10:54:52,780 WARN org.apache.flink.runtime.net.ConnectionUtils >> - Could not find any IPv4 address that is not loopback or >> link-local. Using localhost address. >> >> >> >> The network interface on the machine looks like this - >> >> >> >> >> >> ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9001 >> >> inet 10.16.75.30 netmask 255.255.255.128 broadcast 10.16.75.127 >> >> ether 02:f1:8b:34:75:51 txqueuelen 1000 (Ethernet) >> >> RX packets 69370 bytes 80369110 (80.3 MB) >> >> RX errors 0 dropped 0 overruns 0 frame 0 >> >> TX packets 28787 bytes 2898540 (2.8 MB) >> >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> >> >> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 >> >> inet 127.0.0.1 netmask 255.0.0.0 >> >> loop txqueuelen 1000 (Local Loopback) >> >> RX packets 9562 bytes 1596138 (1.5 MB) >> >> RX errors 0 dropped 0 overruns 0 frame 0 >> >> TX packets 9562 bytes 1596138 (1.5 MB) >> >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> >> >> >> >> Note: On RHEL, the primary network interface was eth0. Could this be the >> issue? >> >> >> >> Here's the full task manager log - https://paste.ubuntu.com/p/vgh96FHzRq/ >> >> >> >> Thanks >> >> Harshith
