fwiw, I've seen this type of error in the past when the system runs out of ephemeral ports. Not saying this definitely the same issue, but I suggest checking to see if you have ephemeral ports available.
On Thu, May 3, 2018 at 8:57 AM, Zhitao Li <zhitaoli...@gmail.com> wrote: > Can you paste the command line of how you started the Mesos agent process? > > On Wed, May 2, 2018 at 9:21 PM, Luke Adolph <kenan3...@gmail.com> wrote: > >> Hi all: >> When mesos slave run task, the stderr file shows >> I0503 04:01:20.488590 9110 logging.cpp:188] INFO level logging started! >> I0503 04:01:20.489073 9110 fetcher.cpp:424] Fetcher Info: >> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/2bcc032f- >> 950b-4c36-bff4-b5552c193dc9-S1\/root","items":[{"action":" >> BYPASS_CACHE","uri":{"extract":true,"value":"file:\/\/\/etc\ >> /.dockercfg"}}],"sandbox_directory":"\/tmp\/mesos\/ >> slaves\/2bcc032f-950b-4c36-bff4-b5552c193dc9-S1\/docker\/ >> links\/b4eabcbb-5769-49f0-9324-b25c3cda8b8c","user":"root"} >> I0503 04:01:20.491297 9110 fetcher.cpp:379] Fetching URI >> 'file:///etc/.dockercfg' >> I0503 04:01:20.491325 9110 fetcher.cpp:250] Fetching directly into the >> sandbox directory >> I0503 04:01:20.491348 9110 fetcher.cpp:187] Fetching URI >> 'file:///etc/.dockercfg' >> I0503 04:01:20.491367 9110 fetcher.cpp:167] Copying resource with >> command:cp '/etc/.dockercfg' '/tmp/mesos/slaves/2bcc032f-95 >> 0b-4c36-bff4-b5552c193dc9-S1/docker/links/b4eabcbb-5769-49f >> 0-9324-b25c3cda8b8c/.dockercfg' >> W0503 04:01:20.495400 9110 fetcher.cpp:272] Copying instead of >> extracting resource from URI with 'extract' flag, because it does not seem >> to be an archive: file:///etc/.dockercfg >> I0503 04:01:20.495728 9110 fetcher.cpp:456] Fetched >> 'file:///etc/.dockercfg' to '/tmp/mesos/slaves/2bcc032f-95 >> 0b-4c36-bff4-b5552c193dc9-S1/docker/links/b4eabcbb-5769-49f >> 0-9324-b25c3cda8b8c/.dockercfg' >> F0503 04:01:21.990416 9202 process.cpp:889] Failed to initialize: Failed >> to bind on 0.0.0.0:0: Address already in use: Address already in use [98] >> *** Check failure stack trace: *** >> @ 0x7f95fc6ef86d google::LogMessage::Fail() >> @ 0x7f95fc6f169d google::LogMessage::SendToLog() >> @ 0x7f95fc6ef45c google::LogMessage::Flush() >> @ 0x7f95fc6ef669 google::LogMessage::~LogMessage() >> @ 0x7f95fc6f05d2 google::ErrnoLogMessage::~ErrnoLogMessage() >> @ 0x7f95fc6955d9 process::initialize() >> @ 0x7f95fc696be2 process::ProcessBase::ProcessBase() >> @ 0x430e9a mesos::internal::docker::Docke >> rExecutorProcess::DockerExecutorProcess() >> @ 0x41916b main >> @ 0x7f95fa60ff45 (unknown) >> @ 0x419c77 (unknown) >> >> When mesos slave initialize, it runs into "Failed to bind on 0.0.0.0:0: >> Address already in use", I run `netstat -nlp`, But there is no port "0" is >> used, full output is >> root@10:~# netstat -nlp >> Active Internet connections (only servers) >> Proto Recv-Q Send-Q Local Address Foreign Address >> State PID/Program name >> tcp 0 0 0.0.0.0:22 0.0.0.0:* >> LISTEN 1153/sshd >> tcp 0 0 0.0.0.0:37786 0.0.0.0:* >> LISTEN 20042/mesos-docker- >> tcp 0 0 0.0.0.0:5051 0.0.0.0:* >> LISTEN 12701/mesos-slave >> tcp 0 0 0.0.0.0:37084 0.0.0.0:* >> LISTEN 19765/mesos-docker- >> tcp 0 0 0.0.0.0:24220 0.0.0.0:* >> LISTEN 28584/ruby >> tcp 0 0 0.0.0.0:8765 0.0.0.0:* >> LISTEN 28353/nginx >> tcp 0 0 0.0.0.0:24224 0.0.0.0:* >> LISTEN 28584/ruby >> tcp 0 0 127.0.0.1:24225 0.0.0.0:* >> LISTEN 28584/ruby >> tcp 0 0 0.0.0.0:46690 0.0.0.0:* >> LISTEN 28932/mesos-docker- >> tcp 0 0 0.0.0.0:42437 0.0.0.0:* >> LISTEN 32184/mesos-docker- >> tcp 0 0 0.0.0.0:34695 0.0.0.0:* >> LISTEN 25862/mesos-docker- >> tcp 0 0 0.0.0.0:37039 0.0.0.0:* >> LISTEN 21273/mesos-docker- >> tcp 0 0 0.0.0.0:46001 0.0.0.0:* >> LISTEN 710/mesos-docker-ex >> tcp6 0 0 :::31765 :::* >> LISTEN 20160/docker-proxy >> tcp6 0 0 :::31605 :::* >> LISTEN 20149/docker-proxy >> tcp6 0 0 :::31327 :::* >> LISTEN 820/docker-proxy >> tcp6 0 0 :::31008 :::* >> LISTEN 32291/docker-proxy >> tcp6 0 0 :::2375 :::* >> LISTEN 28305/node >> tcp6 0 0 :::31690 :::* >> LISTEN 25966/docker-proxy >> tcp6 0 0 :::31211 :::* >> LISTEN 21379/docker-proxy >> tcp6 0 0 :::31245 :::* >> LISTEN 19988/docker-proxy >> tcp6 0 0 :::31121 :::* >> LISTEN 29037/docker-proxy >> udp 0 0 0.0.0.0:24224 0.0.0.0:* >> 28584/ruby >> udp 0 0 192.168.0.1:123 0.0.0.0:* >> 1348/ntpd >> udp 0 0 59.110.24.56:123 0.0.0.0:* >> 1348/ntpd >> udp 0 0 10.25.141.251:123 0.0.0.0:* >> 1348/ntpd >> udp 0 0 127.0.0.1:123 0.0.0.0:* >> 1348/ntpd >> udp 0 0 0.0.0.0:123 0.0.0.0:* >> 1348/ntpd >> udp6 0 0 :::123 :::* >> 1348/ntpd >> Active UNIX domain sockets (only servers) >> Proto RefCnt Flags Type State I-Node PID/Program >> name Path >> unix 2 [ ACC ] STREAM LISTENING 15907 >> 1424/AliYunDun /tmp/Aegis-<Guid(5A2C30A2-A87D >> -490A-9281-6765EDAD7CBA)> >> unix 2 [ ACC ] STREAM LISTENING 178254587 28237/python >> /tmp/supervisor.sock.9 >> unix 2 [ ACC ] STREAM LISTENING 178256956 19165/uwsgi >> /ddserver/ddserver.sock >> unix 2 [ ACC ] STREAM LISTENING 229374127 17729/xin >> /var/lib/alauda/xin/xin.sock >> unix 2 [ ACC ] STREAM LISTENING 15609 >> 983/dbus-daemon /var/run/dbus/system_bus_socket >> unix 2 [ ACC ] STREAM LISTENING 8117 1/init >> @/com/ubuntu/upstart >> unix 2 [ ACC ] STREAM LISTENING 178252397 >> 27819/dockerd /var/run/docker.sock >> unix 2 [ ACC ] STREAM LISTENING 178252401 >> 27855/docker-contai /var/run/docker/libcontainerd/docker-containerd.sock >> unix 2 [ ACC ] STREAM LISTENING 14557 1282/nscd >> /var/run/nscd/socket >> unix 2 [ ACC ] STREAM LISTENING 178252469 >> 27819/dockerd /run/docker/libnetwork/2dc9ef >> 136eb6933a5634dcf439d9fbd4acf96a5de16eb79122b6ddd8aefcf81c.sock >> unix 2 [ ACC ] STREAM LISTENING 178253543 28184/node >> /root/.forever/sock/worker.1524559297488oBZ.sock >> unix 2 [ ACC ] STREAM LISTENING 15908 >> 1424/AliYunDun /usr/local/aegis/Aegis-<Guid(5 >> A2C30A2-A87D-490A-9281-6765EDAD7CBA)> >> unix 2 [ ACC ] SEQPACKET LISTENING 8173 >> 416/systemd-udevd /run/udev/control >> >> So where is the problem? Thanks. >> > > > > -- > Cheers, > > Zhitao Li >