Public bug reported:
We're running in to a case that's very similar to
https://bugs.launchpad.net/dhcp/+bug/1872118 where the DHCP server
periodically crashes. Crashes sometimes take days to happen on
relatively busy DHCP servers. There's not an obvious reason, however the
logs end up showing (hostnames and email address changed slightly to
keep meaning, but not disclose everything).
May 14 05:22:17 is-landlord-04.wm.edu sh[305109]:
../../../../lib/isc/unix/socket.c:4359: fatal error: select() failed: Bad file
descriptor
May 14 05:22:27 hostname.wm.edu systemd[1]: isc-dhcp-server.service: Main
process exited, code=killed, status=6/ABRT
May 14 05:22:32 is-landlord-04.wm.edu sSMTP[1645996]: Sent mail for
nore...@wm.edu (221 2.0.0 Service closing transmission channel) uid=0
username=root ou>
May 14 05:22:32 is-landlord-04.wm.edu systemd[1]: isc-dhcp-server.service:
Failed with result 'signal'.
May 14 05:22:37 is-landlord-04.wm.edu systemd[1]: isc-dhcp-server.service:
Scheduled restart job, restart counter is at 25.
May 14 05:22:37 is-landlord-04.wm.edu systemd[1]: Stopped ISC DHCP IPv4 server.
May 14 05:22:37 is-landlord-04.wm.edu systemd[1]: Started ISC DHCP IPv4 server.
$ sudo cat _usr_sbin_dhcpd.0.crash
ProblemType: Crash
Architecture: amd64
Date: Fri May 14 05:22:21 2021
DistroRelease: Ubuntu 20.04
ExecutablePath: /usr/sbin/dhcpd
ExecutableTimestamp: 1615303185
ProcCmdline: dhcpd -user dhcpd -group dhcpd -f -4 -pf
/run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf
ProcEnviron: Error: [Errno 13] Permission denied: 'environ'
ProcMaps: Error: [Errno 13] Permission denied: 'maps'
ProcStatus:
Name: dhcpd
Umask: 0022
State: D (disk sleep)
Tgid: 305109
Ngid: 0
Pid: 305109
PPid: 1
TracerPid: 0
Uid: 116 116 116 116
Gid: 119 119 119 119
FDSize:256
Groups:
NStgid:305109
NSpid: 305109
NSpgid:305109
NSsid: 305109
VmPeak: 422372 kB
VmSize: 356836 kB
VmLck:0 kB
VmPin:0 kB
VmHWM: 289404 kB
VmRSS: 258028 kB
RssAnon: 255560 kB
RssFile: 2468 kB
RssShmem: 0 kB
VmData: 282632 kB
VmStk: 132 kB
VmExe: 592 kB
VmLib: 5352 kB
VmPTE: 584 kB
VmSwap:2496 kB
HugetlbPages: 0 kB
CoreDumping: 1
THP_enabled: 1
Threads: 4
SigQ: 0/3521
SigPnd:
ShdPnd:
SigBlk:
SigIgn:1000
SigCgt:00018000
CapInh:
CapPrm:
CapEff:
CapBnd:003f
CapAmb:
NoNewPrivs:0
Seccomp: 0
Speculation_Store_Bypass: thread vulnerable
Cpus_allowed: ,
Cpus_allowed_list: 0-63
Mems_allowed:
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0001
Mems_allowed_list: 0
voluntary_ctxt_switches: 94
nonvoluntary_ctxt_switches:831
Signal: 6
Uname: Linux 5.4.0-71-generic x86_64
UserGroups: N/A
Because of the similarities with #1872118 we started by checking various
package versions:
$ sudo dpkg -l | grep -i isc-
ii isc-dhcp-client 4.4.1-2.1ubuntu5.20.04.1
amd64DHCP client for automatically obtaining an IP address
ii isc-dhcp-common 4.4.1-2.1ubuntu5.20.04.1
amd64common manpages relevant to all of the isc-dhcp packages
ii isc-dhcp-server 4.4.1-2.1ubuntu5.20.04.1
amd64ISC DHCP server for automatic IP address assignment
ii libisc-export1105:amd64 1:9.11.16+dfsg-3~ubuntu1
amd64Exported ISC Shared Library
$ sudo dpkg -l | grep -i bind9-libs
ii bind9-libs:amd64 1:9.16.1-0ubuntu2.8
amd64Shared Libraries used by BIND 9
$ sudo dpkg -l | grep -i libisc-*
ii libisc-export1105:amd64 1:9.11.16+dfsg-3~ubuntu1
amd64Exported ISC Shared Library
ii libisccfg-export163 1:9.11.16+dfsg-3~ubuntu1
amd64Exported ISC CFG Shared Library
$ sudo dpkg -l | grep -i libirs-*
ii libirs-export161 1:9.11.16+dfsg-3~ubuntu1
amd64Exported IRS Shared Library
$ sudo dpkg -l | grep -i libdns-*
ii libdns-export11091:9.11.16+dfsg-3~ubuntu1
amd64Exported DNS Shared Library
Finally, we're running Ubuntu 20.04.2:
$ lsb_release -rd
Description:Ubuntu 20.04.2 LTS
Release:20.04
Let us know what else can be provided to help confirm or troubleshoot
the issue.
** Affects: isc-dhcp (Ubuntu)
Importance: Undecided
Status:
