On Sun, Jun 07, 2026 at 01:01:58AM +0200, Matija Nalis wrote: > All logs I've seen are those 5 lines included in original report > (which got logged in /var/log/lighttpd/error.log) and which do not > indicate why that attempt at restart failed.
When server.errorlog is set in lighttpd.conf, then during startup, lighttpd redirects STDERR file descriptor (2) to server.errorlog. That is the intended behavior. During a graceful lighttpd restart, server.errorlog is closed, which is why no further trace is sent to the error log after lighttpd shuts down, preparing to restart. When restarting, server.errorlog in lighttpd.conf might point to a new location, or if it points to the same location, the lighttpd still wants to close and re-open the log to allow for log file rotation. BTW, on startup, lighttpd opens logs after dropping privileges so that if lighttpd creates those log files, then lighttpd will be able to reopen those log files for log rotation without having to restart. > > If those do not provide enough details, then you can strace -p the > > lighttpd pid while you send a SIGUSR1 to the lighttpd process to trigger > > a graceful restart. Look at the system calls which fail. > > > Yes, sending it USR1 manually terminates lighttpd too. > > strace(1) reveals it is due to EPERM opening private key file for > https certificate. > > I guess originally it worked because lighttpd was still running as a > root at the time of system startup. So some suggestions: > > - when lighttpd decides to terminates due to some error, it should > log an error in its /var/log/lighttpd/error.log log. See above why this might be more complicated to do than it seems. lighttpd does try to log to STDERR, but in your environment, STDERR has been closed. While the running lighttpd process could theoretically perform an expensive reload config test, and then throw away that work, that could further delay restarting lighttpd if lighttpd config generation on someone's system included expensive include_shell script directives in lighttpd.conf. You're suggesting that lighttpd, a light and fast daemon, perform an expensive amount of work *every* restart to detect a situtation which fails on your system 100% of the time, i.e. could never worked due to your environment. Please read on for how others generally solve this. > Currently it seems it attempts to write the error only to STDERR > (which won't be seen by sysadmin, as the restart was not invoked > manually from shell but automatically): > > 1513 20:56:17 openat(AT_FDCWD, "/var/lib/acme/live/REDACTED/privkey", > O_RDONLY|O_NOCTTY|O_NONBLOCK|O_CLOEXEC) = -1 EACCES (Permission denied) > 1513 20:56:17 write(2, "2026-06-06 20:56:15: (fdevent.c.946) > fdevent_load_file() /var/lib/acme/live/REDACTED/privkey: Permission > denied\n", 127) = 127 > 1513 20:56:17 write(2, "2026-06-06 20:56:15: (server.c.1696) > Initialization of plugins failed. Going down.\n", 83) = 83 > > (additionally, FD 2 seems to have been /dev/null'ed at the time, so > the error message won't even hit the boot console terminal): > > # lsof -p 13002 | grep -w 2u > lighttpd 13002 www-data 2u CHR 1,3 0t0 4 /dev/null > > > - change of time should not cause the lighttpd to attempt to restart, > especially when it (could) know that restart is not going to be > successful. I think you may be overlooking the common (beneficial) side effects. If lighttpd were to not restart when requested, that could cause other issues, and some could be related to security. Generally speaking, if lighttpd fails to restart after shutting down, and instead exits, then systemd or sysvinit can be configured to restart the lighttpd process. This generally is the best way to recover from a permissions error at restart since the system daemon can restart lighttpd as root, lighttpd can read the privileged certs, and then lighttpd can drop privileges. lighttpd graceful restart become slightly less graceful, turning into a stop and start by the system, but then lighttpd continues serving requests, as before. This choice is not accidental. If restarting the lighttpd service when lighttpd exits is not what you are doing on your system, then I suggest that workaround. If you have any service that should be running, then if that service exits, the service should be restarted (with caveats about rate limiting to avoid restarting the service too quickly). > Is there a reason that time-change is a critical error requiring server > restart? Of course there is. At least one is known, and more could be unknown. lighttpd internally uses monotonic time whenever it can so that lighttpd is generally unaffected by time changes. However, there are libraries used by lighttpd, such as TLS libraries, which can be affected by time jumps. TLS 1.3 with SessionTicket fail for the first 8 hours of 1970 https://redmine.lighttpd.net/issues/3075 https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/d50d4dc0e557d582a0da4f3116ef1c4ab7a0bd78 https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/0ee964268b7476b68146a499c7b2f30f5c4fa08f > I'd prefer if lighttpd just logged an warning in such situation and > continue to function (perhaps make it an option if there are very > good reasons for restart, but it is not absolutely required?) There is already a configuration option: https://redmine.lighttpd.net/projects/lighttpd/wiki/Server_feature-flagsDetails server.feature-flags += ( "server.clock-jump-restart" => 1800 ) #default To disable: server.feature-flags += ( "server.clock-jump-restart" => 0 ) #disable > I do not want lighttpd attempting to restart auto-magically. > (and especially if there is any chance that restart might not work, > and lighttpd might terminate instead). Then please choose to use the configuration above to disable lighttpd restart when lighttpd detects a clock jump. > IOW, if I want it restarted, I want to do it manually (when I'm > ready and logged in to the server). > > Some other servers do safety check first to verify if they are very > certain that subsequent restart is going to successful, and attempt > to restart only in such case. That could be an alternative. Your sysvinit script to start lighttpd should already do this. If it does not, please file a bug with the operating system distribution and forward the bug to me so that I can ask them to add to their distro lighttpd package a lighttpd -tt config test before starting lighttpd. tl;dr: * configure your sysvinit to restart lighttpd service if lighttpd exits. * if you configure lighttpd.conf to disable restart on clock jump, then set up a log watcher to detect and alert you when there is a clock jump, so you can respond how you like. Cheers, Glenn
