All the org-*-idle-seconds func test at some point for org-x11idle-exists-p which is defvar-ed at load time.
For me, org-x11idle-exists-p is always nil at startup, but is set to true if I eval it later on. I guess it's because I'm starting Emacs in daemon mode, (as a systemd user service actually), and org-x11idle-exists-p relies on (eq window-system 'x) which is somehow not set in the terminal when the daemon. For the time being, it manually set org-x11idle-exists-p to true but shouldn't all this tests on (eq-window-system) be run each time a new frame is created ? -- Julien Cubizolles