[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
All sessions appear in closing state, here's another one: $ loginctl show-session 77 Id=77 User=1004 Name=raccess Timestamp=Чт 2018-03-29 20:30:02 MSK TimestampMonotonic=2363185101 VTNr=0 Remote=yes RemoteHost=216.126.59.227 Service=sshd Scope=session-77.scope Leader=3389 Audit=77 Type=tty Class=user Active=yes State=closing IdleHint=no IdleSinceHint=0 IdleSinceHintMonotonic=0 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: Confirmed Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
I am having the same issue on Ubuntu 16.04 now. All sessions appear to be for one user that is used for incoming ssh connections for rsync. Here's an example: $ loginctl show-session 59 Id=59 User=1004 Name=raccess Timestamp=Чт 2018-03-29 20:20:03 MSK TimestampMonotonic=1763641802 VTNr=0 Remote=yes RemoteHost=216.126.59.227 Service=sshd Scope=session-59.scope Leader=2814 Audit=59 Type=tty Class=user Active=yes State=closing IdleHint=no IdleSinceHint=0 IdleSinceHintMonotonic=0 I don't see any processes running under user raccess apart from system ones: $ ps aux | grep raccess | grep -v grep raccess 2822 0.0 1.6 6384 4132 ?Ss 20:20 0:00 /lib/systemd/systemd --user raccess 2823 0.0 0.4 7316 1096 ?S20:20 0:00 (sd-pam) And here's an output of lsof -u raccess: https://pbin.cf/444b2d29 This is happening on two servers. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: Confirmed Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
This only seems to happen for us, on 14.04 LTS at least, if the package
'systemd' is installed rather than merely libsystemd-{daemon,login}0,
and systemd-{services,shim}. It could be that a workaround is simply to
remove the 'systemd' package, assuming it's not required for your
particular setup.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd-shim in Ubuntu.
https://bugs.launchpad.net/bugs/1426362
Title:
logind session files fill up /run space
Status in systemd-shim package in Ubuntu:
Confirmed
Bug description:
Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory
with files.
The files are 240-260 bytes, but there are many
As per default install, /run is on tmpfs with 50 MB size. After a couple of
days, /run/ runs out of free space, which can cause not so funny errors, like
failed kernel update (kernel postinst tries to create files here, then fails)
As I see, thesefiles are releated to ssh sesssions, and yes there are
plenty SSH connection to this host. Shouldn't systemd remove these old
session files?
# lsb_release -rd
Description:Ubuntu 14.10
Release:14.10
# apt-cache policy systemd
systemd:
Installed: 208-8ubuntu8.2
Candidate: 208-8ubuntu8.2
Version table:
*** 208-8ubuntu8.2 0
500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64
Packages
100 /var/lib/dpkg/status
208-8ubuntu8 0
500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
We have same issue :( -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: Confirmed Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: systemd-shim (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: Confirmed Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
I'm noticing this as an issue as well. My scenario is in using Fabric or Ansible to do deployments. A large number of these dangling SSH files are in the /run/systemd/sessions/ folder and eventually cause the "out of disk space" issue. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: New Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
** Package changed: systemd (Ubuntu) => systemd-shim (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd-shim in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd-shim package in Ubuntu: New Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd-shim/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
The server (vm) used as a management gateway to other service. Sessions opened by SSH. Remote machine opens a SSH connection with a key, by SSH force command, a python script opens, do the stuffs, and exits. Sessions are in 'closing' state, but no processes (not even zombies) are exist under this user. root@ad:/run/systemd/sessions# grep -r 'STATE' . | grep closing | wc -l 12419 root@ad:/run/systemd/sessions# grep -r 'STATE' . | grep -v closing | wc -l 1 root@ad:/run/systemd/sessions# loginctl list-users UID USER 1001 ad-sync-prod 0 root root@ad:/run/systemd/sessions# ps -u ad-sync-prod PID TTY TIME CMD root@ad:/run/systemd/sessions# loginctl user-status ad-sync-prod ad-sync-prod (1001) Since: Fri 2015-02-27 13:40:08 CET; 5 days ago State: opening Failed to query ControlGroup: No such interface 'org.freedesktop.systemd1.Slice' Sessions: 35752 35751 35750 35749 35748 35747 35746 35745 35744 35743 35742 35740 35739 35738 35737 35736 35735 35734 35733 35732 35731 35730 35729 35728 35726 35725 35724 35723 35722 Unit: user-1001.slice -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd package in Ubuntu: New Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
I. e. it should remove them as soon as the sessions are actually closed. Otherwise, if you log out, they merely go into state "closing", and wait until all remaining processes are going away. If you start screen sessions or other background jobs, they'll stay around for those. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd package in Ubuntu: New Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1426362] Re: logind session files fill up /run space
It should remove them, yes. I wonder if all these sessions are still open, i. e. are there any leftover processes in them? Can you please give me the output of "loginctl", and then "loginctl " for a few session IDs? (local ones look like "c2", remote ones should just be a number). ** Summary changed: - systemd session files fills up space + logind session files fill up /run space -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1426362 Title: logind session files fill up /run space Status in systemd package in Ubuntu: New Bug description: Under Ubuntu 14.10 (server), systemd fills up /run/systemd/sessions directory with files. The files are 240-260 bytes, but there are many As per default install, /run is on tmpfs with 50 MB size. After a couple of days, /run/ runs out of free space, which can cause not so funny errors, like failed kernel update (kernel postinst tries to create files here, then fails) As I see, thesefiles are releated to ssh sesssions, and yes there are plenty SSH connection to this host. Shouldn't systemd remove these old session files? # lsb_release -rd Description:Ubuntu 14.10 Release:14.10 # apt-cache policy systemd systemd: Installed: 208-8ubuntu8.2 Candidate: 208-8ubuntu8.2 Version table: *** 208-8ubuntu8.2 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages 100 /var/lib/dpkg/status 208-8ubuntu8 0 500 http://hu.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1426362/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
