ustcweizhou opened a new pull request #4869:
URL: https://github.com/apache/cloudstack/pull/4869
### Description
We had critical issue with VR recently. The VRs of shared network or vpc
stops working after some days.
After investigation, I found that the disk space is full
```
root@r-10-VM:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/vda5 2086316 2069932 0 100% /
```
logrotate/ryslog has compresses the log files, but space is not released.
see `lsof |grep deleted`
```
root@r-10-VM:~# lsof |grep deleted
rsyslogd 960 root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
rsyslogd 960 962 in:imuxso root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 962 in:imuxso root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 962 in:imuxso root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 962 in:imuxso root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
rsyslogd 960 963 in:imklog root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 963 in:imklog root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 963 in:imklog root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 963 in:imklog root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
rsyslogd 960 964 in:imfile root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 964 in:imfile root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 964 in:imfile root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 964 in:imfile root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
rsyslogd 960 965 in:imudp root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 965 in:imudp root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 965 in:imudp root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 965 in:imudp root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
rsyslogd 960 966 rs:main root 12w REG 254,5
493060096 137 /var/log/daemon.log.1 (deleted)
rsyslogd 960 966 rs:main root 13w REG 254,5
17715200 110 /var/log/messages.1 (deleted)
rsyslogd 960 966 rs:main root 16w REG 254,5
545968128 342 /var/log/auth.log.1 (deleted)
rsyslogd 960 966 rs:main root 18w REG 254,5
38313984 341 /var/log/cron.log.1 (deleted)
```
workaround: restarting rsyslog to release the space.
```
systemctl restart rsyslog
```
The root cause is, the following command does not work in 4.15 template
```
root@r-10-VM:~# invoke-rc.d rsyslog rotate
[FAIL] Closing open files: rsyslogd failed!
```
Fix: use `/usr/lib/rsyslog/rsyslog-rotate` instead
```
root@r-10-VM:~# /usr/lib/rsyslog/rsyslog-rotate
root@r-10-VM:~# cat /usr/lib/rsyslog/rsyslog-rotate
if [ -d /run/systemd/system ]; then
systemctl kill -s HUP rsyslog.service
else
invoke-rc.d rsyslog rotate > /dev/null
fi
```
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
### Feature/Enhancement Scale or Bug Severity
#### Bug Severity
- [ ] BLOCKER
- [X] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to
-->
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the
[CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md)
document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
