The spool files for my DA syslog forwarder queue have consumed all the inodes 
on my system and aren't being cleaned up by rsyslog:

[root@devhost ~]# df -i .
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/myvg-rootvol
                      971040  971040       0  100% /
[root@devhost ~]# cd /var/spool/rsyslog_spool
[root@devhost rsyslog_spool]# ls -l | wc -l
891185

Here's my queue index file:

[root@devhost rsyslog_spool]# cat aqfwd.qi
<OPB:1:qqueue:1:
+iQueueSize:2:2:34:
+tVars.disk.sizeOnDisk:2:5:22629:
+tVars.disk.bytesRead:2:3:499:
>End
.
<Obj:1:strm:1:
+iCurrFNum:2:2:64:
+pszFName:1:5:aqfwd:
+iMaxFiles:2:8:10000000:
+bDeleteOnClose:2:1:0:
+sType:2:1:1:
+tOperationsMode:2:1:2:
+tOpenMode:2:3:384:
+iCurrOffs:2:1:0:
>End
.
<Obj:1:strm:1:
+iCurrFNum:2:2:27:
+pszFName:1:5:aqfwd:
+iMaxFiles:2:8:10000000:
+bDeleteOnClose:2:1:1:
+sType:2:1:1:
+tOperationsMode:2:1:1:
+tOpenMode:2:3:384:
+iCurrOffs:2:3:499:
>End
.
[root@devhost rsyslog_spool]#

Rsyslog has open file descriptors to these files:

[root@devhost rsyslog_spool]# lsof -p `pgrep rsyslog` | grep rsyslog_spool
rsyslogd 1086 root    4w   REG              253,0      541     522935 
/var/spool/rsyslog_spool/aqfwd.00000064
rsyslogd 1086 root    5r   REG              253,0      499     529874 
/var/spool/rsyslog_spool/aqfwd.00000027
[root@devhost rsyslog_spool]#

[root@devhost rsyslog_spool]# ls -l /var/spool/rsyslog_spool/aqfwd.00000064 
/var/spool/rsyslog_spool/aqfwd.00000027
-rw-------. 1 root root 499 Feb 22 19:12 /var/spool/rsyslog_spool/aqfwd.00000027
-rw-------. 1 root root 541 Mar 11 17:26 /var/spool/rsyslog_spool/aqfwd.00000064
[root@devhost rsyslog_spool]#


Here's the action queue config:

# Forward messages to central log server.
$ActionQueueType LinkedList
$ActionQueueFileName aqfwd
$ActionQueueSize 150000
$ActionQueueLowWaterMark 30000          # 20% of QueueSize
$ActionQueueHighWaterMark 120000        # 80% of QueueSize
$ActionQueueSaveOnShutdown on
$ActionQueueDiscardSeverity 7           # Dont discard messages.
$ActionQueueMaxFileSize 10g             # Ridiculous limit, but dont want to 
lose messages :)
$ActionResumeRetryCount -1              # Keep trying
*.* @@syslog-server:10514;RSYSLOG_ForwardFormat # This the line that actually 
does forwarding


I thought maybe clearing up a few inodes (and a service restart) might give 
rsyslog enough breathing room to start cleaning up these files but that hasn't 
proven to be the case.

This is a RHEL6 machine running rsyslog-5.8.10-6.el6.x86_64.

If there's any additional information needed please let me know.  I still 
relatively new to rsyslog and I didn't write the config above so I can't answer 
any "why" questions about it.

Thanks in advance for any help!

-Sean



_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to