Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Hi, Tom, thanks for your work on this! On Donnerstag, 27. August 2009, Stig Sandbeck Mathisen wrote: Munin and munin-node creates a /var/log/munin directory, perhaps creating /var/log/munin/munin-node-configure.log is a good idea, and send STDOUT and STDERR there? That would be a place I'd look for logs, if something went wrong. sounds good to me. regards, Holger signature.asc Description: This is a digitally signed message part.
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Hi,
Could you please review the attached patch? (It's written against
the svn rev: 2375).
This patch does the following:
* It removes the deletion of the TMPFILE that caused all plugins not
to be activated, in case one of them raised an error.
* It logs the STDOUT and STDERR to /var/log/munin/munin-node-configure.log
The log format is the same as the rest of the munin logs in terms of
date formatting. postinst logs the STDOUT first, and only in case STDERR
was found, it logs the errors. In order to seperate STDOUT STDERR I had
to use an additional TMPFILE, and I also added it to be removed at the
end of the process.
* Reduced a little bit of duplication in the init_plugins() function.
Regarding filtering the STDOUT for '^ln ' lines, I thought about this and I
think this might cause a problem in case munin-node-configure changes in the
future to output additional commands besides 'ln'. As we know that
munin-node-configure seperates the errors to STDERR and that when given the
--shell option it only outputs relevant commands to STDOUT, I think it's safe
to continue with the way things are today (without filtering ln).
Here's the log that's resulting in installing munin-node with the patch
on my machine:
/var/log/munin/munin-node-configure.log:
---
Aug 27 09:40:23 - Starting munin-node-configure --shell
ln -s /usr/share/munin/plugins/cpu /etc/munin/plugins/cpu
ln -s /usr/share/munin/plugins/entropy /etc/munin/plugins/entropy
ln -s /usr/share/munin/plugins/exim_mailqueue /etc/munin/plugins/exim_mailqueue
ln -s /usr/share/munin/plugins/exim_mailstats /etc/munin/plugins/exim_mailstats
ln -s /usr/share/munin/plugins/forks /etc/munin/plugins/forks
ln -s /usr/share/munin/plugins/if_ /etc/munin/plugins/if_eth0
ln -s /usr/share/munin/plugins/if_err_ /etc/munin/plugins/if_err_eth0
ln -s /usr/share/munin/plugins/interrupts /etc/munin/plugins/interrupts
ln -s /usr/share/munin/plugins/iostat /etc/munin/plugins/iostat
ln -s /usr/share/munin/plugins/irqstats /etc/munin/plugins/irqstats
ln -s /usr/share/munin/plugins/load /etc/munin/plugins/load
ln -s /usr/share/munin/plugins/memory /etc/munin/plugins/memory
ln -s /usr/share/munin/plugins/nfs_client /etc/munin/plugins/nfs_client
ln -s /usr/share/munin/plugins/open_files /etc/munin/plugins/open_files
ln -s /usr/share/munin/plugins/open_inodes /etc/munin/plugins/open_inodes
ln -s /usr/share/munin/plugins/processes /etc/munin/plugins/processes
ln -s /usr/share/munin/plugins/swap /etc/munin/plugins/swap
ln -s /usr/share/munin/plugins/vmstat /etc/munin/plugins/vmstat
The following errors were reported by munin-node-configure --shell
# There were some errors:
# Got junk from df_inode: df: no file systems processed
# Got junk from df_inode: Error executing df. Exit code 256
# Got junk from df: df: no file systems processed
# Got junk from df: Error executing df. Exit code 256
---
Thanks,
Tom
Index: munin-node.postinst
===
--- munin-node.postinst (revision 2375)
+++ munin-node.postinst (working copy)
@@ -28,25 +28,35 @@
init_plugins() {
TMPFILE=`mktemp /tmp/munin-node.configure.XX`
+ TMPFILE_STDERR=`mktemp /tmp/munin-node.configure.err.XX`
+ MUNIN_NODE_CONF_LOG=/var/log/munin/munin-node-configure.log
if [ $prevver ]; then
+ MUNIN_NODE_CMD=munin-node-configure --shell --newer ${prevver%-*}
echo -n Initializing new plugins..
- munin-node-configure --shell --newer ${prevver%-*} $TMPFILE || rm -f $TMPFILE
- if [ -f $TMPFILE ] ; then
- sh $TMPFILE
- else
- echo failed.
- fi
else
+ MUNIN_NODE_CMD=munin-node-configure --shell
echo -n Initializing plugins..
- munin-node-configure --shell $TMPFILE || rm -f $TMPFILE
- if [ -f $TMPFILE ] ; then
- sh $TMPFILE
- else
- echo failed.
- fi
+fi
+
+ # munin-node-conf returns 1 if case of partial plugin autoconf
+ # errors. We need to ignore these errors as even one plugin
+ # can fail the entire process. See Debian bug #539886 for details.
+ $MUNIN_NODE_CMD 2$TMPFILE_STDERR $TMPFILE || true
+ if [ -f $TMPFILE ]; then
+ DATE=`date '+%b %d %T'`
+ echo $DATE - Starting $MUNIN_NODE_CMD $MUNIN_NODE_CONF_LOG
+ cat $TMPFILE $MUNIN_NODE_CONF_LOG
+if [ -s $TMPFILE_STDERR ]; then
+ echo The following errors were reported by $MUNIN_NODE_CMD \
+ $MUNIN_NODE_CONF_LOG
+ cat $TMPFILE_STDERR $MUNIN_NODE_CONF_LOG
+fi
+ sh $TMPFILE
+ else
+ echo failed.
fi
echo done.
- rm -f $TMPFILE
+ rm -f $TMPFILE $TMPFILE_STDERR
echo -n Restarting munin-node..
invoke-rc.d munin-node restart
}
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
I've been looking at this bug, and from what I can see here's the problem: The command munin-node-configure (supplied by upstream) when executed with the --shell command line option, goes over each available plugin from /usr/share/munin/plugins/ with autoconf as the first parameter, which basically asks each available plugin whether or not it is capable of running on the current system. For plugins that are able to run, a symbolic link command is printed to STDOUT, and for plugins that are not able to run, a shell remark is printed along with the error. debian/munin-node.postinst is running the following command: munin-node-configure --shell and in case this command returns a non-zero exit code, it does not run the list of symbolic link commands, and thus no plugins are activated after munin-node package plugin. In that case, debian/munin-node.postinst should still apply the list of printed commands, even if some plugins are not able to run on the system, as the output of the plugins that can be installed is still applicable. For example, on my system, munin-node-configure --shell returns the following output: --- ln -s /usr/share/munin/plugins/cpu /etc/munin/plugins/cpu ln -s /usr/share/munin/plugins/entropy /etc/munin/plugins/entropy ln -s /usr/share/munin/plugins/forks /etc/munin/plugins/forks ln -s /usr/share/munin/plugins/if_ /etc/munin/plugins/if_eth0 # There were some errors: # Got junk from df_inode: df: no file systems processed # Got junk from postgres_space_: Can't locate DBI.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at /usr/share/munin/plugins/postgres_space_ line 74. # Got junk from postgres_space_: BEGIN failed--compilation aborted at /usr/share/munin/plugins/postgres_space_ line 74. Even though the command returned a non-zero exit code, this output can still be ran as a valid shell script which activates the compatible plugins for the system, while the ones that returned an error will be skipped as they are commented. As you can see, this is not a df spesific problem, I guess the bug title should be changed to something more generic like: munin-node: maintainer scripts fails to create default set of enabled plugins due to plugin autoconf error The only problem in fixing this is that I'm not sure what's the correct place for the fix: * Should munin-node-configure be patched to return an exit 0 in case of partial errors? * Or should debian/munin-node.postinst be changed to check if munin-node-configure returns either 0 (success) or 1 (partial plugins failure) and continue even if munin-node-configure returned exit code 1? -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Thanks for a good analysis. It matches my view of the issue. I think the correct place to fix this is in munin-node-configure, or in the surrounding code in the munin-node.postinst script. It (and the plugins) has since been rewritten upstream in a more sane manner to run the plugins with autoconf as argument, and use exit codes from each plugin to figure out if the plugin should be installed or not, and the output from stdout used for --suggest --shell symlink hints. In this case, one plugin that exits with a bad return code stops all processing. We should allow a plugin to return with a non-zero return code, and ignore the plugin (but log the output) if so happens. It is possible this may be backported from the upstream munin-node-configure, unless it has been rewritten too much. Attempting to autoconfigure the rest of the plugins may still produce sensible result. -- Stig Sandbeck Mathisen ooo, shiny! -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Hi Stig, Thanks for the quick reply. I'd be happy to write the patch either in the postinst or munin-node-configure. I'll have a look at munin-node-configure from the 1.3 trunk to see how much it has changed to see if it's sensible to try and backport it. (if not we can always do it in the postinst). You mention that we should log the errors from plugins that fail their autoconf. Where should this be logged into? Currently the postinst simply echo's the errors to STDOUT. Thanks, Tom Feiner -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Tom Feiner feiner@gmail.com writes: Thanks for the quick reply. I'd be happy to write the patch either in the postinst or munin-node-configure. A patch would be welcome. I'll have a look at munin-node-configure from the 1.3 trunk to see how much it has changed to see if it's sensible to try and backport it. (if not we can always do it in the postinst). I'd say postinst would be the place for a nice quick-n-dirty hack. :) Most of the things in trunk (but not necessarily yet in branches/1.3) has been completely rewritten using a Munin perl module hierarchy. The plugin autoconfiguration methods have also changed to the better, where these problems should not occur. There seems to be a lot of activity on munin now that people are back from their summer vacations. :) In the munin-node postinst, we should be able to ignore the exit status of munin-node-configure, and check the lines in the output file, and send all lines with ^ln foo bar to the shell. That's a quick and dirty hack. Not pretty, but I think it'll bypass the problem, which is that configuration in postinst stops if munin-node-configure exits non-zero. You mention that we should log the errors from plugins that fail their autoconf. Where should this be logged into? Currently the postinst simply echo's the errors to STDOUT. I'm not sure. I like having logs, and I like having them at predictable locations. I certainly don't want to clutter the dpkg output with it, it'll probably just be ignored on upgrade. Munin and munin-node creates a /var/log/munin directory, perhaps creating /var/log/munin/munin-node-configure.log is a good idea, and send STDOUT and STDERR there? That would be a place I'd look for logs, if something went wrong. -- Stig Sandbeck Mathisen ooo, shiny! -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Package: munin-node Version: 1.2.6-10~lenny1 Severity: normal $ aptitude install munin-node # Got junk from df_inode: df: `/var/lib/postgresql/8.4/main/base': Permission denied # Got junk from df_inode: Error executing df. Exit code 256 # Got junk from df: df: `/var/lib/postgresql/8.4/main/base': Permission denied # Got junk from df: Error executing df. Exit code 256 if you have a mountpoint which is not accessible by munin user, df plugin fails with an error message and no symlinks are created by maintainer script # ls -ald /var/lib/postgresql/8.4/main/base/ drwx-- 8 postgres postgres 4.0K 2009-08-03 12:51 /var/lib/postgresql/8.4/main/base/ # mount | grep postgres /dev/md1 on /var/lib/postgresql/8.4/main/base type ext3 (rw,noatime,data=writeback) # sudo -u munin df Filesystem 1K-blocks Used Available Use% Mounted on df: `/var/lib/postgresql/8.4/main/base': Permission denied the error should be handled and the maintainer script should go on to install other plugins at the moment no plugins are installed, just because df fails on one mount point. ls -al /etc/munin/plugins/ total 8.0K drwxr-xr-x 2 root root 4.0K 2009-03-13 14:10 . drwxr-xr-x 4 root root 4.0K 2009-08-04 10:33 .. kind regards Janning -- System Information: Debian Release: 5.0.2 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.26-2-amd64 (SMP w/8 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages munin-node depends on: ii adduser 3.110add and remove users and groups ii gawk1:3.1.5.dfsg-4.1 GNU awk, a pattern scanning and pr ii libnet-server-perl 0.97-1 An extensible, general perl server ii lsb-base3.2-20 Linux Standard Base 3.2 init scrip ii perl5.10.0-19Larry Wall's Practical Extraction ii procps 1:3.2.7-11 /proc file system utilities Versions of packages munin-node recommends: ii libnet-snmp-perl 5.2.0-1Script SNMP connections Versions of packages munin-node suggests: pn acpi | lm-sensors none (no description available) pn ethtool none (no description available) ii libdbd-pg-perl2.8.7-1Perl DBI driver for the PostgreSQL pn liblwp-useragent-determined-p none (no description available) pn libnet-irc-perl none (no description available) pn libwww-perl none (no description available) pn munin none (no description available) pn munin-plugins-extra none (no description available) pn mysql-client none (no description available) ii python2.5.2-3An interactive high-level object-o pn smartmontools none (no description available) -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#539886: munin-node: maintainer scripts fails to create default set of enabled plugins due to df error
Hi Janning, On Dienstag, 4. August 2009, janning wrote: the error should be handled and the maintainer script should go on to install other plugins at the moment no plugins are installed, just because df fails on one mount point. I agree, thanks for your bug report! regards, Holger signature.asc Description: This is a digitally signed message part.
