Bug#710803: cron: systematically creates a temporary file in /tmp every hour
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-06-07 15:55, Saint Germain wrote: On 5 June 2013 23:34, Christian Kastner deb...@kvr.at wrote: On 2013-06-05 13:12, Saint Germain wrote: Indeed even if cron has nothing to do, a temporary file is always created in /tmp and immediatly deleted. So another workaround for you would be to comment out the hourly stanza from /etc/crontab. Any other solution would be overly complex to implement, I'm afraid. Ok I haven't carefully read the commit, so I didn't understand why this file creation was necessary in the first place (and why other distro don't have this behavior). To prevent data loss. Summarized: one of cron's features is the mailing of output. Regular cron connects to the MTA, runs the job and pipes its output to the MTA. However, some MTAs time out if they don't receive input within a certain amount of time, resulting in the output not being sent by mail, resulting in data loss (see bugs #155109, #443615). With the patch, we write to a tmpfile and only connect to the MTA when the job is finished, thereby preventing this possible data loss. If mounting /tmp on a tmpfs as a workaround is acceptable, probably that creating the temporary file directly in RAM would also be acceptable ? That came to my mind as a possible solution, too. The problem is we don't know how much RAM we might have available (think embedded systems), so consuming possibly large amounts of it periodicalyly may result in Bad Stuff(tm) happening to the system. An alternative could be to control this behaviour (pipe to MTA directly vs pipe to tmpfile) via a command-line flag. Hum, that is a very hot question that was discussed many many times: http://lwn.net/Articles/499534/ I think it is better to avoid discussing too much this particular hot topic /tmp on tmpfs. ;-) Good point :-) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlG0k6oACgkQwSyarY0epjXVlgCfS089H4fOvWeTWOQ7Wm53ALBv /B4AnjGRYT7jyXeLVw7nHx7UOcBGNrXA =seE+ -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
On Sunday, 9 June 2013, Christian Kastner deb...@kvr.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-06-07 15:55, Saint Germain wrote: On 5 June 2013 23:34, Christian Kastner deb...@kvr.at wrote: On 2013-06-05 13:12, Saint Germain wrote: Indeed even if cron has nothing to do, a temporary file is always created in /tmp and immediatly deleted. So another workaround for you would be to comment out the hourly stanza from /etc/crontab. Any other solution would be overly complex to implement, I'm afraid. Ok I haven't carefully read the commit, so I didn't understand why this file creation was necessary in the first place (and why other distro don't have this behavior). To prevent data loss. Summarized: one of cron's features is the mailing of output. Regular cron connects to the MTA, runs the job and pipes its output to the MTA. However, some MTAs time out if they don't receive input within a certain amount of time, resulting in the output not being sent by mail, resulting in data loss (see bugs #155109, #443615). With the patch, we write to a tmpfile and only connect to the MTA when the job is finished, thereby preventing this possible data loss. If mounting /tmp on a tmpfs as a workaround is acceptable, probably that creating the temporary file directly in RAM would also be acceptable ? That came to my mind as a possible solution, too. The problem is we don't know how much RAM we might have available (think embedded systems), so consuming possibly large amounts of it periodicalyly may result in Bad Stuff(tm) happening to the system. An alternative could be to control this behaviour (pipe to MTA directly vs pipe to tmpfile) via a command-line flag. OK got it ! Thanks for having taken the time to explain. A command-line flag would be ideal, I agree. Best regards,
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
Hi Christian,
From memory, I believe debian already patches cron in one other way to
avoid spinning up laptop drives (I believe it's related to stat()ing
/etc/cron.d to check for updated files?). I believe we should aim to
fix this here, too. I'm attaching a very lightly tested patch. Note
this also fixes a bug in the existing cron tmpfile patch regarding
fwrite()ing of the tmpfile buffer to the popen()ed mail command. In
the case of an incomplete write(), the bug causes the beginning of the
buffer to be written again, rather than resuming where it left off,
essentially resulting in data loss off the last portion of the buffer.
--- ../orig/do_command.c2013-06-09 15:37:45.0 -0700
+++ ./do_command.c 2013-06-09 16:37:36.0 -0700
@@ -127,7 +127,7 @@
user*u;
{
int stdin_pipe[2];
- FILE*tmpout;
+ int stdout_pipe[2];
register char *input_data;
char*usernm, *mailto;
int children = 0;
@@ -191,12 +191,8 @@
/* create a pipe to talk to our future child
*/
pipe(stdin_pipe); /* child's stdin */
- /* child's stdout */
- if ((tmpout = tmpfile()) == NULL) {
- log_it(CRON, getpid(), error, create tmpfile);
- exit(ERROR_EXIT);
- }
-
+ pipe(stdout_pipe); /* child's stdout */
+
/* since we are a forked process, we can diddle the command string
* we were passed -- nobody else is going to use it again, right?
*
@@ -284,6 +280,7 @@
* appropriate circumstances.
*/
close(stdin_pipe[WRITE_PIPE]);
+ close(stdout_pipe[READ_PIPE]);
/* grandchild process. make std{in,out} be the ends of
* pipes opened by our daddy; make stderr go to stdout.
@@ -291,14 +288,15 @@
/* Closes are unnecessary -- let dup2() do it */
/* close(STDIN) */; dup2(stdin_pipe[READ_PIPE], STDIN);
- dup2(fileno(tmpout), STDOUT);
/* close(STDERR)*/; dup2(STDOUT, STDERR);
-
/* close the pipe we just dup'ed. The resources will remain.
*/
close(stdin_pipe[READ_PIPE]);
- fclose(tmpout);
+
+ // dup2(fileno(tmpout), STDOUT);
+ dup2(stdout_pipe[WRITE_PIPE], STDOUT);
+ close(stdout_pipe[WRITE_PIPE]);
/* set our login universe. Do this in the grandchild
* so that the child can invoke /usr/lib/sendmail
@@ -404,6 +402,7 @@
* grandchild process...
*/
close(stdin_pipe[READ_PIPE]);
+ close(stdout_pipe[WRITE_PIPE]);
/*
* write, to the pipe connected to child's stdin, any input specified
@@ -465,8 +464,8 @@
children++;
/*
-* read output from the grandchild. it's stderr has been redirected to
-* it's stdout, which has been redirected to our pipe. if there is any
+* read output from the grandchild. its stderr has been redirected to
+* its stdout, which has been redirected to our pipe. if there is any
* output, we'll be mailing it to the user whose crontab this is...
* when the grandchild exits, we'll get EOF.
*/
@@ -506,23 +505,50 @@
}
}
+// Read the output of the command, save it into a tempfile. This
+// doesn't write to a tempfile directly to avoid creating tempfiles
+// and spinning up laptop drives, even when only run-parts is the
+// only, empty job run.
+
+ FILE *tmpout;
+ char buf[19];
+ int ret, remain;
+
+ // if no output, done:
+ if (read(stdout_pipe[READ_PIPE], buf, 1)==0)
+ goto mail_finished;
+
+ // else collect it all in tmpfile:
+ if ((tmpout = tmpfile()) == NULL) {
+ log_it(CRON, getpid(), error, create tmpfile);
+ goto mail_finished;
+ }
+
+ putc(*buf, tmpout);
+ for ( ; ret=read(stdout_pipe[READ_PIPE], buf, sizeof(buf)); ) {
+ char *pos;
+ for (pos=buf, remain=ret; remain!=0; ) {
+ int x=fwrite(buf, 1, remain, tmpout);
+ if (x0) {
+ remain -= ret;
+ pos+=ret;
+ }
+ }
+ }
+
+
// Finally, send any output of the command to the mailer; also, alert
// the user if their job failed. Avoid popening the mailcmd until now
// since sendmail may time out, and to write info about the exit
// status.
-
- long pos;
+ register FILE *mail = NULL;
+ register intbytes = 1;
+
struct stat mcsb;
int statret;
- fseek(tmpout, 0, SEEK_END);
- pos = ftell(tmpout);
- fseek(tmpout, 0,
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
On 5 June 2013 23:34, Christian Kastner deb...@kvr.at wrote:
On 2013-06-05 13:12, Saint Germain wrote:
Indeed even if cron has nothing to do, a temporary file is
always created in /tmp and immediatly deleted.
nothing to do = there is no job (cron.hourly is empty and users
cron are empty)
However it would more elegant for cron to skip this file creation
in case there is no job to execute.
Ah, but there is one job running: run-parts.
The scripts in cron.{hourly,daily,weekly,monthly} are not processed by
the cron daemon directly. Instead, the system crontab /etc/crontab
contains schedules for calls to run-parts /etc/cron.[interval], and
run-parts then processes those scripts. From cron's POV, there is
always exactly 1 process being run.
So another workaround for you would be to comment out the hourly
stanza from /etc/crontab.
Any other solution would be overly complex to implement, I'm afraid.
Ok I haven't carefully read the commit, so I didn't understand why
this file creation was necessary in the first place (and why other
distro don't have this behavior).
If mounting /tmp on a tmpfs as a workaround is acceptable, probably
that creating the temporary file directly in RAM would also be
acceptable ?
Anyway, as you are the maintainer, I think you are much knowledgeable
than me to decide what is best for Debian.
This is only a minor bug though, but I spent quite a long time
tracking it (especially given that only Debian users have this bug)
so I thought that it may be useful to report it.
I agree, thank you.
I can of course mount /tmp on a tmpfs, but it seems to me that it
would be overkill (as cron is the only software I have which
behaves like this). A better workaround would be for me to simply
comment the line cron.hourly in /etc/crontab (as I don't have any
hourly job).
(Off-topic, personal opinion: having a tmpfs on /tmp has many other
benefits, eg: there is no need to clean the filesystem on boot. Most of
the stuff in /tmp is probably going to be cached, ie in RAM, anyway)
Hum, that is a very hot question that was discussed many many times:
http://lwn.net/Articles/499534/
I think it is better to avoid discussing too much this particular hot
topic /tmp on tmpfs. ;-)
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 It seems that the following Debian commit which introduces a temporary file has an unfortunate side-effect: http://anonscm.debian.org/gitweb/?p=pkg-cron/pkg- cron.git;a=commitdiff;h=e61050367866fb1b627e2c004d370914eced70cb Indeed even if cron has nothing to do, a temporary file is always created in /tmp and immediatly deleted. I am trying to reduce the power consumption of my server and this patch had the effect of waking-up the hard-drive every hour. Maybe it is possible to prevent the temporary file creation if cron has nothing to do ? With nothing to do, do mean no jobs are run or jobs do not create output? In any case, a simple solution would be to have a tmpfs on /tmp. This is trivial to do with Wheezy (see /etc/default/tmpfs), and still easy with Squeeze. Christian -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlGu//wACgkQwSyarY0epjX6JgCeNJ9KM7vpudKvueQ+EBERNpQo jeIAnjNXziel3gQ+H2sHRqiR7FOPxijW =5oHu -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
On 5 June 2013 11:08, Christian Kastner deb...@kvr.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 It seems that the following Debian commit which introduces a temporary file has an unfortunate side-effect: http://anonscm.debian.org/gitweb/?p=pkg-cron/pkg- cron.git;a=commitdiff;h=e61050367866fb1b627e2c004d370914eced70cb Indeed even if cron has nothing to do, a temporary file is always created in /tmp and immediatly deleted. I am trying to reduce the power consumption of my server and this patch had the effect of waking-up the hard-drive every hour. Maybe it is possible to prevent the temporary file creation if cron has nothing to do ? With nothing to do, do mean no jobs are run or jobs do not create output? In any case, a simple solution would be to have a tmpfs on /tmp. This is trivial to do with Wheezy (see /etc/default/tmpfs), and still easy with Squeeze. nothing to do = there is no job (cron.hourly is empty and users cron are empty) I can of course mount /tmp on a tmpfs, but it seems to me that it would be overkill (as cron is the only software I have which behaves like this). A better workaround would be for me to simply comment the line cron.hourly in /etc/crontab (as I don't have any hourly job). However it would more elegant for cron to skip this file creation in case there is no job to execute. This is only a minor bug though, but I spent quite a long time tracking it (especially given that only Debian users have this bug) so I thought that it may be useful to report it. Best regards, -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 2013-06-05 13:12, Saint Germain wrote:
Indeed even if cron has nothing to do, a temporary file is
always created in /tmp and immediatly deleted.
I am trying to reduce the power consumption of my server and
this patch had the effect of waking-up the hard-drive every
hour.
Maybe it is possible to prevent the temporary file creation if
cron has nothing to do ?
With nothing to do, do mean no jobs are run or jobs do not
create output?
nothing to do = there is no job (cron.hourly is empty and users
cron are empty)
However it would more elegant for cron to skip this file creation
in case there is no job to execute.
Ah, but there is one job running: run-parts.
The scripts in cron.{hourly,daily,weekly,monthly} are not processed by
the cron daemon directly. Instead, the system crontab /etc/crontab
contains schedules for calls to run-parts /etc/cron.[interval], and
run-parts then processes those scripts. From cron's POV, there is
always exactly 1 process being run.
So another workaround for you would be to comment out the hourly
stanza from /etc/crontab.
Any other solution would be overly complex to implement, I'm afraid.
This is only a minor bug though, but I spent quite a long time
tracking it (especially given that only Debian users have this bug)
so I thought that it may be useful to report it.
I agree, thank you.
I can of course mount /tmp on a tmpfs, but it seems to me that it
would be overkill (as cron is the only software I have which
behaves like this). A better workaround would be for me to simply
comment the line cron.hourly in /etc/crontab (as I don't have any
hourly job).
(Off-topic, personal opinion: having a tmpfs on /tmp has many other
benefits, eg: there is no need to clean the filesystem on boot. Most of
the stuff in /tmp is probably going to be cached, ie in RAM, anyway)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAlGvrsoACgkQwSyarY0epjXFaQCg3PL4cROPSawxFfaRzHDGXUJl
X9cAoI4EgvV4HH4z9K61+wkYcXARj9kF
=P7UF
-END PGP SIGNATURE-
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#710803: cron: systematically creates a temporary file in /tmp every hour
Package: cron Version: 3.0pl1-124 Severity: minor Dear Maintainer, It seems that the following Debian commit which introduces a temporary file has an unfortunate side-effect: http://anonscm.debian.org/gitweb/?p=pkg-cron/pkg- cron.git;a=commitdiff;h=e61050367866fb1b627e2c004d370914eced70cb Indeed even if cron has nothing to do, a temporary file is always created in /tmp and immediatly deleted. I am trying to reduce the power consumption of my server and this patch had the effect of waking-up the hard-drive every hour. Maybe it is possible to prevent the temporary file creation if cron has nothing to do ? Best regards -- Package-specific info: --- EDITOR: --- /usr/bin/editor: /bin/nano --- /usr/bin/crontab: -rwxr-sr-x 1 root crontab 34760 Jul 3 2012 /usr/bin/crontab --- /var/spool/cron: drwxr-xr-x 5 root root 4096 Nov 1 2010 /var/spool/cron --- /var/spool/cron/crontabs: drwx-wx--T 2 root crontab 4096 Oct 16 2010 /var/spool/cron/crontabs --- /etc/cron.d: drwxr-xr-x 2 root root 4096 May 4 20:56 /etc/cron.d --- /etc/cron.daily: drwxr-xr-x 2 root root 4096 May 15 21:43 /etc/cron.daily --- /etc/cron.hourly: drwxr-xr-x 2 root root 4096 Aug 16 2012 /etc/cron.hourly --- /etc/cron.monthly: drwxr-xr-x 2 root root 4096 Aug 16 2012 /etc/cron.monthly --- /etc/cron.weekly: drwxr-xr-x 2 root root 4096 May 15 21:42 /etc/cron.weekly -- System Information: Debian Release: jessie/sid APT prefers testing APT policy: (900, 'testing'), (500, 'oldstable'), (90, 'experimental') Architecture: i386 (i686) Kernel: Linux 3.2.0-2-686-pae (SMP w/4 CPU cores) Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages cron depends on: ii adduser 3.113+nmu3 ii debianutils 4.3.4 ii dpkg1.16.10 ii libc6 2.13-38 ii libpam-runtime 1.1.3-9 ii libpam0g1.1.3-9 ii libselinux1 2.1.9-5 ii lsb-base4.1+Debian9 Versions of packages cron recommends: ii exim4 4.80-7 ii exim4-daemon-light [mail-transport-agent] 4.80-7 Versions of packages cron suggests: ii anacron2.3-19 pn checksecurity none ii logrotate 3.8.1-4 Versions of packages cron is related to: pn libnss-ldap none pn libnss-ldapd none pn libpam-ldap none pn libpam-mount none pn nis none pn nscd none -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
