Re: Race between cron and crontab

2012-02-03 Thread John Baldwin
On Thursday, February 02, 2012 8:03:12 pm Doug Barton wrote:
 On 02/01/2012 04:42, John Baldwin wrote:
  On Tuesday, January 31, 2012 9:23:12 pm Doug Barton wrote:
  On 01/31/2012 08:49, John Baldwin wrote:
  A co-worker ran into a race between updating a cron tab via crontab(8) 
  and 
  cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
  was 
  updated.  The problem is that 1) by default our filesystems only use 
  second 
  granularity for timestamps and 2) cron only caches the seconds portion of 
  a 
  file's timestamp when checking for changes anyway.  This means that cron 
  can 
  miss updates to a spool directory if multiple updates to the directory 
  are 
  performed within a single second and cron wakes up to scan the spool 
  directory 
  within the same second and scans it before all of the updates are 
  complete.
 
  Specifically, when replacing a crontab, crontab(8) first creates a 
  temporary 
  file in /var/cron/tabs and then uses a rename to install it followed by 
  touching the spool directory to update its modification time.  However, 
  the 
  creation of the temporary file already changes the modification time of 
  the 
  directory, and cron may miss the rename if it scans the directory in 
  between 
  the creation of the temporary file and the rename.
 
  The fix I am planning to use locally is to simply force crontab(8) to 
  sleep 
  for a second before it touches the spool directory, thus ensuring that it 
  the 
  touch of the spool directory will use a later modification time than the 
  creation of the temporary file.
 
  If you really want cron to have sub-second granularity I don't see how
  you could do it without using flags.
 
  crontab open   sets flag that it is editing a file
  crontab close  clears editing flag, sets something changed flag
 (if something actually changed of course)
 
  cron   checks existence of something changed flag, pulls the
 update if there is no editing flag, clears changed flag
  
  I don't want it to have sub-second granularity,
 
 Ok, I was interpolating, sorry if I misinterpreted your intentions.
 
  I just want to make
  'crontab -e' more reliable so that cron doesn't miss edits.  cron is
  currently using the mod-time of the spool directory as the 'something
  changed' flag (have you read the cron code?).
 
 I understand the spool behavior from history/experience, and I am
 relying on your excellent summary for the details. :)
 
  The problem is that it
  currently can set the 'something changed' flag non-atomically while it is
  updating a crontab.
 
 That much I understood from your post. My response to what it is I think
 you're trying to achieve is that it's not likely that you can achieve it
 by only using 1 flag, no matter what that 1 flag is. I may be wrong
 about that, but hopefully my suggestion gives you some other ideas to
 consider.

Eh, I think the mod-time on the directory is fine.  It will change anything
anything in that directory changes, so that is good enough, and it's ok to
lose the race in the direction of doing extra scans of the spool directory,
it just needs to avoid not doing enough scans.

 Meanwhile, I was thinking more about this and TMK cron doesn't actually
 *run* jobs with seconds granularity, only minutes, right?  If so then it
 seems that the only really important seconds to care about are :59 and
 :00. That would seem to present a solution that rather than having cron
 wake up every second to see if something has changed that it only do
 that at :59 (or however many seconds in advance of :00 that it needs,
 although if it's more than 1 I'll be surprised). That limits the race to
 someone who writes out a new crontab entry at the point during second
 :59 that is after cron wakes up to look but before :00. So that's not a
 perfect solution to your problem, but it should limit the race to a very
 narrow window without having to modify the code very much.

Hmm, did you see my original e-mail?  I have a one-line fix to crontab(8). :)
Do you think that patch is invalid?

Also, the race is not what you think it is: cron already only checks once a
minute, the problem is a crontab(8) replace happening concurrently with
cron's scan of the spool directory in such a way that cron only sees part of
the actions crontab(8) performs while updating a tab.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Race between cron and crontab

2012-02-02 Thread Doug Barton
On 02/01/2012 04:42, John Baldwin wrote:
 On Tuesday, January 31, 2012 9:23:12 pm Doug Barton wrote:
 On 01/31/2012 08:49, John Baldwin wrote:
 A co-worker ran into a race between updating a cron tab via crontab(8) and 
 cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
 was 
 updated.  The problem is that 1) by default our filesystems only use second 
 granularity for timestamps and 2) cron only caches the seconds portion of a 
 file's timestamp when checking for changes anyway.  This means that cron 
 can 
 miss updates to a spool directory if multiple updates to the directory are 
 performed within a single second and cron wakes up to scan the spool 
 directory 
 within the same second and scans it before all of the updates are complete.

 Specifically, when replacing a crontab, crontab(8) first creates a 
 temporary 
 file in /var/cron/tabs and then uses a rename to install it followed by 
 touching the spool directory to update its modification time.  However, the 
 creation of the temporary file already changes the modification time of the 
 directory, and cron may miss the rename if it scans the directory in 
 between 
 the creation of the temporary file and the rename.

 The fix I am planning to use locally is to simply force crontab(8) to 
 sleep 
 for a second before it touches the spool directory, thus ensuring that it 
 the 
 touch of the spool directory will use a later modification time than the 
 creation of the temporary file.

 If you really want cron to have sub-second granularity I don't see how
 you could do it without using flags.

 crontab open sets flag that it is editing a file
 crontab closeclears editing flag, sets something changed flag
  (if something actually changed of course)

 cron checks existence of something changed flag, pulls the
  update if there is no editing flag, clears changed flag
 
 I don't want it to have sub-second granularity,

Ok, I was interpolating, sorry if I misinterpreted your intentions.

 I just want to make
 'crontab -e' more reliable so that cron doesn't miss edits.  cron is
 currently using the mod-time of the spool directory as the 'something
 changed' flag (have you read the cron code?).

I understand the spool behavior from history/experience, and I am
relying on your excellent summary for the details. :)

 The problem is that it
 currently can set the 'something changed' flag non-atomically while it is
 updating a crontab.

That much I understood from your post. My response to what it is I think
you're trying to achieve is that it's not likely that you can achieve it
by only using 1 flag, no matter what that 1 flag is. I may be wrong
about that, but hopefully my suggestion gives you some other ideas to
consider.

Meanwhile, I was thinking more about this and TMK cron doesn't actually
*run* jobs with seconds granularity, only minutes, right?  If so then it
seems that the only really important seconds to care about are :59 and
:00. That would seem to present a solution that rather than having cron
wake up every second to see if something has changed that it only do
that at :59 (or however many seconds in advance of :00 that it needs,
although if it's more than 1 I'll be surprised). That limits the race to
someone who writes out a new crontab entry at the point during second
:59 that is after cron wakes up to look but before :00. So that's not a
perfect solution to your problem, but it should limit the race to a very
narrow window without having to modify the code very much.


hth,

Doug

-- 

It's always a long day; 86400 doesn't fit into a short.

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Race between cron and crontab

2012-02-02 Thread Ian Lepore
On Thu, 2012-02-02 at 17:03 -0800, Doug Barton wrote:
 On 02/01/2012 04:42, John Baldwin wrote:
  On Tuesday, January 31, 2012 9:23:12 pm Doug Barton wrote:
  On 01/31/2012 08:49, John Baldwin wrote:
  A co-worker ran into a race between updating a cron tab via crontab(8) 
  and 
  cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
  was 
  updated.  The problem is that 1) by default our filesystems only use 
  second 
  granularity for timestamps and 2) cron only caches the seconds portion of 
  a 
  file's timestamp when checking for changes anyway.  This means that cron 
  can 
  miss updates to a spool directory if multiple updates to the directory 
  are 
  performed within a single second and cron wakes up to scan the spool 
  directory 
  within the same second and scans it before all of the updates are 
  complete.
 
  Specifically, when replacing a crontab, crontab(8) first creates a 
  temporary 
  file in /var/cron/tabs and then uses a rename to install it followed by 
  touching the spool directory to update its modification time.  However, 
  the 
  creation of the temporary file already changes the modification time of 
  the 
  directory, and cron may miss the rename if it scans the directory in 
  between 
  the creation of the temporary file and the rename.
 
  The fix I am planning to use locally is to simply force crontab(8) to 
  sleep 
  for a second before it touches the spool directory, thus ensuring that it 
  the 
  touch of the spool directory will use a later modification time than the 
  creation of the temporary file.
 
  If you really want cron to have sub-second granularity I don't see how
  you could do it without using flags.
 
  crontab open   sets flag that it is editing a file
  crontab close  clears editing flag, sets something changed flag
 (if something actually changed of course)
 
  cron   checks existence of something changed flag, pulls the
 update if there is no editing flag, clears changed flag
  
  I don't want it to have sub-second granularity,
 
 Ok, I was interpolating, sorry if I misinterpreted your intentions.
 
  I just want to make
  'crontab -e' more reliable so that cron doesn't miss edits.  cron is
  currently using the mod-time of the spool directory as the 'something
  changed' flag (have you read the cron code?).
 
 I understand the spool behavior from history/experience, and I am
 relying on your excellent summary for the details. :)
 
  The problem is that it
  currently can set the 'something changed' flag non-atomically while it is
  updating a crontab.
 
 That much I understood from your post. My response to what it is I think
 you're trying to achieve is that it's not likely that you can achieve it
 by only using 1 flag, no matter what that 1 flag is. I may be wrong
 about that, but hopefully my suggestion gives you some other ideas to
 consider.
 
 Meanwhile, I was thinking more about this and TMK cron doesn't actually
 *run* jobs with seconds granularity, only minutes, right?  If so then it
 seems that the only really important seconds to care about are :59 and
 :00. That would seem to present a solution that rather than having cron
 wake up every second to see if something has changed that it only do
 that at :59 (or however many seconds in advance of :00 that it needs,
 although if it's more than 1 I'll be surprised). That limits the race to
 someone who writes out a new crontab entry at the point during second
 :59 that is after cron wakes up to look but before :00. So that's not a
 perfect solution to your problem, but it should limit the race to a very
 narrow window without having to modify the code very much.
 
 
 hth,
 
 Doug

I think part of the problem here is that I started typing without
thinking enough... I thought I was offering a different small change
that fixed more than one problem, but it was wrong.  My mistake seems to
be having the unintended effect of sidetracking a perfectly reasonable
small fix for a problem that's known to happen in the real world, just
because it doesn't also fix a theoretical problem that might happen
somewhere some day; that sure wasn't my intention.

-- Ian


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Race between cron and crontab

2012-02-01 Thread John Baldwin
On Tuesday, January 31, 2012 9:23:12 pm Doug Barton wrote:
 On 01/31/2012 08:49, John Baldwin wrote:
  A co-worker ran into a race between updating a cron tab via crontab(8) and 
  cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
  was 
  updated.  The problem is that 1) by default our filesystems only use second 
  granularity for timestamps and 2) cron only caches the seconds portion of a 
  file's timestamp when checking for changes anyway.  This means that cron 
  can 
  miss updates to a spool directory if multiple updates to the directory are 
  performed within a single second and cron wakes up to scan the spool 
  directory 
  within the same second and scans it before all of the updates are complete.
  
  Specifically, when replacing a crontab, crontab(8) first creates a 
  temporary 
  file in /var/cron/tabs and then uses a rename to install it followed by 
  touching the spool directory to update its modification time.  However, the 
  creation of the temporary file already changes the modification time of the 
  directory, and cron may miss the rename if it scans the directory in 
  between 
  the creation of the temporary file and the rename.
  
  The fix I am planning to use locally is to simply force crontab(8) to 
  sleep 
  for a second before it touches the spool directory, thus ensuring that it 
  the 
  touch of the spool directory will use a later modification time than the 
  creation of the temporary file.
 
 If you really want cron to have sub-second granularity I don't see how
 you could do it without using flags.
 
 crontab open  sets flag that it is editing a file
 crontab close clears editing flag, sets something changed flag
   (if something actually changed of course)
 
 cron  checks existence of something changed flag, pulls the
   update if there is no editing flag, clears changed flag

I don't want it to have sub-second granularity, I just want to make
'crontab -e' more reliable so that cron doesn't miss edits.  cron is
currently using the mod-time of the spool directory as the 'something
changed' flag (have you read the cron code?).  The problem is that it
currently can set the 'something changed' flag non-atomically while it is
updating a crontab.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Race between cron and crontab

2012-01-31 Thread Ian Lepore
On Tue, 2012-01-31 at 11:49 -0500, John Baldwin wrote:
 A co-worker ran into a race between updating a cron tab via crontab(8) and 
 cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab was 
 updated.  The problem is that 1) by default our filesystems only use second 
 granularity for timestamps and 2) cron only caches the seconds portion of a 
 file's timestamp when checking for changes anyway.  This means that cron can 
 miss updates to a spool directory if multiple updates to the directory are 
 performed within a single second and cron wakes up to scan the spool 
 directory 
 within the same second and scans it before all of the updates are complete.
 
 Specifically, when replacing a crontab, crontab(8) first creates a temporary 
 file in /var/cron/tabs and then uses a rename to install it followed by 
 touching the spool directory to update its modification time.  However, the 
 creation of the temporary file already changes the modification time of the 
 directory, and cron may miss the rename if it scans the directory in 
 between 
 the creation of the temporary file and the rename.
 
 The fix I am planning to use locally is to simply force crontab(8) to sleep 
 for a second before it touches the spool directory, thus ensuring that it the 
 touch of the spool directory will use a later modification time than the 
 creation of the temporary file.
 
 Note that crontab -r is not affected by this race as it only does one atomic 
 update to the directory (unlink()).
 
 Index: crontab.c
 ===
 --- crontab.c (revision 225431)
 +++ crontab.c (working copy)
 @@ -604,6 +604,15 @@ replace_cmd() {
  
   log_it(RealUser, Pid, REPLACE, User);
  
 + /*
 +  * Creating the 'tn' temp file has already updated the
 +  * modification time of the spool directory.  Sleep for a
 +  * second to ensure that poke_daemon() sets a later
 +  * modification time.  Otherwise, this can race with the cron
 +  * daemon scanning for updated crontabs.
 +  */
 + sleep(1);
 +
   poke_daemon();
  
   return (0);

Maybe this is overly pedantic, but that solution still allows the
possibility of the same sort of race if a user updates their crontab in
the same second as an admin saves a new /etc/crontab, because cron takes
the max timestamp of /etc/crontab and /var/cron/tabs and compares it
against the database-rebuild timestamp.

A possible solution on the daemon side of things might be something like
the attached, but I should state (nay, shout) that I haven't looked
beyond these few lines to see if there are any unintended side effects
to such a change.

-- Ian

diff -r eb5f4971de86 usr.sbin/cron/cron/database.c
--- usr.sbin/cron/cron/database.c	Fri Jan 20 16:12:15 2012 -0700
+++ usr.sbin/cron/cron/database.c	Tue Jan 31 10:48:32 2012 -0700
@@ -72,7 +72,7 @@ load_database(old_db)
 	 * so is guaranteed to be different than the stat() mtime the first
 	 * time this function is called.
 	 */
-	if (old_db-mtime == TMAX(statbuf.st_mtime, syscron_stat.st_mtime)) {
+	if (old_db-mtime  TMAX(statbuf.st_mtime, syscron_stat.st_mtime)) {
 		Debug(DLOAD, ([%d] spool dir mtime unch, no load needed.\n,
 			  getpid()))
 		return;
@@ -83,7 +83,7 @@ load_database(old_db)
 	 * actually changed.  Whatever is left in the old database when
 	 * we're done is chaff -- crontabs that disappeared.
 	 */
-	new_db.mtime = TMAX(statbuf.st_mtime, syscron_stat.st_mtime);
+	new_db.mtime = 1 + TMAX(statbuf.st_mtime, syscron_stat.st_mtime);
 	new_db.head = new_db.tail = NULL;
 
 	if (syscron_stat.st_mtime) {
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Race between cron and crontab

2012-01-31 Thread John Baldwin
On Tuesday, January 31, 2012 12:57:50 pm Ian Lepore wrote:
 On Tue, 2012-01-31 at 11:49 -0500, John Baldwin wrote:
  A co-worker ran into a race between updating a cron tab via crontab(8) and 
  cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
  was 
  updated.  The problem is that 1) by default our filesystems only use second 
  granularity for timestamps and 2) cron only caches the seconds portion of a 
  file's timestamp when checking for changes anyway.  This means that cron 
  can 
  miss updates to a spool directory if multiple updates to the directory are 
  performed within a single second and cron wakes up to scan the spool 
  directory 
  within the same second and scans it before all of the updates are complete.
  
  Specifically, when replacing a crontab, crontab(8) first creates a 
  temporary 
  file in /var/cron/tabs and then uses a rename to install it followed by 
  touching the spool directory to update its modification time.  However, the 
  creation of the temporary file already changes the modification time of the 
  directory, and cron may miss the rename if it scans the directory in 
  between 
  the creation of the temporary file and the rename.
  
  The fix I am planning to use locally is to simply force crontab(8) to 
  sleep 
  for a second before it touches the spool directory, thus ensuring that it 
  the 
  touch of the spool directory will use a later modification time than the 
  creation of the temporary file.
  
  Note that crontab -r is not affected by this race as it only does one 
  atomic 
  update to the directory (unlink()).
  
  Index: crontab.c
  ===
  --- crontab.c   (revision 225431)
  +++ crontab.c   (working copy)
  @@ -604,6 +604,15 @@ replace_cmd() {
   
  log_it(RealUser, Pid, REPLACE, User);
   
  +   /*
  +* Creating the 'tn' temp file has already updated the
  +* modification time of the spool directory.  Sleep for a
  +* second to ensure that poke_daemon() sets a later
  +* modification time.  Otherwise, this can race with the cron
  +* daemon scanning for updated crontabs.
  +*/
  +   sleep(1);
  +
  poke_daemon();
   
  return (0);
 
 Maybe this is overly pedantic, but that solution still allows the
 possibility of the same sort of race if a user updates their crontab in
 the same second as an admin saves a new /etc/crontab, because cron takes
 the max timestamp of /etc/crontab and /var/cron/tabs and compares it
 against the database-rebuild timestamp.

Hmm, I'm not sure I see the race in that case.  If the /etc/crontab file
matches the timestamp of the spool directory before the utimes() call
after the one-second sleep, then it will still rescan it on the next
check when it notices a newer timestamp on the spool directory.  If
it is the same timestamp as the second timestamp on the spool directory,
then the scan is guaranteed to have not started before that second began,
meaning that the crontab(8) process editing the user's crontab must have
passed the rename, so the scan will see the user's new crontab.

 A possible solution on the daemon side of things might be something like
 the attached, but I should state (nay, shout) that I haven't looked
 beyond these few lines to see if there are any unintended side effects
 to such a change.

I think this patch doesn't change anything at all actually.  It is 
certainly subject to the original race I described if you do not use
the patch in crontab(8) itself.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Race between cron and crontab

2012-01-31 Thread Ian Lepore
On Tue, 2012-01-31 at 13:30 -0500, John Baldwin wrote:
 On Tuesday, January 31, 2012 12:57:50 pm Ian Lepore wrote:
  On Tue, 2012-01-31 at 11:49 -0500, John Baldwin wrote:
   A co-worker ran into a race between updating a cron tab via crontab(8) 
   and 
   cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab 
   was 
   updated.  The problem is that 1) by default our filesystems only use 
   second 
   granularity for timestamps and 2) cron only caches the seconds portion of 
   a 
   file's timestamp when checking for changes anyway.  This means that cron 
   can 
   miss updates to a spool directory if multiple updates to the directory 
   are 
   performed within a single second and cron wakes up to scan the spool 
   directory 
   within the same second and scans it before all of the updates are 
   complete.
   
   Specifically, when replacing a crontab, crontab(8) first creates a 
   temporary 
   file in /var/cron/tabs and then uses a rename to install it followed by 
   touching the spool directory to update its modification time.  However, 
   the 
   creation of the temporary file already changes the modification time of 
   the 
   directory, and cron may miss the rename if it scans the directory in 
   between 
   the creation of the temporary file and the rename.
   
   The fix I am planning to use locally is to simply force crontab(8) to 
   sleep 
   for a second before it touches the spool directory, thus ensuring that it 
   the 
   touch of the spool directory will use a later modification time than the 
   creation of the temporary file.
   
   Note that crontab -r is not affected by this race as it only does one 
   atomic 
   update to the directory (unlink()).
   
   Index: crontab.c
   ===
   --- crontab.c (revision 225431)
   +++ crontab.c (working copy)
   @@ -604,6 +604,15 @@ replace_cmd() {

 log_it(RealUser, Pid, REPLACE, User);

   + /*
   +  * Creating the 'tn' temp file has already updated the
   +  * modification time of the spool directory.  Sleep for a
   +  * second to ensure that poke_daemon() sets a later
   +  * modification time.  Otherwise, this can race with the cron
   +  * daemon scanning for updated crontabs.
   +  */
   + sleep(1);
   +
 poke_daemon();

 return (0);
  
  Maybe this is overly pedantic, but that solution still allows the
  possibility of the same sort of race if a user updates their crontab in
  the same second as an admin saves a new /etc/crontab, because cron takes
  the max timestamp of /etc/crontab and /var/cron/tabs and compares it
  against the database-rebuild timestamp.
 
 Hmm, I'm not sure I see the race in that case.  If the /etc/crontab file
 matches the timestamp of the spool directory before the utimes() call
 after the one-second sleep, then it will still rescan it on the next
 check when it notices a newer timestamp on the spool directory.  If
 it is the same timestamp as the second timestamp on the spool directory,
 then the scan is guaranteed to have not started before that second began,
 meaning that the crontab(8) process editing the user's crontab must have
 passed the rename, so the scan will see the user's new crontab.
 
  A possible solution on the daemon side of things might be something like
  the attached, but I should state (nay, shout) that I haven't looked
  beyond these few lines to see if there are any unintended side effects
  to such a change.
 
 I think this patch doesn't change anything at all actually.  It is 
 certainly subject to the original race I described if you do not use
 the patch in crontab(8) itself.
 

You're right about my patch not fixing anything; I didn't think hard
enough before I started typing.  

But I think the problem I was trying to get at with /etc/crontab still
exists with your patch; it would be triggered if a user updated their
crontab and after the 1 second sleep the directory timestamp gets
updated and cron rebuilds the database, and then right after that but
still in the same second /etc/crontab gets written.  (That's why I was
trying, however feebly, to move the solution into the daemon.)

Maybe the simple answer is for admins to be sure not to save changes
to /etc/crontab during the xx:xx:00 second, or with your patch, :01.
(I'm kidding, of course.  The fact that cron likes to wake up at the top
of minute is no g'tee that it actually does so every time.)

Once you start thinking along the no g'tee lines you realize that two
users can be updating their tabs concurrently, and one arrives in
poke_daemon() and grabs the current time (let's say it's noon) then gets
preempted for a long while before calling utimes(), the other user runs
through poke_daemon() and updates the directory timestamp to 12:00:01,
then the first process gets some cycles again and re-updates the
timestamp to 12:00:00.  Ooopsie, we've achieved a complex and annoying
proof of the idea that 

Re: Race between cron and crontab

2012-01-31 Thread John Baldwin
On Tuesday, January 31, 2012 3:13:34 pm Ian Lepore wrote:
 On Tue, 2012-01-31 at 13:30 -0500, John Baldwin wrote:
  On Tuesday, January 31, 2012 12:57:50 pm Ian Lepore wrote:
   On Tue, 2012-01-31 at 11:49 -0500, John Baldwin wrote:
A co-worker ran into a race between updating a cron tab via crontab(8) 
and 
cron(8) yesterday.  Specifically, cron(8) failed to notice that a 
crontab was 
updated.  The problem is that 1) by default our filesystems only use 
second 
granularity for timestamps and 2) cron only caches the seconds portion 
of a 
file's timestamp when checking for changes anyway.  This means that 
cron can 
miss updates to a spool directory if multiple updates to the directory 
are 
performed within a single second and cron wakes up to scan the spool 
directory 
within the same second and scans it before all of the updates are 
complete.

Specifically, when replacing a crontab, crontab(8) first creates a 
temporary 
file in /var/cron/tabs and then uses a rename to install it followed 
by 
touching the spool directory to update its modification time.  
However, the 
creation of the temporary file already changes the modification time 
of the 
directory, and cron may miss the rename if it scans the directory in 
between 
the creation of the temporary file and the rename.

The fix I am planning to use locally is to simply force crontab(8) 
to sleep 
for a second before it touches the spool directory, thus ensuring that 
it the 
touch of the spool directory will use a later modification time than 
the 
creation of the temporary file.

Note that crontab -r is not affected by this race as it only does one 
atomic 
update to the directory (unlink()).

Index: crontab.c
===
--- crontab.c   (revision 225431)
+++ crontab.c   (working copy)
@@ -604,6 +604,15 @@ replace_cmd() {
 
log_it(RealUser, Pid, REPLACE, User);
 
+   /*
+* Creating the 'tn' temp file has already updated the
+* modification time of the spool directory.  Sleep for a
+* second to ensure that poke_daemon() sets a later
+* modification time.  Otherwise, this can race with the cron
+* daemon scanning for updated crontabs.
+*/
+   sleep(1);
+
poke_daemon();
 
return (0);
   
   Maybe this is overly pedantic, but that solution still allows the
   possibility of the same sort of race if a user updates their crontab in
   the same second as an admin saves a new /etc/crontab, because cron takes
   the max timestamp of /etc/crontab and /var/cron/tabs and compares it
   against the database-rebuild timestamp.
  
  Hmm, I'm not sure I see the race in that case.  If the /etc/crontab file
  matches the timestamp of the spool directory before the utimes() call
  after the one-second sleep, then it will still rescan it on the next
  check when it notices a newer timestamp on the spool directory.  If
  it is the same timestamp as the second timestamp on the spool directory,
  then the scan is guaranteed to have not started before that second began,
  meaning that the crontab(8) process editing the user's crontab must have
  passed the rename, so the scan will see the user's new crontab.
  
   A possible solution on the daemon side of things might be something like
   the attached, but I should state (nay, shout) that I haven't looked
   beyond these few lines to see if there are any unintended side effects
   to such a change.
  
  I think this patch doesn't change anything at all actually.  It is 
  certainly subject to the original race I described if you do not use
  the patch in crontab(8) itself.
  
 
 You're right about my patch not fixing anything; I didn't think hard
 enough before I started typing.  
 
 But I think the problem I was trying to get at with /etc/crontab still
 exists with your patch; it would be triggered if a user updated their
 crontab and after the 1 second sleep the directory timestamp gets
 updated and cron rebuilds the database, and then right after that but
 still in the same second /etc/crontab gets written.  (That's why I was
 trying, however feebly, to move the solution into the daemon.)

What I would do for this case is change the daemon to remove all the TMAX
crap and just cache both timestamps and rebuild the database any time
either one changes.

 Maybe the simple answer is for admins to be sure not to save changes
 to /etc/crontab during the xx:xx:00 second, or with your patch, :01.
 (I'm kidding, of course.  The fact that cron likes to wake up at the top
 of minute is no g'tee that it actually does so every time.)
 
 Once you start thinking along the no g'tee lines you realize that two
 users can be updating their tabs concurrently, and one arrives in
 poke_daemon() and grabs the current time (let's 

Re: Race between cron and crontab

2012-01-31 Thread Doug Barton
On 01/31/2012 08:49, John Baldwin wrote:
 A co-worker ran into a race between updating a cron tab via crontab(8) and 
 cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab was 
 updated.  The problem is that 1) by default our filesystems only use second 
 granularity for timestamps and 2) cron only caches the seconds portion of a 
 file's timestamp when checking for changes anyway.  This means that cron can 
 miss updates to a spool directory if multiple updates to the directory are 
 performed within a single second and cron wakes up to scan the spool 
 directory 
 within the same second and scans it before all of the updates are complete.
 
 Specifically, when replacing a crontab, crontab(8) first creates a temporary 
 file in /var/cron/tabs and then uses a rename to install it followed by 
 touching the spool directory to update its modification time.  However, the 
 creation of the temporary file already changes the modification time of the 
 directory, and cron may miss the rename if it scans the directory in 
 between 
 the creation of the temporary file and the rename.
 
 The fix I am planning to use locally is to simply force crontab(8) to sleep 
 for a second before it touches the spool directory, thus ensuring that it the 
 touch of the spool directory will use a later modification time than the 
 creation of the temporary file.

If you really want cron to have sub-second granularity I don't see how
you could do it without using flags.

crontab opensets flag that it is editing a file
crontab close   clears editing flag, sets something changed flag
(if something actually changed of course)

cronchecks existence of something changed flag, pulls the
update if there is no editing flag, clears changed flag


Doug

-- 

It's always a long day; 86400 doesn't fit into a short.

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org