Re: Inheriting the nodump flag

[Dima Dorfman]

> > Attached below is a port of NetBSD's patch to FreeBSD's dump(8).
> > dump's tree walker is a little weird, so the patch is a little more
> > complicated than calling fts_set with FTS_SKIP.  For the technical
> > details of what it does, see:
> > http://lists.openresources.com/NetBSD/tech-kern/msg00453.html.
> [.]
> > Comments?
> 
> Just that I noticed a typo in a comment where ``inherited'' is misspelt 
> ``herited'' going via your URL but it's fixed in the attached 
> patch :-)  It's probably worth updating the web page.

That page is the archive of the [EMAIL PROTECTED] mailing list, and
as such can't be changed (much less by me).  I provided it because it
offers a very good technical explanation of the patch, which I didn't
want to repeat.  I took the patch there almost verbatim, except for,
obviously, the exact placement of the code, and the comments; I didn't
take the latter because the person who wrote it isn't a natural
English speaker (as far as I can tell, anyway), and as such didn't
write such great comments ("such great" being a reference to the
English grammar and spelling, not their technical accuracy).

> 
> I'd certainly like to see this committed - I'm sick of backing up 
> release directories, cvs repositories and /usr/obj :-)

Yes, this was my motivation for porting this, too. :-)

Thanks

Dima Dorfman
[EMAIL PROTECTED]

P.S.  I just sent this patch to -audit per request of rwatson.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message

Re: Inheriting the nodump flag

[Brian Somers]

> Attached below is a port of NetBSD's patch to FreeBSD's dump(8).
> dump's tree walker is a little weird, so the patch is a little more
> complicated than calling fts_set with FTS_SKIP.  For the technical
> details of what it does, see:
> http://lists.openresources.com/NetBSD/tech-kern/msg00453.html.
[.]
> Comments?

Just that I noticed a typo in a comment where ``inherited'' is misspelt 
``herited'' going via your URL but it's fixed in the attached 
patch :-)  It's probably worth updating the web page.

I'd certainly like to see this committed - I'm sick of backing up 
release directories, cvs repositories and /usr/obj :-)

> Thanks in advance
> 
>   Dima Dorfman
>   [EMAIL PROTECTED]

Cheers.
-- 
Brian <[EMAIL PROTECTED]>
     
Don't _EVER_ lose your sense of humour !



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message

Re: Inheriting the nodump flag

[Dima Dorfman]

> 1) This method of handling recursive nodump is far superior to any actual
>inheritence of the flag as part of file system operations, as currently
>no other file flags are inherited from the parent directory -- the only
>property that is inherited is the group.  With ACLs, the parent's
>default ACL will also play a role in the new access ACL.  In any case,
>there is no precedent for file flag inheritence.

I'm not sure if this is supposed to be a confirmation, but, just to
clear things up, the patch doesn't cause the nodump flag to be
inherited in the filesystem per se.  I.e., you won't see entire trees
of files marked nodump that weren't marked before after running dump
with this patch applied.  It is inherited in terms of dump's internal
maps; perhaps propagated would be a better word to describe its
behavior.

> 
> 2) Please run the patch by freebsd-audit -- there have been a fair number
>of vulnerabilities in the fts code in the past due to race conditions
>of various sorts, and it's important that any modifications be
>carefully scrutinized to prevent the reintroduction of vulnerabilites.

dump doesn't use fts; I used calling fts_set as an example because in
a program that uses fts, pruning a directory and everything under it
is a matter of one library call.  In dump's case, it's not that
simple.

Nevertheless (or should that be consequently?), your point is well
taken.  I will send this to -audit in a few days barring any
objections here.

Thanks again

Dima Dorfman
[EMAIL PROTECTED]


> 
> However, the general idea sounds very useful, and something that I'd find
> applicable on a daily basis :-).
> 
> Robert N M Watson FreeBSD Core Team, TrustedBSD Project
> [EMAIL PROTECTED]  NAI Labs, Safeport Network Services
> 
> On Mon, 26 Feb 2001, Dima Dorfman wrote:
> 
> > Hello -hackers
> > 
> > Some time ago, on -arch, phk proposed that the nodump flag should be
> > inherited (see 'inheriting the "nodump" flag ?' around Dec. 2000).
> > This was generally considered a good idea, however, the patch to the
> > kernel he proposed was thought an ugly hack.  In addition, jeroen
> > pointed out that NetBSD had implemented this functionality the Right
> > Way(tm), in dump(8).
> > 
> > Attached below is a port of NetBSD's patch to FreeBSD's dump(8).
> > dump's tree walker is a little weird, so the patch is a little more
> > complicated than calling fts_set with FTS_SKIP.  For the technical
> > details of what it does, see:
> > http://lists.openresources.com/NetBSD/tech-kern/msg00453.html.
> > 
> > I've been using this on two of my hosts for a while, and it works as
> > expected.  Given the additional fact that NetBSD has had this for
> > almost two years, and that the patch below looks very similar to the
> > one they applied, I doubt it significantly breaks anything.
> > 
> > Comments?
> > 
> > Thanks in advance
> > 
> > Dima Dorfman
> > [EMAIL PROTECTED]
> > 
> > 
> > [ Patch snipped to conserve bandwidth of those who have to pay for
> > it; it's available at
> > http://www.unixfreak.org/~dima/home/nodump.diff or the mailing
> > list archives if you're interested. ]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message

Re: Inheriting the nodump flag

[Robert Watson]

I won't have a chance to look at the patch below until later this week,
but had two comments--

1) This method of handling recursive nodump is far superior to any actual
   inheritence of the flag as part of file system operations, as currently
   no other file flags are inherited from the parent directory -- the only
   property that is inherited is the group.  With ACLs, the parent's
   default ACL will also play a role in the new access ACL.  In any case,
   there is no precedent for file flag inheritence.

2) Please run the patch by freebsd-audit -- there have been a fair number
   of vulnerabilities in the fts code in the past due to race conditions
   of various sorts, and it's important that any modifications be
   carefully scrutinized to prevent the reintroduction of vulnerabilites.

However, the general idea sounds very useful, and something that I'd find
applicable on a daily basis :-).

Robert N M Watson FreeBSD Core Team, TrustedBSD Project
[EMAIL PROTECTED]  NAI Labs, Safeport Network Services

On Mon, 26 Feb 2001, Dima Dorfman wrote:

> Hello -hackers
> 
> Some time ago, on -arch, phk proposed that the nodump flag should be
> inherited (see 'inheriting the "nodump" flag ?' around Dec. 2000).
> This was generally considered a good idea, however, the patch to the
> kernel he proposed was thought an ugly hack.  In addition, jeroen
> pointed out that NetBSD had implemented this functionality the Right
> Way(tm), in dump(8).
> 
> Attached below is a port of NetBSD's patch to FreeBSD's dump(8).
> dump's tree walker is a little weird, so the patch is a little more
> complicated than calling fts_set with FTS_SKIP.  For the technical
> details of what it does, see:
> http://lists.openresources.com/NetBSD/tech-kern/msg00453.html.
> 
> I've been using this on two of my hosts for a while, and it works as
> expected.  Given the additional fact that NetBSD has had this for
> almost two years, and that the patch below looks very similar to the
> one they applied, I doubt it significantly breaks anything.
> 
> Comments?
> 
> Thanks in advance
> 
>   Dima Dorfman
>   [EMAIL PROTECTED]
> 
> 
> Index: traverse.c
> ===
> RCS file: /st/src/FreeBSD/src/sbin/dump/traverse.c,v
> retrieving revision 1.11
> diff -u -r1.11 traverse.c
> --- traverse.c2000/04/14 06:14:59 1.11
> +++ traverse.c2001/02/20 01:39:06
> @@ -74,9 +74,11 @@
>  typedef  long fsizeT;
>  #endif
>  
> -static   int dirindir __P((ino_t ino, daddr_t blkno, int level, long *size));
> +static   int dirindir __P((ino_t ino, daddr_t blkno, int level, long *size,
> +long *tapesize, int nodump));
>  static   void dmpindir __P((ino_t ino, daddr_t blk, int level, fsizeT *size));
> -static   int searchdir __P((ino_t ino, daddr_t blkno, long size, long 
>filesize));
> +static   int searchdir __P((ino_t ino, daddr_t blkno, long size, long filesize,
> +long *tapesize, int nodump));
>  
>  /*
>   * This is an estimation of the number of TP_BSIZE blocks in the file.
> @@ -152,10 +154,14 @@
>   dp = getino(ino);
>   if ((mode = (dp->di_mode & IFMT)) == 0)
>   continue;
> - SETINO(ino, usedinomap);
> + /*
> +  * All dirs go in dumpdirmap; only inodes that are to
> +  * be dumped go in usedinomap and dumpinomap, however.
> +  */
>   if (mode == IFDIR)
>   SETINO(ino, dumpdirmap);
>   if (WANTTODUMP(dp)) {
> + SETINO(ino, usedinomap);
>   SETINO(ino, dumpinomap);
>   if (mode != IFREG && mode != IFDIR && mode != IFLNK)
>   *tapesize += 1;
> @@ -192,9 +198,10 @@
>   long *tapesize;
>  {
>   register struct dinode *dp;
> - register int i, isdir;
> + register int i, isdir, nodump;
>   register char *map;
>   register ino_t ino;
> + struct dinode di;
>   long filesize;
>   int ret, change = 0;
>  
> @@ -204,24 +211,34 @@
>   isdir = *map++;
>   else
>   isdir >>= 1;
> - if ((isdir & 1) == 0 || TSTINO(ino, dumpinomap))
> + /*
> +  * If a directory has been removed from usedinomap, it
> +  * either has the nodump flag set, or has inherited
> +  * it.  Although a directory can't be in dumpinomap if
> +  * it isn't in usedin

Inheriting the nodump flag

[Dima Dorfman]

Hello -hackers

Some time ago, on -arch, phk proposed that the nodump flag should be
inherited (see 'inheriting the "nodump" flag ?' around Dec. 2000).
This was generally considered a good idea, however, the patch to the
kernel he proposed was thought an ugly hack.  In addition, jeroen
pointed out that NetBSD had implemented this functionality the Right
Way(tm), in dump(8).

Attached below is a port of NetBSD's patch to FreeBSD's dump(8).
dump's tree walker is a little weird, so the patch is a little more
complicated than calling fts_set with FTS_SKIP.  For the technical
details of what it does, see:
http://lists.openresources.com/NetBSD/tech-kern/msg00453.html.

I've been using this on two of my hosts for a while, and it works as
expected.  Given the additional fact that NetBSD has had this for
almost two years, and that the patch below looks very similar to the
one they applied, I doubt it significantly breaks anything.

Comments?

Thanks in advance

Dima Dorfman
[EMAIL PROTECTED]


Index: traverse.c
===
RCS file: /st/src/FreeBSD/src/sbin/dump/traverse.c,v
retrieving revision 1.11
diff -u -r1.11 traverse.c
--- traverse.c  2000/04/14 06:14:59 1.11
+++ traverse.c  2001/02/20 01:39:06
@@ -74,9 +74,11 @@
 typedeflong fsizeT;
 #endif
 
-static int dirindir __P((ino_t ino, daddr_t blkno, int level, long *size));
+static int dirindir __P((ino_t ino, daddr_t blkno, int level, long *size,
+long *tapesize, int nodump));
 static void dmpindir __P((ino_t ino, daddr_t blk, int level, fsizeT *size));
-static int searchdir __P((ino_t ino, daddr_t blkno, long size, long filesize));
+static int searchdir __P((ino_t ino, daddr_t blkno, long size, long filesize,
+long *tapesize, int nodump));
 
 /*
  * This is an estimation of the number of TP_BSIZE blocks in the file.
@@ -152,10 +154,14 @@
dp = getino(ino);
if ((mode = (dp->di_mode & IFMT)) == 0)
continue;
-   SETINO(ino, usedinomap);
+   /*
+* All dirs go in dumpdirmap; only inodes that are to
+* be dumped go in usedinomap and dumpinomap, however.
+*/
if (mode == IFDIR)
SETINO(ino, dumpdirmap);
if (WANTTODUMP(dp)) {
+   SETINO(ino, usedinomap);
SETINO(ino, dumpinomap);
if (mode != IFREG && mode != IFDIR && mode != IFLNK)
*tapesize += 1;
@@ -192,9 +198,10 @@
long *tapesize;
 {
register struct dinode *dp;
-   register int i, isdir;
+   register int i, isdir, nodump;
register char *map;
register ino_t ino;
+   struct dinode di;
long filesize;
int ret, change = 0;
 
@@ -204,24 +211,34 @@
isdir = *map++;
else
isdir >>= 1;
-   if ((isdir & 1) == 0 || TSTINO(ino, dumpinomap))
+   /*
+* If a directory has been removed from usedinomap, it
+* either has the nodump flag set, or has inherited
+* it.  Although a directory can't be in dumpinomap if
+* it isn't in usedinomap, we have to go through it to
+* propogate the nodump flag.
+*/
+   nodump = (TSTINO(ino, usedinomap) == 0);
+   if ((isdir & 1) == 0 || (TSTINO(ino, dumpinomap) && !nodump))
continue;
dp = getino(ino);
-   filesize = dp->di_size;
+   di = *dp;   /* inode buf may change in searchdir(). */
+   filesize = di.di_size;
for (ret = 0, i = 0; filesize > 0 && i < NDADDR; i++) {
-   if (dp->di_db[i] != 0)
-   ret |= searchdir(ino, dp->di_db[i],
+   if (di.di_db[i] != 0)
+   ret |= searchdir(ino, di.di_db[i],
(long)dblksize(sblock, dp, i),
-   filesize);
+   filesize, tapesize, nodump);
if (ret & HASDUMPEDFILE)
filesize = 0;
else
filesize -= sblock->fs_bsize;
}
for (i = 0; filesize > 0 && i < NIADDR; i++) {
-   if (dp->di_ib[i] == 0)
+   if (di.di_ib[i] == 0)
continue;
-   ret |= dirindir(ino, dp->di_ib[i], i, &filesize);
+