Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-21 Thread Anton Salikhmetov
2008/1/21, Peter Staubach <[EMAIL PROTECTED]>: > Linus Torvalds wrote: > > On Fri, 18 Jan 2008, Ingo Oeser wrote: > > > >> Can we get "if the write to the page hits the disk, the mtime has hit the > >> disk > >> already no less than SOME_GRANULARITY before"? > >> > >> That is very important for co

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-21 Thread Peter Staubach
Linus Torvalds wrote: On Fri, 18 Jan 2008, Ingo Oeser wrote: Can we get "if the write to the page hits the disk, the mtime has hit the disk already no less than SOME_GRANULARITY before"? That is very important for computer forensics. Esp. in saving your ass! Ok, now back again to making t

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-19 Thread Matt Mackall
On Sat, 2008-01-19 at 11:22 +0100, Miklos Szeredi wrote: > > Reminds me, I've got a patch here for addressing that problem with loop > > mounts: > > > > Writes to loop should update the mtime of the underlying file. > > > > Signed-off-by: Matt Mackall <[EMAIL PROTECTED]> > > > > Index: l/drive

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-19 Thread Miklos Szeredi
> Reminds me, I've got a patch here for addressing that problem with loop > mounts: > > Writes to loop should update the mtime of the underlying file. > > Signed-off-by: Matt Mackall <[EMAIL PROTECTED]> > > Index: l/drivers/block/loop.c >

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Rik van Riel
On Fri, 18 Jan 2008 18:50:03 -0600 Matt Mackall <[EMAIL PROTECTED]> wrote: > On Fri, 2008-01-18 at 17:54 -0500, Rik van Riel wrote: > > Backup programs not seeing an updated mtime is a really big deal. > > And that's fixed with the 4-line approach. > > Reminds me, I've got a patch here for addre

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Matt Mackall
On Fri, 2008-01-18 at 17:54 -0500, Rik van Riel wrote: > On Fri, 18 Jan 2008 14:47:33 -0800 (PST) > Linus Torvalds <[EMAIL PROTECTED]> wrote: > > > - keep it simple. Let's face it, Linux has never ever given those > >guarantees before, and it's not is if anybody has really cared. Even > >

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Rik van Riel
On Fri, 18 Jan 2008 14:47:33 -0800 (PST) Linus Torvalds <[EMAIL PROTECTED]> wrote: > - keep it simple. Let's face it, Linux has never ever given those >guarantees before, and it's not is if anybody has really cared. Even >now, the issue seems to be more about paper standards conformance

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Ingo Oeser wrote: > > Can we get "if the write to the page hits the disk, the mtime has hit the disk > already no less than SOME_GRANULARITY before"? > > That is very important for computer forensics. Esp. in saving your ass! > > Ok, now back again to making that fast :-)

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Anton Salikhmetov
2008/1/19, Linus Torvalds <[EMAIL PROTECTED]>: > > > On Sat, 19 Jan 2008, Anton Salikhmetov wrote: > > > > The page_check_address() function is called from the > > page_mkclean_one() routine as follows: > > .. and the page_mkclean_one() function is totally different. > > Lookie here, this is the co

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Ingo Oeser
Hi Linus, On Friday 18 January 2008, Linus Torvalds wrote: > On Fri, 18 Jan 2008, Miklos Szeredi wrote: > > > > What I'm saying is that the times could be left un-updated for a long > > time if program doesn't do munmap() or msync(MS_SYNC) for a long time. > > Sure. > > But in those circumstanc

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Sat, 19 Jan 2008, Anton Salikhmetov wrote: > > The page_check_address() function is called from the > page_mkclean_one() routine as follows: .. and the page_mkclean_one() function is totally different. Lookie here, this is the correct and complex sequence: > entry = ptep_cl

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Anton Salikhmetov
2008/1/19, Linus Torvalds <[EMAIL PROTECTED]>: > > > On Sat, 19 Jan 2008, Anton Salikhmetov wrote: > > > > Before using pte_wrprotect() the vma_wrprotect() routine uses the > > pte_offset_map_lock() macro to get the PTE and to acquire the ptl > > spinlock. Why did you say that this code was not SMP

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Sat, 19 Jan 2008, Anton Salikhmetov wrote: > > Before using pte_wrprotect() the vma_wrprotect() routine uses the > pte_offset_map_lock() macro to get the PTE and to acquire the ptl > spinlock. Why did you say that this code was not SMP-safe? It should > be atomic, I think. It's atomic WITH R

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Anton Salikhmetov
2008/1/18, Linus Torvalds <[EMAIL PROTECTED]>: > > > On Fri, 18 Jan 2008, Anton Salikhmetov wrote: > > > > The current solution doesn't hit the performance at all when compared to > > the competitor POSIX-compliant systems. It is faster and does even more > > than the POSIX standard requires. > > Y

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Anton Salikhmetov wrote: > > The current solution doesn't hit the performance at all when compared to > the competitor POSIX-compliant systems. It is faster and does even more > than the POSIX standard requires. Your current patches have two problems: - they are simply unn

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Anton Salikhmetov
2008/1/18, Linus Torvalds <[EMAIL PROTECTED]>: > > > On Fri, 18 Jan 2008, Miklos Szeredi wrote: > > > > What I'm saying is that the times could be left un-updated for a long > > time if program doesn't do munmap() or msync(MS_SYNC) for a long time. > > Sure. > > But in those circumstances, the prog

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Miklos Szeredi wrote: > > What I'm saying is that the times could be left un-updated for a long > time if program doesn't do munmap() or msync(MS_SYNC) for a long time. Sure. But in those circumstances, the programmer cannot depend on the mtime *anyway* (because there is

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> > > > But then background writeout, sync(2), etc, wouldn't update the times. > > Sure it would, but only when doing the final unmap. > > Did you miss the "on unmap and msync" part? No :) What I'm saying is that the times could be left un-updated for a long time if program doesn't do munmap()

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Miklos Szeredi wrote: > > But then background writeout, sync(2), etc, wouldn't update the times. Sure it would, but only when doing the final unmap. Did you miss the "on unmap and msync" part? Linus -- To unsubscribe from this list: send the line "unsubscr

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> > That would need a new page flag (PG_mmap_dirty?). Do we have one > > available? > > Yeah, that would be bad. We probably have flags free, but those page flags > are always a pain. Scratch that. > > How about just setting a per-vma dirty flag, and then instead of updating > the mtime when t

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> > > And even in that four-liner, I suspect that the *last* two lines are > > > actually incorrect: there's no point in updating the file time when the > > > page *becomes* dirty, > > > > Actually all four lines do that. The first two for a write access on > > a present, read-only pte, the oth

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Miklos Szeredi wrote: > > That would need a new page flag (PG_mmap_dirty?). Do we have one > available? Yeah, that would be bad. We probably have flags free, but those page flags are always a pain. Scratch that. How about just setting a per-vma dirty flag, and then inste

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Rik van Riel
On Fri, 18 Jan 2008 19:11:47 +0100 Miklos Szeredi <[EMAIL PROTECTED]> wrote: > > And even in that four-liner, I suspect that the *last* two lines are > > actually incorrect: there's no point in updating the file time when the > > page *becomes* dirty, > > Actually all four lines do that. The f

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> And even in that four-liner, I suspect that the *last* two lines are > actually incorrect: there's no point in updating the file time when the > page *becomes* dirty, Actually all four lines do that. The first two for a write access on a present, read-only pte, the other two for a write on a

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Linus Torvalds
On Fri, 18 Jan 2008, Peter Zijlstra wrote: > > Bah, and will break on s390... so we'd need a page_mkclean() variant > that doesn't actually clear dirty. No, we simply want to not play all these very expensive games with dirty in the first place. Guys, mmap access times aren't important enough

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> Possibly, I didn't see a quick way to break that iteration. > >From a quick glance at prio_tree.c the iterator isn't valid anymore > after releasing i_mmap_lock. Fixing that would be,.. 'fun'. Maybe i_mmap_lock isn't needed at all, since msync holds mmap_sem, which protects the prio tree as well

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Peter Zijlstra
On Fri, 2008-01-18 at 12:17 +0100, Miklos Szeredi wrote: > > diff --git a/mm/msync.c b/mm/msync.c > > index 144a757..a1b3fc6 100644 > > --- a/mm/msync.c > > +++ b/mm/msync.c > > @@ -14,6 +14,122 @@ > > #include > > #include > > > > +unsigned long masync_pte_range(struct vm_area_struct *vma,

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> diff --git a/mm/msync.c b/mm/msync.c > index 144a757..a1b3fc6 100644 > --- a/mm/msync.c > +++ b/mm/msync.c > @@ -14,6 +14,122 @@ > #include > #include > > +unsigned long masync_pte_range(struct vm_area_struct *vma, pmd_t *pdm, > + unsigned long addr, unsigned long end) > +{ > +

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Peter Zijlstra
On Fri, 2008-01-18 at 11:38 +0100, Miklos Szeredi wrote: > > On Fri, 2008-01-18 at 10:51 +0100, Miklos Szeredi wrote: > > > > > > diff --git a/mm/msync.c b/mm/msync.c > > > > index a4de868..a49af28 100644 > > > > --- a/mm/msync.c > > > > +++ b/mm/msync.c > > > > @@ -13,11 +13,33 @@ > > > > #incl

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Anton Salikhmetov
2008/1/18, Peter Zijlstra <[EMAIL PROTECTED]>: > > On Fri, 2008-01-18 at 11:15 +0100, Peter Zijlstra wrote: > > On Fri, 2008-01-18 at 10:51 +0100, Miklos Szeredi wrote: > > > > > > diff --git a/mm/msync.c b/mm/msync.c > > > > index a4de868..a49af28 100644 > > > > --- a/mm/msync.c > > > > +++ b/mm/m

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> On Fri, 2008-01-18 at 10:51 +0100, Miklos Szeredi wrote: > > > > diff --git a/mm/msync.c b/mm/msync.c > > > index a4de868..a49af28 100644 > > > --- a/mm/msync.c > > > +++ b/mm/msync.c > > > @@ -13,11 +13,33 @@ > > > #include > > > > > > /* > > > + * Scan the PTEs for pages belonging to the

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Peter Zijlstra
On Fri, 2008-01-18 at 11:15 +0100, Peter Zijlstra wrote: > On Fri, 2008-01-18 at 10:51 +0100, Miklos Szeredi wrote: > > > > diff --git a/mm/msync.c b/mm/msync.c > > > index a4de868..a49af28 100644 > > > --- a/mm/msync.c > > > +++ b/mm/msync.c > > > @@ -13,11 +13,33 @@ > > > #include > > > > >

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Peter Zijlstra
On Fri, 2008-01-18 at 10:51 +0100, Miklos Szeredi wrote: > > diff --git a/mm/msync.c b/mm/msync.c > > index a4de868..a49af28 100644 > > --- a/mm/msync.c > > +++ b/mm/msync.c > > @@ -13,11 +13,33 @@ > > #include > > > > /* > > + * Scan the PTEs for pages belonging to the VMA and mark them rea

Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-18 Thread Miklos Szeredi
> Updating file times at write references to memory-mapped files and > forcing file times update at the next write reference after > calling the msync() system call with the MS_ASYNC flag. > > Signed-off-by: Anton Salikhmetov <[EMAIL PROTECTED]> > --- > mm/memory.c |6 ++ > mm/msync.c |

[PATCH -v6 2/2] Updating ctime and mtime for memory-mapped files

2008-01-17 Thread Anton Salikhmetov
Updating file times at write references to memory-mapped files and forcing file times update at the next write reference after calling the msync() system call with the MS_ASYNC flag. Signed-off-by: Anton Salikhmetov <[EMAIL PROTECTED]> --- mm/memory.c |6 ++ mm/msync.c | 52 +++