Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Christoph Hellwig
On Mon, Mar 05, 2007 at 12:02:59PM -0800, Mingming Cao wrote: > Yep, I think it makes sense to use preallocation for defragmentation. > After all both preallocation and defragmentation shall call underlying > filesystem multiple block allocator to try to allocate a chunk of > contiguous blocks on

Re: e2fsck and human intervention

2007-03-05 Thread Andreas Dilger
On Mar 05, 2007 11:26 -0500, Daniel Drake wrote: > This usually happens during e2fsck's regular "check every X mounts" > thing, as opposed to immediately after booting up after power loss, so > to begin with it's not immediately obvious why there is a problem. That's because the post-boot e2fsck

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Eric Sandeen
Jörn Engel wrote: > Does the allocation have to be persistent beyond lifetime of the file > descriptor? It would be fairly simple to support the write guarantee > while the file is open (or rather the inode remains cached) and drop it > afterwards. "The posix_fallocate() function shall ensure tha

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Eric Sandeen
Jan Kara wrote: >> I am wondering if it is useful to add another mode to advise block >> allocation policy? Something like indicating which physical block/block >> group to allocate from (goal), and whether ask for strict contigous >> blocks. This will help preallocation or reservation to choos

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Mingming Cao
Jan Kara wrote: On Fri, 02 Mar 2007 09:40:54 +1100 Nathan Scott <[EMAIL PROTECTED]> wrote: On Thu, 2007-03-01 at 14:25 -0800, Andrew Morton wrote: On Fri, 2 Mar 2007 00:04:45 +0530 "Amit K. Arora" <[EMAIL PROTECTED]> wrote: This is to give a heads up on few patches that we will be soon

Re: e2fsck and human intervention

2007-03-05 Thread Sev Binello
Theodore Tso wrote: On Mon, Mar 05, 2007 at 11:26:57AM -0500, Daniel Drake wrote: Hi, I'm working with ext3 partitions in a product environment, where numerous embedded Linux systems will be shipped to various locations. In testing we occasionally find that system boot is halted by e2fsck w

Re: e2fsck and human intervention

2007-03-05 Thread Daniel Drake
Hi Ted, Thanks for the quick response. On Mon, 2007-03-05 at 11:48 -0500, Theodore Tso wrote: > Actually, power loss by itself should *not* cause any corruption when > you are using ext3; that's the whole point of the journal. If there > is, you probably have some other problem that you might do

Re: e2fsck and human intervention

2007-03-05 Thread Theodore Tso
On Mon, Mar 05, 2007 at 11:26:57AM -0500, Daniel Drake wrote: > Hi, > > I'm working with ext3 partitions in a product environment, where > numerous embedded Linux systems will be shipped to various locations. > > In testing we occasionally find that system boot is halted by e2fsck > with an "UNEX

e2fsck and human intervention

2007-03-05 Thread Daniel Drake
Hi, I'm working with ext3 partitions in a product environment, where numerous embedded Linux systems will be shipped to various locations. In testing we occasionally find that system boot is halted by e2fsck with an "UNEXPECTED INCONSISTENCY" error message. This is while running in preen mode. T

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Ulrich Drepper
Theodore Tso wrote: > [...] although the libc > implementation still wouldn't be able to go away for long time due to > the need to be backwards compatible with older kernels that didn't > have this support. It's better than that. If somebody compiles glibc to not run on older kernels at all (tes

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Theodore Tso
On Mon, Mar 05, 2007 at 07:15:33AM -0800, Ulrich Drepper wrote: > Well, I'm sure the kernel can do better than the code we have in libc > now. The kernel has access to the bitmasks which say which blocks have > already been allocated. The libc code does not and we have to be very > simple-minded

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Ulrich Drepper
Jörn Engel wrote: >> Of course. You call posix_fallocate once for the lifetime of the file >> when it is created to ensure that all future uses will work. > > That part is not quite clear from the manpage but I trust most people > would assume the same. Not only that, it is what this function is

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Jörn Engel
On Mon, 5 March 2007 07:08:03 -0800, Ulrich Drepper wrote: > Jörn Engel wrote: > > Does the allocation have to be persistent beyond lifetime of the file > > descriptor? > > Of course. You call posix_fallocate once for the lifetime of the file > when it is created to ensure that all future uses wi

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Christoph Hellwig
On Mon, Mar 05, 2007 at 07:15:33AM -0800, Ulrich Drepper wrote: > Theodore Tso wrote: > > Given that glibc already has to support this for older kernels, I > > would argue that there's no point putting in generic support for > > filesystem that can't support a more advanced way of doing things. >

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Ulrich Drepper
Jörn Engel wrote: > The bad news for posix_fallocate() is that even if libc is smart enough > to write random data, mmap() can still cause problems. This is not smart, quite to the contrary. The standard guarantees that all not-yet-written-to places in the file are zero. And if a block has alrea

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Ulrich Drepper
Jörn Engel wrote: > Does the allocation have to be persistent beyond lifetime of the file > descriptor? Of course. You call posix_fallocate once for the lifetime of the file when it is created to ensure that all future uses will work. It seems your filesystem will not be able to support this unl

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Ulrich Drepper
Theodore Tso wrote: > Given that glibc already has to support this for older kernels, I > would argue that there's no point putting in generic support for > filesystem that can't support a more advanced way of doing things. Well, I'm sure the kernel can do better than the code we have in libc now.

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Anton Altaparmakov
On 5 Mar 2007, at 14:37, Theodore Tso wrote: On Sun, Mar 04, 2007 at 11:22:06PM +, Anton Altaparmakov wrote: And I specifically did NOT update the initialized size in the inode thus it will remain at its old value thus all new allocated blocks will be considered as present but not initialize

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Theodore Tso
On Sun, Mar 04, 2007 at 11:22:06PM +, Anton Altaparmakov wrote: > And I specifically did NOT update the initialized size in the inode > thus it will remain at its old value thus all new allocated blocks > will be considered as present but not initialized thus a read will > always return z

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Jörn Engel
On Mon, 5 March 2007 00:32:14 +, Anton Altaparmakov wrote: > > I don't know how your compression algorithm works [...] LogFS is designed for flash media, so it does not have to worry much about reducing disk seeks. It is log-structured, which simplifies compression further. When writing a b

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Christoph Hellwig
On Sat, Mar 03, 2007 at 11:45:32PM +0100, Arnd Bergmann wrote: > > I'd be more happy to have the write out zeroes loop in glibc. ?And > > glibc needs to have it anyway, for older kernels. > > A generic_fallocate makes sense to me iff we can do it in the kernel > more significantly more efficiently

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Jörn Engel
On Mon, 5 March 2007 01:36:36 +0100, Arnd Bergmann wrote: > > Using the current glibc implementation on a compressed file system ideally > should be a very expensive no-op because you won't actually allocate much > space for a file when writing zeroes to it. You also don't benefit of a > contiguou

Re: [RFC] Heads up on sys_fallocate()

2007-03-05 Thread Jan Kara
> >On Fri, 02 Mar 2007 09:40:54 +1100 > >Nathan Scott <[EMAIL PROTECTED]> wrote: > > > > > >>On Thu, 2007-03-01 at 14:25 -0800, Andrew Morton wrote: > >> > >>>On Fri, 2 Mar 2007 00:04:45 +0530 > >>>"Amit K. Arora" <[EMAIL PROTECTED]> wrote: > >>> > >>> > This is to give a heads up on few patche