[RFC] ext3 freeze feature ver 0.2

2008-02-26 Thread Takashi Sato
Hi, Takashi Sato wrote: >>> Instead, I'd like the sec to timeout on freeze API in order to thaw >>> the filesystem automatically. It can prevent a filesystem from staying >>> frozen forever. >>> (Because a freezer may cause a deadlock by accessing the fr

Re: [RFC] ext3 freeze feature

2008-02-15 Thread Takashi Sato
Hi, Christoph Hellwig wrote: On Fri, Feb 08, 2008 at 08:26:57AM -0500, Andreas Dilger wrote: You may as well make the common ioctl the same as the XFS version, both by number and parameters, so that applications which already understand the XFS ioctl will work on other filesystems. Yes. In f

Re: [RFC] ext3 freeze feature

2008-02-13 Thread Takashi Sato
Hi, P.S. Oh yeah, it should be noted that freezing at the filesystem layer does *not* guarantee that changes to the block device aren't happening via mmap()'ed files. The LVM needs to freeze writes the block device level if it wants to guarantee a completely stable snapshot image. So the prop

Re: [RFC] ext3 freeze feature

2008-02-08 Thread Takashi Sato
Hi, Ted wrote: And I do agree that we probably should just implement this in filesystem independent way, in which case all of the filesystems that support this already have super_operations functions write_super_lockfs() and unlockfs(). So if this is done using a new system call, there should b

Re: [RFC] ext3 freeze feature

2008-02-06 Thread Takashi Sato
Hi, What you *could* do is to start putting processes to sleep if they attempt to write to the frozen filesystem, and then detect the deadlock case where the process holding the file descriptor used to freeze the filesystem gets frozen because it attempted to write to the filesystem --- at which

Re: [RFC] ext3 freeze feature

2008-01-28 Thread Takashi Sato
Hi, What you *could* do is to start putting processes to sleep if they attempt to write to the frozen filesystem, and then detect the deadlock case where the process holding the file descriptor used to freeze the filesystem gets frozen because it attempted to write to the filesystem --- at which

Re: [RFC] ext3 freeze feature

2008-01-28 Thread Takashi Sato
Hi, Thank you for your comments. That's inherently unsafe - you can have multiple unfreezes running in parallel which seriously screws with the bdev semaphore count that is used to lock the device due to doing multiple up()s for every down. Your timeout thingy guarantee that at some point you

Re: [RFC] ext3 freeze feature

2008-01-25 Thread Takashi Sato
Hi, I am also wondering whether we should have system call(s) for these: On Jan 25, 2008 12:59 PM, Takashi Sato <[EMAIL PROTECTED]> wrote: + case EXT3_IOC_FREEZE: { + case EXT3_IOC_THAW: { And just convert XFS to use them too? I think it is reasonable to implement it

[RFC] ext3 freeze feature

2008-01-25 Thread Takashi Sato
d to the delayed work queue to unfreeze the filesystem automatically after the lapse of the specified time. Any comments are very welcome. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> --- diff -uprN -X linux-2.6.24-rc8/Documentation/dontdiff linux-2.6.24-rc8/fs/ext3/ioctl.c linux-2.6.24-

[RFC][PATCH 10/10] Online defrag command

2007-06-20 Thread Takashi Sato
device-name Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- /* * e4defrag.c - ext4 filesystem defragmenter */ #ifndef _LARGEFILE_SOURCE #define _LARGEFILE_SOURCE #endif #ifndef _LARGEFILE64_SOURCE #define _LARGEFILE64_SOURCE #en

[RFC][PATCH 9/10] Fix bugs in multi-block allocation and locality-group

2007-06-20 Thread Takashi Sato
- Move lg_list to s_locality_dirty in ext4_lg_sync_single_group() to flush all of dirty inodes. - Fix ext4_mb_new_blocks() to return err value when defrag failed. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-

[RFC][PATCH 8/10] Release reserved block

2007-06-20 Thread Takashi Sato
- Release reserved blocks if defrag failed. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.6.19-rc6-git-BLOCK_RELEASE/fs/ext4/exten

[RFC][PATCH 7/10] Reserve freed blocks

2007-06-20 Thread Takashi Sato
- Reserve the free blocks in the target area, not to be used by other process. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.

[RFC][PATCH 6/10] Move files from target block group to other block group

2007-06-20 Thread Takashi Sato
- To make contiguous free blocks, move files from the target block group to other block group. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_

[RFC][PATCH 5/10] Get all extents information of specified inode number

2007-06-20 Thread Takashi Sato
- Get all extents information of specified inode number to calculate the combination of extents which should be moved to other block group. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.

[RFC][PATCH 4/10] Get free blocks distribution of the target block group

2007-06-20 Thread Takashi Sato
- Get free blocks distribution of the target block group to know how many free blocks it has. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_

[RFC][PATCH 2/10] Move the file data to the new blocks

2007-06-20 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato <[EMAIL PROTEC

[RFC][PATCH 3/10] Get block group information

2007-06-20 Thread Takashi Sato
- Get s_blocks_per_group and s_inodes_per_group of target filesystem. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> Signed-off-by: Akira Fujita <[EMAIL PROTECTED]> --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr linux-2.6.19-rc6-test1/fs/ext4/ball

[RFC][PATCH 1/10] Allocate new contiguous blocks

2007-06-20 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. "[RFC] delayed allocation, mballoc, etc" http://marc.theaimsgroup.com/?l=linux-ext4&m=116493228301966&w=2 Signed-

[RFC][PATCH 0/10] ext4 online defrag (ver 0.5)

2007-06-20 Thread Takashi Sato
Hi all, I have updated my online defrag patchset for addition of a new function. This function is defragmentation for free space. If filesytem has insufficient contiguous free blocks, defrag tries to move other files to make sufficient space and reallocates the contiguous blocks for the target fil

[RFC][PATCH 2/4] Move the file data to the new blocks

2007-04-26 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato <[EMAIL PROTEC

[RFC][PATCH 3/4] Online defrag command

2007-04-26 Thread Takashi Sato
The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> --- /* * e4defrag

ext4 online defrag (ver 0.4)

2007-04-26 Thread Takashi Sato
Hi all, I have made following changes to the previous online defrag patchset to improve it. Note that there is no functional change. 1. Change the handling of temporary inode. Now ext4_ext_defrag() calls ext4_new_inode()/iput() pair instead of new_inode()/delete_ext_defrag_inode(). Because

[RFC][PATCH 4/4] ext4_locality_group bug fix

2007-04-26 Thread Takashi Sato
Move lg_list to s_locality_dirty and mark lg as dirty if nr_to_write(total page count which has not written in disk yet) is 0 or less and lg_io is not empty in ext4_lg_sync_single_group(). This makes sure that inode is written to disk. Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> --

[RFC][PATCH 1/4] Allocate new contiguous blocks

2007-04-26 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. "[RFC] delayed allocation, mballoc, etc" http://marc.theaimsgroup.com/?l=linux-ext4&m=116493228301966&w=2 Signed-

Re: [RFC][PATCH 3/3] Online defrag command

2007-02-09 Thread Takashi Sato
Hi, On Thu, Feb 08 2007, Takashi Sato wrote: The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Would it be possible to

[RFC][PATCH 3/3] Online defrag command

2007-02-08 Thread Takashi Sato
The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Signed-off-by: Takashi Sato <[EMAIL PROTECTED]> --- /* * e4defrag

[RFC][PATCH 2/3] Move the file data to the new blocks

2007-02-08 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato <[EMAIL PROTEC

[RFC][PATCH 1/3] Allocate new contiguous blocks

2007-02-08 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. "[RFC] delayed allocation, mballoc, etc" http://marc.theaimsgroup.com/?l=linux-ext4&m=116493228301966&w=2 Signed-

[RFC][PATCH 0/3] ext4 online defrag (ver 0.3)

2007-02-08 Thread Takashi Sato
Hi all, I've updated my online defrag patches to support leaf node which is filled with extents and has no space for new extent any more. Any comments from reviews or tests are very welcome. Implementation: 1. When the leaf node is filled with extents and there is no space for additional exten

Re: [RFC][PATCH 2/3] Move the file data to the new blocks

2007-02-07 Thread Takashi Sato
Hi, +ext4_ext_replace_branches(struct inode *org_inode, struct inode *dest_inode, + pgoff_t from_page, pgoff_t dest_from_page, + pgoff_t count_page, unsigned long *delete_start) +{ + struct ext4_ext_path *org_path = NULL; + struct ext4_ext_path *dest_path = NULL; + struct ext4_extent *oext, *

Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-19 Thread Takashi Sato
Hi, Thank you for your comment. >>>1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical >>> block number of the specified file. With this ioctl, a command >>> gets the specified directory's. >> >>Maybe I don't understand, but how is this different from the long-time >>FIBMAP

Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-18 Thread Takashi Sato
Hi, On Jan 16, 2007 21:03 +0900, [EMAIL PROTECTED] wrote: 1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical block number of the specified file. With this ioctl, a command gets the specified directory's. Maybe I don't understand, but how is this different from the long

Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-17 Thread Takashi Sato
Hi, On Jan 16, 2007 21:03 +0900, [EMAIL PROTECTED] wrote: 1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical block number of the specified file. With this ioctl, a command gets the specified directory's. Maybe I don't understand, but how is this different from the long

Re: [RFC][PATCH 0/3] Extent base online defrag

2006-11-17 Thread Takashi Sato
Hi, > - Specify the target area in a file using the following structure: > struct ext3_ext_defrag_data { > loff_t start_offset; /* start offset to defrag in bytes */ > loff_t defrag_size; /* size of defrag in bytes */ > } > It uses loff_t so that the size of the structure is identical