Re: [PATCH V3 17/18] Btrfs: Full direct I/O and AIO read implementation.

2010-03-25 Thread jim owens
Andi Kleen wrote: On Wed, Mar 24, 2010 at 11:08:07PM -0400, jim owens wrote: Agree, so the write design needs to prevent bad checksums. How? Do you have a plan for that? Yes... have Josef do it. ;) The options I considered are: 1 - buffer always for uncompressed, the same as compressed. 2

Re: [PATCH V3 17/18] Btrfs: Full direct I/O and AIO read implementation.

2010-03-24 Thread jim owens
sorry, this time with cc to list Andi Kleen wrote: On Tue, Mar 23, 2010 at 05:40:00PM -0400, jim owens wrote: Andi Kleen wrote: With checksums enabled, uncompressed reads aligned on the 4k block are classic direct IO to user memory except at EOF. Hmm, but what happens if the user

Re: [PATCH V3 17/18] Btrfs: Full direct I/O and AIO read implementation.

2010-03-23 Thread jim owens
Andi Kleen wrote: One thing that stroke me while reading is that, except for the out of line no data checksum case, this isn't real classical zero copy direct IO because you always have to copy through some buffer. Uh no, unless I really messed up or don't understand what you mean.

[PATCH] Btrfs: change btrfs_get_extent for direct I/O merges.

2010-03-21 Thread jim owens
Direct I/O passes unaligned start and length values that must be adjusted to block boundaries for lookup and merging when an add_extent_mapping fails with EEXIST. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/inode.c | 56 - 1

[PATCH] Btrfs: change ordered data tree search to use start and length.

2010-03-21 Thread jim owens
Users of btrfs_lookup_first_ordered_extent() want to detect only a real overlap, so searching for the range instead of just an address prevents taking references on entries we don't want. This also simplifies the code. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/file.c

[PATCH] Btrfs: fix direct I/O handling of extent map errors.

2010-03-21 Thread jim owens
Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/dio.c | 22 ++ 1 files changed, 14 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index b76b227..b6934be 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c @@ -486,8 +486,13 @@ getlock

[PATCH] Btrfs: change direct I/O read to not use i_mutex.

2010-03-21 Thread jim owens
This depends on the change to ordered data search. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/dio.c | 150 +++- 1 files changed, 104 insertions(+), 46 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index b6934be

[PATCH V3 02/18] Btrfs: Do not limit RAID1 and DUP transfer length to one stripe.

2010-03-21 Thread jim owens
The 65k stripe length should be ignored as the stripes are physically contiguous on disk so transfers can span stripes. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/volumes.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs

[PATCH V3 03/18] Btrfs: __btrfs_map_block should not set length more than input

2010-03-21 Thread jim owens
Returning a value greater than the caller's is ugly and prone to dangerous future coding mistakes. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/extent-tree.c |3 --- fs/btrfs/inode.c |5 ++--- fs/btrfs/volumes.c |9 + 3 files changed, 7 insertions

[PATCH V3 05/18] Btrfs: btrfs_discard_extent must use WRITE with btrfs_map_block.

2010-03-21 Thread jim owens
WRITE is needed on btrfs_map_block() to fetch all raid stripes, READ only returns one device and we want to discard all copies. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/extent-tree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/extent

[PATCH V3 06/18] Btrfs: remove dead code for unplugging pages.

2010-03-21 Thread jim owens
The code in btrfs_unplug_io_fn() to selectively unplug just the stripes affected by a page had been previously disabled. The unneeded code for unplug_page in __btrfs_map_block made changes for btrfs directio more difficult so I'm removing all of it. Signed-off-by: jim owens owens6...@gmail.com

[PATCH V3 07/18] Btrfs: reduce memory use in compress/decompress workspace.

2010-03-21 Thread jim owens
Compression and decompression can use a single z_stream and z_stream.workspace memory as the operations always begin from scratch and do not overlap in one thread's execution. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/zlib.c | 146

[PATCH V3 08/18] Btrfs: split btrfs_map_block into two parts for direct I/O.

2010-03-21 Thread jim owens
creation of the btrfs_multi_bio struct now calls btrfs_map_to_stripe(). Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/volumes.c | 85 +++ fs/btrfs/volumes.h | 10 ++ 2 files changed, 62 insertions(+), 33 deletions(-) diff --git a/fs

[PATCH V3 09/18] Btrfs: add direct I/O helper routines to decode em map_lookup.

2010-03-21 Thread jim owens
Direct I/O needs the total number of block devices and the struct block_device for each to allocate control structures prior to building bios for an extent. When building a bio, the device's base physical address of the chunk is needed. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs

[PATCH V3 10/18] Btrfs: add direct I/O helper to process inline compressed extents.

2010-03-21 Thread jim owens
Use access_extent_buffer_page() to point at btree location of inline compressed data so it can be inflated without a memcopy. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/extent_io.c | 16 fs/btrfs/extent_io.h |8 ++-- 2 files changed, 22 insertions(+), 2

[PATCH V3 11/18] Btrfs: add direct I/O checksum option to btrfs_lookup_csums_range.

2010-03-21 Thread jim owens
Direct I/O needs to efficiently fetch an extent range of checksums so an option is added to copy raw checksums into a buffer instead of locally allocating and returning btrfs_ordered_sum structs. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/ctree.h |2 +- fs/btrfs/file

[PATCH V3 12/18] Btrfs: add direct I/O helper to return mirror count from extent_map.

2010-03-21 Thread jim owens
Direct I/O retry needs to know the number of mirror copies. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/volumes.c | 22 ++ fs/btrfs/volumes.h |1 + 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c

[PATCH V3 13/18] Btrfs: add decompression code for direct I/O.

2010-03-21 Thread jim owens
. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/compression.h | 20 +++ fs/btrfs/zlib.c| 142 +--- 2 files changed, 154 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 421f5b4

[PATCH V3 15/18] Btrfs: change btrfs_get_extent for direct I/O merges.

2010-03-21 Thread jim owens
Direct I/O passes unaligned start and length values that must be adjusted to block boundaries for lookup and merging when an add_extent_mapping fails with EEXIST. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/inode.c | 56 - 1

[PATCH V3 16/18] Btrfs: change ordered data tree search to use start and length.

2010-03-21 Thread jim owens
Users of btrfs_lookup_first_ordered_extent() want to detect only a real overlap, so searching for the range instead of just an address prevents taking references on entries we don't want. This also simplifies the code. Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/file.c

[PATCH V3 18/18] Btrfs: enable direct I/O read support.

2010-03-21 Thread jim owens
Signed-off-by: jim owens owens6...@gmail.com --- fs/btrfs/Makefile |2 +- fs/btrfs/inode.c |7 ++- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile index a35eb36..5cbe798 100644 --- a/fs/btrfs/Makefile +++ b/fs/btrfs/Makefile

extent map merge bad block_len

2010-03-17 Thread jim owens
Chris, Something that probably should be fixed is how merging extent maps with block_len == -1 produces illegal lengths, as in 8191. I saw it with holes in directIO and it is not the cause of my current problems so I'll hope someone else decides to fix. jim -- To unsubscribe from this list:

[RFC 0/2] removing hard coded 512 byte size from direct I/O.

2010-03-05 Thread jim owens
The following patches add the field for tracking the smallest device block size in the filesystem and using it instead of the hard coded 512 byte values in dio.c. I also implemented a simpler test for user misalignment on devices with larger block sizes. It passes fsx, but I have not tested

[PATCH 1/2] Btrfs: add multi-device minimum logical block size for direct I/O.

2010-03-05 Thread jim owens
In a multi-device filesystem, it is possible to have devices with different block sizes, as in 512, 1024, 2048, 4096. DirectIO read will check user request alignment is valid for at least one device. Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/volumes.c | 24

[PATCH 2/2] Btrfs: change dio.c to use dio_min_blocksize instead of 512.

2010-03-05 Thread jim owens
Instead of hard coding the minimum I/O alignment, use the smallest bdev_logical_blocksize in the filesystem. Also change the alignment tests to determine the real user request minimum alignment and make all eof tail and device checks on that user blocksize. Signed-off-by: jim owens jim6

[PATCH 2/2] Btrfs: change dio.c to use dio_min_blocksize instead of 512.

2010-03-05 Thread jim owens
Instead of hard coding the minimum I/O alignment, use the smallest bdev_logical_blocksize in the filesystem. Also change the alignment tests to determine the real user request minimum alignment and make all eof tail and device checks on that user blocksize. Signed-off-by: jim owens jim6

[PATCH V2] Btrfs: remove dead code for unplugging pages.

2010-03-03 Thread jim owens
The code in btrfs_unplug_io_fn() to selectively unplug just the stripes affected by a page had been previously disabled. The unneeded code for unplug_page in __btrfs_map_block made changes for btrfs directio more difficult so I'm removing all of it. Signed-off-by: jim owens jow...@hp.com Signed

[PATCH V2] Btrfs: add direct I/O helper to process inline compressed extents.

2010-03-03 Thread jim owens
Use access_extent_buffer_page() to point at btree location of inline compressed data so it can be inflated without a memcopy. Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 fixes whitespace checkpatch warning fs/btrfs/extent_io.c | 16

PATCH V2] Btrfs: add direct I/O checksum option to btrfs_lookup_csums_range.

2010-03-03 Thread jim owens
Direct I/O needs to efficiently fetch an extent range of checksums so an option is added to copy raw checksums into a buffer instead of locally allocating and returning btrfs_ordered_sum structs. Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 changes

[PATCH V2] Btrfs: add direct I/O helper to return mirror count from extent_map.

2010-03-03 Thread jim owens
Direct I/O retry needs to know the number of mirror copies. Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 fixes whitespace checkpatch warning fs/btrfs/volumes.c | 22 ++ fs/btrfs/volumes.h |1 + 2 files changed, 15

[PATCH V2] Btrfs: add decompression code for direct I/O.

2010-03-03 Thread jim owens
. Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 fixes whitespace checkpatch warning and removes JIM testing printk fs/btrfs/compression.h | 20 +++ fs/btrfs/zlib.c| 142 +--- 2 files changed, 154

[PATCH V2] Btrfs: split btrfs_map_block into two parts for direct I/O.

2010-03-03 Thread jim owens
creation of the btrfs_multi_bio struct now calls btrfs_map_to_stripe(). Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 fixes long line checkpatch warning fs/btrfs/volumes.c | 85 +++ fs/btrfs/volumes.h

[PATCH V2] Btrfs: add direct I/O helper to process inline compressed extents.

2010-03-03 Thread jim owens
Use access_extent_buffer_page() to point at btree location of inline compressed data so it can be inflated without a memcopy. Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens jim6...@gmail.com --- V2 fixes long line checkpatch warnings fs/btrfs/extent_io.c | 16

[PATCH dio.c V2 1/6] Btrfs: fix indents and checkpatch issues in dio.c.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c | 247 1 files changed, 142 insertions(+), 105 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index 3315cc9..84277a9 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs

[PATCH dio.c V2 2/6] Btrfs: change size_t to u32 in dio.c for 32-bit compile.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c | 24 1 files changed, 12 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index 84277a9..97ab77f 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c @@ -1487,8 +1487,8 @@ fail

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-03-03 Thread jim owens
The following 6 patches [dio.c V2 #/6] are code cleanup from your review and my testing on a 32 bit laptop. These patches do not have any changes to remove the hard coded 511/512 or i_mutex. That will come later. These patches are only to make it easier to see what I have done, I will roll them

[PATCH dio.c V2 3/6] Btrfs: minor code corrections in dio.c from Josef.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c | 23 --- 1 files changed, 12 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index 97ab77f..9f53577 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c @@ -199,7 +199,7 @@ static void

[PATCH dio.c V2 4/6] Btrfs: remove blocksize from diocb.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c | 72 1 files changed, 36 insertions(+), 36 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index 9f53577..ad9b58f 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c

[PATCH dio.c V2 5/6] Btrfs: remove lockend from diocb.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c | 21 ++--- 1 files changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index ad9b58f..05a7445 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c @@ -127,7 +127,6 @@ struct

[PATCH dio.c V2 6/6] Btrfs: fix goto fail mistakes in btrfs_dio_read.

2010-03-03 Thread jim owens
Signed-off-by: jim owens jim6...@gmail.com --- fs/btrfs/dio.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c index 05a7445..b1beafc 100644 --- a/fs/btrfs/dio.c +++ b/fs/btrfs/dio.c @@ -459,7 +459,7 @@ getlock: end

Re: BackupPC, per-dir hard link limit, Debian packaging

2010-03-02 Thread jim owens
Hubert Kario wrote: On Tuesday 02 March 2010 03:29:05 Robert Collins wrote: As I say, I realise this is queued to get addressed anyway, but it seems like a realistic thing for people to do (use BackupPC on btrfs) - even if something better still can be written to replace the BackupPC store in

Re: fsck.btrfs dumps core

2010-02-25 Thread jim owens
Adam Kłobukowski wrote: Hello I'm running Ubuntu Lucid (2.6.32-14-generic-pae #20-Ubuntu SMP Sat Feb 20 07:07:46 UTC 2010 i686 GNU/Linux) Doing online fsck.btrfs on 2TB volume dumps core: fsck with the fs mounted is not supported. there is a pending patch for fsck.btrfs to prevent it from

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-16 Thread jim owens
Chris Mason wrote: [ use i_mutex for reads? ] But, we already need the code that btrfs_page_mkwrite uses. It should be enough to wait for the ordered extents and have the extent range locked. You don't mean have the lock_extent active while I issue the btrfs_wait_ordered_range as I found

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-16 Thread jim owens
Josef Bacik wrote: On Wed, Feb 10, 2010 at 01:53:50PM -0500, jim owens wrote: + +static int btrfs_dio_hole_read(struct btrfs_diocb *diocb, u64 hole_len) +{ + int err = 0; + diocb-umc.todo = hole_len; + while (diocb-umc.todo) { + struct bio_vec uv

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-15 Thread jim owens
Christoph Hellwig wrote: On Mon, Feb 15, 2010 at 11:42:55AM -0500, Chris Mason wrote: We can record the smallest alignment while scanning the devices and just use that. In general 4K is good enough, although some other database company might be more worried about smaller numbers. How is

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-15 Thread jim owens
rk wrote: http://www.osnews.com/story/22872/Linux_Not_Fully_Prepared_for_4096-Byte_Sector_Hard_Drives Yes, partition alignment to the 4k drive block is critical, but that is not an direct IO issue, it is mkfs.btrfs and just as important for cache IO. And in any case, many people will use 512

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-15 Thread jim owens
Christoph Hellwig wrote: On Mon, Feb 15, 2010 at 05:26:34PM -0500, jim owens wrote: My understanding is the current 4k drives normally operate in 512 byte read/write access mode unless you set them to run as 4k only. In 512 byte mode, they buffer internally on writes. It is probably just

Re: [PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-13 Thread jim owens
Josef Bacik wrote: On Wed, Feb 10, 2010 at 01:53:50PM -0500, jim owens wrote: + /* traditional 512-byte device sector alignment is the +* minimum required. if they have a larger sector disk +* (possibly multiple sizes in the filesystem) and need + * a larger alignment

[PATCH V2] Btrfs: Full direct I/O and AIO read implementation.

2010-02-10 Thread jim owens
Signed-off-by: jim owens jow...@hp.com Signed-off-by: jim owens owens6...@gmail.com --- V2 is a merge of my original file: http://article.gmane.org/gmane.comp.file-systems.btrfs/4530 and the fixes produced from Josef Bacik's fsx testing: http://article.gmane.org/gmane.comp.file-systems.btrfs

Re: RAID-10 arrays built with btrfs md report 2x difference in available size?

2010-01-29 Thread jim owens
RK wrote: I think so too -- I have six 1TB drives on RAID-10 btrfs and it shows that I have 5.5TB free space .. how that can be ? # df -h FilesystemSize Used Avail Use% Mounted on /dev/sde1 66G 3.8G 59G 7% / /dev/sda 5.5T 28K 5.5T 1%

Re: [RFC INFO PATCH 02/03] export use_mm and unuse_mm for filesystem modules.

2010-01-20 Thread jim owens
Liuwenyi wrote: Maybe, you should send this patch to Andrew Morton a...@linux-foundation.org Andrea Arcangeli aarca...@redhat.com Michael S. Tsirkin m...@redhat.com linux-ker...@vger.kernel.org linux...@kvack.org I said in [00] that I sent this patch to Andrew. What I did not say was it

Re: [RFC INFO PATCH 02/03] export use_mm and unuse_mm for filesystem modules.

2010-01-20 Thread jim owens
jim owens wrote: What I did not say was it was sent on December 9, 2009 to every addess on your list, except for Andrea. OK, as josef told me, the patch is corrupt and will not apply. I am sending a new one I just made against btrfs-unstable to linux-btrfs and will create a new patch

[RFC PATCH 00/03] Btrfs Direct I/O for .34 maybe

2010-01-19 Thread jim owens
This is the completed code for direct I/O read. It handles all extent types, validates checksums, and does retries. I have tested all the paths (on X86-64) but it needs testing with automation in a larger setup. If this design is acceptable, Eric Whitney will run those tests. The code supports

[RFC PATCH 01/03] Btrfs: Full direct I/O and AIO read implementation.

2010-01-19 Thread jim owens
Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/dio.c | 1902 1 files changed, 1902 insertions(+), 0 deletions(-) create mode 100644 fs/btrfs/dio.c diff --git a/fs/btrfs/dio.c b/fs/btrfs/dio.c new file mode 100644 index 000

[RFC PATCH 03/03] Btrfs: enable direct I/O read support.

2010-01-19 Thread jim owens
Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/Makefile |2 +- fs/btrfs/inode.c |7 ++- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile index a35eb36..5cbe798 100644 --- a/fs/btrfs/Makefile +++ b/fs/btrfs/Makefile @@ -7,4

Re: Metadata Duplication on a Single Disk Btrfs Setup

2010-01-12 Thread jim owens
Mitch Harder wrote: On a single disk btrfs setup, such as for a desktop computer, what are the implecations of creating your btrfs partition with '-m single'? At first, I assumed I would want a single disk desktop setup to be configured as 'single'. But that may not be the case for

[RFC PATCH 01/01] Btrfs: add decompression code for direct I/O.

2010-01-08 Thread jim owens
. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/compression.h | 20 +++ fs/btrfs/zlib.c| 143 +--- 2 files changed, 155 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 421f5b4..afd262d

Re: What protection does btrfs checksumming currently give? (Was Re: btrfs volume mounts and dies (was Re: Segfault in btrfsck))

2010-01-07 Thread jim owens
Steve Freitas wrote: Hi all, I was under the mistaken impression that btrfs checksumming, in its current default configuration, protected your data from bitrot. It appears this is not the case: On Wed, 2010-01-06 at 18:24 +0100, Johannes Hirte wrote: Am Mittwoch 06 Januar 2010 16:59:55

[RFC 01/12 RESEND PATCH] Btrfs: Do not limit RAID1 and DUP transfer length to one stripe.

2010-01-04 Thread jim owens
The 65k stripe length should be ignored as the stripes are physically contiguous on disk so transfers can span stripes. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/volumes.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs

[RFC 02/12 RESEND PATCH] Btrfs: __btrfs_map_block should not set length more than input length.

2010-01-04 Thread jim owens
Returning a value greater than the caller's is ugly and prone to dangerous future coding mistakes. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/extent-tree.c |3 --- fs/btrfs/inode.c |5 ++--- fs/btrfs/volumes.c |9 + 3 files changed, 7 insertions(+), 10

[RFC 07/12 PATCH] Btrfs: split btrfs_map_block into two parts for direct I/O.

2010-01-04 Thread jim owens
creation of the btrfs_multi_bio struct now calls btrfs_map_to_stripe(). Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/volumes.c | 84 +++ fs/btrfs/volumes.h | 10 ++ 2 files changed, 61 insertions(+), 33 deletions(-) diff --git a/fs/btrfs

[RFC 08/12 PATCH] Btrfs: add direct I/O helper routines to decode em map_lookup.

2010-01-04 Thread jim owens
Direct I/O needs the total number of block devices and the struct block_device for each to allocate control structures prior to building bios for an extent. When building a bio, the device's base physical address of the chunk is needed. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs

[RFC 09/12 PATCH] Btrfs: add direct I/O helper to process inline compressed extents.

2010-01-04 Thread jim owens
Use access_extent_buffer_page() to point at btree location of inline compressed data so it can be inflated without a memcopy. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/extent_io.c | 16 fs/btrfs/extent_io.h |3 +++ 2 files changed, 19 insertions(+), 0 deletions

[RFC 10/12 PATCH] Btrfs: add direct I/O checksum option to btrfs_lookup_csums_range.

2010-01-04 Thread jim owens
Direct I/O needs to efficiently fetch an extent range of checksums so an option is added to copy raw checksums into a buffer instead of locally allocating and returning btrfs_ordered_sum structs. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/ctree.h |2 +- fs/btrfs/file-item.c

[RFC 00/12] btrfs core patches for direct I/O

2010-01-04 Thread jim owens
As if a Monday after a long holiday was not bad enough... this series has some simple patches in existing btrfs code that I want to make for btrfs directio code. The RESEND patches 1-5 were sent to Chris in December because IMO they have value without my directio code. You need them to apply

[RFC 11/12 PATCH] Btrfs: add direct I/O helper btrfs_lookup_one_csum().

2010-01-04 Thread jim owens
Direct I/O needs to fetch a single checksum based on the extent information instead of page struct content. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/ctree.h |2 ++ fs/btrfs/file-item.c | 45 + 2 files changed, 47 insertions(+), 0

[RFC 05/12 RESEND PATCH] Btrfs: remove dead code for unplugging pages.

2010-01-04 Thread jim owens
The code in btrfs_unplug_io_fn() to selectively unplug just the stripes affected by a page had been previously disabled. The unneeded code for unplug_page in __btrfs_map_block made changes for btrfs directio more difficult so I'm removing all of it. Signed-off-by: jim owens jow...@hp.com

Re: [Jfs-discussion] benchmark results

2009-12-27 Thread jim owens
Christian Kujau wrote: On 26.12.09 08:00, jim owens wrote: I was using sync to make sure that the data should be on the disks Good, but not good enough for many tests... info sync [...] On Linux, sync is only guaranteed to schedule the dirty blocks for writing; it can

PATCH] Do not limit RAID1 and DUP transfer length to one stripe

2009-12-17 Thread jim owens
The 65k stripe length should be ignored as the stripes are physically contiguous on disk so transfers can span stripes. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/volumes.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs

[PATCH] __btrfs_map_block should not set length more than input length.

2009-12-17 Thread jim owens
Returning a value greater than the caller's is ugly and prone to dangerous future coding mistakes. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/extent-tree.c |3 --- fs/btrfs/inode.c |5 ++--- fs/btrfs/volumes.c |9 + 3 files changed, 7 insertions(+), 10

Btrfs: btrfs_discard_extent must use WRITE with btrfs_map_block.

2009-12-17 Thread jim owens
WRITE is needed on btrfs_map_block() to fetch all raid stripes, READ only returns one device and we want to discard all copies. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/extent-tree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs

Re: UI issues around RAID1

2009-11-17 Thread jim owens
Andrey Kuzmin wrote: On Tue, Nov 17, 2009 at 12:48 AM, jim owens jow...@hp.com wrote: But as we have said many times... if we have different raid types active on different files, any attempt to make Late question, but could you please explain it a bit further (or point me to respective

Re: Reporting free space to userspace programs

2009-10-30 Thread jim owens
Leszek Ciesielski wrote: Hi, the results of running 'df' against a btrfs volume are somewhat unintuitive from a user point of view. On a single drive btrfs volume, created with 'mkfs.btrfs -m raid1 -d raid1 /dev/sda6', I am getting the following result: /dev/sda6 1.4T 594G 804G

Re: Mass-Hardlinking Oops

2009-10-12 Thread jim owens
Goffredo Baroncelli wrote: I don't know a software which need so many hard links. But it easy to find some similar cases. For example under my /usr/bin I have 478 _soft links_ to _different_ files. Hard link is not used in place of soft link... soft link is a different and preferred

[PATCH] don't OOPs when we are not raid56

2009-09-02 Thread jim owens
David Woodhouse wrote: http://git.infradead.org/users/dwmw2/btrfs-raid56.git Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/volumes.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 95babc1..913c29f 100644 --- a/fs

Re: [RFC] Early look at btrfs directIO read code

2009-09-02 Thread jim owens
Still not ready for review or for anyone else to try it, this V2 update fixes the bugs I have found in my testing. New things that now seem to work: * multiple disks (btrfs raid 0,1,10,5,6) * AIO * multiple buffer vectors * files that are not a multiple of 512 bytes jim

Re: New idea about RAID and SSD

2009-09-01 Thread jim owens
Massimo Maggi wrote: Hi, SSDs have low latency but a high price per GB, Traditional hard disks have high latency, but high sequential read/write speed and low price per GB. Is possibile to use a SSD for metadata, which requires many seeks and is relatively small, in a special RAID mode with a

[RFC] Early look at btrfs directIO read code

2009-08-24 Thread jim owens
This is my still-working-on-it code for btrfs directIO read. I'm posting it so people can see the progress being made on the project and can take an early shot at telling me this is just a bad idea and I'm crazy if they want to, or point out where I made some stupid mistake with btrfs core

Re: [RFC] Early look at btrfs directIO read code

2009-08-24 Thread jim owens
Arrrgh... If I did not already scare you enough to stay away from trying this code, I already hit another bug. Proving the axiom that you will find a bad bug as soon as you send your code to anyone. * non-inline file size must be a multiple of device sector size. jim -- To unsubscribe from

Re: Read errors while benchmarking

2009-06-24 Thread jim owens
Roy Sigurd Karlsbakk wrote: Currently, btrfs doesn't support using direct I/O for read. Is anyone working on this? It /is/ a rather basic feature, and testing with iozone would be nice. Yes, I am working on it. I will be away for the next 10 days but hope to have something for read tests

Re: Data Deduplication with the help of an online filesystem check

2009-04-28 Thread jim owens
Andrey Kuzmin wrote: On Tue, Apr 28, 2009 at 2:02 PM, Chris Mason chris.ma...@oracle.com wrote: On Tue, 2009-04-28 at 07:22 +0200, Thomas Glanzmann wrote: Hello Chris, There is a btrfs ioctl to clone individual files, and this could be used to implement an online dedup. But, since it is

Re: Btrfs TODO

2009-04-09 Thread jim owens
I'm working on O_DIRECT support... but with checksumming. I've also been looking at AIO but directio is first. jim -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

[PATCH] remove unused ftrace include

2009-03-25 Thread jim owens
ftrace does not exist in earlier kernels and requires backport patching Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/async-thread.c |1 - fs/btrfs/delayed-ref.c |1 - 2 files changed, 0 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async

Re: metadata copied/data not copied?

2009-03-16 Thread jim owens
CSights wrote: Hi everyone, I'm curious what would happen in btrfs if the following commands were issued: # cp file1 file2 # chown newuser:newgroup file2 Where file1 was owned by olduser:oldgroup. If I understand copy-on-write correctly the cp would merely create a new pointer (or

Re: experimental branch rebased

2009-03-09 Thread jim owens
Lee Trager wrote: Sorry it took so long to reply its been a crazy week for me. You're lucky it was only a week, I often have crazy months! I can't believe I made such a stupid mistake. Back when I started in computers, I thought 16 bits would be more than I would ever need to count the

Re: [PATCH] Btrfs: take block group fragmentation into account for allocation

2009-03-09 Thread jim owens
Josef Bacik wrote: On Mon, Mar 09, 2009 at 03:21:06PM -, Oliver Mattos wrote: So the idea of the function is to return an integer in the range [0,100]? Why are we using a range of 0 to 100 anyway? 100 seems like an arbitary value for kernel space - why not just keep it as a value in the

Re: experimental branch rebased

2009-03-03 Thread jim owens
Lee, The 02/26/2009 version is missing some of the patches that were in the 02/20/2009 version so it will not compile on 26. I found the bug in the patch that breaks on 26: +#if LINUX_VERSION_CODE = KERNEL_VERSION(2, 6, 27) if (!trylock_page(page)) +#else +

Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch

2009-02-24 Thread jim owens
Lee Trager wrote: The more and more I look at this problem the more I tend to think that the issue is because of some change in the way the VFS or something interacts with the file system. Does anyone know of any big changes? Why is the inode being marked dirty? Is there some kind of read error.

Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch

2009-02-20 Thread jim owens
lee, A couple of thoughts about your .26 lockup: - I assume you are on the same hardware with 27. - Are you using a module or builtin btrfs (I build it in). - It looks like you are using vmware... when I'm doing kernel stuff I want to be on the iron, not trusting virtual machine code.

Re: Warning and BUG with btrfs and corrupted image

2009-02-05 Thread jim owens
Pavel Machek wrote: If you don't want it, don't compile it in. The Kconfig text is very clear. No, I'd not expect that option to panic systems. That's why I suggested: diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig index 29228f5..b7ac847 100644 --- a/fs/xfs/Kconfig +++ b/fs/xfs/Kconfig @@

[PATCH] Btrfs: support security xattr with SELinux enabled

2009-01-07 Thread jim owens
Add call to LSM security initialization and save resulting security xattr for new inodes. Add xattr support to symlink inode ops. Set inode-i_op for existing special files. Signed-off-by: jim owens jow...@hp.com --- fs/btrfs/inode.c | 23 +++ fs/btrfs/xattr.c | 32

Re: [PATCH] Btrfs: support security xattr with SELinux enabled

2009-01-07 Thread jim owens
Josef Bacik wrote: On Wed, Jan 07, 2009 at 03:19:38PM -0500, jim owens wrote: +int btrfs_xattr_security_init(struct inode *inode, struct inode *dir) +{ + int err; + size_t len; + void *value; + char *suffix; + char *name; + + err

Re: Selective Compression/Encryption

2008-12-09 Thread jim owens
Diego Calleja wrote: El Tue, 9 Dec 2008 13:09:51 -0500, Lee Trager [EMAIL PROTECTED] escribió: It does seem that doing it with volumes would limit user control and add lots of complexity to such a simple task. IMHO, WRT compression it's the contrary. Compression on a per-file basis has never

Re: Some very basic questions

2008-10-22 Thread jim owens
Ric Wheeler wrote: Matthias Wächter wrote: On 10/22/2008 3:50 PM, Chris Mason wrote: Let me reword my answer ;). The next write will always succeed unless the drive is out of remapping sectors. If the drive is out, it is only good for reads and holding down paper on your desk. I

Re: Some very basic questions

2008-10-22 Thread jim owens
Michel Salim wrote: Though it would be nice to have a tool that would provide enough information to make a warranty claim -- does btrfs keep enough information for such a tool to be written? Failed device I/O (rather than bad checksums and other fs-specific error detections) should be logged

Re: Btrfs conf call

2008-08-13 Thread jim owens
Morey Roof wrote: Could someone write a quick recap of this for those of us who are interested and happened to miss it? Everyone on the call seems to be waiting for someone else to respond. I'll do it just to make up for dedup :) - Chris said he saw major performance problems from pdflush

Re: Btrfs conf call

2008-08-13 Thread jim owens
Morey Roof wrote: Thank you. Also, did they get around to talking about when they are going to merge in some of the queued up patches? We have never discussed code merges on any calls. Probably because Chris does all that merge work. I would not expect anything until he finishes the flush

Re: Hot topics for the next release

2008-08-06 Thread jim owens
Improved allocator threading I wanted to work on the allocator with a larger scope where threading is only a minor part of trying to address these items from the Project_ideas that I think could change disk format in some way (to fix it before v1.0): - Different sector sizes - Multiple chunk

Re: Hot topics for the next release

2008-08-06 Thread jim owens
Chris Mason wrote: Josef's allocator fix is on the list because we currently fall over in some workloads at 100% cpu time when the FS is 60% full. The space indexing is complex and strange, it just needs to be redone. I don't understand. Do you mean josef wants someone to fix multithreading