Re: Recommended why to use btrfs for production?

On 06/03/2016 04:05 PM, Chris Murphy wrote: > Make certain the kernel command timer value is greater than the driver > error recovery timeout. The former is found in sysfs, per block > device, the latter can be get and set with smartctl. Wrong > configuration is common (it's actually the default)

Re: btrfs

On Sat, Jun 4, 2016 at 4:43 PM, Christoph Anton Mitterer wrote: > On Sat, 2016-06-04 at 13:13 -0600, Chris Murphy wrote: >> mdadm supports DDF. > > Sure... it also supports IMSM,... so what? Neither of them are the > default for mdadm, nor does it change the used

Re: RAID1 vs RAID10 and best way to set up 6 disks

On Sat, Jun 4, 2016 at 7:10 PM, Christoph Anton Mitterer wrote: > Well the RAID1 was IMHO still bad choice as it's pretty ambiguous. That's ridiculous. It isn't incorrect to refer to only 2 copies as raid1. You have to explicitly ask both mdadm and lvcreate for the number

RE: Recommended why to use btrfs for production?

On 06/05/2016 10:46 AM, Mladen Milinkovic wrote: > On 06/03/2016 04:05 PM, Chris Murphy wrote: > > Make certain the kernel command timer value is greater than the driver > > error recovery timeout. The former is found in sysfs, per block > > device, the latter can be get and set with smartctl.

[PATCH 45/45] block, drivers, fs: rename REQ_FLUSH to REQ_PREFLUSH

From: Mike Christie To avoid confusion between REQ_OP_FLUSH, which is handled by request_fn drivers, and upper layers requesting the block layer perform a flush sequence along with possibly a WRITE, this patch renames REQ_FLUSH to REQ_PREFLUSH. Signed-off-by: Mike Christie

[PATCH 42/45] block, fs, drivers: remove REQ_OP compat defs and related code

From: Mike Christie This patch drops the compat definition of req_op where it matches the rq_flag_bits definitions, and drops the related old and compat code that allowed users to set either the op or flags for the operation. We also then store the operation in the

[PATCH 43/45] block, drivers: add REQ_OP_FLUSH operation

From: Mike Christie This adds a REQ_OP_FLUSH operation that is sent to request_fn based drivers by the block layer's flush code, instead of sending requests with the request->cmd_flags REQ_FLUSH bit set. Signed-off-by: Mike Christie Reviewed-by:

[PATCH 19/45] mpage: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have the mpage code set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 26/45] drbd: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have drbd set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- v8: 1.

[PATCH 01/45] block/fs/drivers: remove rw argument from submit_bio

From: Mike Christie This has callers of submit_bio/submit_bio_wait set the bio->bi_rw instead of passing it in. This makes that use the same as generic_make_request and how we set the other bio fields. Signed-off-by: Mike Christie --- v8: 1. Fix bug

[PATCH 05/45] block, drivers, cgroup: use op_is_write helper instead of checking for REQ_WRITE

From: Mike Christie We currently set REQ_WRITE/WRITE for all non READ IOs like discard, flush, writesame, etc. In the next patches where we no longer set up the op as a bitmap, we will not be able to detect a operation direction like writesame by testing if REQ_WRITE is set.

[PATCH 18/45] hfsplus: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have gfs2 set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 14/45] btrfs: use bio fields for op and flags

From: Mike Christie The bio REQ_OP and bi_rw rq_flag_bits are now always setup, so there is no need to pass around the rq_flag_bits bits too. btrfs users should should access the bio insead. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig

[PATCH 30/45] block: copy bio op to request op

From: Mike Christie The bio users should now always be setting up the bio op. This patch has the block layer copy that to the request. Signed-off-by: Mike Christie --- block/blk-core.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff

[PATCH 04/45] fs: have ll_rw_block users pass in op and flags separately

From: Mike Christie This has ll_rw_block users pass in the operation and flags separately, so ll_rw_block can setup the bio op and bi_rw flags on the bio that is submitted. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig

[PATCH 16/45] gfs2: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have gfs2 set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 32/45] block: prepare mq request creation to use REQ_OPs

From: Mike Christie This patch modifies the blk mq request creation code to use separate variables for the operation and flags, because in the the next patches the struct request users will be converted like was done for bios. Signed-off-by: Mike Christie

[PATCH 22/45] pm: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have the pm code set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 06/45] dm: use op_is_write instead of checking for REQ_WRITE

From: Mike Christie We currently set REQ_WRITE/WRITE for all non READ IOs like discard, flush, writesame, etc. In the next patches where we no longer set up the op as a bitmap, we will not be able to detect a operation direction like writesame by testing if REQ_WRITE is set.

[PATCH 17/45] xfs: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have xfs set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie --- v8: 1. Handled changes due to rebase and dropped signed offs due to upstream changes since last

[PATCH 27/45] md: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have md set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 03/45] fs: have submit_bh users pass in op and flags separately

From: Mike Christie This has submit_bh users pass in the operation and flags separately, so submit_bh_wbc can setup the bio op and bi_rw flags on the bio that is submitted. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig

[PATCH 24/45] dm: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have dm set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie v8: - Moved op_is_write changes to its own patch. - Dropped signed offs due to changes in dm. ---

[PATCH 20/45] nilfs: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have nilfs set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Acked-by:

[PATCH 25/45] bcache: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have bcache set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie --- drivers/md/bcache/btree.c | 4 ++-- drivers/md/bcache/debug.c | 4 ++--

[PATCH 29/45] xen: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have xen set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 02/45] block: add REQ_OP definitions and helpers

From: Mike Christie The following patches separate the operation (WRITE, READ, DISCARD, etc) from the rq_flag_bits flags. This patch adds definitions for request/bio operations (REQ_OPs) and adds request/bio accessors to get/set the op. In this patch the REQ_OPs match the

[PATCH 31/45] block: prepare request creation/destruction code to use REQ_OPs

From: Mike Christie This patch prepares *_get_request/*_put_request and freed_request, to use separate variables for the operation and flags. In the next patches the struct request users will be converted like was done for bios where the op and flags are set separately.

check if hardware checksumming works or not

Hi, I'm running Debian ARM on a Marvell Kirkwood-based 2-disk NAS. Kirkwood SoCs have a XOR engine that can hardware-accelerate crc32c checksumming, and from what I see in kernel mailing lists it seems to have a linux driver and should be supported. I wanted to ask if there is a way to test

[PATCH 07/45] bcache: use op_is_write instead of checking for REQ_WRITE

From: Mike Christie We currently set REQ_WRITE/WRITE for all non READ IOs like discard, flush, writesame, etc. In the next patches where we no longer set up the op as a bitmap, we will not be able to detect a operation direction like writesame by testing if REQ_WRITE is set.

Re: btrfs

>> > - OTOH, defrag seems to be viable for important use cases (VM >> > images, >> > DBs,... everything where large files are internally re-written >> > randomly). >> > Sure there is nodatacow, but with that one effectively completely >> > looses one of the core features/promises of btrfs

Re: btrfs

On Sun, 2016-06-05 at 09:51 -0600, Chris Murphy wrote: > Why is mdadm the reference point for terminology? I haven't said it is,... I just said it mdadm, original paper, WP use it the common/historic way. And since all of these were there before btrfs, and in the case of mdadm/MD "in" the

Re: [PATCH 42/45] block, fs, drivers: remove REQ_OP compat defs and related code

Hi, [auto build test WARNING on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: btrfs

On Sun, 2016-06-05 at 22:39 +0200, Henk Slager wrote: > > So the point I'm trying to make: > > People do probably not care so much whether their VM image/etc. is > > COWed or not, snapshots/etc. still work with that,... but they may > > likely care if the integrity feature is lost. > > So IMHO,

[PATCH 35/45] block: convert merge/insert code to check for REQ_OPs.

From: Mike Christie This patch converts the block layer merging code to use separate variables for the operation and flags, and to check req_op for the REQ_OP. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by:

Re: check if hardware checksumming works or not

Hi Alberto, On 5 June 2016 at 15:37, Alberto Bursi wrote: > > Hi, I'm running Debian ARM on a Marvell Kirkwood-based 2-disk NAS. > > Kirkwood SoCs have a XOR engine that can hardware-accelerate crc32c > checksumming, and from what I see in kernel mailing lists it seems

Re: RAID1 vs RAID10 and best way to set up 6 disks

On Sun, 2016-06-05 at 09:36 -0600, Chris Murphy wrote: > That's ridiculous. It isn't incorrect to refer to only 2 copies as > raid1. No, if there are only two devices then not. But obviously we're talking about how btrfs does RAID1, in which even with n>2 devices there are only 2 copies - that's

Re: [PATCH 01/45] block/fs/drivers: remove rw argument from submit_bio

Hi, [auto build test ERROR on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: btrfs

On Sun, Jun 05, 2016 at 10:56:45PM +0200, Christoph Anton Mitterer wrote: > On Sun, 2016-06-05 at 22:39 +0200, Henk Slager wrote: > > > So the point I'm trying to make: > > > People do probably not care so much whether their VM image/etc. is > > > COWed or not, snapshots/etc. still work with

[PATCH 28/45] target: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have the target layer set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie --- drivers/target/target_core_iblock.c | 29 ++---

[PATCH 41/45] block, drivers, fs: shrink bi_rw from long to int

From: Mike Christie We don't need bi_rw to be so large on 64 bit archs, so reduce it to unsigned int. Signed-off-by: Mike Christie --- block/blk-core.c | 2 +- drivers/md/dm-flakey.c | 2 +- drivers/md/raid5.c | 6 +++---

[PATCH 33/45] block: prepare elevator to use REQ_OPs.

From: Mike Christie This patch converts the elevator code to use separate variables for the operation and flags, and to check req_op for the REQ_OP. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke

[PATCH 21/45] ocfs2: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have ocfs2 set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 44/45] block: do not use REQ_FLUSH for tracking flush support

From: Mike Christie The last patch added a REQ_OP_FLUSH for request_fn drivers and the next patch renames REQ_FLUSH to REQ_PREFLUSH which will be used by file systems and make_request_fn drivers so they can send a write/flush combo. This patch drops xen's use of REQ_FLUSH

[PATCH 38/45] blktrace: use op accessors

From: Mike Christie Have blktrace use the req/bio op accessor to get the REQ_OP. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- v8: 1. Fix REQ_OP_WRITE_SAME handling, so it is

[PATCH 34/45] blkg_rwstat: separate op from flags

From: Mike Christie The bio and request operation and flags are going to be separate definitions, so we cannot pass them in as a bitmap. This patch converts the blkg_rwstat code and its caller, cfq, to pass in the values separately. Signed-off-by: Mike Christie

[PATCH 39/45] ide cd: do not set REQ_WRITE on requests.

From: Mike Christie The block layer will set the correct READ/WRITE operation flags/fields when creating a request, so there is not need for drivers to set the REQ_WRITE flag. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig

[PATCH 23/45] dm: pass dm stats data dir instead of bi_rw

From: Mike Christie It looks like dm stats cares about the data direction (READ vs WRITE) and does not need the bio/request flags. Commands like REQ_FLUSH, REQ_DISCARD and REQ_WRITE_SAME are currently always set with REQ_WRITE, so the extra check for REQ_DISCARD in

[PATCH 37/45] drivers: use req op accessor

From: Mike Christie The req operation REQ_OP is separated from the rq_flag_bits definition. This converts the block layer drivers to use req_op to get the op from the request struct. Signed-off-by: Mike Christie --- drivers/block/loop.c |

[PATCH 36/45] block: convert is_sync helpers to use REQ_OPs.

From: Mike Christie This patch converts the is_sync helpers to use separate variables for the operation and flags. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 40/45] block: move bio io prio to a new field

From: Mike Christie In the next patch, we move drop the compat code and make the op a separate value that is hidden in bi_rw. To give the op and rq bits flags room to grow this moves prio to its own field. Signed-off-by: Mike Christie ---

[PATCH 11/45] btrfs: have submit_one_bio users use bio op accessors

From: Mike Christie This patch has btrfs's submit_one_bio users set the bio op using bio_set_op_attrs and get the op using bio_op. The next patches will continue to convert btrfs, so submit_bio_hook and merge_bio_hook related code will be modified to take only the bio. I

[PATCH 09/45] block discard: use bio set op accessor

From: Mike Christie This converts the block issue discard helper and users to use the bio_set_op_attrs accessor and only pass in the operation flags like REQ_SEQURE. Signed-off-by: Mike Christie --- block/blk-lib.c| 13 +++--

[PATCH 15/45] f2fs: use bio op accessors

From: Mike Christie Separate the op from the rq_flag_bits and have f2fs set/get the bio using bio_set_op_attrs/bio_op. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke ---

[PATCH 13/45] btrfs: update __btrfs_map_block for REQ_OP transition

From: Mike Christie We no longer pass in a bitmap of rq_flag_bits bits to __btrfs_map_block. It will always be a REQ_OP, or the btrfs specific REQ_GET_READ_MIRRORS, so this drops the bit tests. Signed-off-by: Mike Christie Reviewed-by: Christoph

[PATCH 00/45] v8: separate operations from flags in the bio/request structs

The following patches begin to cleanup the request->cmd_flags and bio->bi_rw mess. We currently use cmd_flags to specify the operation, attributes and state of the request. For bi_rw we use it for similar info and also the priority but then also have another bi_flags field for state. At some

[PATCH 08/45] block, fs, mm, drivers: use bio set/get op accessors

From: Mike Christie This patch converts the simple bi_rw use cases in the block, drivers, mm and fs code to set/get the bio operation using bio_set_op_attrs/bio_op These should be simple one or two liner cases, so I just did them in one patch. The next patches handle the

[PATCH 12/45] btrfs: use bio op accessors

From: Mike Christie This should be the easier cases to convert btrfs to bio_set_op_attrs/bio_op. They are mostly just cut and replace type of changes. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig Reviewed-by: Hannes

[PATCH 10/45] direct-io: use bio set/get op accessors

From: Mike Christie This patch has the dio code use a REQ_OP for the op and rq_flag_bits for bi_rw flags. To set/get the op it uses the bio_set_op_attrs/bio_op accssors. It also begins to convert btrfs's dio_submit_t because of the dio submit_io callout use. The next

Re: Recommended why to use btrfs for production?

05.06.2016 19:33, James Johnston пишет: > On 06/05/2016 10:46 AM, Mladen Milinkovic wrote: >> On 06/03/2016 04:05 PM, Chris Murphy wrote: >>> Make certain the kernel command timer value is greater than the driver >>> error recovery timeout. The former is found in sysfs, per block >>> device, the

Re: [PATCH 01/45] block/fs/drivers: remove rw argument from submit_bio

Hi, [auto build test WARNING on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: btrfs

On Sun, 2016-06-05 at 21:07 +, Hugo Mills wrote: >    The problem is that you can't guarantee consistency with > nodatacow+checksums. If you have nodatacow, then data is overwritten, > in place. If you do that, then you can't have a fully consistent > checksum -- there are always race

Re: [PATCH 42/45] block, fs, drivers: remove REQ_OP compat defs and related code

Hi, [auto build test ERROR on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: RAID1 vs RAID10 and best way to set up 6 disks

On Sun, Jun 5, 2016 at 2:31 PM, Christoph Anton Mitterer wrote: > On Sun, 2016-06-05 at 09:36 -0600, Chris Murphy wrote: >> That's ridiculous. It isn't incorrect to refer to only 2 copies as >> raid1. > No, if there are only two devices then not. > But obviously we're

Re: [PATCH 42/45] block, fs, drivers: remove REQ_OP compat defs and related code

Hi, [auto build test ERROR on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH 01/45] block/fs/drivers: remove rw argument from submit_bio

Hi, [auto build test WARNING on v4.7-rc1] [cannot apply to dm/for-next md/for-next next-20160603] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

btrfs device delete /dev/sdc1 /mnt/raid1 user experience

Hi there, I planned to remove one of my disks, so that I can take it from Singapore to the UK and then re-establish another remote RAID1 store. delete is an alias of remove, so I added a new disk (devid 3) and proceeded to run: btrfs device delete /dev/sdc1 /mnt/raid1 (devid 1) nuc:~$ uname

Is it possible to block any writer inside a fs?

Hi, Is it possible to block any new writer from a fs? Just like what we do in remounting fs to readonly, although it's done in VFS. The case is, for example, we have an ioctl to control how buffered write works.(inband dedupe) And when changing/disabling such behavior, we need to ensure

Re: btrfs

On Sun, Jun 5, 2016 at 3:31 PM, Christoph Anton Mitterer wrote: > On Sun, 2016-06-05 at 21:07 +, Hugo Mills wrote: >>The problem is that you can't guarantee consistency with >> nodatacow+checksums. If you have nodatacow, then data is overwritten, >> in place. If you

Re: btrfs

On Fri, Jun 3, 2016 at 7:51 PM, Christoph Anton Mitterer wrote: > I think I remember that you've claimed that last time already, and as > I've said back then: > - what counts is probably the common understanding of the term, which > is N disks RAID1 = N disks mirrored >

Re: Recommended why to use btrfs for production?

On Sun, Jun 5, 2016 at 4:45 AM, Mladen Milinkovic wrote: > On 06/03/2016 04:05 PM, Chris Murphy wrote: >> Make certain the kernel command timer value is greater than the driver >> error recovery timeout. The former is found in sysfs, per block >> device, the latter can be

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

At 06/03/2016 10:02 PM, Josef Bacik wrote: On 06/01/2016 01:48 AM, Lu Fengqi wrote: Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was referred by different offset of same file, it should also be identified as

RE: Recommended why to use btrfs for production?

On 06/06/2016 at 01:47, Chris Murphy wrote: > On Sun, Jun 5, 2016 at 4:45 AM, Mladen Milinkovic > wrote: > > On 06/03/2016 04:05 PM, Chris Murphy wrote: > >> Make certain the kernel command timer value is greater than the driver > >> error recovery timeout. The former is