[PATCH 7/9] btrfs: Add support for recovery for a RAID 5 btrfs profiles.

2018-09-26 Thread Goffredo Baroncelli
On 25/09/2018 21.10, Daniel Kiper wrote: > On Wed, Sep 19, 2018 at 08:40:38PM +0200, Goffredo Baroncelli wrote: >> From: Goffredo Baroncelli >> >> Add support for recovery for a RAID 5 btrfs profile. In addition >> it is added some code as preparatory work for RAID 6

Re: [PATCH 9/9] btrfs: Add RAID 6 recovery for a btrfs filesystem.

2018-09-26 Thread Goffredo Baroncelli
On 25/09/2018 21.20, Daniel Kiper wrote: > On Wed, Sep 19, 2018 at 08:40:40PM +0200, Goffredo Baroncelli wrote: >> From: Goffredo Baroncelli >> [] >> * - stripe_offset is the disk offset, >> * - csize is the "potential"

[PATCH 4/9] btrfs: Avoid a rescan for a device which was already not found.

2018-09-26 Thread Goffredo Baroncelli
On 25/09/2018 19.29, Daniel Kiper wrote: > On Wed, Sep 19, 2018 at 08:40:35PM +0200, Goffredo Baroncelli wrote: >> From: Goffredo Baroncelli >> >> If a device is not found, do not return immediately but >> record this failure by storing NULL in data->devices_a

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-19 Thread Goffredo Baroncelli
On 18/09/2018 19.15, Goffredo Baroncelli wrote: >> b. The bootloader code, would have to have sophisticated enough Btrfs >> knowledge to know if the grubenv has been reflinked or snapshot, >> because even if +C, it may not be valid to overwrite, and COW must >> still happen

[PATCH 7/9] btrfs: Add support for recovery for a RAID 5 btrfs profiles.

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Add support for recovery for a RAID 5 btrfs profile. In addition it is added some code as preparatory work for RAID 6 recovery code. Signed-off-by: Goffredo Baroncelli --- grub-core/fs/btrfs.c | 169 +-- 1 file changed, 164

[PATCH 6/9] btrfs: Refactor the code that read from disk

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Move the code in charge to read the data from disk into a separate function. This helps to separate the error handling logic (which depends on the different raid profiles) from the read from disk logic. Refactoring this code increases the general readability too. This

[PATCH 2/9] btrfs: Add helper to check the btrfs header.

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli This helper is used in a few places to help the debugging. As conservative approach the error is only logged. This does not impact the error handling. Signed-off-by: Goffredo Baroncelli Reviewed-by: Daniel Kiper --- grub-core/fs/btrfs.c | 24

[PATCH 5/9] btrfs: Move logging code in grub_btrfs_read_logical()

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli A portion of the logging code is moved outside of internal for(;;). The part that is left inside is the one which depends on the internal for(;;) index. This is a preparatory patch. The next one will refactor the code inside the for(;;) into an another function

[PATCH 4/9] btrfs: Avoid a rescan for a device which was already not found.

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli If a device is not found, do not return immediately but record this failure by storing NULL in data->devices_attached[]. This way we avoid unnecessary devices rescan, and speedup the reads in case of a degraded array. Signed-off-by: Goffredo Baroncelli --- grub-c

[PATCH 8/9] btrfs: Make more generic the code for RAID 6 rebuilding

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The original code which handles the recovery of a RAID 6 disks array assumes that all reads are multiple of 1 << GRUB_DISK_SECTOR_BITS and it assumes that all the I/O is done via the struct grub_diskfilter_segment. This is not true for the btrfs code. In order to

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
On 18/09/2018 20.52, Chris Murphy wrote: > On Tue, Sep 18, 2018 at 11:15 AM, Goffredo Baroncelli > wrote: >> On 18/09/2018 06.21, Chris Murphy wrote: >>> b. The bootloader code, would have to have sophisticated enough Btrfs >>> knowledge to know if the grubenv

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
.. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-20 Thread Goffredo Baroncelli
On 07/20/2018 07:17 AM, Andrei Borzenkov wrote: > 18.07.2018 22:42, Goffredo Baroncelli пишет: >> On 07/18/2018 09:20 AM, Duncan wrote: >>> Goffredo Baroncelli posted on Wed, 18 Jul 2018 07:59:52 +0200 as >>> excerpted: >>> >>>> On 07/17/2018 1

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-20 Thread Goffredo Baroncelli
On 07/19/2018 09:10 PM, Austin S. Hemmelgarn wrote: > On 2018-07-19 13:29, Goffredo Baroncelli wrote: [...] >> >> So until now you are repeating what I told: the only useful raid profile are >> - striping >> - mirroring >> - striping+paring (even limiting the numbe

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-19 Thread Goffredo Baroncelli
On 07/19/2018 01:43 PM, Austin S. Hemmelgarn wrote: > On 2018-07-18 15:42, Goffredo Baroncelli wrote: >> On 07/18/2018 09:20 AM, Duncan wrote: >>> Goffredo Baroncelli posted on Wed, 18 Jul 2018 07:59:52 +0200 as >>> excerpted: >>> >>>> On 0

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-18 Thread Goffredo Baroncelli
On 07/18/2018 09:20 AM, Duncan wrote: > Goffredo Baroncelli posted on Wed, 18 Jul 2018 07:59:52 +0200 as > excerpted: > >> On 07/17/2018 11:12 PM, Duncan wrote: >>> Goffredo Baroncelli posted on Mon, 16 Jul 2018 20:29:46 +0200 as >>> excerpted: >>>

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-17 Thread Goffredo Baroncelli
On 07/17/2018 11:12 PM, Duncan wrote: > Goffredo Baroncelli posted on Mon, 16 Jul 2018 20:29:46 +0200 as > excerpted: > >> On 07/15/2018 04:37 PM, waxhead wrote: > >> Striping and mirroring/pairing are orthogonal properties; mirror and >> parity are mutually exc

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-16 Thread Goffredo Baroncelli
ROR2 > RAID1c4 / R3.S0.P0 / MIRROR3 > RAID10  / R1.Sm.P0 / STRIPE.MIRROR1 > RAID5   / R1.Sm.P1 / STRIPE.PARITY1 > RAID6   / R1.Sm.P2 / STRIPE.PARITY2 > > And i think this is much more readable, but others may disagree. And as a > side note... from a (hobby) coders perspective this is

Re: [PATCH 2/4] btrfs: add support for 3-copy replication (raid1c3)

2018-07-13 Thread Goffredo Baroncelli
#define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \ > diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h > index aff1356c2bb8..fa75b63dd928 100644 > --- a/include/uapi/linux/btrfs_tree.h > +++ b/include/uapi/linux/btrfs_tree.h > @@ -836,6 +836,7 @@ struct btrfs_dev_replace_item { > #defi

Re: Major design flaw with BTRFS Raid, temporary device drop will corrupt nodatacow files

2018-06-28 Thread Goffredo Baroncelli
filesystem (with cow enabled of course !) I think that this is a good example of why a HW Raid controller battery backed could be better than a SW raid. Of course the likelihood of a lot of problems could be reduced using a power supply. BR -- gpg @keyserver.linux.it: Goffredo Baroncelli K

Re: unsolvable technical issues?

2018-06-24 Thread Goffredo Baroncelli
hnology. I fully agree with the above sentences... > > The other difference, AFAIK, is that stratis is specifically a > corporation making it a/the main money product, whereas btrfs was always > something the btrfs devs used at their employers (oracle, facebook), who > have other t

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2018-05-03 Thread Goffredo Baroncelli
ripe boundary. For a filesystem this is complicate to grant, however for a journal it would be more simple to do; BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line &quo

Re: RAID56 - 6 parity raid

2018-05-03 Thread Goffredo Baroncelli
org/majordomo-info.html > > His patch was about a generic library to do RAID6, it wasn't directly > for btrfs. > > To actually use that for btrfs someone would have to actually port btrfs > to that library. In the past Andrea ported this library to btrfs too https://lwn.n

Re: RAID56 - 6 parity raid

2018-05-03 Thread Goffredo Baroncelli
you start a scrub once per week, the fact that grub requires 1 hr, or 1 day doesn't impact the reliability, because in any case you have 1 week of un-scrubbed data. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
t calculated > on the data+checksum ?! > Data is checksummed per block. The parity is invisible to the checksum. The parity are allocated in an "address space" parallel to the data "address space" exposed by the BG. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffred

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
On 05/02/2018 09:29 PM, Austin S. Hemmelgarn wrote: > On 2018-05-02 13:25, Goffredo Baroncelli wrote: >> On 05/02/2018 06:55 PM, waxhead wrote: >>>> >>>> So again, which problem would solve having the parity checksummed ? On the >>>> best of

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
On 05/02/2018 08:17 PM, waxhead wrote: > Goffredo Baroncelli wrote: >> On 05/02/2018 06:55 PM, waxhead wrote: >>>> >>>> So again, which problem would solve having the parity checksummed ? On the >>>> best of my knowledge nothing. In any case the dat

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
r than the gain, and the likelihood the gain is very lower compared to the likelihood (=100% or always) of the cost. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line &

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
ase both the code complexity and the write amplification, because every time a part of the stripe is touched not only the parity has to be updated, but also the checksum has too.. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0

Re: [RFC] Add support for BTRFS raid5/6 to GRUB

2018-04-23 Thread Goffredo Baroncelli
On 04/23/2018 01:50 PM, Daniel Kiper wrote: > On Tue, Apr 17, 2018 at 09:57:40PM +0200, Goffredo Baroncelli wrote: >> Hi All, >> >> Below you can find a patch to add support for accessing files from >> grub in a RAID5/6 btrfs filesystem. This is a RFC because it is

[RFC] Add support for BTRFS raid5/6 to GRUB

2018-04-17 Thread Goffredo Baroncelli
. Comments are welcome. BR G.Baroncelli --- commit 8c80a1b7c913faf50f95c5c76b4666ed17685666 Author: Goffredo Baroncelli Date: Tue Apr 17 21:40:31 2018 +0200 Add initial support for btrfs raid5/6 chunk diff --git a/grub-core/fs/btrfs.c b/grub-core/fs/btrfs.c index be195448d..4c5632acb 100644

Re: How to dump/find parity of RAID-5 file?

2018-04-17 Thread Goffredo Baroncelli
On 02/06/2017 09:40 PM, Goffredo Baroncelli wrote: > On 2017-02-03 11:44, Lakshmipathi.G wrote: >> Hi. >> >> Came across this thread >> https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html >> Exploring possibility of adding test-scripts arou

Re: [PATCH v2] btrfs-progs: dump-super: Refactor print function and add extra check

2018-04-11 Thread Goffredo Baroncelli
t this is quite verbose... however if this is the constraint, why not print it > > 3) C macro usage >Just a minor problem, macro usage such as SZ_4K/SZ_64K may not looks >good for new developers. I find it quite clear and intuitive even if this is the first time that

Re: [PATCH v2] btrfs-progs: dump-super: Refactor print function and add extra check

2018-04-10 Thread Goffredo Baroncelli
Hi Qu, On 04/10/2018 04:00 AM, Qu Wenruo wrote: > > > On 2018年04月10日 05:50, Goffredo Baroncelli wrote: >> Hi Qu, >> >> On 04/09/2018 11:19 AM, Qu Wenruo wrote: >>> When manually patching super blocks, current validation check is pretty >>> weak (

Re: [PATCH v2] btrfs-progs: dump-super: Refactor print function and add extra check

2018-04-09 Thread Goffredo Baroncelli
ign(&sb->dev_item)); > - printf("dev_item.io_width\t%u\n", (unsigned int) > -btrfs_stack_device_io_width(&sb->dev_item)); > - printf("dev_item.sector_size\t%u\n", (unsigned int) > -btrfs_stack_device_sector_size(&sb->dev_item)); > - printf("dev_item.devid\t\t%llu\n", > -btrfs_stack_device_id(&sb->dev_item)); > - printf("dev_item.dev_group\t%u\n", (unsigned int) > -btrfs_stack_device_group(&sb->dev_item)); > - printf("dev_item.seek_speed\t%u\n", (unsigned int) > -btrfs_stack_device_seek_speed(&sb->dev_item)); > - printf("dev_item.bandwidth\t%u\n", (unsigned int) > -btrfs_stack_device_bandwidth(&sb->dev_item)); > - printf("dev_item.generation\t%llu\n", (unsigned long long) > -btrfs_stack_device_generation(&sb->dev_item)); > + /* For embedded device item, don't do extra check, just like kernel */ > + print_super_dev(sb, type); > + print_super_dev(sb, total_bytes); > + print_super_dev(sb, bytes_used); > + print_super_dev(sb, io_align); > + print_super_dev(sb, io_width); > + print_super_dev(sb, sector_size); > + print_super_dev(sb, id); > + print_super_dev(sb, group); > + print_super_dev(sb, seek_speed); > + print_super_dev(sb, bandwidth); > + print_super_dev(sb, generation); > if (full) { > printf("sys_chunk_array[%d]:\n", BTRFS_SYSTEM_CHUNK_ARRAY_SIZE); > print_sys_chunk_array(sb); > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Status of RAID5/6

2018-04-04 Thread Goffredo Baroncelli
On 04/04/2018 08:01 AM, Zygo Blaxell wrote: > On Wed, Apr 04, 2018 at 07:15:54AM +0200, Goffredo Baroncelli wrote: >> On 04/04/2018 12:57 AM, Zygo Blaxell wrote: [...] >> Before you pointed out that the non-contiguous block written has >> an impact on performance. I a

Re: Status of RAID5/6

2018-04-03 Thread Goffredo Baroncelli
avoid writing the end of an extent without further data (e.g. the gap to the stripe has to be filled in the same transaction) BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Status of RAID5/6

2018-04-03 Thread Goffredo Baroncelli
On 04/03/2018 02:31 AM, Zygo Blaxell wrote: > On Mon, Apr 02, 2018 at 06:23:34PM -0400, Zygo Blaxell wrote: >> On Mon, Apr 02, 2018 at 11:49:42AM -0400, Austin S. Hemmelgarn wrote: >>> On 2018-04-02 11:18, Goffredo Baroncelli wrote: >>>> I thought that a possible

Re: Status of RAID5/6

2018-04-02 Thread Goffredo Baroncelli
empty the BG #1, and #2. Otherwise a new BG should be allocated. The cost should be comparable to the logging/journaling (each data shorter than a full-stripe, has to be written two times); the implementation should be quite easy, because already NOW btrfs support BG with different set of disks. B

Re: Status of RAID5/6

2018-03-31 Thread Goffredo Baroncelli
he "old" data should be untouched; or you are saying that the "old" data is rewritten with the same data ? BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: se

Re: Status of RAID5/6

2018-03-30 Thread Goffredo Baroncelli
sider a power failure+device missing case. However in this case even logging the "new data" would be not sufficient. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send t

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > On 2018/03/30 2:35, Goffredo Baroncelli wrote: >> Hi Misono, > > Hello, [...] >> My conclusion, is that your work is not consistent with the current >> implementation; so I am suggesting to create a new subcommand (&q

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > [Un]fortunately, the stock "btrfs sub list" has the capability to show all > the subvolumes, even the ones not mounted, so this can be entirely replaced > by your API. s/can be entirely/can't be entirely/ -- gpg @keys

Re: Status of RAID5/6

2018-03-30 Thread Goffredo Baroncelli
t; on-disk format, much less implemented. It could go even further; it would be sufficient to track which *partial* stripes update will be performed before a commit, in one of the btrfs logs. Then in case of a mount of an unclean filesystem, a scrub on these stripes would be sufficient. BR G.Baronc

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-29 Thread Goffredo Baroncelli
t; root and normal user > > btrfs-list.c | 376 > +++-- > btrfs-list.h | 7 +- > cmds-subvolume.c | 14 +- > ioctl.h | 86 +++ > tests/cli-tests/009-subvo

Re: [PATCH v2 0/3] Allow rmdir(2) to delete a subvolume

2018-03-26 Thread Goffredo Baroncelli
() by using btrfs_delete_subvolume() > > fs/btrfs/ctree.h | 5 +- > fs/btrfs/inode.c | 199 > ++- > fs/btrfs/ioctl.c | 185 +-- > 3 files changed, 200 insertions(+), 1

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-22 Thread Goffredo Baroncelli
On 03/22/2018 01:15 PM, Austin S. Hemmelgarn wrote: > On 2018-03-21 16:38, Goffredo Baroncelli wrote: >> On 03/21/2018 12:47 PM, Austin S. Hemmelgarn wrote: >>> I agree as well, with the addendum that I'd love to see a new ioctl that >>> does proper permission

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-21 Thread Goffredo Baroncelli
he files permissions, and only if all the permissions are OK, you can delete the subvolume. However this cannot be a "quick" subvolume delete BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- T

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-20 Thread Goffredo Baroncelli
> - err = inode_permission(inode, MAY_WRITE | MAY_EXEC); > - if (err) > - goto out_dput; > + err = inode_permission(inode, MAY_WRITE | MAY_EXEC); > + if (err) > + goto out_dp

Re: Crashes running btrfs scrub

2018-03-20 Thread Goffredo Baroncelli
nt > > David, any idea? > > thanks, > liubo > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord.

Re: [RFC PATCH v3 5/7] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

2018-03-19 Thread Goffredo Baroncelli
full of root_info objects, but we need to fill > - * in their path names within the subvol that is referencing each one. > - */ > - ret = list_subvol_fill_paths(fd, root_lookup); > return ret; > } > > @@ -1598,12 +1920,12 @@ int btrfs_get_subvol(int fd,

Re: Crashes running btrfs scrub

2018-03-18 Thread Goffredo Baroncelli
On 03/18/2018 08:57 AM, Goffredo Baroncelli wrote: > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE = 2048 > sizeof(struct btrfs_chunk)) = 48 > sizeof(struct btrfs_stripe) = 32 > > So > > (2048/2-48)/32+1 = 31 > > If my math is correct my math was wr

Re: Crashes running btrfs scrub

2018-03-18 Thread Goffredo Baroncelli
48)/32+1 = 31 If my math is correct gb -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org Mor

Re: [RFC PATCH v2 6/8] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

2018-03-17 Thread Goffredo Baroncelli
,19 @@ static int cmd_subvol_list(int argc, char **argv) > goto out; > } > > + ret = check_perm_for_tree_search(fd); > + if (ret < 0) { > + ret = -1; > + error("can't check the permission for tree search: %s&quo

Re: [RFC PATCH v2 3/8] btrfs-progs: sub list: Add helper function which checks the permission for tree search ioctl

2018-03-17 Thread Goffredo Baroncelli
dex 6e5fc778..6225311d 100644 > --- a/btrfs-list.h > +++ b/btrfs-list.h > @@ -176,5 +176,6 @@ char *btrfs_list_path_for_root(int fd, u64 root); > int btrfs_list_get_path_rootid(int fd, u64 *treeid); > int btrfs_get_subvol(int fd, struct root_info *the_ri); > int btrfs_get_toplevel_subv

Re: Ongoing Btrfs stability issues

2018-03-14 Thread Goffredo Baroncelli
On 03/14/2018 08:27 PM, Austin S. Hemmelgarn wrote: > On 2018-03-14 14:39, Goffredo Baroncelli wrote: >> On 03/14/2018 01:02 PM, Austin S. Hemmelgarn wrote: >> [...] >>>> >>>> In btrfs, a checksum mismatch creates an -EIO error during the reading. In >

Re: Ongoing Btrfs stability issues

2018-03-14 Thread Goffredo Baroncelli
> >> Even in this case I am curious about dm-integrity would sole this issue. > dm-integrity uses journaling, and actually based on the testing I've done, > will typically have much worse performance than the overhead of just enabling > COW on files on BTRFS and manually defragment

Re: Ongoing Btrfs stability issues

2018-03-13 Thread Goffredo Baroncelli
On 03/12/2018 10:48 PM, Christoph Anton Mitterer wrote: > On Mon, 2018-03-12 at 22:22 +0100, Goffredo Baroncelli wrote: >> Unfortunately no, the likelihood might be 100%: there are some >> patterns which trigger this problem quite easily. See The link which >> I posted in my

Re: Ongoing Btrfs stability issues

2018-03-12 Thread Goffredo Baroncelli
On 03/11/2018 11:37 PM, Christoph Anton Mitterer wrote: > On Sun, 2018-03-11 at 18:51 +0100, Goffredo Baroncelli wrote: >> >> COW is needed to properly checksum the data. Otherwise is not >> possible to ensure the coherency between data and checksum (however I >> have to

Re: Ongoing Btrfs stability issues

2018-03-11 Thread Goffredo Baroncelli
of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [*] https://www.spinics.net/lists/linux-btrfs/msg69185.html -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 --

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-07 Thread Goffredo Baroncelli
On 03/07/2018 01:40 AM, Misono, Tomohiro wrote: > On 2018/03/07 5:29, Goffredo Baroncelli wrote: >> On 03/06/2018 09:30 AM, Misono, Tomohiro wrote: >>> Add new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO) which searches >>> and returns only subvolume related item (ROOT

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-06 Thread Goffredo Baroncelli
t; + if (ret == -EOVERFLOW) > + ret = 0; > + > + if (ret == 0 && copy_to_user(&uargs->key, &sk, sizeof(sk))) > + ret = -EFAULT; > + return ret; > +} > + > static noinline int btrfs_ioctl_snap_destroy(struct file *file, >

Re: [PATCH RFC] Btrfs: expose bad chunks in sysfs

2018-02-09 Thread Goffredo Baroncelli
On 02/08/2018 08:07 PM, Liu Bo wrote: > On Thu, Feb 08, 2018 at 07:52:05PM +0100, Goffredo Baroncelli wrote: >> On 02/06/2018 12:15 AM, Liu Bo wrote: >> [...] >>> One way to mitigate the data loss pain is to expose 'bad chunks', >>> i.e. degraded ch

Re: [PATCH RFC] Btrfs: expose bad chunks in sysfs

2018-02-08 Thread Goffredo Baroncelli
so on. Checking "[...]/devs", it would be easy understand if the chunk is in "degraded" mode or not. However I have to admit that I don't know how feasible is iterate over a sysfs directory which is a map of a kernel objects list. I think that if these interface would

Re: degraded permanent mount option

2018-01-27 Thread Goffredo Baroncelli
another matter). > This can't be done sanely from outside the kernel. Adding finer control > would be reasonable ("wait and block" vs "try and return immediately") but > that's about all. It's be also wrong to have a different interface for > daemo

Re: [PATCH 11/26] libbtrfsutil: add btrfs_util_create_snapshot()

2018-01-27 Thread Goffredo Baroncelli
On 01/27/2018 06:45 AM, Omar Sandoval wrote: > On Sat, Jan 27, 2018 at 01:00:58PM +0800, Qu Wenruo wrote: >> >> >> On 2018年01月27日 03:46, Omar Sandoval wrote: >>> On Fri, Jan 26, 2018 at 08:31:06PM +0100, Goffredo Baroncelli wrote: >>>> On 01/26/2018 07:40

Re: [PATCH 11/26] libbtrfsutil: add btrfs_util_create_snapshot()

2018-01-26 Thread Goffredo Baroncelli
const char *name, int flags, > + uint64_t *async_transid, > + struct > btrfs_util_qgroup_inherit *qgroup_inherit) > +{ > + struct btrfs_ioctl_vol_args_

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
On 12/05/2017 09:17 PM, Austin S. Hemmelgarn wrote: > On 2017-12-05 14:09, Goffredo Baroncelli wrote: >> On 12/05/2017 07:46 PM, Graham Cobb wrote: >>> On 05/12/17 18:01, Goffredo Baroncelli wrote: >>>> On 12/05/2017 04:42 PM, Graham Cobb wrote: >> [] &

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
On 12/05/2017 07:46 PM, Graham Cobb wrote: > On 05/12/17 18:01, Goffredo Baroncelli wrote: >> On 12/05/2017 04:42 PM, Graham Cobb wrote: [] >>>>> Then no impact to kernel, all complex work is done in user space. >>>> Exactly how hard is it to just ch

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
d 2 new non-root ioctls to get subvolume/quota info for accessible > subvolumes > == > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.ker

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
ode if you don't have write access to the parent directory. I assume that this is a POSIX requirement; and if so this should be true even for BTRFS. This means that in order to remove a subvolume and you are not root, you should check all the contained directories. It is not sufficient to chec

Re: [PATCH v2] btrfs: handle dynamically reappearing missing device

2017-11-20 Thread Goffredo Baroncelli
On 11/20/2017 09:19 AM, Anand Jain wrote: > > > On 11/18/2017 09:52 PM, Goffredo Baroncelli wrote: >> On 11/17/2017 01:36 PM, Anand Jain wrote: >>> If the device is not present at the time of (-o degrade) mount, >>> the mount context will create a dummy missi

Re: [PATCH v2] btrfs: handle dynamically reappearing missing device

2017-11-18 Thread Goffredo Baroncelli
Because the filesystem is a raid1, btrfs may read from /dev/sdb (updated data) or /dev/sda (old data), or worse read something from the former and something from the later (metadata from sda and data from sdb ) ??? Am I missing something ? BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo B

Re: [PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-04 Thread Goffredo Baroncelli
On 10/04/2017 07:13 PM, Liu Bo wrote: > On Wed, Oct 04, 2017 at 04:22:28PM +0200, David Sterba wrote: >> On Tue, Oct 03, 2017 at 07:31:10PM +0200, Goffredo Baroncelli wrote: >>> From: Goffredo Baroncelli >>> >>> Jean-Denis Girard noticed commit c821e7f3 "

[PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-03 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Jean-Denis Girard noticed commit c821e7f3 "pass bytes to btrfs_bio_alloc" (https://patchwork.kernel.org/patch/9763081/) introduces a regression on 32 bit machines. When CONFIG_LBDAF is _not_ defined (CONFIG_LBDAF == Support for large (2TB+) block devices

Re: Mount error on 32 bits, ok on 64 bits

2017-10-02 Thread Goffredo Baroncelli
On 10/02/2017 11:46 PM, Goffredo Baroncelli wrote: > which on 32bit is 32 bit if CONFIG_LBDAF is _not_ defined (CONFIG_LBDAF == > Support for large (2TB+) block devices and files) Only now I noticed that in the first email you attached the config; in fact in your config CONFIG_LBDAF is n

Re: Mount error on 32 bits, ok on 64 bits

2017-10-02 Thread Goffredo Baroncelli
bio = btrfs_bio_alloc(bdev, sector << 9); with bio = btrfs_bio_alloc(bdev, (u64)sector << 9); [1] http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/extent_io.c#L2762 [2] http://elixir.free-electrons.com/linux/latest/source/include/linux/types.h#L133 > >

Re: Why do full balance and deduplication reduce available free space?

2017-10-02 Thread Goffredo Baroncelli
a lot (about 24+7+4 = 35 times) > > Thanks, > Niccolò BR G.Baroncelli > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at  http://vger.kernel.org/majordomo-

Re: What means "top level" in "btrfs subvolume list" ?

2017-10-02 Thread Goffredo Baroncelli
subvolume in the filesystem through fstab; - however I am still searching a way to handle the snapshots which fully satisfy me (putting them under /, is not the best) BR -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To u

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-10-02 Thread Goffredo Baroncelli
#x27; is too small to make a usable filesystem ERROR: minimum size for each btrfs device is 41943040 you have to create a big enough ghigo@venice:/tmp$ truncate -s 10G disk.img ghigo@venice:/tmp$ mkfs.btrfs --root test disk.img btrfs-progs v4.12 See http://btrfs.wiki.kernel.org for more information.

What means "top level" in "btrfs subvolume list" ?

2017-09-30 Thread Goffredo Baroncelli
/tmp/test1/test3 ID = 535 /debian/tmp/test1.snap2 ID = 536 /test ID = 537 /test/sub1 ID = 538 BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsu

What means "top id" in "btrfs subvolume list" ?

2017-09-30 Thread Goffredo Baroncelli
sub1 ID = 538 BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Can't remove device -> I/O error

2017-09-30 Thread Goffredo Baroncelli
nvironment, but I suppose that also on the bare metal it will be quite quick. BR G.Baroncelli > > > > Am 30.09.2017 um 09:16 schrieb Goffredo Baroncelli: >> On 09/30/2017 01:06 AM, DocMAX wrote: >>>>> Did you removed the disk before mounting (physically or d

Re: Can't remove device -> I/O error

2017-09-30 Thread Goffredo Baroncelli
> same i/o error > Did you switch off the machine ? If not, before mounting in degraded mode, do "echo 1 >/sys/block/xxx/device/delete". After the monting do a btrfs dev del missing > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DA

Re: Can't remove device -> I/O error

2017-09-29 Thread Goffredo Baroncelli
unless i > re-format. No, it should not be necessary. > > > > Am 29.09.2017 um 23:04 schrieb Goffredo Baroncelli: >> On 09/29/2017 10:00 PM, Dirk Diggler wrote: >>> Hi, >>> >>> is there any chance to get my device removed? >> I simulated a dev

Re: Can't remove device -> I/O error

2017-09-29 Thread Goffredo Baroncelli
Device size: 1.82TiB >Device slack: 0.00B >Data,RAID5:333.00GiB >Data,RAID5:955.00GiB >Data,RAID5: 5.57GiB >Metadata,RAID1: 3.00GiB >Unallocated: 566.44GiB > > Thanks, &

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-09-29 Thread Goffredo Baroncelli
On 09/28/2017 02:00 AM, Qu Wenruo wrote: > > > On 2017年09月28日 00:20, David Sterba wrote: >> On Mon, Sep 25, 2017 at 07:15:30AM -0400, Austin S. Hemmelgarn wrote: >>> On 2017-09-24 10:08, Goffredo Baroncelli wrote: >>>> On 09/24/2017 12:10 PM, Anand Jain

[PATCH] New ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Add a new ioctl to get info about chunk without requiring the root privileges. This allow to a non root user to know how the space of the filesystem is allocated. Signed-off-by: Goffredo Baroncelli --- fs/btrfs/ioctl.c | 215

[RFC] btrfs: new ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
pi is a good api. In particular I am not sure that returning -EAGAIN is the right thing to do when further data is available but the buffer is not enough. Comments are welcome BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9

[PATCH 2/2] Use the new ioctl BTRFS_IOC_GET_CHUNK_INFO for load_chunk_info()

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Use the new ioctl BTRFS_IOC_GET_CHUNK_INFO in load_chunk_info() instead of the BTRFS_IOC_TREE_SEARCH. The old method is still present as fallback for compatibility reason. The goal is to avoid BTRFS_IOC_GET_CHUNK_INFO because it requires root privileges. Signed-off-by

[RFC] btrfs-progs: use the new ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
oncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://

[PATCH 1/2] get_partition_size() doens't require root privileges

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Allow the get_partition_size() to be called without root privileges. Signed-off-by: Goffredo baroncelli --- utils.c | 27 +++ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/utils.c b/utils.c index 7a2710fe..ee3ed544 100644

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-09-24 Thread Goffredo Baroncelli
>  Pls don't remove this feature, it needs fix such as #2 (above) to improve on > #1 (above) as in your list. > > Thanks, Anand > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-18 Thread Goffredo Baroncelli
rted in kernel log > 2) read(2) returns successfully with some content being 0x01. > > In order to fix it, we need to report its checksum mismatch error to > the upper layer (dio layer in this case) as well. > > Signed-off-by: Liu Bo > Reported-by: Goffredo Baroncelli

[PATCH] btrfs-progs: load_chunk_info() wrong set search key 'type'

2017-09-17 Thread Goffredo Baroncelli
The function load_chunk_info() doesn't initialize correctly the sk->min/max_type when it calls the TREE_SEARCH ioctl: these keys are swapped. Moreover this function assumes that all the items contained in the tree BTRFS_CHUNK_TREE_OBJECTID are of type BTRFS_CHUNK_ITEM_KEY, however some items are of

[PATCH] load_chunk_info() : wrong search key 'type'

2017-09-17 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The function load_chunk_info() doesn't initialize correctly the sk->min/max_type when it calls the TREE_SEARCH ioctl: these keys are swapped. Moreover this function assumes that all the items contained in the tree BTRFS_CHUNK_TREE_OBJECTID are

Re: subvolume list as user?

2017-09-16 Thread Goffredo Baroncelli
t; SEARCH ioctl to find it's info. > > This is the explantion why it does not work, but it does not help me. I > still have the problem: how can a user get a list of his subvolumes? He > may created them some time ago and forget it. He now wants to have a list > of them. Far

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-16 Thread Goffredo Baroncelli
or 0+0 records in 0+0 records out 0 bytes copied, 0.000404156 s, 0.0 kB/s ghigo@venice:~/btrfs/crash-o-direct/t$ dmesg -T | tail -1 [Sat Sep 16 13:56:41 2017] BTRFS warning (device sdd5): csum failed root 5 ino 257 off 0 csum 0x98f94189 expected csum 0x0ab6be80 mirror 1 > > Signed-off-b

Re: A user cannot remove his readonly snapshots?!

2017-09-16 Thread Goffredo Baroncelli
rfs sub del", so an user can remove an _empty_ subvolume https://www.spinics.net/lists/linux-btrfs/msg06522.html > > BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list:

Re: A user cannot remove his readonly snapshots?!

2017-09-16 Thread Goffredo Baroncelli
uot;ro" property to false before deleting it btrfs property set /test/tux/zz/.snapshot/2017-09-15_1824.test ro false BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: sen

<    1   2   3   4   5   6   7   8   9   10   >