Re: [systemd-devel] install Fedora systemd-nspawn container on btrfs

2015-04-23 Thread Andrei Borzenkov
On Thu, Apr 23, 2015 at 2:50 PM, Lennart Poettering lenn...@poettering.net wrote: On Thu, 23.04.15 13:45, arnaud gaboury (arnaud.gabo...@gmail.com) wrote: Not sure what I did wrong, but I can't install/boot my nspawn container. Here is my setup: Archlinux box- updated 1- created 3 btrfs

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-13 Thread Andrei Borzenkov
В Sat, 13 Jun 2015 17:35:53 +0800 Anand Jain anand.j...@oracle.com пишет: Thanks for your reply Andrei and Goffredo. more below... On 06/13/2015 04:08 AM, Goffredo Baroncelli wrote: On 2015-06-12 20:04, Andrei Borzenkov wrote: В Fri, 12 Jun 2015 21:16:30 +0800 Anand Jain anand.j

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-17 Thread Andrei Borzenkov
В Wed, 17 Jun 2015 23:02:02 +0200 Lennart Poettering lenn...@poettering.net пишет: On Wed, 17.06.15 21:10, Goffredo Baroncelli (kreij...@libero.it) wrote: Well, /bin/mount is not a daemon, and it should not be one. My helper is not a deamon; you was correct the first time: it blocks

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-12 Thread Andrei Borzenkov
В Fri, 12 Jun 2015 21:16:30 +0800 Anand Jain anand.j...@oracle.com пишет: BTRFS_IOC_DEVICES_READY is to check if all the required devices are known by the btrfs kernel, so that admin/system-application could mount the FS. It is checked against a device in the argument. However the

Re: Pointers to mirroring partitions (w/ encryption?) help?

2016-06-04 Thread Andrei Borzenkov
04.06.2016 20:31, B. S. пишет: >>> >>> Yeah, when it comes to FDE, you either have to make your peace with >>> trusting the manufacturer, or you can't. If you are going to boot >>> your system with a traditional boot loader, an unencrypted partition >>> is mandatory. >> >> No, it is not with grub2

Re: Pointers to mirroring partitions (w/ encryption?) help?

2016-06-04 Thread Andrei Borzenkov
04.06.2016 22:05, Chris Murphy пишет: ... >> >> Yeah, when it comes to FDE, you either have to make your peace with >> trusting the manufacturer, or you can't. If you are going to boot your >> system with a traditional boot loader, an unencrypted partition is >> mandatory. > > /boot can be

Re: btrfs

2016-06-04 Thread Andrei Borzenkov
04.06.2016 20:00, Chris Murphy пишет: > On Sat, Jun 4, 2016 at 1:24 AM, Andrei Borzenkov <arvidj...@gmail.com> wrote: >> 04.06.2016 04:51, Christoph Anton Mitterer пишет: >> ... >>> >>>> The only extant systems that support higher >>>> level

Re: Recommended why to use btrfs for production?

2016-06-05 Thread Andrei Borzenkov
05.06.2016 19:33, James Johnston пишет: > On 06/05/2016 10:46 AM, Mladen Milinkovic wrote: >> On 06/03/2016 04:05 PM, Chris Murphy wrote: >>> Make certain the kernel command timer value is greater than the driver >>> error recovery timeout. The former is found in sysfs, per block >>> device, the

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Andrei Borzenkov
13.06.2016 01:49, Henk Slager пишет: > On Sun, Jun 12, 2016 at 11:22 PM, Maximilian Böhm wrote: >> Hi there, I did something terribly wrong, all blame on me. I wanted to >> write to an USB stick but /dev/sdc wasn't the stick in this case but >> an attached HDD with GPT and an 8

Re: "No space left on device" and balance doesn't work

2016-06-04 Thread Andrei Borzenkov
02.06.2016 15:56, Austin S. Hemmelgarn пишет: > > In your particular situation, what's happened is that you have all the > space allocated to chunks, but have free space within those chunks. > Balance never puts data in existing chunks, and you can't allocate any > new chunks, so you can't run a

Re: btrfs

2016-06-04 Thread Andrei Borzenkov
04.06.2016 04:51, Christoph Anton Mitterer пишет: ... > >> The only extant systems that support higher >> levels of replication and call it RAID-1 are entirely based on MD >> RAID >> and it's poor choice of naming. > > Not true either, show me any single hardware RAID controller that does >

Re: Pointers to mirroring partitions (w/ encryption?) help?

2016-06-04 Thread Andrei Borzenkov
04.06.2016 04:39, Justin Brown пишет: > Here's some thoughts: > >> Assume a CD sized (680MB) /boot > > Some distros carry patches for grub that allow booting from Btrfs, > so no separate /boot file system is required. (Fedora does not; > Ubuntu -- and therefore probably all Debians -- does.) >

Re: [PATCH] btrfs,vfs: allow FILE_EXTENT_SAME on a file opened ro

2016-05-29 Thread Andrei Borzenkov
29.05.2016 03:56, Zygo Blaxell пишет: >> >> I don't think this can happen on btrfs: the superblock is updated only after >> a barrier when both the data and extent refs are already on the disk. > > If and only if the filesystem is mounted with the flushoncommit option, > that's true. This is not

Re: Hot data tracking / hybrid storage

2016-05-29 Thread Andrei Borzenkov
20.05.2016 20:59, Austin S. Hemmelgarn пишет: > On 2016-05-20 13:02, Ferry Toth wrote: >> We have 4 1TB drives in MBR, 1MB free at the beginning, grub on all 4, >> then 8GB swap, then all the rest btrfs (no LVM used). The 4 btrfs >> partitions are in the same pool, which is in btrfs RAID10 format.

Re: Adventures in btrfs raid5 disk recovery

2016-06-24 Thread Andrei Borzenkov
On Fri, Jun 24, 2016 at 11:50 AM, Hugo Mills <h...@carfax.org.uk> wrote: > On Fri, Jun 24, 2016 at 07:02:34AM +0300, Andrei Borzenkov wrote: >> 24.06.2016 04:47, Zygo Blaxell пишет: >> > On Thu, Jun 23, 2016 at 06:26:22PM -0600, Chris Murphy wrote: >> >> On Th

Re: Adventures in btrfs raid5 disk recovery

2016-06-24 Thread Andrei Borzenkov
On Fri, Jun 24, 2016 at 1:16 PM, Hugo Mills <h...@carfax.org.uk> wrote: > On Fri, Jun 24, 2016 at 12:52:21PM +0300, Andrei Borzenkov wrote: >> On Fri, Jun 24, 2016 at 11:50 AM, Hugo Mills <h...@carfax.org.uk> wrote: >> > On Fri, Jun 24, 2016 at 07:02:34AM

Re: Adventures in btrfs raid5 disk recovery

2016-06-24 Thread Andrei Borzenkov
On Fri, Jun 24, 2016 at 8:20 AM, Chris Murphy wrote: > [root@f24s ~]# filefrag -v /mnt/5/* > Filesystem type is: 9123683e > File size of /mnt/5/a.txt is 16383 (4 blocks of 4096 bytes) > ext: logical_offset:physical_offset: length: expected: flags: >0:

Re: Adventures in btrfs raid5 disk recovery

2016-06-24 Thread Andrei Borzenkov
24.06.2016 20:06, Chris Murphy пишет: > On Fri, Jun 24, 2016 at 3:52 AM, Andrei Borzenkov <arvidj...@gmail.com> wrote: >> On Fri, Jun 24, 2016 at 11:50 AM, Hugo Mills <h...@carfax.org.uk> wrote: >> eta)data and RAID56 parity is not data. >>> >>>Chec

Re: Adventures in btrfs raid5 disk recovery

2016-06-23 Thread Andrei Borzenkov
24.06.2016 04:47, Zygo Blaxell пишет: > On Thu, Jun 23, 2016 at 06:26:22PM -0600, Chris Murphy wrote: >> On Thu, Jun 23, 2016 at 1:32 PM, Goffredo Baroncelli >> wrote: >>> The raid5 write hole is avoided in BTRFS (and in ZFS) thanks to the >>> checksum. >> >> Yeah I'm kinda

Re: Adventures in btrfs raid5 disk recovery

2016-06-26 Thread Andrei Borzenkov
26.06.2016 00:52, Chris Murphy пишет: > Interestingly enough, so far I'm finding with full stripe writes, i.e. > 3x raid5, exactly 128KiB data writes, devid 3 is always parity. This > is raid4. That's not what code suggests and what I see in practice - parity seems to be distributed across all

Undelete deleted subvolume?

2016-05-12 Thread Andrei Borzenkov
I accidentally deleted wrong snapshot using SUSE snapper. Is it possible to undelete subvolume? I know that it is possible to extract files from old tree (although SLES12 does not seem to offer btrfs-find-root), but is it possible to "reconnect" subvolume back? -- To unsubscribe from this list:

Re: fsck: to repair or not to repair

2016-05-16 Thread Andrei Borzenkov
16.05.2016 14:17, Austin S. Hemmelgarn пишет: > On 2016-05-13 17:35, Chris Murphy wrote: >> On Fri, May 13, 2016 at 9:28 AM, Nikolaus Rath wrote: >>> On May 13 2016, Duncan <1i5t5.dun...@cox.net> wrote: Because btrfs can be multi-device, it needs some way to track which

Re: /etc/fstab rootfs options vs grub2 rootflags cmdline

2016-05-04 Thread Andrei Borzenkov
04.05.2016 18:54, Duncan пишет: > Niccolò Belli posted on Wed, 04 May 2016 15:52:39 +0200 as excerpted: > >> Thanks, >> Now my fstab option are >> rw,noatime,compress=lzo,discard,autodefrag,subvolid=257,subvol=/@ >> I tried to add rootflags=noatime,compress=lzo,discard,autodefrag to >>

Re: [PATCH] btrfs-progs: udev: add rules for dm devices

2016-05-07 Thread Andrei Borzenkov
06.05.2016 22:27, Jeff Mahoney пишет: > Systemd's btrfs rule runs btrfs dev ready on each device > as it's discovered. The btrfs command is executed as a builtin > command via an IMPORT{builtin} rule, which means it gets > executed at rule evaluation time, not rule execution time. That > means

Re: Force recalculation of a data block checksum

2016-07-23 Thread Andrei Borzenkov
24.07.2016 07:32, Tomasz Melcer пишет: > On 24.07.2016 04:16, Chris Murphy wrote: >> On Sat, Jul 23, 2016 at 8:03 PM, Tomasz Melcer wrote: >>> How can I ask btrfs to recompute the checksum of a data block as it is >>> stored on the drive? >> >> Since btrfs-progs 3.17 'btrfs

Re: Likelihood of read error, recover device failure raid10

2016-08-14 Thread Andrei Borzenkov
14.08.2016 19:20, Chris Murphy пишет: > > As an aside, I'm finding the size information for the data chunk in > 'fi us' confusing... > > The sample file system contains one file: > [root@f24s ~]# ls -lh /mnt/0 > total 1.4G > -rw-r--r--. 1 root root 1.4G Aug 13 19:24 >

Re: Likelihood of read error, recover device failure raid10

2016-08-14 Thread Andrei Borzenkov
14.08.2016 19:20, Chris Murphy пишет: ... > > This volume now has about a dozen chunks created by kernel code, and > the stripe X to devid Y mapping is identical. Using dd and hexdump, > I'm finding that stripe 0 and 1 are mirrored pairs, they contain > identical information. And stripe 2 and 3

Re: checksum error in metadata node - best way to move root fs to new drive?

2016-08-11 Thread Andrei Borzenkov
On Thu, Aug 11, 2016 at 1:23 AM, Chris Murphy wrote: > On Wed, Aug 10, 2016 at 4:01 PM, Dave T wrote: > >> I will be very disappointed if I cannot use btrfs + dm-crypt. As far >> as I can see, there is no alternative given that I need to use >>

Re: [PATCH] btrfs-progs: add option to run balance as daemon

2016-07-12 Thread Andrei Borzenkov
12.07.2016 15:25, Austin S. Hemmelgarn пишет: > > I'm not changing my init system just to add functionality that should > already exist in btrfs-progs. The fact that the balance ioctl is > synchronous was a poor design choice, and we need to provide the option > to work around that independent

Re: Status of SMR with BTRFS

2016-07-21 Thread Andrei Borzenkov
On Thu, Jul 21, 2016 at 4:34 PM, Chris Murphy wrote: > > Do programs have a way to communicate what portion of a data file is > modified, so that only changed blocks are COW'd? When I change a > single pixel in a 400MiB image and do a save (to overwrite the > original

Re: Size reported differently between profiles

2016-07-16 Thread Andrei Borzenkov
17.07.2016 05:09, Sébastien Luttringer пишет: > Hello, > > «btrfs fi usage» report size differently between > single,RAID0,RAID1,RAID5,RAID6 > and RAID10. > > The test is done with 2 files of 1.4GiB each on 4x10GiB devices. I used > balance > to get size between profiles. > > Data,single:

Re: [BUG] Btrfs scrub sometime recalculate wrong parity in raid5: take two

2016-07-15 Thread Andrei Borzenkov
15.07.2016 19:29, Chris Mason пишет: > >> However I have to point out that this kind of test is very >> difficult to do: the file-cache could lead to read an old data, so please >> suggestion about how flush the cache are good (I do some sync, >> unmount the filesystem and perform "echo 3

Re: [BUG] Btrfs scrub sometime recalculate wrong parity in raid5: take two

2016-07-15 Thread Andrei Borzenkov
15.07.2016 16:20, Chris Mason пишет: >>> >>> Interesting, thanks for taking the time to write this up. Is the >>> failure specific to scrub? Or is parity rebuild in general also failing >>> in this case? >>> >> >> How do you rebuild parity without scrub as long as all devices appear to >> be

Re: [BUG] Btrfs scrub sometime recalculate wrong parity in raid5: take two

2016-07-14 Thread Andrei Borzenkov
15.07.2016 00:20, Chris Mason пишет: > > > On 07/12/2016 05:50 PM, Goffredo Baroncelli wrote: >> Hi All, >> >> I developed a new btrfs command "btrfs insp phy"[1] to further >> investigate this bug [2]. Using "btrfs insp phy" I developed a script >> to trigger the bug. The bug is not always

Re: Bug in 'btrfs filesystem du' ?

2016-06-28 Thread Andrei Borzenkov
28.06.2016 19:55, Henk Slager пишет: > On Tue, Jun 28, 2016 at 2:56 PM, M G Berberich > wrote: >> Hello, >> >> Am Montag, den 27. Juni schrieb Henk Slager: >>> On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich >>> wrote: Am Montag, den 27.

Re: Bug in 'btrfs filesystem du' ?

2016-06-28 Thread Andrei Borzenkov
28.06.2016 20:20, Andrei Borzenkov пишет: > 28.06.2016 19:55, Henk Slager пишет: >> On Tue, Jun 28, 2016 at 2:56 PM, M G Berberich <bt...@oss.m-berberich.de> >> wrote: >>> Hello, >>> >>> Am Montag, den 27. Juni schrieb Henk Slager: >>>>

Re: Unable to mount degraded RAID5

2016-07-04 Thread Andrei Borzenkov
04.07.2016 23:43, Chris Murphy пишет: > > Have you done a scrub on this file system and do you know if anything > was fixed or if it always found no problem? > scrub on degraded RAID5 cannot fix anything by definition, because even if scrub finds discrepancies, it does not have enough data to

Re: 64-btrfs.rules and degraded boot

2016-07-06 Thread Andrei Borzenkov
On Wed, Jul 6, 2016 at 2:45 PM, Austin S. Hemmelgarn <ahferro...@gmail.com> wrote: > On 2016-07-06 05:51, Andrei Borzenkov wrote: >> >> On Tue, Jul 5, 2016 at 11:10 PM, Chris Murphy <li...@colorremedies.com> >> wrote: >>> >>> I started a syste

Re: 64-btrfs.rules and degraded boot

2016-07-06 Thread Andrei Borzenkov
Отправлено с iPhone > 6 июля 2016 г., в 15:14, Austin S. Hemmelgarn <ahferro...@gmail.com> > написал(а): > >> On 2016-07-06 07:55, Andrei Borzenkov wrote: >> On Wed, Jul 6, 2016 at 2:45 PM, Austin S. Hemmelgarn >> <ahferro...@gmail.com> wrote: >>

Re: rollback to a snapshot and delete old top volume - missing of "@"

2016-07-08 Thread Andrei Borzenkov
07.07.2016 15:17, Kai Herlemann пишет: > Hi, > > I want to rollback a snapshot and have done this by execute "btrfs sub > set-default / 618". > Now I want to delete the old top volume to save space, but google and > manuals didn't helped. > > I mounted for the following the root volume at

Re: 64-btrfs.rules and degraded boot

2016-07-06 Thread Andrei Borzenkov
On Wed, Jul 6, 2016 at 8:19 PM, Chris Murphy wrote: > > I'm mainly concerned with rootfs. And I'm mainly concerned with a very > simple 2 disk raid1. With a simple user opt in using > rootflags=degraded, it should be possible to boot the system. Right > now it's not

Re: 64-btrfs.rules and degraded boot

2016-07-06 Thread Andrei Borzenkov
On Wed, Jul 6, 2016 at 9:23 PM, Chris Murphy wrote: >>> [root@f24s ~]# btrfs fi show >>> warning, device 2 is missing >>> Label: none uuid: 96240fd9-ea76-47e7-8cf4-05d3570ccfd7 >>> Total devices 3 FS bytes used 2.26GiB >>> devid3 size 50.00GiB used 3.01GiB

Re: rollback to a snapshot and delete old top volume - missing of "@"

2016-07-08 Thread Andrei Borzenkov
09.07.2016 00:50, Chris Murphy пишет: >> >> Instead those utilities should employ rootflags=subvol or subvolid to >> explicitly use a particular fs tree for rootfs, rather that hide this >> fact by using subvolume set-default. > > The only distro installer I know that works this way out of the

Re: raid1 has failing disks, but smart is clear

2016-07-08 Thread Andrei Borzenkov
07.07.2016 09:40, Corey Coughlin пишет: > Hi Tomasz, > Thanks for the response! I should clear some things up, though. > > On 07/06/2016 03:59 PM, Tomasz Kusmierz wrote: >>> On 6 Jul 2016, at 23:14, Corey Coughlin >>> wrote: >>> >>> Hi all, >>> Hoping you

Re: [PATCH v2] btrfs-progs: du: fix to skip not btrfs dir/file

2016-07-08 Thread Andrei Borzenkov
07.07.2016 12:43, Wang Shilong пишет: > 'btrfs file du' is a very useful tool to watch my system > file usage information with snapshot aware. > > when trying to run following commands: > [root@localhost btrfs-progs]# btrfs file du / > Total Exclusive Set shared Filename > ERROR: Failed

Re: raid1 has failing disks, but smart is clear

2016-07-08 Thread Andrei Borzenkov
08.07.2016 04:24, Duncan пишет: > Corey Coughlin posted on Wed, 06 Jul 2016 23:40:30 -0700 as excerpted: > >> Well yeah, if I was mounting all the disks to different mount points, I >> would definitely use UUIDs to get them mounted. But I haven't seen any >> way to set up a "mkfs.btrfs" command

Re: 64-btrfs.rules and degraded boot

2016-07-06 Thread Andrei Borzenkov
On Tue, Jul 5, 2016 at 11:10 PM, Chris Murphy wrote: > I started a systemd-devel@ thread since that's where most udev stuff > gets talked about. > > https://lists.freedesktop.org/archives/systemd-devel/2016-July/037031.html > Before discussing how to implement it in

Re: possible bug - wrong path in 'btrfs subvolume show' when snapshot is in path below subvolume.

2016-08-06 Thread Andrei Borzenkov
05.08.2016 02:32, Peter Holm пишет: > 'btrfs subvolumee show' gives no path to btrfs system root (volid=5) > when snapshot is in the folder of subvolume. > > Step to reproduce. > 1.btrfs subvolume create xyz > 2.btrfs subvolume snapshot xyz xyz/xyz > 3.btrfs subvolume snapshot /xyz This last

Re: possible bug - wrong path in 'btrfs subvolume show' when snapshot is in path below subvolume.

2016-08-06 Thread Andrei Borzenkov
inal when > executing thoose commands. > https://s31.postimg.org/9f0d7xb7f/is_this_a_bug.png > > /Peter Holm > > > 2016-08-06 7:14 GMT+02:00, Andrei Borzenkov <arvidj...@gmail.com>: >> 05.08.2016 02:32, Peter Holm пишет: >>> 'btrfs subvolumee show' give

Re: raid1: cannot add disk to replace faulty because can only mount fs as read-only.

2017-01-28 Thread Andrei Borzenkov
27.01.2017 23:03, Austin S. Hemmelgarn пишет: > On 2017-01-27 11:47, Hans Deragon wrote: >> On 2017-01-24 14:48, Adam Borowski wrote: >> >>> On Tue, Jan 24, 2017 at 01:57:24PM -0500, Hans Deragon wrote: >>> If I remove 'ro' from the option, I cannot get the filesystem mounted because of

Re: man filesystems(5) doesn't contain Btrfs

2017-02-17 Thread Andrei Borzenkov
16.02.2017 23:47, Austin S. Hemmelgarn пишет: > On 2017-02-16 15:36, Chris Murphy wrote: >> Hi, >> >> This man page contains a list for pretty much every other file system, >> with a oneliner description: ext4, XFS is in there, and even NTFS, but >> not Btrfs. >> >> Also, /etc/filesystems doesn't

Re: Raid 1 recovery

2017-01-19 Thread Andrei Borzenkov
On Thu, Jan 19, 2017 at 10:15 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Chris Murphy posted on Wed, 18 Jan 2017 14:30:28 -0700 as excerpted: > >> On Wed, Jan 18, 2017 at 2:07 PM, Jon wrote: >>> So, I had a raid 1 btrfs system setup on my laptop. Recently I upgraded >>> the

Re: Mixing partitioned and non-partitioned discs in a RAID?

2016-08-20 Thread Andrei Borzenkov
20.08.2016 07:00, Bearcat Şándor пишет: > I have a btrfs non-raid file system that i'd like to convert to > raid10. This single device has my efi boot partion on it, so it's > partitioned into sda1 and sda2. I have 3 other discs (sdc-sde) that > i'd like to make partition-less systems and then add

"parent transid verify failed" out of blue sky

2017-02-27 Thread Andrei Borzenkov
This is VM under QEMU/KVM running openSUSE Tumbleweed. I boot it infrequently for short time to test something. Last time it installed quite a lot of updates including kernel (I think 4.9.11 was the last version); I do not remember whether I rebooted it after that. Today I booted it to check

Cross subvolume "cp --reflink"?

2016-10-19 Thread Andrei Borzenkov
I get "Failed to clone: Invalid cross-device link". Is it expected? Basically this is (on openSUSE TW which has root on subvolume) mount -o subvol=/ /dev/vda1 /mnt btrfs sub create /mnt/var/cache cp -a --reflink=always /var/cache/* /mnt/var/cache Kernel 4.7.5-1-default. -- To unsubscribe from

Re: Cross subvolume "cp --reflink"?

2016-10-19 Thread Andrei Borzenkov
19.10.2016 20:04, Hugo Mills пишет: > On Wed, Oct 19, 2016 at 07:52:14PM +0300, Andrei Borzenkov wrote: >> I get "Failed to clone: Invalid cross-device link". Is it expected? >> Basically this is (on openSUSE TW which has root on subvolume) >> >> mount -o

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

2016-11-27 Thread Andrei Borzenkov
28.11.2016 06:37, Christoph Anton Mitterer пишет: > On Sat, 2016-11-26 at 14:12 +0100, Goffredo Baroncelli wrote: >> I cant agree. If the filesystem is mounted read-only this behavior >> may be correct; bur in others cases I don't see any reason to not >> correct wrong data even in the read case.

Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots

2016-10-16 Thread Andrei Borzenkov
15.10.2016 01:58, Alberto Bursi пишет: > > > On 10/15/2016 12:17 AM, Chris Murphy wrote: >> It should be -e can accept a listing of all the subvolumes you want to >> send at once. And possibly an -r flag, if it existed, could >> automatically populate -e. But the last time I tested -e I just got

What are "parent" and "parent_uuid" in "btrfs sub list"?

2016-10-17 Thread Andrei Borzenkov
I try to understand how to build a tree of snapshots (i.e. - which subvolume was used to snapshot/clone other subvolume). What is the correct way to determine it? In particular, "btrfs sub list -p" always prints something for "parent snapshot", while btrfs sub list -q" only prints parent_uuid for

Re: Metadata balance fails ENOSPC

2016-12-02 Thread Andrei Borzenkov
01.12.2016 18:48, Chris Murphy пишет: > On Thu, Dec 1, 2016 at 7:10 AM, Stefan Priebe - Profihost AG > <s.pri...@profihost.ag> wrote: >> >> Am 01.12.2016 um 14:51 schrieb Hans van Kranenburg: >>> On 12/01/2016 09:12 AM, Andrei Borzenkov wrote: >>>> O

Re: Metadata balance fails ENOSPC

2016-12-01 Thread Andrei Borzenkov
On Thu, Dec 1, 2016 at 10:49 AM, Stefan Priebe - Profihost AG wrote: ... > > Custom 4.4 kernel with patches up to 4.10. But i already tried 4.9-rc7 > which does the same. > > >>> # btrfs filesystem show /ssddisk/ >>> Label: none uuid: a69d2e90-c2ca-4589-9876-234446868adc

Re: Will fstrim discard unused parts of chunks?

2017-01-03 Thread Andrei Borzenkov
03.01.2017 00:02, Jeff Mahoney пишет: > On 1/2/17 4:55 AM, Andrei Borzenkov wrote: >> I try to understand what exactly is trimmed in case of btrfs. Using >> installation in QEMU I see that host file size is about 9GB, allocated >> size in guest approximately matches it an

Will fstrim discard unused parts of chunks?

2017-01-02 Thread Andrei Borzenkov
I try to understand what exactly is trimmed in case of btrfs. Using installation in QEMU I see that host file size is about 9GB, allocated size in guest approximately matches it and used space in guest is 7.6GB. After some experimenting it looks like host size follows "Device allocated" value in

Re: Qgroups are not applied when snapshotting a subvol?

2017-03-26 Thread Andrei Borzenkov
27.03.2017 03:39, Qu Wenruo пишет: > > > At 03/26/2017 06:03 AM, Moritz Sichert wrote: >> Hi, >> >> I tried to configure qgroups on a btrfs filesystem but was really >> surprised that when you snapshot a subvolume, the snapshot will not be >> assigned to the qgroup the subvolume was in. >> >> As

Re: send snapshot from snapshot incremental

2017-03-29 Thread Andrei Borzenkov
On Wed, Mar 29, 2017 at 1:01 AM, Jakob Schürz wrote: > > There is Subvolume A on the send- and the receive-side. > There is also Subvolume AA on the send-side from A. > The parent-ID from send-AA is the ID from A. > The received-ID from A on received-side A is the ID

Re: Qgroups are not applied when snapshotting a subvol?

2017-03-27 Thread Andrei Borzenkov
27.03.2017 22:32, Chris Murphy пишет: > How about if qgroups are enabled, then non-root user is prevented from > creating new subvolumes? > > Or is there a way for a new nested subvolume to be included in its > parent's quota, rather than the new subvolume having a whole new quota > limit? >

Re: Is btrfs-convert able to deal with sparse files in a ext4 filesystem?

2017-04-02 Thread Andrei Borzenkov
02.04.2017 03:59, Duncan пишет: > > 4) In fact, since an in-place convert is almost certainly going to take > more time than a blow-away and restore from backup, This caught my eyes. Why? In-place convert just needs to recreate metadata. If you have multi-terabyte worth of data copying them

Re: Btrfs disk layout question

2017-04-12 Thread Andrei Borzenkov
12.04.2017 14:20, Austin S. Hemmelgarn пишет: > On 2017-04-12 00:18, Chris Murphy wrote: >> On Tue, Apr 11, 2017 at 3:00 PM, Adam Borowski >> wrote: >>> On Tue, Apr 11, 2017 at 12:15:32PM -0700, Amin Hassani wrote: I am working on a project with Btrfs and I was wondering

Re: Btrfs disk layout question

2017-04-12 Thread Andrei Borzenkov
12.04.2017 20:21, Chris Murphy пишет: > btrfs-map-logical is the tool that will convert logical to physical > and also give what device it's on; but the device notation is copy 1 > and copy 2, so you have to infer what device that is, it's not > explicit. > Quickly checking output - for my

Re: Btrfs disk layout question

2017-04-15 Thread Andrei Borzenkov
12.04.2017 20:21, Chris Murphy пишет: > btrfs-map-logical is the tool that will convert logical to physical > and also give what device it's on; but the device notation is copy 1 > and copy 2, so you have to infer what device that is, it's not > explicit. > Actually I just checked with

Re: raid1 degraded mount still produce single chunks, writeable mount not allowed

2017-03-02 Thread Andrei Borzenkov
02.03.2017 16:41, Duncan пишет: > Chris Murphy posted on Wed, 01 Mar 2017 17:30:37 -0700 as excerpted: > >> [1717713.408675] BTRFS warning (device dm-8): missing devices (1) >> exceeds the limit (0), writeable mount is not allowed >> [1717713.446453] BTRFS error (device dm-8): open_ctree failed

Re: [PATCH v3 7/7] btrfs: Enhance missing device kernel message

2017-03-07 Thread Andrei Borzenkov
08.03.2017 05:41, Qu Wenruo пишет: > For missing device, btrfs will just refuse to mount with almost > meaningless kernel message like: > > BTRFS info (device vdb6): disk space caching is enabled > BTRFS info (device vdb6): has skinny extents > BTRFS error (device vdb6): failed to read the

Re: btrfs device ready purpose

2017-07-07 Thread Andrei Borzenkov
07.07.2017 19:42, Chris Murphy пишет: > I'm digging through piles of list emails and not really finding an > answer to this. Maybe it's Friday and I'm just confused... > > > [root@f26s ~]# btrfs device ready /dev/sda1 > [root@f26s ~]# echo $? > 0 > [root@f26s ~]# btrfs device ready

Re: bind mounts on btrfs cause mountinfo confusion

2017-07-18 Thread Andrei Borzenkov
18.07.2017 10:24, Duncan пишет: > > Before the kernel started printing that subvol= stuff, this information > was missing, as it still is for non-btrfs. > Huh? tail -1 /proc/self/mountinfo 215 25 8:4 /tmp/from /tmp/to rw,relatime shared:1 - ext4 /dev/sda4 rw,errors=remount-ro,data=ordered

Re: Destination's Received UUID and source's UUID are not the same

2017-07-16 Thread Andrei Borzenkov
16.07.2017 02:03, Cerem Cem ASLAN пишет: > A few days ago I tried to copy my backups to another machine by using > my external hard drive. `btrfs send my-backup | btrfs receive > /path/to/backup/dir` didn't go right since destination snapshot's > "Received UUID" does not match with source's

Re: btrfs device ready purpose

2017-07-21 Thread Andrei Borzenkov
21.07.2017 17:36, Chris Murphy пишет: >> >> The command is just a simple wrapper around the DEVICES_READY ioctl, but >> now that systemd has it's own wrapper tool, there are probably no users >> of that subcommand in 'btrfs' tool itself. We can enhance the >> documentation to state the expected

How to detect "orphaned" subvolume attachment point in snapshot?

2017-07-22 Thread Andrei Borzenkov
Here is structure of snapshots in openSUSE; all snapshots of root volume are created under /.snapshots subvolume: linux-gtrk:/host/home/src/python-btrfs/examples # sudo mount -o ro,subvol=/ /dev/sda3 /mnt linux-gtrk:/host/home/src/python-btrfs/examples # ./show_directory_contents.py /mnt/

Re: Deleting mounted subvolumes

2017-07-03 Thread Andrei Borzenkov
04.07.2017 02:21, Chris Murphy пишет: > It's more like a bind mount of a directory, as far as what's going on > under the hood. I take it it's possible to delete a directory that is > bind mounted elsewhere? Yes, it is. Usual rules apply - it must be empty, but "rm -r" works as well (and is more

Re: [PATCH] btrfs: check options during subsequent mount

2017-04-28 Thread Andrei Borzenkov
28.04.2017 12:14, Anand Jain пишет: > We allow recursive mounts with subvol options such as [1] > > [1] > mount -o rw,compress=lzo /dev/sdc /btrfs1 > mount -o ro,subvol=sv2 /dev/sdc /btrfs2 > > And except for the btrfs-specific subvol and subvolid options > all-other options are just ignored

Re: Can I see what device was used to mount btrfs?

2017-05-03 Thread Andrei Borzenkov
03.05.2017 14:26, Austin S. Hemmelgarn пишет: > On 2017-05-02 15:50, Goffredo Baroncelli wrote: >> On 2017-05-02 20:49, Adam Borowski wrote: It could be some daemon that waits for btrfs to become complete. Do we have something? >>> Such a daemon would also have to read the chunk tree.

Re: Can I see what device was used to mount btrfs?

2017-05-03 Thread Andrei Borzenkov
03.05.2017 21:43, Chris Murphy пишет: > If I understand the bug report correctly, the user specifies mounting > by label which then systemd is converting into /dev/dm-0 (because it's > a two LUKS devices Btrfs volume). > No, that's not the problem. The actual reason for report is that systemd

Re: Creating btrfs RAID on LUKS devs makes devices disappear

2017-05-13 Thread Andrei Borzenkov
12.05.2017 20:07, Chris Murphy пишет: > On Thu, May 11, 2017 at 5:24 PM, Ochi wrote: >> Hello, >> >> here is the journal.log (I hope). It's quite interesting. I rebooted the >> machine, performed a mkfs.btrfs on dm-{2,3,4} and dm-3 was missing >> afterwards (around timestamp 66.*).

Re: Creating btrfs RAID on LUKS devs makes devices disappear

2017-05-13 Thread Andrei Borzenkov
13.05.2017 18:28, Ochi пишет: > Hello, > > okay, I think I now have a repro that is stupidly simple, I'm not even > sure if I overlook something here. No multi-device btrfs involved, but > notably it does happen with btrfs, but not with e.g. ext4. > I could not reproduce it with single device

Re: Exactly what is wrong with RAID5/6

2017-06-21 Thread Andrei Borzenkov
21.06.2017 09:51, Marat Khalili пишет: > On 21/06/17 06:48, Chris Murphy wrote: >> Another possibility is to ensure a new write is written to a new*not* >> full stripe, i.e. dynamic stripe size. So if the modification is a 50K >> file on a 4 disk raid5; instead of writing 3 64K data strips + 1 64K

Re: Exactly what is wrong with RAID5/6

2017-06-21 Thread Andrei Borzenkov
21.06.2017 16:41, Austin S. Hemmelgarn пишет: > On 2017-06-21 08:43, Christoph Anton Mitterer wrote: >> On Wed, 2017-06-21 at 16:45 +0800, Qu Wenruo wrote: >>> Btrfs is always using device ID to build up its device mapping. >>> And for any multi-device implementation (LVM,mdadam) it's never a >>>

Re: BTRFS converted from EXT4 becomes read-only after reboot

2017-05-23 Thread Andrei Borzenkov
24.05.2017 00:49, Marc MERLIN пишет: > On Tue, May 23, 2017 at 03:38:01PM -0600, Chris Murphy wrote: >>> I've tried an ext4 to btrfs conversion 3 times in the last 3 years, it >>> never worked properly any of those times, sadly. >> >> Since the 4.6 total rewrite? There are also recent bug fixes

Can I see what device was used to mount btrfs?

2017-04-29 Thread Andrei Borzenkov
I'm chasing issue with btrfs mounts under systemd (https://github.com/systemd/systemd/issues/5781) - to summarize, systemd waits for the final device that makes btrfs complete and mounts it using this device name. But in /proc/self/mountinfo we actually see another device name. Due to

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Andrei Borzenkov
On Tue, May 2, 2017 at 4:58 PM, Adam Borowski <kilob...@angband.pl> wrote: > On Sun, Apr 30, 2017 at 08:47:43AM +0300, Andrei Borzenkov wrote: >> I'm chasing issue with btrfs mounts under systemd >> (https://github.com/systemd/systemd/issues/5781) - to summarize, systemd &

Re: send | receive: received snapshot is missing recent files

2017-09-11 Thread Andrei Borzenkov
11.09.2017 20:53, Axel Burri пишет: > On 2017-09-08 06:44, Dave wrote: >> I'm referring to the link below. Using "btrfs subvolume snapshot -r" >> copies the Received UUID from the source into the new snapshot. The >> btrbk FAQ entry suggests otherwise. Has something changed? > > I don't think

Re: Storage and snapshots as historical yearly

2017-09-11 Thread Andrei Borzenkov
11.09.2017 21:17, Senén Vidal Blanco пишет: > I am trying to implement a system that stores the data in a unit (A) with > BTRFS format that is untouchable and that future files and folders created or > modified are stored in another physical unit (B) with BTRFS format. > Each year the new files

Re: snapshots of encrypted directories?

2017-09-15 Thread Andrei Borzenkov
15.09.2017 15:35, Austin S. Hemmelgarn пишет: > On 2017-09-14 23:45, Andrei Borzenkov wrote: >> 14.09.2017 18:32, Hugo Mills пишет: >>> On Thu, Sep 14, 2017 at 04:57:39PM +0200, Ulli Horlacher wrote: >>>> I use encfs on top of btrfs. >>>> I can create

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-15 Thread Andrei Borzenkov
15.09.2017 08:50, Goffredo Baroncelli пишет: > On 09/15/2017 05:55 AM, Andrei Borzenkov wrote: >> 15.09.2017 01:00, Goffredo Baroncelli пишет: >>> >>> 2) The second bug, is a more severe bug. If during a writing of a buffer >>> with O_DIRECT, the buffer is

Re: snapshots of encrypted directories?

2017-09-14 Thread Andrei Borzenkov
14.09.2017 18:32, Hugo Mills пишет: > On Thu, Sep 14, 2017 at 04:57:39PM +0200, Ulli Horlacher wrote: >> I use encfs on top of btrfs. >> I can create btrfs snapshots, but I have no suggestive access to the files >> in these snaspshots, because they look like: >> >> drwx-- framstag users

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-14 Thread Andrei Borzenkov
15.09.2017 01:00, Goffredo Baroncelli пишет: > > 2) The second bug, is a more severe bug. If during a writing of a buffer with > O_DIRECT, the buffer is updated at the same time by a second process, the > checksum may be incorrect. > Is it btrfs specific? If buffer is updated before it was

Re: ERROR: parent determination failed (btrfs send-receive)

2017-09-18 Thread Andrei Borzenkov
18.09.2017 11:45, Graham Cobb пишет: > On 18/09/17 07:10, Dave wrote: >> For my understanding, what are the restrictions on deleting snapshots? >> >> What scenarios can lead to "ERROR: parent determination failed"? > > The man page for btrfs-send is reasonably clear on the requirements > btrfs

Re: ERROR: parent determination failed (btrfs send-receive)

2017-09-19 Thread Andrei Borzenkov
18.09.2017 09:10, Dave пишет: > I use snap-sync to create and send snapshots. > > GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external > drive > https://github.com/wesbarnett/snap-sync > Are you trying to backup top-level subvolume? I just reproduced this behavior with

Re: difference between -c and -p for send-receive?

2017-09-19 Thread Andrei Borzenkov
On Tue, Sep 19, 2017 at 1:24 PM, Graham Cobb wrote: > On 19/09/17 01:41, Dave wrote: >> Would it be correct to say the following? > > Like Duncan, I am just a user, and I haven't checked the code. I > recommend Duncan's explanation, but in case you are looking for > something

Re: difference between -c and -p for send-receive?

2017-09-19 Thread Andrei Borzenkov
19.09.2017 03:41, Dave пишет: > new subject for new question > > On Mon, Sep 18, 2017 at 1:37 PM, Andrei Borzenkov <arvidj...@gmail.com> wrote: > >>>> What scenarios can lead to "ERROR: parent determination failed"? >>> >>> The man p

Re: difference between -c and -p for send-receive?

2017-09-20 Thread Andrei Borzenkov
20.09.2017 22:05, Antoine Belvire пишет: > Hello, > >> All snapshots listed in -c options and snapshot that we want to >> transfer must have the same parent uuid, unless -p is explicitly >> provided. > > It's rather the same mount point than the same parent uuid, like cp > --reflink, isn't it?

Re: ERROR: parent determination failed (btrfs send-receive)

2017-09-17 Thread Andrei Borzenkov
18.09.2017 05:31, Dave пишет: > Sometimes when using btrfs send-receive, I get errors like this: > > ERROR: parent determination failed for > > When this happens, btrfs send-receive backups fail. And all subsequent > backups fail too. > > The issue seems to stem from the fact that an automated

  1   2   >