Re: 5.2.0 rc5, circular lock warning systemd-journal and btrfs_page_mkwrite

2019-06-18 Thread Chris Murphy
On Tue, Jun 18, 2019 at 11:45 AM Chris Murphy wrote: > > This still happens on RC5, attaching full dmesg to this email. > > [8.432777] BTRFS info (device sda6): use zstd compression, level 3 > [8.432791] BTRFS info (device sda6): using free space tree > [8.581762] sy

Re: Issues with btrfs send/receive with parents

2019-06-13 Thread Chris Murphy
do incrementals between the most recent two snapshots. i.e. the snapshot that follows -p should be the snapshot most recently successfully received on the destination. That represents the least amount of increment needed to update the destination. It will work the way you're doing it now, with -p being an older snapshot, but you're unnecessarily sending a lot more data every time. -- Chris Murphy

Re: APFS improvements (e.g. firm links, volume w/ subvols replication) as ideas for Btrfs?

2019-06-12 Thread Chris Murphy
everything at its level. I'm not > suggesting Btrfs do it for everything, but the filesystem needs some > intelligence about subvolume handling that it doesn't have now. ZFS is way more limited in this regard than Btrfs. Snapshots are always children of datasets, they are always read only. You can clone a snapshot into a dataset. I don't think (?) there is such a thing as nested datasets or snapshots. Datasets can't be deleted until all children (snapshots) are deleted first. In many ways ZFS limitations keep users from doing things like many nested subvolumes like we see on Btrfs and then it turns into logical problems for users and developers alike. So a non-Btrfs solution is going to need higher level advancements anyway, either VFS over overlayfs or some combination of the two. -- Chris Murphy

Re: APFS improvements (e.g. firm links, volume w/ subvols replication) as ideas for Btrfs?

2019-06-11 Thread Chris Murphy
as it might seem from the pool, compared to what a ZFS dataset is, or I guess it's called a volume is in APFS. To do this on Btrfs probably is another disk format change. My guess is something based on seed-sprout feature, but without the mandatory 2nd block device for the spout. i.e. freeze all the trees. -- Chris Murphy

Re: Issues with btrfs send/receive with parents

2019-06-10 Thread Chris Murphy
worked around. But I considered it untenable and fixed it with a good quality self-powered USB hub (don't rely on bus power), or perhaps more specifically one that comes with a high amp power adapter. It needs to be able to drive all the drives in their read/write usage, which for laptop drives is ~0.35A each. You really shouldn't be getting link resets like the above, even though I suspect it's unrelated to the current problem report. -- Chris Murphy

5.2rc2, circular lock warning systemd-journal and btrfs_page_mkwrite

2019-06-04 Thread Chris Murphy
01 R09: 55d0300e8160 [7.887550] fmac.local kernel: R10: 7ffc79f6a080 R11: 35fc R12: 7ffc79ed78c8 [7.887551] fmac.local kernel: R13: 7ffc79ed78c0 R14: 55d0300efa60 R15: 00c3d9ef -- Chris Murphy

leaf size and compression

2019-05-30 Thread Chris Murphy
ing compression? All the prior benchmarks back in ancient times when default node size was changed from 4KiB to 16KIB I'm pretty sure were predicated on uncompressed data. It could be a question of how many angels can dance on the head of a pin, but there it is. -- Chris Murphy

Re: parent transid verify failed on 604602368 wanted 840641 found 840639

2019-05-27 Thread Chris Murphy
han what exist on the filesystem, which suggests file system metadata was dropped. It's not certain from this information what caused that: device, or some layer in between like bcache, or Btrfs. -- Chris Murphy

Re: Citation Needed: BTRFS Failure Resistance

2019-05-23 Thread Chris Murphy
On Thu, May 23, 2019 at 10:34 AM Adam Borowski wrote: > > On Thu, May 23, 2019 at 10:24:28AM -0600, Chris Murphy wrote: > > On Thu, May 23, 2019 at 5:19 AM Austin S. Hemmelgarn > > > BTRFS explicitly requests write barriers to prevent that type of > > > reordering

Re: Citation Needed: BTRFS Failure Resistance

2019-05-23 Thread Chris Murphy
ier Note: This option has been deprecated as of kernel v4.10; in that version, integrity operations are always performed and the mount option is ignored. These mount options will be removed no earlier than kernel v4.15. Since they're getting rid of it, I wonder if it's sane for most any sane file system use case. -- Chris Murphy

Re: Re[2]: [Samba] Fw: Btrfs Samba and Quotas

2019-05-21 Thread Chris Murphy
explain why I'm not affected as Samba only sees one set of inodes, no duplicates, per mount. >From the same Btrfs volume, I do share other subvolumes, and therefore there's a repeat of inodes, but they're each in their own mountpoints+shares. So far I've seen no evidence of Samba confusion. But I also don't use quotas. -- Chris Murphy

Re: Btrfs remote reflink with Samba

2019-05-20 Thread Chris Murphy
und (ctrl-x, ctrl-v) > files in Nautilus within the same share without making new copies. I just did ctrl-c, ctrl-v for a file in one dir to another dir, and it takes forever. It's clearly being copied over the network to my local machine and then pushed back to the server. Three minutes to copy a 2GiB file. Server side: kernel 5.1.0-1.fc31.x86_64 samba-4.9.5-0.fc29.x86_64 smb.conf contains 'vfs objects = btrfs' for this share Client side: samba-client-4.10.2-1.1.fc30.x86_64 gvfs-smb-1.40.1-2.fc30.x86_64 nautilus-3.32.1-1.fc30.x86_64 -- Chris Murphy

'watch btrfs fi show' crash while 'btrfs device delete'

2019-05-20 Thread Chris Murphy
evice, and then there's a crash. -- Chris Murphy

Re: Unbootable root btrfs

2019-05-18 Thread Chris Murphy
On Fri, May 17, 2019 at 10:39 PM Robert White wrote: > > On 5/18/19 4:06 AM, Chris Murphy wrote: > > On Fri, May 17, 2019 at 2:18 AM Lee Fleming > > wrote: > >> > >> I didn't see that particular warning. I did see a warning that it could > >>

Re: Massive filesystem corruption after balance + fstrim on Linux 5.1.2

2019-05-17 Thread Chris Murphy
rror Huh. Any kernel message at the same time? I would expect any fstrim user space error message to also have a kernel message. Any i/o error suggests some kind of storage stack failure - which could be hardware or software, you can't know without seeing the kernel messages. -- Chris Murphy

Re: Unbootable root btrfs

2019-05-17 Thread Chris Murphy
reproducer. Is it using QCOW2 or RAW file backing, or LVM, or plain partition? What is the qemu command for the VM? You can get that with 'ps -aux | grep qemu' and it should show all the options used including the kind of block devices and caching. And then what is the workload inside the VM? -- Chris Murphy

Re: Massive filesystem corruption after balance + fstrim on Linux 5.1.2

2019-05-16 Thread Chris Murphy
does however, with pvmove. But that makes the testing more complicated, introduces more factors. So...I still vote for bisect. But even if you can't bisect, if you can reproduce, that might help someone else who can do the bisect. Your stack looks like this? Btrfs LUKS/dmcrypt LVM Samsung SSD -- Chris Murphy

Re: Unbootable root btrfs

2019-05-16 Thread Chris Murphy
;btrfs restore' which is a way to scrape data out of an unmountable file system. It's better than nothing if the data is important, but ideal if at least ro mount can work. -- Chris Murphy

Re: Howto read btrfs stack trace?

2019-05-07 Thread Chris Murphy
om here you can do a mount. If you previously cancelled the balance, you can use 'btrfs balance resume' to resume, and see if it runs into the same problem. If it does, then yeah it's still a bug, if not maybe it's fixed. But again, I'm not sure if there is a backport of the fix to older kernels or what version the backport is found in. -- Chris Murphy

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-05-07 Thread Chris Murphy
iles, it checksums file extents in 4KiB increments. And I don't even think there's an ioctl to extract only checksums, in order to do a comparison in user space. The checksums are, as far as I'm aware, only used internally within Btrfs kernel space. -- Chris Murphy

Re: Hibernation into swap file

2019-05-05 Thread Chris Murphy
done by the kernel just after initramfs is unpacked, well before the file system is ro mounted. I think it's probably a good idea to make sure bootloader kernel command line contains the 'ro' parameter. I don't know if Btrfs ro is guaranteed to change on-disk metadata, I know logreplay affects the in-memory state of Btrfs but I don't think it changes on-disk metadata. But yeah, worth testing and making sure the backups are kept current. -- Chris Murphy

Re: ref mismatch / root not found in extent tree / backpointer mismatch / owner ref check failed

2019-05-04 Thread Chris Murphy
ated: 30545889550336 > referenced 31161532510208 Hopefully a developer has an idea how serious it is and whether it's safe to try to use --repair. In the meantime, I strongly advice ensuring your backups are up to date while you wait for a reply. Extent tree problems can be serious. Also, what do you get for: # btrfs dev stats If unmounted use device node, if mounted use the mountpoint. -- Chris Murphy

Re: Re[4]: Rough (re)start with btrfs

2019-05-04 Thread Chris Murphy
ce you have a reproducer, then you can change the scheduler and see if your reproduce steps still reproduce the problem. > > >Also, what are the mount options? > rw,noatime,nospace_cache,subvolid=5,subvol=/ > But noatime and nospace_cache I added just today. OK that all looks good. -- Chris Murphy

Re: Re[2]: Rough (re)start with btrfs

2019-05-02 Thread Chris Murphy
etadata. The Btrfs call traces during the blocked task are INFO, not warnings or errors, so the file system and data is likely fine. There's no read, write, corruption, or generation errors in the dmesg; but you can also check 'btfs dev stats ' which is a persistent counter for this particular device. -- Chris Murphy

Re: Rough (re)start with btrfs

2019-05-02 Thread Chris Murphy
he free space cache, blocking the whole fs to be > committed. > > If you still want to try btrfs, you could try "nosapce_cache" mount option. > Free space cache of btrfs is just an optimization, you can completely > ignore that with minor performance drop. I should have read this before replying earlier. You can also do a one time clean mount with '-o clear_cache,space_cache=v2' which will remove the v1 (default) space cache, and create a v2 cache. Subsequent mount will see the flag for this feature and always use the v2 cache. It's a totally differently implementation and shouldn't have this problem. -- Chris Murphy

Re: Re[2]: Rough (re)start with btrfs

2019-05-02 Thread Chris Murphy
nge nothing else. Now try to reproduce. Let's see if it still happens. Also, what are the mount options? -- Chris Murphy

Re: Rough (re)start with btrfs

2019-05-01 Thread Chris Murphy
drive or the link. Can you attach the entire dmesg as a file? Also the output from # smartctl -l gplog,0x13 /dev/sdX If there's an error or it outputs nothing useful, then # smarctl -x /dev/sdX Also attach that as a file. It's better if these things are file attachments, they're easier to read, most MUAs do terrible line wraps. -- Chris Murphy

Re: recommended way to allow mounting of degraded array at boot

2019-04-30 Thread Chris Murphy
d in 30 seconds, good chance they won't appear, and then either fail at this rule or continue on with the mount attempt (which then fails). But either way, the user gets to a prompt, and can troubleshoot the problem from there rather than being stuck with force poweroff being the only alternative. -- Chris Murphy

Re: Some file lost when send back snapshot.

2019-04-27 Thread Chris Murphy
ace. I'm in agreement, and would like to see 'btrfs property set <> ro false> removed. It's confusing. Once a subvolume is read-only, it should remain read-only, and if the use case requires rw, then just snapshot it (default rw). -- Chris Murphy

Re: reading/writing btrfs volume regularly freezes system

2019-04-22 Thread Chris Murphy
sysrq and writing out the trigger command in a console (either a tty if you have physical access, or netconsole), then reproduce the hang, and then hit return on the console with the pre-typed sysrq command. Sometimes the sysrq output is quite a lot for the kernel buffer and will overflow dmesg, so you'll either need to use `log_buf_len=1M` boot parameter, or you can get sysrq output from journalctl if it's a system system. -- Chris Murphy

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-04-07 Thread Chris Murphy
id1 profile instead binding them in a RAID5 and "giving" just a > single device to btrfs. Not necessarily. If corruption happens early enough, it gets baked into all copies of the metadata. -- Chris Murphy

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-04-05 Thread Chris Murphy
There is a valid superblock however. So it restored something, just not everything, not sure. Might be related to create failed success! -- Chris Murphy

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-04-04 Thread Chris Murphy
up so it doesn't really matter. I did file a bug. https://bugzilla.kernel.org/show_bug.cgi?id=202717 -- Chris Murphy

Re: help request for an unmountable raid1 filesystem

2019-04-04 Thread Chris Murphy
suggest that's a bug, just because btrfs restore needs to be expected to do better than this. I suggest taking a btrfs-image while waiting for btrfs-progs 5.0. # btrfs-image -c9 -t4 -ss -w /dev/ /pathtoimage/ -- Chris Murphy

Re: btrfs balance hangs

2019-04-04 Thread Chris Murphy
has some bug or Btrfs, and I would expect the firmware can't be responsible for such a huge transid difference. But yeah maybe Qu has an idea, also he knows about qgroups, I don't know anything about them other than it can make things very slow. There are some fixes on the way soon but it

Re: btrfs balance hangs

2019-04-04 Thread Chris Murphy
ached. I can't parse that, maybe Qu has an idea. In the meantime I suggest btrfs-progs 4.20.2 and running: # btrfs check --readonly /dev/ Attach that output. Balance is supposed to be COW. And skip_balance and clear_cache are supposed to be safe. So I'd say you've found a bug but I

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-04-04 Thread Chris Murphy
t it's an incomplete image. > The smaller system let me create an image, but the size of the file, > resulting from "btrfs-image -c 9 /dev/sdXY ...", is surprisingly small - > only 536576B. I guess this is conform with the man-page: "All data will > be zeroed, but metadata and the like is preserved. Mainly used for > debugging purposes." > > I shall send you a link to the image (in a private mail) as soon as > possible. Please, respect any private data in case you manage to recover > something. You should use -ss option for this reason. -- Chris Murphy

life time of backup roots

2019-04-02 Thread Chris Murphy
backup_total_bytes:1000207335424 backup_bytes_used:439356653568 backup_num_devices: 2 -- Chris Murphy

Re: interest in post-mortem examination of a BTRFS system and improving the btrfs-code?

2019-04-02 Thread Chris Murphy
available), no writing commands were issued. > > --repair will cause write, unless it even failed to open the filesystem. It consider `--repair --readonly` is a contradictory request, and it's ambiguous what the user wants (it's user error) and the command should fail with "conflicting options" error. -- Chris Murphy

Re: help request for an unmountable raid1 filesystem

2019-04-01 Thread Chris Murphy
On Sun, Mar 31, 2019 at 11:48 PM Glenn Trigg wrote: > > Hi Chris, > > After booting the fedora usb stick (running rc2), I got the same results. > > On Mon, 1 Apr 2019 at 08:35, Chris Murphy wrote: > > > > On Sat, Mar 30, 2019 at 5:43 PM Glenn Trigg wrote: >

Re: volumes.c:1762: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1

2019-04-01 Thread Chris Murphy
than they were before the repair was tried, and in the process the file system is so badly damaged it's not even useful for user or developer. -- Chris Murphy

Re: btrfs balance hangs

2019-04-01 Thread Chris Murphy
that means there's a kernel deadlock. > The output of dmesg after running echo w >/proc/sysrq-trigger is attached. > > kernel: 5.0.3-arch1-1-ARCH x86_64 > btrfs-progs version: v4.20.2 > Are qgroups enabled? -- Chris Murphy

Re: help request for an unmountable raid1 filesystem

2019-03-31 Thread Chris Murphy
On Sat, Mar 30, 2019 at 5:43 PM Glenn Trigg wrote: > > Hi Chris, > > Thanks for replying. > > On Fri, 29 Mar 2019 at 13:27, Chris Murphy wrote: > ... > > Seem in conflict. I don't really understand how the kernel complains > > about a bad super and yet user

Re: help request for an unmountable raid1 filesystem

2019-03-28 Thread Chris Murphy
On Thu, Mar 28, 2019 at 8:27 PM Chris Murphy wrote: >>So I suggest 5.0.4, or 4.19.32, or you can be brave and > download this, image it to a USB stick (dd if=file of=/dev/ bs=1M > oflag=direct) which of course will erase everything on the stick. > > https://kojipkgs.fedorap

Re: help request for an unmountable raid1 filesystem

2019-03-28 Thread Chris Murphy
; Where do I go from here? If it can't be mounted, then the only chance is `btrfs-find-tree` and `btrfs restore` to try and scrape out whatever data you need that isn't already backed up. The priority before trying to repair it, is to get anything important off because trying to repair it has a good chance of permanent data loss. Definitely the latest tools are recommended for repair, kernel doesn't matter so much. -- Chris Murphy

Re: backup uuid_tree generation not consistent across multi device (raid0) btrfs - won´t mount

2019-03-27 Thread Chris Murphy
Removing Andrei and Qu. On Wed, Mar 27, 2019 at 12:52 PM Chris Murphy wrote: > > And then also in the meantime, prepare for having to rebuild this array. a. Check with the manufacturer of the hardware raid for firmware updates for all the controllers. Also check if the new version is ba

Re: backup uuid_tree generation not consistent across multi device (raid0) btrfs - won´t mount

2019-03-27 Thread Chris Murphy
recover properly on their own, but... And then also in the meantime, prepare for having to rebuild this array. -- Chris Murphy

Re: backup uuid_tree generation not consistent across multi device (raid0) btrfs - won´t mount

2019-03-26 Thread Chris Murphy
On Tue, Mar 26, 2019 at 11:38 AM Chris Murphy wrote: > > On Tue, Mar 26, 2019 at 12:44 AM Andrei Borzenkov wrote: > > > > > > He has btrfs raid0 profile on top of hardware RAID6 devices. > > sys_chunk_array[2048]: > item 0 key (FI

Re: backup uuid_tree generation not consistent across multi device (raid0) btrfs - won´t mount

2019-03-26 Thread Chris Murphy
io_align 4096 io_width 4096 sector_size 4096 num_stripes 1 Pretty sure the metadata profiles is "single". From the super, I can't tell what profile the data block groups use. -- Chris Murphy

Re: Fw: kernel oops when mounting btrfs

2019-03-24 Thread Chris Murphy
7;s some problem already, before clearing space cache, that means more bugs: Bug 3, btrfs check doesn't find the problem, reports no errors Bug 4, btrfs kernel code doesn't find the problem during scrub, reports no errors -- Chris Murphy

Re: BTRFS partition over a SSD got totally broken on Ubuntu 18.10 -> errno=-17 Object already exists (Failed to recover log tree)

2019-03-16 Thread Chris Murphy
I don't know any distro doing backports, so it's reasonable to just use the latest upstream version once it's a couple weeks old. For the purpose of zeroing the log, I don't think you need to update btrfs-progs. -- Chris Murphy

psa, wiki needs updating now that Btrfs supports swapfiles in 5.0

2019-03-14 Thread Chris Murphy
Hi, If you install btrfs-progs 4.20+ you'll see the documentation for supporting swapfiles on Btrfs, supported in kernel 5.0+. `man 5 btrfs` Anyone with access to the wiki should update the FAQ https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F -- Chris Murphy

Re: Balancing raid5 after adding another disk does not move/use any data on it

2019-03-14 Thread Chris Murphy
making changes for now and make sure your backups are up to date as a top priority. And then it's safer to poke this with a stick and see what's going on and how to get it to cooperate. -- Chris Murphy

Re: Allow sending of rw-subvols if file system is mounted ro

2019-03-12 Thread Chris Murphy
hich is just a single bit change to the containing node, and also updating the checksum for that node. But, kinda sorta what you want is built into btrfs rescue. You can scrape data out of specific subvolumes. So, while you can't do an incremental send/receive style recovery; you can at least get the data out. -- Chris Murphy

Re: confusing behavior when supers mismatch

2019-03-10 Thread Chris Murphy
device name = /dev/mapper/vg-f30test superblock bytenr = 65536 device name = /dev/mapper/vg-f30test superblock bytenr = 67108864 [All bad supers]: All supers are valid, no need to recover $ -- Chris Murphy

Re: confusing behavior when supers mismatch

2019-03-10 Thread Chris Murphy
On Sun, Mar 10, 2019 at 7:18 PM Qu Wenruo wrote: > > > > On 2019/3/11 上午7:09, Chris Murphy wrote: > > In the case where superblock 0 at 65536 is valid but stale (older than > > the others): > > Then this means either the fs is fuzzed, or the FUA implementation of &g

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-10 Thread Chris Murphy
On Sat, Mar 2, 2019 at 11:18 AM Chris Murphy wrote: > > Sending URL for dump-tree output offlist. Conversation should still be > on-list. Any more information required from me at this point? -- Chris Murphy

Re: confusing behavior when supers mismatch

2019-03-10 Thread Chris Murphy
Described behavior observed with: btrfs-progs 4.20.2 kernel 4.20.12 On Sun, Mar 10, 2019 at 5:09 PM Chris Murphy wrote: > > In the case where superblock 0 at 65536 is valid but stale (older than > the others): > > 1. btrfs check doesn't complain, the stale super is used for

confusing behavior when supers mismatch

2019-03-10 Thread Chris Murphy
s are read at mount time, and if there's discrepancy that either there's code to suspiciously sanity check the latest roots in the newest super, or it flat out fails to mount. Mounting based on stale super data seems risky doesn't it? -- Chris Murphy

Re: btrfs filesystem failing with 'No space left on device' after 4 hours

2019-03-06 Thread Chris Murphy
ing regular snapshots (for a 'previous versions' feature), and many of > the failures were just after a snapshot was created. However, I have now > disabled the snapshot creation and I am still seeing regular failures. Could be one of the edge cases that was fixed in 4.12 but off ha

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
ay, if you need more than just the extent tree, I need a hint on how to sanitize file names. I tried looking through the archives, I thought you posted a sed trick to do that with debug-tree but I can't find it. And I know you don't like btrfs-image -s or -ss. --- Chris Murphy

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
(FREE_SPACE UNTYPED 568038785024) block 933724160 gen 4067 # I guess I don't understand why 'btrfs check' would have successfully written six updated copies of the super, when the operation hadn't yet succeeded. I guess this isn't an atomic operation, and perhaps it can't be. --- Chris Murphy

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
On Fri, Mar 1, 2019 at 6:05 PM Qu Wenruo wrote: > > > > On 2019/3/2 上午8:20, Chris Murphy wrote: > > Problem happens with btrfsprogs 4.19.1 > > > > https://bugzilla.kernel.org/show_bug.cgi?id=202717 > > > > That bug report includes the trace in user space;

[bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
leave it alone since it's a backup volume anyway. -- Chris Murphy

Re: Corrupted filesystem, looking for guidance

2019-02-23 Thread Chris Murphy
important that you stop making changes to the file system in the > > meantime. Just gather information. Be deliberate. > It's a pity that there is yet no solution without involving a human. I'll not > request developer time which could be used to improve the filesystem. :) Well a lot of times they're able to improve the file system but figuring out how to fix edge cases resulting in problems. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
e send/receive stuff go away, and you can ensure your data is replicated pretty much immediately, and is always available for all computers. *shrug* That's off topic but I'm curious if there are ways to simplify this for your use case. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
On Mon, Feb 18, 2019 at 5:16 PM Chris Murphy wrote: > rsync'd subvolumes across volumes aren't consistent identical by btrfs s/consistent/considered -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
around. Btrfs send-receive does. > About the efficiency I'm not planning to remove large amounts of data > that is used by child subvolumes (although some will be updated). But > given the unpredictability of what files will be used by child > subvolumes i might remove large unused amounts of data. OK? -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
rtant; but by using it as the -p reference snapshot, you're asking send to do a comparison that includes a lot of metadata you don't care about. It's more efficient to diff the incremental snapshots of the changing state of "childofmaster" subvolume. -- Chris Murphy

Re: raid 1 filesystem corruption

2019-02-18 Thread Chris Murphy
ully repair all types of filesystem corruption. Eg. some other software or hardware bugs can fatally damage a volume. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
190218T1905 btrfs send child -p childofmaster.20190218-initial childofmaster.20190218T1905 | btrfs receive /destination/ cp --reflink master/yetmorefiles childofmaster/ btrfs sub snap -r childofmaster childofmaster.20190219T1301 btrfs send child childofmaster.20190218T1905 childofmaster.20190218T1905 | btrfs receive /destination/ -- Chris Murphy

Re: raid 1 filesystem corruption

2019-02-18 Thread Chris Murphy
x27;smartctl -x' for both drives 'smartctl -l sct erc' for both drives 'cat /sys/block/sdX/device/timeout' for both drives -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
maybe just adopted the same logic due in large part to more limitations on the ZFS implementation: snapshots are always read only, incremental send is only done between snapshots of the same dataset, datasets can't be deleted until all snapshots are deleted, there are no reflinks, and no doubt some things I'm forgetting. -- Chris Murphy

Re: Corrupted filesystem, looking for guidance

2019-02-18 Thread Chris Murphy
On Mon, Feb 18, 2019 at 1:14 PM Sébastien Luttringer wrote: > > On Tue, 2019-02-12 at 15:40 -0700, Chris Murphy wrote: > > On Mon, Feb 11, 2019 at 8:16 PM Sébastien Luttringer > > wrote: > > > > FYI: This only does full stripe reads, recomputes parity and overw

Re: raid1 mount as read only

2019-02-18 Thread Chris Murphy
veOS. Unfortunately I'm not certain what kernel this behavior changed in, I know for sure kernel 5.0.0 rc's have it, and I'm almost positive 4.20 has it. I'm not sure if 4.18 has it. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
e usage. At the moment, based on Andrei's explanation and example, what you're trying to do is valid. He also explained why you're seeing different results in send stream output in your dd vs wget example: it's the files themselves not the application. "The last extent i

Re: Btrfs send with parent different size depending on source of files.

2019-02-18 Thread Chris Murphy
On Sat, Feb 16, 2019 at 1:26 PM Andrei Borzenkov wrote: > > 15.02.2019 22:11, Chris Murphy пишет: > > The proven way it works, is as I've described, and many emails over a > > decade on this list, inferred from the man page, and the step by step > > recipe on t

Re: Btrfs corruption: Cannot mount partition

2019-02-15 Thread Chris Murphy
ou're at this point, it's close to check --repair time and that can sometimes make things worse. -- Chris Murphy

Re: Btrfs corruption: Cannot mount partition

2019-02-15 Thread Chris Murphy
es while you have the chance in case repair attempts don't work or things get worse. Obviously the less you change the system (zero log, and check --repair both write to the file system and change it) the better off you are until a developer can give advice. But it's also Friday and it might be a few days before one gets to reply. So, if you can wait, do that. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-15 Thread Chris Murphy
On Fri, Feb 15, 2019 at 10:45 AM Andrei Borzenkov wrote: > > 15.02.2019 1:37, Chris Murphy пишет: > > On Thu, Feb 14, 2019 at 4:37 AM André Malm wrote: > >> > >> Hello, > >> > >> I'm not sure this is the right forum to ask on but I'll try

Re: Btrfs send with parent different size depending on source of files.

2019-02-15 Thread Chris Murphy
om/v1/objects/20c069d373e77265aaeeedb733f7051e294325a3/server.jar > > The resulting out file is 34MB. Well I'd say maybe use -vvv and --no-data instead of -f and see what it's doing. It sounds like the former has no payload, just difference information, and the latter has a payload. I don't know why. -- Chris Murphy

Re: Btrfs send with parent different size depending on source of files.

2019-02-14 Thread Chris Murphy
el.org/index.php/Incremental_Backup#Doing_it_by_hand.2C_step_by_step Depending on your use case if you can describe it, might be tolerated with some adjustments by using the -c clone option instead. -- Chris Murphy

Re: circular lock warning at: btrfs_record_root_in_trans, kernel 5.0.0-0.rc4+

2019-02-14 Thread Chris Murphy
On Thu, Feb 14, 2019 at 1:06 AM Chris Murphy wrote: > > I'm still running into this with 5.0.0-rc6 and sssd. I'm not sure what > the issue is but it often does prevent sssd from starting up, so > that's not good. > > https://bugzilla.redhat.com/show_bug.cgi

Re: circular lock warning at: btrfs_record_root_in_trans, kernel 5.0.0-0.rc4+

2019-02-14 Thread Chris Murphy
I'm still running into this with 5.0.0-rc6 and sssd. I'm not sure what the issue is but it often does prevent sssd from starting up, so that's not good. https://bugzilla.redhat.com/show_bug.cgi?id=1677163 [ 53.313230] localhost.localdomain polkitd[721]: Started polkitd version 0.115 [ 53.46095

Re: Corrupted filesystem, looking for guidance

2019-02-12 Thread Chris Murphy
ectly suggests hardware issues. The linux-raid list is usually quite helpful tracking down such problems, including which devices are suspect, but they're going to ask the same questions about SCT ERC and SCSI command timer values I mentioned earlier, and will want to figure out why you're continuing to see mismatches even after a repair scrub - not normal. --- Chris Murphy

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-12 Thread Chris Murphy
On Tue, Feb 12, 2019 at 3:11 PM Zygo Blaxell wrote: > > On Tue, Feb 12, 2019 at 02:48:38PM -0700, Chris Murphy wrote: > > Is it possibly related to the zlib library being used on > > Debian/Ubuntu? That you've got even one reproducer with the exact same > > hash for t

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-12 Thread Chris Murphy
e test but change to zstd or lzo? No error? Strictly zlib? -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-11 Thread Chris Murphy
ore systemd won't issue a mount command for sysroot. -- Chris Murphy

Re: Cannot 'mount -o degraded /dev/replacement' after a replace

2019-02-10 Thread Chris Murphy
ssible if there were no data writes while degraded; if there were degraded writes, a scrub is also necessary to make sure any single mirror raid1 chunk writes to the degraded device are replicated to the replacement. Yep - kinda ugly. And not intuitive. But that's the present status. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-10 Thread Chris Murphy
o research this > a bit - thanks for the info! It doesn't, not directly. It's from the previously mentioned udev rule. For md, the assembly, delays, and fall back to running degraded, are handled in dracut. But the reason why this is in udev is to prevent a mount failure just because one or more devices are delayed; basically it inserts a pause until the devices appear, and then systemd issues the mount command. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-08 Thread Chris Murphy
nt, it's a fatal startup error. > I would also like for BTRFS to be over-aggressively safe, but I also > want it to be over-aggressively always running or even limping if that > is what it needs to do. While I understand that's a metaphor, someone limping along is not a stable situation. They are more likely to trip and fall. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-08 Thread Chris Murphy
sible for the code to understand your expectations and act accordingly. At least you're discovering the limitations before you end up in trouble. The job of a sysadmin is to find out the difference between expectations and actual feature set, because maybe the technology being evaluated isn't a good match for the use case. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-08 Thread Chris Murphy
, it's just not as significant > > because they re-add and re-sync missing devices automatically when they > > reappear, which makes such split-brain scenarios much less likely. > why does btrfs don't do that? It's a fair question but the simplest answer is, features don't grow on trees, they're written by developers and no one has yet done that work. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-07 Thread Chris Murphy
On Thu, Feb 7, 2019 at 10:37 AM Martin Steigerwald wrote: > > Chris Murphy - 07.02.19, 18:15: > > > So please change the normal behavior > > > > In the case of no device loss, but device delay, with 'degraded' set > > in fstab you risk a non-de

Re: btrfs as / filesystem in RAID1

2019-02-07 Thread Chris Murphy
It's just begging for data loss. That's why it's not the default. That's why it's not recommended. -- Chris Murphy

Re: btrfs as / filesystem in RAID1

2019-02-04 Thread Chris Murphy
On Mon, Feb 4, 2019 at 11:46 PM Chris Murphy wrote: > > After remounting both devices and scrubbing, it's dog slow. 14 minutes > to scrub a 4GiB file system, complaining the whole time about > checksums on the files not replicated. All it appears to be doing is > replicating

Re: btrfs as / filesystem in RAID1

2019-02-04 Thread Chris Murphy
On Mon, Feb 4, 2019 at 3:19 PM Patrik Lundquist wrote: > > On Mon, 4 Feb 2019 at 18:55, Austin S. Hemmelgarn > wrote: > > > > On 2019-02-04 12:47, Patrik Lundquist wrote: > > > On Sun, 3 Feb 2019 at 01:24, Chris Murphy wrote: > > >> > > >&

Re: RAID1 filesystem not mounting

2019-02-03 Thread Chris Murphy
> I have been able to successfully recover files via "btrfs restore ...", and > there doesn't seem to be anything essential missing from its full output with > -D, so if that's necessary to use to offload the entire filesystem, it at > least seems possible if it can't be recovered directly. If the data is important, and if this is the only copy, I always argue in favor of urgently making a backup. Set aside all other troubleshooting. -- Chris Murphy

Re: RAID1 filesystem not mounting

2019-02-02 Thread Chris Murphy
uld recommend that you > check your RAM first before trying anything else that would write to > your filesystem (including btrfs check --repair). Good catch! I think that can account for the corrupt and generation errors. I don't know that memory errors can account for the large number of read and write errors, however. So there may be more than one problem. -- Chris Murphy

Re: RAID1 filesystem not mounting

2019-02-02 Thread Chris Murphy
le system with repair attempts, or trying to mount it read write, the better the chance of recovery. Right now there isn't enough information to tell you what to do other than do as little as possible. -- Chris Murphy

<    1   2   3   4   5   6   7   8   9   10   >