Re: Help recover from btrfs error

2021-04-19 Thread Chris Murphy
root' and plug in the address for roots (try most recent first, and then go older) into the 'btrfs restore -t' option. Basically you're pointing it to an older root that hopefully doesn't have damage. The older back you go though, the more stale the trees are and they could have been overwritten. So you pretty much have to try roots in order from most recent, one by one. Might be easier to ask on irc.freenode.net, #btrfs. -- Chris Murphy

5.12-rc7 occasional btrfs splat when rebooting

2021-04-18 Thread Chris Murphy
f49ce1c5688 R14: 0001 R15: [16339.626109] kernel: wlp108s0: deauthenticating from f8:a0:97:6e:c7:e8 by local choice (Reason: 3=DEAUTH_LEAVING) [16340.238863] kernel: kauditd_printk_skb: 93 callbacks suppressed -- Chris Murphy

Re: Design strangeness of incremental btrfs send/recieve

2021-04-16 Thread Chris Murphy
snapshot myvolume-1 > ERROR: cannot find parent subvolume What about using -c instead of -p? -- Chris Murphy

Re: Dead fs on 2 Fedora systems: block=57084067840 write time tree block corruption detected

2021-04-15 Thread Chris Murphy
ction to corruption on Btrfs is to update backups while you still can, while it's still mounted or can be mounted. Then try repair once the underlying problem has been rectified. -- Chris Murphy

Re: Dead fs on 2 Fedora systems: block=57084067840 write time tree block corruption detected

2021-04-15 Thread Chris Murphy
rward problem like a bit flip, that it's risky to attempt a repair which can end up causing worse corruption. What about the mount options for both file systems? (cat /proc/mounts or /etc/fstab) -- Chris Murphy

Re: Parent transid verify failed (and more): BTRFS for data storage in Xen VM setup

2021-04-10 Thread Chris Murphy
rtitions. Right. The block device (partition containing the Btrfs file system) must be exclusively used by one kernel, host or guest. Dom0 or DomU. Can't be both. The only exception I'm aware of is virtiofs or virtio-9p, but I haven't messed with that stuff yet. -- Chris Murphy

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
7; bolt, using the overlay driver. However, I do still get Bail out! ERROR:../tests/test-common.c:1413:test_io_dir_is_empty: 'empty' should be FALSE -- Chris Murphy

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
On Sat, Apr 10, 2021 at 1:43 PM Chris Murphy wrote: > > On Sat, Apr 10, 2021 at 1:42 PM Chris Murphy wrote: > > > > On Sat, Apr 10, 2021 at 1:36 PM Chris Murphy > > wrote: > > > > > > $ sudo mount -o remount,userxattr /home > > >

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
On Sat, Apr 10, 2021 at 1:42 PM Chris Murphy wrote: > > On Sat, Apr 10, 2021 at 1:36 PM Chris Murphy wrote: > > > > $ sudo mount -o remount,userxattr /home > > mount: /home: mount point not mounted or bad option. > > > > [ 92.573364] BTRFS error (dev

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
On Sat, Apr 10, 2021 at 1:36 PM Chris Murphy wrote: > > $ sudo mount -o remount,userxattr /home > mount: /home: mount point not mounted or bad option. > > [ 92.573364] BTRFS error (device sda6): unrecognized mount option > 'userxattr' > [ 63.320831] BTRFS er

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
On Sat, Apr 10, 2021 at 11:55 AM Amir Goldstein wrote: > > On Sat, Apr 10, 2021 at 8:36 PM Chris Murphy wrote: > > > > I can reproduce the bolt testcase problem in a podman container, with > > overlay driver, using ext4, xfs, and btrfs. So I think I can drop > >

Re: btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-10 Thread Chris Murphy
t; My guess is it has to do with changes related to mounting overlayfs > inside userns, but I couldn't find any immediate suspects. > > Do you have any idea since when the regression appeared? > A bisect would have been helpful here. Yep. All good ideas. Thanks for the fast reply. I'll report back once this has been narrowed down futher. -- Chris Murphy

btrfs+overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

2021-04-09 Thread Chris Murphy
btrfs interaction. Could this be related and just not yet merged? https://lore.kernel.org/linux-unionfs/20210309162654.243184-1-amir7...@gmail.com/ Thanks, -- Chris Murphy

5.12-rc6 splat, MAX_LOCKDEP_CHAIN_HLOCKS too low, Workqueue: btrfs-delalloc btrfs_work_helper

2021-04-09 Thread Chris Murphy
Got this while building bolt in a podman container. I've got reproduce steps and test files here https://bugzilla.redhat.com/show_bug.cgi?id=1948054 [ 3229.119497] overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off. [ 3229.155339] overlayfs: upper fs does not s

Re: Any ideas what this warnings are about?

2021-03-31 Thread Chris Murphy
and a weekly fstrim is generally sufficient for the fast majority of workloads. Conversely a heavy write and delete workload translates into a backlog of trim that gets issued all at once, once a week, and can make an SSD bog down after it's issued. So you just have to test it with your particular workload to know. Discard=async exists because a weekly fstrim, and discard=sync can supply way too much hinting all at once to the drive about what blocks are no longer needed and are ready for garbage collection. But again, it's workload specific, and even hardware specific. Some hardware is sufficiently overprovisioned that there's no benefit to issuing discards at all, and normal usage gives the drive firmware all it needs to know about what blocks are ready for garbage collection (and erasing blocks to prepare them for future writes). -- Chris Murphy

Re: Re[4]: Filesystem sometimes Hangs

2021-03-31 Thread Chris Murphy
m, but also makes me wonder why something is enabling sync discards on a HDD? Anway I think you're on the right track to try 5.11.11 and if you experience a hang again, use sysrq+w and that will dump the blocked task trace into dmesg. Also include a description of the workload at the time of the hang, and recent commands issued. -- Chris Murphy

Re: Re[2]: Filesystem sometimes Hangs

2021-03-30 Thread Chris Murphy
a while to see if that's the problem and then deliberately re-introduce just that one single change so you can monitor it for problems. -- Chris Murphy

Re: Support demand on Btrfs crashed fs.

2021-03-30 Thread Chris Murphy
8 wanted 524940 found 524941 parent transid verify failed on 23079040831488 wanted 524940 found 524941 Ignoring transid failure parent transid verify failed on 23079040319488 wanted 524931 found 524939 Ignoring transid failure This is a dry-run, no files are going to be restored Reached the end of the tree searching the directory -- Chris Murphy

Re: Re: Help needed with filesystem errors: parent transid verify failed

2021-03-30 Thread Chris Murphy
On Tue, Mar 30, 2021 at 2:44 AM B A wrote: > > > > Gesendet: Dienstag, 30. März 2021 um 00:07 Uhr > > Von: "Chris Murphy" > > An: "B A" > > Cc: "Btrfs BTRFS" > > Betreff: Re: Help needed with filesystem errors: parent tran

Re: Help needed with filesystem errors: parent transid verify failed

2021-03-29 Thread Chris Murphy
On Sun, Mar 28, 2021 at 9:41 AM B A wrote: > > * Samsung 840 series SSD (SMART data looks fine) EVO or PRO? And what does its /proc/mounts line look like? Total_LBAs_Written? -- Chris Murphy

Re: help needed with raid 6 filesystem with errors

2021-03-29 Thread Chris Murphy
the filesystem has no errors (see attached > btrfs_sdd_check.txt). I'd take advantage of it's cooperative moment by making sure backups are fresh in case things get worse. > Not sure what to do next, so seeking your advice! The important data on > the drive is backed up, and I'll be running a verify to see if there > are any corruptions overnight. Would still like to try to save the > filesystem if possible though. -- Chris Murphy

Re: Filesystem sometimes Hangs

2021-03-29 Thread Chris Murphy
[1298340.857643] > btrfs_create+0x58/0x1f0 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336160] task:btrfs-transacti > state:D stack:0 pid:20080 ppid: 2 flags:0x4000 > Mar 28 20:58:34 homeserver kernel: [1300153.336215] > btrfs_commit_transaction+0x92b/0xa50 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336246] > transaction_kthread+0x15d/0x180 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336273] ? > btrfs_cleanup_transaction+0x590/0x590 [btrfs] > > > What could I do to find the cause? What kernel version? -- Chris Murphy

Re: Re: Help needed with filesystem errors: parent transid verify failed

2021-03-29 Thread Chris Murphy
Maybe. The generation on these two leaves look recent. But kernels since ~5.3 have a write time tree checker designed to catch metadata errors before they are written. What do you get for: btrfs insp dump-s -f /dev/dm-0 Hopefully Qu or Josef will have an idea. -- Chris Murphy

Re: Help needed with filesystem errors: parent transid verify failed

2021-03-28 Thread Chris Murphy
On Sun, Mar 28, 2021 at 7:02 PM Chris Murphy wrote: > > Can you post the output from both: > > btrfs insp dump-t -b 1144783093760 /dev/dm-0 > btrfs insp dump-t -b 1144881201152 /dev/dm-0 I'm not sure if those dumps will contain filenames, so check them. It's ok to remove

Re: Help needed with filesystem errors: parent transid verify failed

2021-03-28 Thread Chris Murphy
ng and if it's safe to try to repair it. In the meantime I highly recommend refreshing backups of /home in case this can't be repaired. It might be easier to do this with a Live USB boot of Fedora 33, and use 'mount -o ro,subvol=home /dev/dm-0 /mnt/home' to mount your home read-only to get a backup. Live environment will be more cooperative. -- Chris Murphy

Re: 5.12-rc4: rm directory hangs for > 1m on an idle system

2021-03-27 Thread Chris Murphy
Fresh boot, this time no compression, everything else the same. Time to delete both directories takes as long as it takes to copy one of them ~1m17s. This time I took an early and late sysrq t pair, and maybe caught some extra stuff. [ 1190.094618] kernel: Workqueue: events_unbound btrfs_preempt_r

5.12-rc4: rm directory hangs for > 1m on an idle system

2021-03-27 Thread Chris Murphy
w?usp=sharing It isn't nearly as bad deleting one directory at once ~15s. -- Chris Murphy

Re: parent transid verify failed / ERROR: could not setup extent tree

2021-03-23 Thread Chris Murphy
you can just do a 'btrfs check' and see if that comes up clean now. It's basically a kind of rollback and if it worked, there will be no inconsistencies found by btrfs check. -- Chris Murphy

Re: parent transid verify failed / ERROR: could not setup extent tree

2021-03-22 Thread Chris Murphy
On Mon, Mar 22, 2021 at 12:32 AM Dave T wrote: > > On Sun, Mar 21, 2021 at 2:03 PM Chris Murphy wrote: > > > > On Sat, Mar 20, 2021 at 11:54 PM Dave T wrote: > > > > > > # btrfs check -r 2853787942912 /dev/mapper/xyz > > > Opening filesystem to c

Re: parent transid verify failed / ERROR: could not setup extent tree

2021-03-21 Thread Chris Murphy
security about the drive. The simple theory of write order is data->metadata->sync->super->sync. It shouldn't ever be the case that a newer superblock generation is on stable media before the metadata it points to. -- Chris Murphy

Re: parent transid verify failed / ERROR: could not setup extent tree

2021-03-20 Thread Chris Murphy
e to make sure it's always applied). Another way is add a different make/model drive to it, and convert to raid1 profile. And hopefully they won't have overlapping firmware bugs. -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-03-16 Thread Chris Murphy
reading, for example, if the writeback percent is 50%, and the cache device is 128G, at any given time 64G is *only* on the SSD. There's no idle time flushing to the backing device that eventually makes the backing device possibly a self sufficient storage device on its own, it always needs the cache device. -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-03-16 Thread Chris Murphy
igure out a way to get the data out of a week old snapshot. But that's speculation on my part. I really have no idea how long it could take for bcache in writeback mode to flush to the backing device. -- Chris Murphy On Tue, Mar 16, 2021 at 3:35 AM Sebastian Roller wrote: > > Hi again.

Re: BTRFS error (device sda1): bdev /dev/sdb1 errs: wr 2702175, rd 2719033, flush 0, corrupt 6, gen 0

2021-03-13 Thread Chris Murphy
oesn't end on a 4KiB block boundary or why there's a gap before the start of sda2...but at least it's benign. -- Chris Murphy

Re: BTRFS error (device sda1): bdev /dev/sdb1 errs: wr 2702175, rd 2719033, flush 0, corrupt 6, gen 0

2021-03-12 Thread Chris Murphy
udo smartctl -x /dev/sdb That might require installing the smartmontools package. -- Chris Murphy -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-03-09 Thread Chris Murphy
fs.wiki.kernel.org/index.php/Changelog > > btrfs insp dump-t -t 1 /dev/sdi1 > > > > And you'll need to look for a snapshot name in there, find its bytenr, > > and let's first see if just using that works. If it doesn't then maybe > > combining it with the next most recent root tree will work. > > I am working backwards right now using btrfs restore -f in combination > with -t. So far no success. Yep. I think it comes down to the chunk tree needing to be reasonable first, before anything else is possible. -- Chris Murphy

Re: btrfs fails to mount on kernel 5.11.4 but works on 5.10.19

2021-03-07 Thread Chris Murphy
ollowing previous degraded mount - the scrub is needed to get the missing device caught up with the rest. -- Chris Murphy

Re: btrfs fails to mount on kernel 5.11.4 but works on 5.10.19

2021-03-07 Thread Chris Murphy
checking whether AMD-GPU > hangs are fixed). Now I read that -rc1 is a btrfs-killer. I have swap > partition, not swap file, and 64G or RAM, so normally swap is not used, > though. That bug should not have affected the dedication swap partition case. -- Chris Murphy

Re: btrfs fails to mount on kernel 5.11.4 but works on 5.10.19

2021-03-07 Thread Chris Murphy
8.03GiB path /dev/nvme0n1p1 This seems to be a somewhat risky setup or at least highly performance variable. Any single device that fails will result in boot failure. -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-03-07 Thread Chris Murphy
isn't a reason to immediately start overwriting it. I'm thinking maybe the new one got interrupted by the failure and the old one is intact. Ok so the next step is to find a snapshot you want to restore. btrfs insp dump-t -t 1 /dev/sdi1 And you'll need to look for a snapshot name in there, find its bytenr, and let's first see if just using that works. If it doesn't then maybe combining it with the next most recent root tree will work. -- Chris Murphy

convert and scrub: spanning stripes, attempt to access beyond end of device

2021-03-05 Thread Chris Murphy
e2image -Q from the loop mounted rollback image file for inspection? Thanks -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-03-04 Thread Chris Murphy
equally bad shape, having been on the same controller? Are they both btrfs? What do you get for btrfs insp dump-s -f /dev/sdXY There might be a backup tree root in there that can be used with btrfs restore -t Also, sometimes easier to do this on IRC on freenode.net in the channel #btrfs -- Chris Murphy

Re: [report] lockdep warning when mounting seed device

2021-02-26 Thread Chris Murphy
1 $dev1 No mount or copying data to the file system after mkfs and before setting the seed flag? I wonder if that's related to the splat, even though it shouldn't happen. -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-02-26 Thread Chris Murphy
works. Kernel 5.11 has a new feature, mount -o ro,rescue=all that is more tolerant of mounting when there are various kinds of problems. But there's another thread where a failed controller is thwarting recovery, and that code is being looked at for further enhancement. https://lore.kernel.org/linux-btrfs/CAEg-Je-DJW3saYKA2OBLwgyLU6j0JOF7NzXzECi0HJ5hft_5=a...@mail.gmail.com/ -- Chris Murphy

Re: All files are damaged after btrfs restore

2021-02-24 Thread Chris Murphy
On Wed, Feb 24, 2021 at 10:40 PM Chris Murphy wrote: > > I think you best chance is to start out trying to restore from a > recent snapshot. As long as the failed controller wasn't writing > totally spurious data in random locations, that snapshot should be > intact. i.e. t

Re: All files are damaged after btrfs restore

2021-02-24 Thread Chris Murphy
nel: BTRFS error (device sdf1): open_ctree failed I think you best chance is to start out trying to restore from a recent snapshot. As long as the failed controller wasn't writing totally spurious data in random locations, that snapshot should be intact. If there are no recent snapshots, and it's unknown what the controller was doing while it was failing or how long it was failing for? Recovery can be difficult. Try using btrfs-find-root to find older roots, and use that value with btrfs restore -t option. These are not as tidy as snapshots though, the older they are, the more they dead end into more recent overwrites. So you want to start out with the most recent roots you can and work backwards in time. -- Chris Murphy

Re: 5.11 free space tree remount warning

2021-02-21 Thread Chris Murphy
rnal is also not logging this for the same reason. I see it on the console on reboots when there's enough of a delay to notice it, and "warning" pretty much always catches my eye. -- Chris Murphy

5.11 free space tree remount warning

2021-02-19 Thread Chris Murphy
v1 present, it's not a change. -- Chris Murphy

Re: ERROR: failed to read block groups: Input/output error

2021-02-18 Thread Chris Murphy
Also I think ignoring csum for btrfs inspect would be useful. > > $ btrfs inspect dump-tree -b 21057050689536 /dev/sda > btrfs-progs v5.10.1 > node 21057050689536 level 1 items 281 free space 212 generation > 2262739 owner EXTENT_TREE > node 21057050689536 flags 0x1(WRITTEN) backref revision 1 > fs uuid 8aef11a9-beb6-49ea-9b2d-7876611a39e5 > chunk uuid 4ffec48c-28ed-419d-ba87-229c0adb2ab9 > [...] > key (19264654909440 EXTENT_ITEM 524288) block 21057101103104 gen 2262739 > [...] > > > $ btrfs inspect dump-tree -b 21057101103104 /dev/sda > btrfs-progs v5.10.1 > checksum verify failed on 21057101103104 found 00B9 wanted 0075 > checksum verify failed on 21057101103104 found 009C wanted 0075 > checksum verify failed on 21057101103104 found 00B9 wanted 0075 > Csum didn't match > ERROR: failed to read tree block 21057101103104 > > > Thanks! What do you get for btrfs rescue super -v /dev/ btrfs check -b /dev/ You might try kernel 5.11 which has a new mount option that will skip bad roots and csums. It's 'mount -o ro,rescue=all' and while it won't let you fix it, in the off chance it mounts, it'll let you get data out before trying to repair the file system, which sometimes makes things worse. -- Chris Murphy

Re: corrupt leaf, unexpected item end, unmountable

2021-02-18 Thread Chris Murphy
On Thu, Feb 18, 2021 at 6:12 PM Daniel Dawson wrote: > > On 2/18/21 3:57 PM, Chris Murphy wrote: > > metadata raid6 as well? > > Yes. Once everything else is figured out, you should consider converting metadata to raid1c3. https://lore.kernel.org/linux-btrfs/2

Re: corrupt leaf, unexpected item end, unmountable

2021-02-18 Thread Chris Murphy
; devid1 size 457.64GiB used 39.56GiB path /dev/sda3 > devid2 size 457.64GiB used 39.56GiB path /dev/sdb3 > devid4 size 457.64GiB used 39.53GiB path /dev/sdd3 This is confusing. devid 3 is claimed to be missing, but fi show isn't showing any missing devices. If none of sd[abcd] are devid 3, then what dev node is devid 3 and where is it? But yeah you're probably best off not trying to fix this file system until the memory is sorted out. -- Chris Murphy

Re: Recovering Btrfs from a freak failure of the disk controller

2021-02-14 Thread Chris Murphy
On Sun, Feb 14, 2021 at 4:24 PM Neal Gompa wrote: > > On Sun, Feb 14, 2021 at 5:11 PM Chris Murphy wrote: > > > > On Sun, Feb 14, 2021 at 1:29 PM Neal Gompa wrote: > > > > > > Hey all, > > > > > > So one of my main computers recently had a d

Re: Recovering Btrfs from a freak failure of the disk controller

2021-02-14 Thread Chris Murphy
d to do -o recovery,ro mount and get the same issue. I can't > seem to find any reasonably good information on how to do recovery in > this scenario, even to just recover enough to copy data off. > > I'm on Fedora 33, the system was on Linux kernel version 5.9.16 and > the

Re: Recovering Btrfs from a freak failure of the disk controller

2021-02-14 Thread Chris Murphy
Can you also include: btrfs insp dump-s I wonder if log replay is indicated by non-zero value for log_root in the super block. If so, you check if: ro,nologreplay or ro,nologreplay,usebackuproot work. -- Chris Murphy

Re: Recovering Btrfs from a freak failure of the disk controller

2021-02-14 Thread Chris Murphy
n and see if it can skip over this kind of problem. The "parent transid verify failed" are pretty serious, again not the same thing here. But I'm not sure how resilient repair is for either off by one errors, or bitflips still. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-11 Thread Chris Murphy
case, you only get relief once it's rotated. It'd be nice to find an decent, not necessarily perfect, way for them to not get so fragmented in the first place. Or just defrag once a file has 16M of non-contiguous extents. Estimating extents though is another issue, especially with compression enabled. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-10 Thread Chris Murphy
> > compression, and will have iops equivalent to btrfs fi defrag. Or switch to a cow-friendly format that's no worse on overwriting file systems, but improves things on Btrfs and ZFS. RocksDB does well. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-10 Thread Chris Murphy
tently the file made from multiples of 8MB contiguous blocks, even on HDD the seek latency here can't be worth defraging the file. I think defrag makes sense (a) datacow journals, i.e. the default nodatacow is inhibited (b) HDD. In that case the fragmentation is quite considerable, hundreds to thousands of extents. It's sufficiently bad that it'd be probably be better if they were defragmented automatically with a trigger that tests for number of non-contiguous small blocks that somehow cheaply estimates latency reading all of them. Since the files are interleaved, doing something like "systemctl status dbus" might actually read many blocks even if the result isn't a whole heck of alot of visible data. But on SSD, cow or nocow, and HDD nocow - I think just leave them alone. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-10 Thread Chris Murphy
knobs just influence the outcome. Not sure. If the device is HDD, it might be nice if the nodatacow journals are datacow again so they could be compressed. But my evaluation shows that nodatacow journals stick to an 8MB extent pattern, correlating to fallocated append as they grow. It's not significantly fragmented to start out with, whether HDD or SSD. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
t think we can say it's ineffective, it might be an intentional no op either because it's nodatacow or it sees that these many extents are mostly contiguous and not worth defragmenting (which would be good for keeping write amplification down). So I don't know, maybe it's not wrong. -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
On Tue, Feb 9, 2021 at 11:13 AM Goffredo Baroncelli wrote: > > On 2/9/21 1:42 AM, Chris Murphy wrote: > > Perhaps. Attach strace to journald before --rotate, and then --rotate > > > > https://pastebin.com/UGihfCG9 > > I looked to this strace. > > in line 115:

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
On Tue, Feb 9, 2021 at 12:45 PM Goffredo Baroncelli wrote: > > On 2/9/21 8:01 PM, Chris Murphy wrote: > > On Tue, Feb 9, 2021 at 11:13 AM Goffredo Baroncelli > > wrote: > >> > >> On 2/9/21 1:42 AM, Chris Murphy wrote: > >>> Perhaps. Attach strac

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-08 Thread Chris Murphy
ll *shrug*. Maybe they should use BTRFS_IOC_DEFRAG_RANGE and specify an 8MB exent? Because in the nodatacow case, that's what they already have and it'd be a no op. And then for datacow case... well I don't like unconditional write amplification on SSDs just to satisfy the HDD case. But it'd be avoidable by just using default (nodatacow for the journals). -- Chris Murphy

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-08 Thread Chris Murphy
On Mon, Feb 8, 2021 at 3:11 PM Goffredo Baroncelli wrote: > > On 2/7/21 11:06 PM, Chris Murphy wrote: > > systemd-journald journals on Btrfs default to nodatacow, upon log > > rotation it's submitted for defragmenting with BTRFS_IOC_DEFRAG. The > > result looks c

is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-07 Thread Chris Murphy
er the reads to arrive at the least amount of seeks? -- Chris Murphy

Re: btrfs becomes read only on removal of folders

2021-02-04 Thread Chris Murphy
ckup, reformat, restore. Hence the backup needs to be the top priority; and checking the memory the second priority. -- Chris Murphy

Re: Need help for my Unraid cache drive

2021-02-03 Thread Chris Murphy
at sdb1 device part of the unraid? Is there a device missing? The 'allowing degraded mounts' message along with 'open_ctree failed' suggests that there's still too many devices missing. I suggest a relatively recent btrfs-progs, 5.7 or higher, and provide the output from: btrfs insp dump-s /dev/sdb1 -- Chris Murphy

Re: is back and forth incremental send/receive supported/stable?

2021-02-01 Thread Chris Murphy
don't know if it'll set received uuid, such that subsequent send can use -p instead of -c. -c generally still confuses me... in particular multiple instances of -c -- Chris Murphy

Re: ENOSPC in btrfs_run_delayed_refs with 5.10.8 + zstd

2021-01-27 Thread Chris Murphy
ce4f878af44c3987bce5331.00MiB Can you mount or remount with enospc_debug, and reproduce the problem? That'll include some debug info that might be helpful to a developing coming across this report. Also it might help: cd /sys/fs/btrfs/$UUID/allocation grep -R . And post that too. The $UUID is the file system UUID for this specific file system, as reported by blkid or lsblk -f. -- Chris Murphy

Re: Only one subvolume can be mounted after replace/balance

2021-01-27 Thread Chris Murphy
On Wed, Jan 27, 2021 at 6:10 AM Jakob Schöttl wrote: > > Thank you Chris, it's resolved now, see below. > > Am 25.01.21 um 23:47 schrieb Chris Murphy: > > On Sat, Jan 23, 2021 at 7:50 AM Jakob Schöttl wrote: > >> Hi, > >> > >> In short: > &

Re: btrfs becomes read-only

2021-01-27 Thread Chris Murphy
story of the file system? What do you get for: btrfs insp dump-s -f /dev/sdg -- Chris Murphy

Re: btrfs becomes read-only

2021-01-27 Thread Chris Murphy
through and get information for each drive, so that you don't have to put it in JBOD mode. But I'm not familiar with how to do that. Anyway it's a good idea to find out if there's SMART reporting any problems about any drive, but not urgent. -- Chris Murphy

Re: btrfs becomes read-only

2021-01-26 Thread Chris Murphy
--readonly /dev/sdg After that I suggest https://btrfs.wiki.kernel.org/index.php/Restore And try to get any important data out if it's not backed up. You can try btrfs-find-root to get a listing of roots, most recent to oldest. Start at the top, and plug that address in as 'btrfs restore -

Re: Only one subvolume can be mounted after replace/balance

2021-01-25 Thread Chris Murphy
o refresh the backups. That's the top priority, not fixing the file system. Next let us know the following: kernel version btrfs-progs version Output from commands: btrfs fi us /mnt btrfs check --readonly -- Chris Murphy

Re: Recover data from damage disk in "array"

2021-01-22 Thread Chris Murphy
at could easily result in loss or corruption of both copies of metadata and the whole file system can implode. -- Chris Murphy

Re: nodatacow mount option is disregarded when mounting subvolume into same filesystem

2021-01-17 Thread Chris Murphy
no compress. If you want per subvolume options then you need to use 'chattr +C' per subvolume or directory for nodatacow. And for compression you can use +c (small c) which implies zlib, or use 'btrfs property set /path/to/sub-dir-file compression zstd' -- Chris Murphy

Re: received uuid not set btrfs send/receive

2021-01-17 Thread Chris Murphy
don't understand why the "received uuid" on the > remote site in blank. > I tried it locally with smaller volumes and it worked. I suggest using -v or -vv on the receive side to dig into why the receive is failing. Setting the received uuid is one of the last things performed on receive, so if it's not set it suggests the receive isn't finished. -- Chris Murphy

btrfs: shrink delalloc pages instead of full inodes, for 5.10.8?

2021-01-14 Thread Chris Murphy
Hi, It looks like this didn't make it to 5.10.7. I see the PR for 5.11-rc4. Is it likely it'll make it into 5.10.8? e076ab2a2ca70a0270232067cd49f76cd92efe64 btrfs: shrink delalloc pages instead of full inodes Thanks, -- Chris Murphy

Re: Reading files with bad data checksum

2021-01-10 Thread Chris Murphy
(chattr +C) or don't use O_DIRECT until there's a proper fix. > What's the best way to recover the data? I'd say, kernel 5.11's new "mount -o ro,rescue=ignoredatacsums" feature. You can copy it out normally, no special tools. The alternative is 'btrfs restore'. -- Chris Murphy

Re: btrfs receive eats CoW attributes

2021-01-04 Thread Chris Murphy
e any" because these features aren't mkfs or mount time defaults, so I'd make preservation explicitly opt in like they were on the original file system. -- Chris Murphy

Re: tldr; no BTRFS on dev, after a forced shutdown, help

2021-01-04 Thread Chris Murphy
.generation 0 > > This as nothing to do with btrfs anymore, but: do you think a tool like > foremost can recover the files, it'll be a mess, but better then nothing > and I've used it before in a ntfs. No idea. You could scan the entire drive for the Btrfs magic, which is inside the superblock. It will self identify its offset, the first superblock is the one you want, which is offset 65536 (64KiB) from the start of the block device/partition. And that superblock also says the device size. -- Chris Murphy

Re: tldr; no BTRFS on dev, after a forced shutdown, help

2021-01-03 Thread Chris Murphy
of what you did. A forced shutdown does not make Btrfs unreadable, although if writes are happening at the time of the shutdown and the drive firmware doesn't properly honor write order, then it might be 'btrfs restore' territory. What do you get for: btrfs filesystem show kernel messages (dmesg) that appear when you try to mount the volume but it fails. -- Chris Murphy

Re: [BUG] 500-2000% performance regression w/ 5.10

2021-01-03 Thread Chris Murphy
The problem is worse on SSD than on HDD. It actually makes the SSD *slower* than an HDD, on 5.10. For this workload HDD 5.9.16-200.fc33.x86_64 mq-deadline kyber [bfq] none $ time tar -xf /tmp/firefox-85.0b4.source.tar.xz && time sync real1m27.299s user0m27.294s sys0m14.134s real

Re: cp --reflink of inline extent results in two DATA_EXTENT entries

2020-12-23 Thread Chris Murphy
On Tue, Dec 22, 2020 at 11:05 PM Andrei Borzenkov wrote: > > 23.12.2020 06:48, Chris Murphy пишет: > > Hi, > > > > kernel is 5.10.2 > > > > cp --reflink hi hi2 > > > > This results in two EXTENT_DATA items with different offsets, > > therefo

cp --reflink of inline extent results in two DATA_EXTENT entries

2020-12-22 Thread Chris Murphy
unconfined_u:object_r:unlabeled_t:s0 item 13 key (258 EXTENT_DATA 0) itemoff 15364 itemsize 53 generation 435179 type 0 (inline) inline extent data size 32 ram_bytes 174 compression 3 (zstd) total bytes 31005392896 bytes used 20153282560 -- Chris Murphy

memory bit flip not detected by write time tree check

2020-12-21 Thread Chris Murphy
, resulting in mount failure. I'm double checking with the user, but I'm pretty sure it had only seen writes with relatively recent (5.8+) kernels. -- Chris Murphy

what determines what /dev/ is mounted?

2020-12-18 Thread Chris Murphy
/proc/mounts. But this isn't scientific. Is there a predictable logic? Is it always the lowest devid? -- Chris Murphy

Re: feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Chris Murphy
On Tue, Oct 22, 2019 at 1:33 PM Roman Mamedov wrote: > > On Tue, 22 Oct 2019 11:00:07 +0200 > Chris Murphy wrote: > > > Hi, > > > > So XFS has these > > > > [49621.415203] XFS (loop0): Mounting V5 Filesystem > > [49621.58] XFS (loop0): Ending c

Re: feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Chris Murphy
nt code. >From the prior example, I can't tell that at all, there just isn't enough information. -- Chris Murphy

Re: feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Chris Murphy
(resending to list, I don't know why but I messed up the reply directly to Nikolay) On Tue, Oct 22, 2019 at 11:16 AM Nikolay Borisov wrote: > > On 22.10.19 =D0=B3. 12:00 =D1=87., Chris Murphy wrote: > > Hi, > > > > So XFS has these > > > > [49621.41

feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Chris Murphy
#x27;m also not sure what the insertion point for these messages would be. Looking at the mount code in particular, it's a little complicated. And maybe with some of the sanity checking and debug options it could get more complicated, and wouldn't want to conflict with that - or any multiple device use case either. -- Chris Murphy

Re: MD RAID 5/6 vs BTRFS RAID 5/6

2019-10-20 Thread Chris Murphy
is a read only scrub of metadata, as all metadata is needed to be read for that. Of course it's more expensive than just confirm checksums are OK, because it's also doing a bunch of sanity and logical tests that take much longer. -- Chris Murphy

Re: MD RAID 5/6 vs BTRFS RAID 5/6

2019-10-20 Thread Chris Murphy
On Thu, Oct 17, 2019 at 8:23 PM Graham Cobb wrote: > > On 17/10/2019 16:57, Chris Murphy wrote: > > On Wed, Oct 16, 2019 at 10:07 PM Jon Ander MB > > wrote: > >> > >> It would be interesting to know the pros and cons of this setup that > >> you

Re: MD RAID 5/6 vs BTRFS RAID 5/6

2019-10-17 Thread Chris Murphy
lem), and a crash or powerloss, and at the same time one or more device failures. In that case there's no time for a partial scrub to fix the problem leading to the write hole. So even if the corruption is detected, it's too late to fix it. But at least an automatic partial scrub, even degraded, will mean the user would be flagged of the uncorrectable problem before they get too far along. -- Chris Murphy

Re: 5.3.0 deadlock: btrfs_sync_file / btrfs_async_reclaim_metadata_space / btrfs_page_mkwrite

2019-10-14 Thread Chris Murphy
On Mon, Oct 14, 2019 at 7:05 PM James Harvey wrote: > > On Sun, Oct 13, 2019 at 9:46 PM Chris Murphy wrote: > > > > On Sat, Oct 12, 2019 at 5:29 PM James Harvey > > wrote: > > > > > > Was using a temporary BTRFS volume to compile mongodb, which is quit

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-10-14 Thread Chris Murphy
ernel.org kernel? If so, what > version should be safe? ex: > https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.3.6 > > With 180 raw TB in raid1 I just want to be explicit. Thanks! It's fixed upstream stable since 5.2.15, and includes all 5.3.x series. -- Chris Murphy

Re: 5.3.0 deadlock: btrfs_sync_file / btrfs_async_reclaim_metadata_space / btrfs_page_mkwrite

2019-10-13 Thread Chris Murphy
tions, but it'd be interesting if any of the following make a difference: - space_cache=v2 - noatime -- Chris Murphy

Re: BTRFS Raid5 error during Scrub.

2019-10-03 Thread Chris Murphy
and it's actually a benign message. It's normal for nocow files to be missing csums. To confirm you can use 'find /pathtosubvol/ -inum 81781' to find the file, then lsattr it and see if +C is set. You have a few options but the first thing is to refresh backups and prepare to lose this file system: a. bail now, and just create a new Btrfs from scratch and restore from backup b. try 'btrfs check --repair' to see if the transid problems are fixed; if not c. try 'btrfs check --repair --init-extent-tree' there's a good chance this fails and makes things worse but probably faster to try than restoring from backup -- Chris Murphy

Re: BTRFS Raid5 error during Scrub.

2019-10-01 Thread Chris Murphy
s: 18733842432 > total fs tree bytes: 130121728 > total extent tree bytes: 466305024 > btree space waste bytes: 1100711497 > file data blocks allocated: 3891333279744 > referenced 1669470507008 What do you get for # btrfs insp dump-t -b 48781340082176 /dev/ It's possible there will be filenames, it's OK to sanitize them by just deleting the names from the output before posting it. -- Chris Murphy

Re: BTRFS checksum mismatch - false positives

2019-09-26 Thread Chris Murphy
nd 50/50 shot that smart will predict a drive failure in advance. So my suggestion again, without delay, is to make sure the NAS is backed up, and keep those backups fresh. You can recreate the NAS when you have free time - but these problems likely will get worse. --- Chris Murphy

  1   2   3   4   5   6   7   8   9   10   >