WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hello
On my HP Compaq dc5800 with Ubuntu 13.04 and their 3.8.0-19-lowlatency
kernel, I've got quite some kernel traces in the syslog. You can find
them below or at http://pastebin.com/bLXPBX67 (to avoid line breaks…).
These kernel traces all begin with:
WARNING: at fs/btrfs/free-space-cache.c:921
__btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Most of the time, it starts with:
Call Trace:
[] warn_slowpath_common+0x7f/0xc0
But once, I also had:
Call Trace:
[] warn_alloc_failed+0xe9/0x140
It happens most of the time, when I copied something on a btrfs. But
it also happened right now, when the system was basically idle. The
fs is on a "normal" SATA disk (ie. no SSD or usb or such).
I created the btrfs on the linux 3.8.0-19-lowlatency kernel with:
a@ewzw032:~$ btrfs version
Btrfs v0.20-rc1
Line 921 of free-space-cache.c (-> http://pastebin.com/yNQMPG3P) is
the WARN_ON(1) in this block:
/* Make sure we can fit our crcs into the first page */
if (io_ctl.check_crcs &&
(io_ctl.num_pages * sizeof(u32)) >= PAGE_CACHE_SIZE) {
WARN_ON(1);
goto out_nospc;
}
Well - what can be done about it?
Here are my kernel traces:
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016478] [ cut here
]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016507] WARNING: at
fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0
[btrfs]()
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016509] Hardware name: HP
Compaq dc5800 Microtower
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016510] Modules linked in: arc4
md4 parport_pc ppdev bnep rfcomm bluetooth nls_utf8 cifs fscache ext2
snd_hda_codec_analog snd_hda_intel snd_hda_codec coretemp tpm_infineon
snd_hwdep snd_pcm snd_page_alloc dm_multipath snd_seq_midi
snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device
snd_timer scsi_dh snd psmouse soundcore gpio_ich hp_wmi sparse_keymap
serio_raw lpc_ich mei wmi microcode tpm_tis nvidia(POF) mac_hid lp
parport btrfs zlib_deflate libcrc32c hid_generic usbhid hid
usb_storage floppy e1000e
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016537] Pid: 4608, comm:
btrfs-transacti Tainted: PF W O 3.8.0-19-lowlatency #13-Ubuntu
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016538] Call Trace:
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016544] []
warn_slowpath_common+0x7f/0xc0
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016546] []
warn_slowpath_null+0x1a/0x20
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016556] []
__btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016559] [] ?
set_next_entity+0x80/0x90
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016570] []
btrfs_write_out_cache+0x95/0xf0 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016579] []
btrfs_write_dirty_block_groups+0x527/0x620 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016588] []
commit_cowonly_roots+0x15a/0x22c [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016598] []
btrfs_commit_transaction+0x62c/0xb00 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016601] [] ?
finish_wait+0x80/0x80
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016610] []
transaction_kthread+0x1ad/0x230 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016620] [] ?
write_dev_flush.part.107+0xc0/0xc0 [btrfs]
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016622] []
kthread+0xc0/0xd0
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016624] [] ?
kthread_create_on_node+0x120/0x120
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016628] []
ret_from_fork+0x7c/0xb0
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016629] [] ?
kthread_create_on_node+0x120/0x120
Apr 29 12:59:31 ewzw032 kernel: [ 6275.016631] ---[ end trace
3afa1c596d559b57 ]---
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268618] [ cut here
]
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268648] WARNING: at
fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0
[btrfs]()
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268650] Hardware name: HP
Compaq dc5800 Microtower
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268651] Modules linked in: arc4
md4 parport_pc ppdev bnep rfcomm bluetooth nls_utf8 cifs fscache ext2
snd_hda_codec_analog snd_hda_intel snd_hda_codec coretemp tpm_infineon
snd_hwdep snd_pcm snd_page_alloc dm_multipath snd_seq_midi
snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device
snd_timer scsi_dh snd psmouse soundcore gpio_ich hp_wmi sparse_keymap
serio_raw lpc_ich mei wmi microcode tpm_tis nvidia(POF) mac_hid lp
parport btrfs zlib_deflate libcrc32c hid_generic usbhid hid
usb_storage floppy e1000e
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268678] Pid: 4608, comm:
btrfs-transacti Tainted: PF W O 3.8.0-19-lowlatency #13-Ubuntu
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268679] Call Trace:
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268685] []
warn_slowpath_common+0x7f/0xc0
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268687] []
warn_slowpath_null+0x1a/0x20
Apr 29 13:05:18 ewzw032 kernel: [ 6621.268698] []
__btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]
Apr 29 13
Re: WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hugo Mills carfax.org.uk> writes: >The differences in btrfs between the two are very small, and even > I(*) wouldn't call 3.8.0 "very old" quite yet, given that 3.9 was only > released yesterday. From memory, there's one btrfs patch in the 3.8 > stable series. > >Your problem is "just" a warning, and appears to be something to do > with running out of space, or having too many CRCs... I don't really > know the free space cache code at all well, so I'm mostly guessing > here, from looking at the WARN_ON in __btrfs_write_out_cache. Yes, I'm aware that this just a warning, but I'm a bit scared because of the big number of those warnings. FWIW, I also got that exact same kernel trace on the Ubuntu kernel 3.5.0-28-lowlatency; see http://pastebin.com/rmPAqcTu or here: [ cut here ] WARNING: at fs/btrfs/free-space-cache.c:922 __btrfs_write_out_cache+0x6b9/0x990 [btrfs]() Hardware name: HP Compaq dc5800 Microtower Modules linked in: btrfs zlib_deflate libcrc32c des_generic md4 nvidia(PO) snd_hda_codec_analog arc4 tpm_infineon coretemp snd_hda_intel rtl8192cu snd_hda_codec kvm_intel rtl8192c_common rtlwifi kvm snd_hwdep snd_pcm parport_pc snd_seq_midi bnep rfcomm mac80211 ppdev snd_rawmidi bluetooth snd_seq_midi_event snd_seq binfmt_misc snd_timer snd_seq_device cfg80211 dm_multipath psmouse hp_wmi snd scsi_dh gpio_ich sparse_keymap soundcore microcode serio_raw tpm_tis mei mac_hid snd_page_alloc wmi lpc_ich lp parport nls_utf8 cifs fscache ext2 hid_generic usbhid hid usb_storage floppy e1000e Pid: 25834, comm: btrfs Tainted: P O 3.5.0-28-lowlatency #31- Ubuntu Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] warn_slowpath_null+0x1a/0x20 [] __btrfs_write_out_cache+0x6b9/0x990 [btrfs] [] ? __find_space_info+0x85/0xa0 [btrfs] [] ? _raw_spin_unlock+0xe/0x40 [] ? btrfs_run_delayed_refs+0x1ec/0x470 [btrfs] [] btrfs_write_out_cache+0x95/0xf0 [btrfs] [] btrfs_write_dirty_block_groups+0x51f/0x5f0 [btrfs] [] commit_cowonly_roots+0xfd/0x1c7 [btrfs] [] ? btrfs_run_delayed_items+0xd1/0x150 [btrfs] [] btrfs_commit_transaction+0x5d5/0xb00 [btrfs] [] ? finish_wait+0x80/0x80 [] ? __d_instantiate+0xd2/0x110 [] ? security_d_instantiate+0x1b/0x30 [] create_subvol+0x4d1/0x4f8 [btrfs] [] btrfs_mksubvol+0x3a5/0x400 [btrfs] [] btrfs_ioctl_snap_create_transid+0xbe/0x180 [btrfs] [] btrfs_ioctl_snap_create+0x56/0x80 [btrfs] [] btrfs_ioctl+0xc0a/0x1280 [btrfs] [] ? do_page_fault+0x1b4/0x4b0 [] do_vfs_ioctl+0x97/0x530 [] ? vfs_write+0x105/0x180 [] sys_ioctl+0x99/0xa0 [] ? do_device_not_available+0xe/0x10 [] system_call_fastpath+0x16/0x1b ---[ end trace fadd80d2b7f6ec9f ]--- Both traces (from 3.8.0 and 3.5.0) are in fs/btrfs/free-space-cache.c and have warn_slowpath_common as the first function. Strange, isn't it? Since the problem exists for a long time, I pretty much doubt that just updating the kernel source would help. I'd rather not, I'd rather stay with the kernel of my distribution. I should have mentioned in my OP, that the btrfs is on a LVM. Right now, I've got two distinct btrfs filesystems on the system; both on LVM and on seperate logical volumes. I get this kernel trace / warning only on one of these filesystems. The LV existed also "back in kernel 3.5.0 times". I've since ran a btrfs scrub, but it didn't find any errors: a@ewzw032:~$ sudo btrfs scrub status -d /data scrub status for 8009e99c-726d-46fb-b68c-be57fb66ca05 scrub device /dev/mapper/system-Data (id 1) history scrub started at Tue Apr 30 08:55:27 2013 and finished after 2989 seconds total bytes scrubbed: 146.82GB with 0 errors a@ewzw032:~$ sudo btrfs scrub status -R /data scrub status for 8009e99c-726d-46fb-b68c-be57fb66ca05 scrub started at Tue Apr 30 08:55:27 2013 and finished after 2989 seconds data_extents_scrubbed: 2564422 tree_extents_scrubbed: 107996 data_bytes_scrubbed: 157200080896 tree_bytes_scrubbed: 442351616 read_errors: 0 csum_errors: 0 verify_errors: 0 no_csum: 0 csum_discards: 0 super_errors: 0 malloc_errors: 0 uncorrectable_errors: 0 unverified_errors: 0 corrected_errors: 0 last_physical: 251959246848 Best regards, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hello Josef On Tue, Apr 30, 2013 at 3:58 PM, Josef Bacik wrote: > So we deal with this case fine, but it really shouldn't be happening, it only > happens if your block groups are way too large, which again shouldn't be > happening. Can you run fsck on this device and see if it complains? Thanks, a@ewzw032:~$ sudo btrfsck /dev/system/Data checking extents checking fs roots checking root refs found 115308904448 bytes used err is 0 total csum bytes: 112056692 total tree bytes: 521891840 total fs tree bytes: 353947648 btree space waste bytes: 122297974 file data blocks allocated: 114787012608 referenced 114787000320 Btrfs v0.20-rc1 No errors. Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hello Josef On Tue, Apr 30, 2013 at 7:53 PM, Josef Bacik wrote: > Can you run this patch and capture the output when you get the warning? You > should see some mesages before the -- [ cut here ] -- part, make sure to > capture > those. Thanks, Sure. There you go (also on http://pastebin.com/7MHmgpPU): num_pages is 1, blockgroup? yes block group offset=83999326208, size=167959920640 [ cut here ] WARNING: at fs/btrfs/free-space-cache.c:925 __btrfs_write_out_cache+0x9e2/0xa10 [btrfs]() Hardware name: HP Compaq dc5800 Microtower Modules linked in: arc4 md4 parport_pc ppdev rfcomm bnep bluetooth nls_utf8 cifs fscache ext2 tpm_infineon snd_hda_codec_analog coretemp kvm_intel kvm hp_wmi sparse_keymap gpio_ich snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc dm_multipath snd_seq_midi scsi_dh tpm_tis wmi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer nvidia(POF) mac_hid snd psmouse microcode serio_raw lpc_ich mei soundcore lp parport btrfs zlib_deflate libcrc32c hid_generic usbhid hid floppy e1000e usb_storage Pid: 2803, comm: sync Tainted: PF O 3.8.0-19-lowlatency #13 Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] warn_slowpath_null+0x1a/0x20 [] __btrfs_write_out_cache+0x9e2/0xa10 [btrfs] [] ? cpuacct_charge+0x75/0x80 [] ? __switch_to+0x181/0x4d0 [] btrfs_write_out_cache+0x95/0xf0 [btrfs] [] btrfs_write_dirty_block_groups+0x527/0x620 [btrfs] [] commit_cowonly_roots+0x15a/0x22c [btrfs] [] btrfs_commit_transaction+0x62c/0xb00 [btrfs] [] ? finish_wait+0x80/0x80 [] ? fdatawait_one_bdev+0x20/0x20 [] btrfs_sync_fs+0x62/0x110 [btrfs] [] sync_fs_one_sb+0x20/0x30 [] iterate_supers+0xe9/0xf0 [] sys_sync+0x55/0x90 [] system_call_fastpath+0x1a/0x1f ---[ end trace 88dd9bc54a43ce1a ]--- I don't think it has to do with how much data I write at that time. I also get this error, when I copy a 6kb file and then invoke "sync" to force a write. But… Hm… No, I think it IS size related. As I said, I get this error even with a file which is 6520 bytes "big". BUT I do NOT get this error when I copy a file which is just 1129 bytes "small". Some more tests - I start to get these errors with filesizes > 3916 bytes. Filesizes <= 3916 bytes -> no error. Does that make sense to you? Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Best Practice - Partition, or not?
Hello If I want to manage a complete disk with btrfs, what's the "Best Practice"? Would it be best to create the btrfs filesystem on "/dev/sdb", or would it be better to create just one partition from start to end and then do "mkfs.btrfs /dev/sdb1"? Would the same recomendation hold true, if we're talking about huge disks, like 4TB or so? Cross platform compatibility (OS X, Windows, FreeBSD, …) is of no matter to me. Thanks, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Best Practice - Partition, or not?
Hello dima dima parallels.com> writes: > (NB: grub will not boot from "/dev/sdb", selinux will) Fair point :) I don't plan to boot from the disk, though. It's a data disk, if you will. But, yeah, for a "best practice", that's certainly something to keep in mind. BR, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hello Josef On Wed, May 1, 2013 at 8:57 PM, Josef Bacik wrote: > and build that and then run > > ./btrfs-image -c 9 -t 4 /dev/whatever somefile.img > > and upload the image somewhere so I can take a look at it. Thanks, Sure thing ;) You can download it from my Copy share: https://copy.com/6UUFqWdalibY I gpg encrypted it symmetrically; I'll send you the password in private mail. Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs subv list -> ERROR: Failed to lookup path for root 0 - No such file or directory
Hello I'm on Ubuntu 13.04 with Ubuntu kernel 3.8.0-19-lowlatency and Btrfs v0.20-rc1. Sometimes, I'm unable to list the subvolumes of a filesystem, like so: root@ask-home:~# btrfs subv list /home ERROR: Failed to lookup path for root 0 - No such file or directory This was after I removed a few snapshots of this subvolume. To be able to list again, I've got to do a "list -a": root@ask-home:~# btrfs subv list -a /home ID 256 gen 28692 top level 5 path /home ID 257 gen 28695 top level 256 path a ID 258 gen 28695 top level 5 path /Cloud ID 276 gen 28695 top level 5 path /Media ID 277 gen 28695 top level 5 path /Cloud/Dropbox ID 283 gen 28695 top level 5 path /VirtualBox Then the normal list works again: root@ask-home:~# btrfs subv list /home ID 257 gen 28695 top level 256 path a Reg. the "" - that's (I guess...) because I mounted the subvolume at /home; see /etc/fstab: root@ask-home:~# grep /home /etc/fstab /dev/ssd/Data /home btrfs defaults,noatime,ssd,discard,subvol=home 0 0 Why's that happening? Should I report this as a bug on bugzilla? Regards, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Listing EVERYTHING BUT read only snapshots; and what are snapshots if compared with ZFS?
Hello once more... I'm on Ubuntu 13.04 with Ubuntu kernel 3.8.0-19-lowlatency and Btrfs v0.20-rc1. First a clarification question - if I compare btrfs with zfs (which I know better...), how do snapshots compare? A read only snapshot is what ZFS calls snapshots, and a read/write snapshot would be a clone. Is that about right? Now to the real question - how can I *easily* list everything but the read only snapshots? Is there command which does that? With "btrfs subv list -r /home", I can list all read only snapshots under /home. It lists just the read only snapshots. But with "btrfs subv list /home", I see all subvolumes; not only the read/write snaphots. Here's what I get: root@ask-home:~# btrfs subv list /home ID 257 gen 28776 top level 256 path a root@ask-home:~# btrfs subv list -r /home root@ask-home:~# btrfs subv snaps -r /home /home/foo Create a readonly snapshot of '/home' in '/home/foo' root@ask-home:~# btrfs subv list /home ID 257 gen 28778 top level 256 path a ID 347 gen 28778 top level 256 path foo root@ask-home:~# btrfs subv list -r /home ID 347 gen 28778 top level 256 path foo Now I'd like to have a command which only shows me the "real" subvolumes. How would I do that? For a script, I came up with this: ( btrfs subv list -r "$mp" ; btrfs subv list "$mp" ) | sort | uniq -u Eg: ( btrfs subv list -r /home ; btrfs subv list /home ) | sort | uniq -u ID 257 gen 28788 top level 256 path a Works, but is "somewhat" complicated (logic behind that - list only read only subvolumes + list all subvolumes and then show only those, which appear just once)... Is there an easier, more straight forward way? Thanks, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Creating recursive snapshots for all filesystems
Hello once more...
I'm on Ubuntu 13.04 with Ubuntu kernel 3.8.0-19-lowlatency and
Btrfs v0.20-rc1.
(Did I say that before...? *G*)
Okay, ATM I'm writing a script for creating snapshots for "backups" of
all my btrfs filesystems. I come from a FreeBSD / Solaris background
with heavy use of ZFS.
In ZFS, it's very easy to create snapshots of all "zpools"
("filesystems") of the system, like so:
SnapName=backup.`date +%Y%m%d--%H%M%S`
zpool list -Ho name | while read zpool; do
zfs snapshot -r $zpool@$SnapName &
done ; wait
Well… How to do something like this with btrfs? I do _not_ want to
have to manually list the filesystems/subvolumes I've got in some
script or configuration file.
Where I'm hanging right now, is that I can't seem to figure out a
"bullet proof" way to find all the subvolumes of the filesystems I
might have.
I've got this:
root@ask-home:~# btrfs fi show
failed to read /dev/sr0
Label: 'Data' uuid: 7d2eb10f-aced-4d41-bb7f-7badbf075b6a
Total devices 1 FS bytes used 27.96GB
devid1 size 35.00GB used 35.00GB path /dev/dm-0
Label: 'KernBtrfs' uuid: 8b16bc2a-43e3-40da-89f5-7b333c6682f3
Total devices 1 FS bytes used 655.05MB
devid1 size 11.85GB used 6.04GB path /dev/sda11
Btrfs v0.20-rc1
root@ask-home:~# mount -t btrfs
/dev/mapper/ssd-Data on /data type btrfs (rw,noatime,ssd,discard)
/dev/mapper/ssd-Data on /home type btrfs (rw,noatime,ssd,discard,subvol=home)
/dev/sda11 on /data/Kernel/KernBtrfs type btrfs (rw,noatime)
Now, how would I find all the subvolumes of the btrfs with the label
"Data" (or uuid 7d...) or /dev/dm-0? "btrfs subvolume list" only seems
to operate on where a btrfs is mounted. It works on "path".
I could do "btrfs subv list -a /data", but how would I figure out,
that "/data" is the "root" of the filesystem with uuid 7d...?
For now, I do something like this (-> http://pastebin.com/u08ub1i8):
SnapName=backup.`date +%Y%m%d--%H%M%S`
btrfs fi show 2>/dev/null | awk '/ path / {print $NF}' | while read path; do
SafePath=`echo "$path" | tr / .`
TmpMountDir=`mktemp -d /tmp/.btrfs.mount.$SafePath.XX`
mount -t btrfs $path $TmpMountDir
(btrfs subv list -ar $TmpMountDir; btrfs subv list -a $TmpMountDir)
| sort | uniq -u | while read _id Id _gen Gen _top _level Toplevel
_path Path; do
btrfs subv snaps -r "$TmpMountDir/$Path" "$TmpMountDir/$Path.$SnapName"
done
umount $TmpMountDir
rmdir $TmpMountDir
done
Now, this works, but seems "somewhat" complicated... But
maybe I'm just spoiled by ZFS ;)
Is there an easier way to achieve what I want? I want to achieve:
Creating recursive snapshots for all filesystems
;)
Thanks,
Alexander
--
=>Google+ => http://plus.skwar.me <==
=> Chat (Jabber/Google Talk) => a.sk...@gmail.com <==
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs subv list -> ERROR: Failed to lookup path for root 0 - No such file or directory
Hi Russel Russell Coker coker.com.au> writes: > I asked a similar question about 10 days ago and got the below response which > solved it for me. Thanks a lot. This solved it for me as well. Cheers, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Script for creating/managing snapshots of all subvolumes of all filesystems
Hi
FWIW, I've also written a script which creates and "manages"
(ie. deletes old) snapshots.
It figures out all the available filesystems and creates snaps
for all the available (sub)volumes.
It's also on https://copy.com/WI9AXqTH2nD4 and http://pastebin.com/YX8WKcsR
to avoid line break issues and also with comments.
Regards,
Alexander
- cut here
#!/bin/sh
echo "Usage: $0 SNAPSHOT_TAG NUM_SNAPSHOTS
Create hourly, daily, weekly, and monthly snapshots of btrfs filesystems.
Based somewhat on http://article.gmane.org/gmane.comp.file-
systems.btrfs/12609
Here's my crontab:
00,15,30,45 * * * * $0 frequently 4
38 * * * * $0 hourly 24
08 00 * * * $0 daily 7
08 12 * * 0 $0 weekly 4"
exit 1
fi
SNAPSHOT_TAG="$1"
NUM_SNAPSHOTS="$2"
snap_prefix="snapshot:$SNAPSHOT_TAG:"
snap_date="`date +%Y-%m-%d--%H.%M.%S.%N`"
script_name=`basename "$0"`
log_fac="local5"
log_tag="$script_name"
btrfs_progs_dev_path="/home/a/Copy/Computerkram/Programme/btrfs-
progs.dev/bin"
PATH="$btrfs_progs_dev_path:$PATH"
btrfs fi show 2>/dev/null | awk '/ path / {print $NF}' | while read dev; do
set -- `btrfs fi show 2>/dev/null | grep -B2 " path $dev" | \
grep "Label:" | sed 's,.*: \(.*\) uuid: \(.*\),\1 \2,'`
label="$1"
uuid="$2"
logger -t "$log_tag" -p "$log_fac.info" -- \
"Processing filesystem with label $label and uuid $uuid on $dev"
safe_dev=`echo "$dev" | tr / .`
tmp_mount_dir=`mktemp -d "/tmp/.btrfs.mount.$uuid.$safe_dev.XX"`
if ! mount -t btrfs "$dev" "$tmp_mount_dir"; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: mount -t btrfs $dev $tmp_mount_dir"
exit 1
fi
_snap_name="$tmp_mount_dir/,$snap_prefix$snap_date"
if ! btrfs subv snaps -r "$tmp_mount_dir" "$_snap_name" > /dev/null;
then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: btrfs subv snaps -r $tmp_mount_dir
$_snap_name"
exit 1
else
logger -t "$log_tag" -p "$log_fac.info" -- \
"Created snapshot $Path,$snap_prefix$snap_date for root volume of fs
with uuid $uuid"
fi
(btrfs subv list -r "$tmp_mount_dir" | grep " path ,$snap_prefix" \
| tail -"$NUM_SNAPSHOTS"
btrfs subv list -r "$tmp_mount_dir" | grep " path ,$snap_prefix") \
| sort | uniq -u \
| while read __id IdDel __gen GenDel __top __level ToplevelDel __path
PathDel; do
if ! btrfs subv del "$tmp_mount_dir/$PathDel" > /dev/null; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: btrfs subv del $tmp_mount_dir/$PathDel"
exit 1
else
logger -t "$log_tag" -p "$log_fac.info" -- \
"Removed snapshot $PathDel"
fi
done
(btrfs subv list -ar $tmp_mount_dir; btrfs subv list -a $tmp_mount_dir)
\
| sort | uniq -u \
| while read _id Id _gen Gen _top _level Toplevel _path Path; do
_snap_name="$tmp_mount_dir/$Path,$snap_prefix$snap_date"
if ! btrfs subv snaps -r "$tmp_mount_dir/$Path" "$_snap_name" >
/dev/null; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: btrfs subv snaps -r $tmp_mount_dir/$Path
$_snap_name"
exit 1
else
logger -t "$log_tag" -p "$log_fac.info" -- \
"Created snapshot $Path,$snap_prefix$snap_date for subvolume
$Path"
fi
(btrfs subv list -r "$tmp_mount_dir" \
| grep " path $Path,$snap_prefix" | tail -"$NUM_SNAPSHOTS"
btrfs subv list -r "$tmp_mount_dir"|grep " path
$Path,$snap_prefix") \
| sort | uniq -u \
| while read __id IdDel __gen GenDel __top __level ToplevelDel
__path PathDel; do
if ! btrfs subv del "$tmp_mount_dir/$PathDel" > /dev/null; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: btrfs subv del
$tmp_mount_dir/$PathDel"
exit 1
else
logger -t "$log_tag" -p "$log_fac.info" -- \
"Removed snapshot $PathDel"
fi
done
done
if ! umount "$tmp_mount_dir"; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: umount$tmp_mount_dir"
fi
if ! rmdir "$tmp_mount_dir"; then
logger -t "$log_tag" -p "$log_fac.err" -- \
"Error! Could not do: rmdir $tmp_mount_dir"
fi
done
exit 0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Creating recursive snapshots for all filesystems
Hi Sander humilis.net> writes: > > Alexander Skwar wrote (ao): > > Where I'm hanging right now, is that I can't seem to figure out a > > "bullet proof" way to find all the subvolumes of the filesystems I > > might have. > > > Is there an easier way to achieve what I want? I want to achieve: > > > > Creating recursive snapshots for all filesystems > > Not sure if this helps, but I have subvolid=0, which contains all my > subvolumes, mounted under /.root/ Hm, not quite what I'm after and not nearly as easy as ZFS... "Problem" with your approach: The admin has to maintain this. I was looking for something, which "maints itself", so to say. And your approach also wouldn't scale if there are sub-subvolumes. ZFS really is so much easier (at least regarding that). Thanks a lot, though. It's a worthwhile idea. Regards, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs subv list -> ERROR: Failed to lookup path for root 0 - No such file or directory
Hi again. Well, it still doesn't work reliably. a@ask-home:~$ sudo `which btrfs` subv list /data ERROR: Failed to lookup path for root 0 - No such file or directory While I ran the "list" command, I had, in another terminal, "btrfs subv snapshot" or "btrfs subv delete" running. At the very same time. Which additional information should I provide? Thanks, Alexander On Fri, May 3, 2013 at 1:24 PM, Alexander Skwar wrote: > Hi Russel > > Russell Coker coker.com.au> writes: > >> I asked a similar question about 10 days ago and got the below response > which >> solved it for me. > > > Thanks a lot. This solved it for me as well. > > Cheers, > Alexander > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Creating recursive snapshots for all filesystems
Hello On Sun, May 5, 2013 at 1:05 PM, Kai Krakow wrote: > Alexander Skwar schrieb: > >> Where I'm hanging right now, is that I can't seem to figure out a >> "bullet proof" way to find all the subvolumes of the filesystems I >> might have. > > What about this: > > # btrfs sub list -a / How do I know that "/" is a btrfs? How do I know, that there are not also other btrfs filesystems? :) > You still need to iterate through the mounted btrfs filesystems if you are > using more than one: > > # btrfs fi show | fgrep uuid > Label: 'usb-backup' uuid: 7038c8fa-4293-49e9-b493-a9c46e5663ca > Label: 'system' uuid: d2bb232a-2e8f-4951-8bcc-97e237f1b536 > > Then translate the uuid back to an fspath somehow. Yep. That's basically what I'm doing ATM. > > Another option would be to use blkid: > > # blkid -t TYPE=btrfs Ah, that's cool! Didn't know that blkid had this option. > Still needs translation back to fspathes. But that could be done with > grep/head/lsblk trickery... Hm, I didn't know lsblk until now, but it seems that it doesn't handle LVM well, does it? (See http://pastebin.com/fuZ8HHQi for better readable version.) a@ask-home:~$ blkid -t TYPE=btrfs | grep mapper /dev/mapper/ssd-Data: LABEL="Data" UUID="7d2eb10f-aced-4d41-bb7f-7badbf075b6a" UUID_SUB="582b64f5-edd5-48f2-978e-24df9a839b5b" TYPE="btrfs" a@ask-home:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:00 167.7G 0 disk ├─sda1 8:10 100M 0 part ├─sda2 8:20 10.9G 0 part /dell-recovery ├─sda3 8:30 39.1G 0 part /windows ├─sda4 8:40 1K 0 part ├─sda5 8:50 102M 0 part /boot ├─sda6 8:60 8.1G 0 part [SWAP] ├─sda7 8:70 12.2G 0 part │ ├─ssd-Data (dm-0) 252:0035G 0 lvm │ └─ssd-UbRoot1304 (dm-1) 252:1010G 0 lvm / ├─sda8 8:80 12.2G 0 part │ └─ssd-Data (dm-0) 252:0035G 0 lvm ├─sda9 8:90 12.2G 0 part │ └─ssd-Data (dm-0) 252:0035G 0 lvm ├─sda10 8:10 0 12.2G 0 part │ └─ssd-Data (dm-0) 252:0035G 0 lvm ├─sda11 8:11 0 11.9G 0 part ├─sda12 8:12 0 11.9G 0 part /data/Kernel/KernExt4 ├─sda13 8:13 0 11.9G 0 part ├─sda14 8:14 0 11.9G 0 part /data/Kernel/KernReiserfs └─sda15 8:15 0 11.9G 0 part /data/Kernel/KernXfs sr011:01 1024M 0 rom a@ask-home:~$ lsblk -P|grep Data NAME="ssd-Data (dm-0)" MAJ:MIN="252:0" RM="0" SIZE="35G" RO="0" TYPE="lvm" MOUNTPOINT="" NAME="ssd-Data (dm-0)" MAJ:MIN="252:0" RM="0" SIZE="35G" RO="0" TYPE="lvm" MOUNTPOINT="" NAME="ssd-Data (dm-0)" MAJ:MIN="252:0" RM="0" SIZE="35G" RO="0" TYPE="lvm" MOUNTPOINT="" NAME="ssd-Data (dm-0)" MAJ:MIN="252:0" RM="0" SIZE="35G" RO="0" TYPE="lvm" MOUNTPOINT="" So I guess I'd still need to mount the root volume temporarily somewhere to do the translation. ZFS really wipes the floor with btrfs regarding ease of use as far as that's concerned… That blkid trick was quite useful, though. Thanks! Cheers, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Creating recursive snapshots for all filesystems
Hi Kai On Sun, May 5, 2013 at 6:03 PM, Kai Krakow wrote: > That brings in the idea how bedup seems to handle this. Maybe you want to > take one or the other idea from there as it also has to enumerate all btrfs > filesystems and snapshots: Sure, will have a look. There are _certainly_ more clever ways to do that, than what I came up with ☺ BR, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Script for creating/managing snapshots of all subvolumes of all filesystems
Hi On Sun, May 5, 2013 at 7:36 PM, Kai Krakow wrote: > Alexander Skwar schrieb: > >> FWIW, I've also written a script which creates and "manages" >> (ie. deletes old) snapshots. >> >> It figures out all the available filesystems and creates snaps >> for all the available (sub)volumes. >> >> It's also on https://copy.com/WI9AXqTH2nD4 and >> http://pastebin.com/YX8WKcsR to avoid line break issues and also with >> comments. > > You should change the calls of "btrfs subv del" etc to use unabbreviated > versions of the commands. Otherwise you expose your script to erratic Oh, yes. Very good point. And there really isn't any benefit in using abbreviated versions of the command names in a script anyway. On the command line: Yes, less to type = better. But in a script? Thanks. Very good point indeed! Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs subv list -> ERROR: Failed to lookup path for root 0 - No such file or directory
Hi Wang On Sat, May 4, 2013 at 4:50 PM, Wang Shilong wrote: > I use the latest btrfs-progs, and i can not reproduce the problem, would you > please > double check with this url: > https://git.kernel.org/cgit/linux/kernel/git/mason/btrfs-progs.git/ Must be so, that I by accident used the "stable" version that got shipped with Ubuntu. If I use the one from git, I also cannot reproduce. Thanks Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at fs/btrfs/free-space-cache.c:921 __btrfs_write_out_cache+0x6b9/0x9a0 [btrfs]()
Hello Josef On Wed, May 8, 2013 at 10:47 PM, Josef Bacik wrote: > On Wed, May 08, 2013 at 11:34:40AM -0600, Alexander Skwar wrote: >> Hello Josef >> >> Did you have a chance to look at that image? Did you find anything? >> >> Or should I simply create a new filesystem and forget about the issue? >> > > So the file system isn't corrupt, you just got a giant block group for some > reason. Hmm... Sure? I did start a "btrfs balance" and it consistently fails. See my dmesg output at http://pastebin.com/7XgAhZ1s At first, there are a few WARNING messages: [ 300.422702] WARNING: at /build/buildd/linux-3.8.0/fs/btrfs/delayed-ref.c:454 update_existing_ref+0x119/0x150 [btrfs]() [ 300.422960] WARNING: at /build/buildd/linux-3.8.0/fs/btrfs/delayed-ref.c:454 update_existing_ref+0x119/0x150 [btrfs]() … But there's also an error: [ 300.425633] BTRFS error (device dm-5) in __btrfs_inc_extent_ref:1935: Object already exists [ 300.425634] btrfs is forced readonly What can I do? Other than destroying that filesystem? Regards, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Determining size of subvolumes
Hello What's the easiest way to determine the size of snapshots/subvolumes? As I come from a Solaris/FreeBSD background with ZFS, I compare the feature sets and in ZFS I can easily do this: # zfs list -t snapshot,filesystem -r datapool/home/ftp_example NAMEUSED AVAIL REFER MOUNTPOINT datapool/home/ftp_example 442M 4.66G 346M /home/ftp_example/ datapool/home/ftp_example@Tue 11.7M - 344M - datapool/home/ftp_example@Wed 6.94M - 345M - datapool/home/ftp_example@Thu 5.00M - 344M - datapool/home/ftp_example@Fri 6.87M - 345M - datapool/home/ftp_example@Sat 7.47M - 346M - datapool/home/ftp_example@Sun 5.14M - 346M - datapool/home/ftp_example@Mon 5.02M - 346M - This shows me, that in the "ftp_example" filesystem, there are 346 MB and in the "Wed"nesday snapshot, 6.94 MB got changed (or so…). How could I do something like this with btrfs? PS: On Ubuntu Linux 13.04 with their 3.8.0-20-generic kernel. Btrfs v0.20-rc1-253-g7854c8b-dirty tools (from Git). Thanks a lot, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Filesystem "somewhat" destroyed - need help for recovery/fixing
Hello I think, I somewhat destroyed my btrfs filesystem on my Ubuntu 13.04 kernel 3.8.0-25-lowlatency system. It got destroyed, because the system was hanging for some other reason and I had to remove power... When I try to mount my filesystem (there's only one, with a few subfilesystems), the system crashes. Also btrfsck dies; always like this: a@ask-home:~$ sudo /btrfs-progs.dev/bin/btrfsck /dev/ssd/Data parent transid verify failed on 33327525888 wanted 53973 found 53972 parent transid verify failed on 33327525888 wanted 53973 found 53972 Ignoring transid failure Checking filesystem on /dev/ssd/Data UUID: 7d2eb10f-aced-4d41-bb7f-7badbf075b6a checking extents checking fs roots root 325 inode 31590 errors 400 extent buffer leak: start 33327525888 len 4096 *** Error in `btrfs check': corrupted double-linked list: 0x007a2600 *** a@ask-home:~$ /btrfs-progs.dev/bin/btrfs version Btrfs v0.20-rc1-253-g7854c8b-dirty btrfsck always dies with a corrupted double-linked list. When the system crashes when I try to mount the filesystem, I get a kernel oop. Nothing in syslog, though - too fast :/ On http://wir.myds.me/photo/photo_thumb.php? dir=53637265656e73686f74732f4372617368206d6f756e74206274726673 or http://imgur.com/a/UA7fF I tried to capute a few photos of the crash message. "Screenshots", of sorts... I was able to recover most of the files with btrfs-recover. But not all of it correctly - at least one VirtualBox image is now broken. Well - what do I do now? Could someone help, please? Thanks a lot, Alexander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Filesystem "somewhat" destroyed - need help for recovery/fixing
Hello Josef On Mon, Jun 17, 2013 at 11:21 PM, Josef Bacik wrote: > Pull down my tree > > git://github.com/josefbacik/btrfs-progs.git > > and build and run the fsck in there and see if it's a bit more friendly. I just gave it a try, but wasn't successful, it seems… Kernel still crashes. Maybe checkout the screenphotos at http://goo.gl/DWkRH or http://imgur.com/a/00pTx Thanks, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Filesystem "somewhat" destroyed - need help for recovery/fixing
Hi On Mon, Jun 17, 2013 at 11:43 PM, Alexander Skwar wrote: > Hello Josef > > On Mon, Jun 17, 2013 at 11:21 PM, Josef Bacik wrote: > >> Pull down my tree >> >> git://github.com/josefbacik/btrfs-progs.git >> >> and build and run the fsck in there and see if it's a bit more friendly. > > I just gave it a try, but wasn't successful, it seems… Kernel still > crashes. > Maybe checkout the screenphotos at http://goo.gl/DWkRH or > http://imgur.com/a/00pTx Any other ideas, about what I might be able to do, to revive my btrfs filesystem? Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Filesystem
Hello Rodrigo, On Wed, Jun 26, 2013 at 5:44 PM, Rodrigo Dias Cruz wrote: > I had the very same problem some days ago. > > I have not yet found out how to fix the broken btrfs filesystem. However, I > have been able to recover all my files from the filesystem and copy them to > a brand new ext4 filesystem, that I am using now. Well, yeah, I also tried to do "btrfs-restore". Did not work well. It restored all files, but didn't do so in a good way. An unknown number of files got corrupted (but btrfs-restore went thru anyway). For example a VirtualBox image got destroyed and my Hamster time tracking database. And probably some more. > At the end, all your files should be found at the directory "/backup". If They were. > you wish, you can recreate the filesystem on "/dev/sda1" (using mkfs.btrfs > or mkfs.ext4) and copy the files back to there. I think, I got cured from btrfs… As of now, not ready for daily use, unless you are a real hacker. It does have nice features, but an ugly userland (in comparison to ZFS) and is unstable with real danger of data loss. Maybe in a few years. Regards, Alexander -- =>Google+ => http://plus.skwar.me <== => Chat (Jabber/Google Talk) => a.sk...@gmail.com <== -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
